Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re^2: how to speed up dupe checking of arrays

by ultibuzz (Monk)
on Jul 31, 2007 at 14:58 UTC ( #629845=note: print w/replies, xml ) Need Help??


in reply to Re: how to speed up dupe checking of arrays
in thread how to speed up dupe checking of arrays

we have a oracle 10g on a hp superdome loading all data with sqlloader and direct=true as option takes longer then the dupechecking with perl and then load filtered data in.
loading the data with perl into the db or normal sqlloader without tunig woud take ages.
with direct=true the sqlloader pumps in 10million rows in less then 20 sec, without direct=true it takes 10 minutes+ because orcale set commit points and check for data correctness ^^.
other point is we need several indexes and partiotion groups on it wich woud take hours to create if we used the unfiltered data.

kd ultibuzz

  • Comment on Re^2: how to speed up dupe checking of arrays

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://629845]
help
Chatterbox?
[Corion]: The sad thing is that my script sits at the end of the pipeline and can only look at the payments due today or tomorrow basically, while there are many more systems further up in the pipeline
[GotToBTru]: better late than never, I guess
[Corion]: GotToBTru: Sure - there is a long and sad story of many frantic cleanups that led us to implement this notification ;)

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (7)
As of 2017-03-29 11:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Should Pluto Get Its Planethood Back?



    Results (347 votes). Check out past polls.