Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Performance : process lakhs of database records

by selva (Scribe)
on Jun 03, 2010 at 07:30 UTC ( #842866=perlquestion: print w/ replies, xml ) Need Help??
selva has asked for the wisdom of the Perl Monks concerning the following question:


Dear Monks,

I have a requirement to process lakhs of recorda which
is in remote machine. Kindly instruct me the best
solution to process it.


Thanks in Advance

Comment on Performance : process lakhs of database records
Re: Performance : process lakhs of database records
by marto (Chancellor) on Jun 03, 2010 at 07:32 UTC

      "Lakh" is a unit much used in India and thereabouts, and is equal to 100,000.

      Paul

        Also known as 100 grand pianos, right? :)

        Yeah, I linked to the wiki entry in my post under the suspicion that was what OP meant.

Re: Performance : process lakhs of database records
by JavaFan (Canon) on Jun 03, 2010 at 09:40 UTC
    I fail to pinpoint a particular problem in your question. Is it the "how do I connect to the database" issue you have a problem with? Is it "how do I process records"? Is it "how do I write a Perl program"? Is it "please write a program for me?". Please elaborate. And tell us what you've done so far.

      Currently i take nearly hundred thousands of record and load it in hash.
      After that compare the record values to another table.
      It leads to performance issue. Is there any efficient way to improve this.

        Why not simply do a database query via DBI and compare the values between the two tables?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://842866]
Approved by moritz
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (10)
As of 2014-07-10 08:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (203 votes), past polls