Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Make my script faster and more efficient

by nagalenoj (Friar)
on Dec 03, 2008 at 06:24 UTC ( #727595=note: print w/ replies, xml ) Need Help??


in reply to Make my script faster and more efficient

Hi,

Your problem would have been solved by a reply posted(The code without using the inner loop).

As you have told that you are going to work with huge data, It's better to use the DBM hash instead of hash. Storing more records in the hash, will take more memory. Using DBM hash the data will stored in a database file, and at the same time it is as easy as working with hashes.

For clarity refer the following sample code to work with DBM hashes,

Example: #!/usr/bin/perl #===================================================================== +========== # FILE: dbm_test.pl # USAGE: ./dbm_test.pl # DESCRIPTION: To test DBM. #===================================================================== +========== use strict; use warnings; my %hash; # To open a dbmfile and associate with hash dbmopen(%hash, "nagalenoj", '0666') || die "Can't open database bookdb +!"; $hash{'one'}="1"; $hash{'two'}="2"; $hash{'three'}="3"; #Retrieving values print $hash{'three'}; dbmclose %hash; #===================================================================== +==========

And I noted that you have opened all the files in the beginning. You can avoid that. open the second file after finishing the process with first file. This will help you to improve the efficiency and coding standard

Instead of using print and exit, you could use die.


Comment on Re: Make my script faster and more efficient
Download Code
Re^2: Make my script faster and more efficient
by tilly (Archbishop) on Dec 03, 2008 at 17:57 UTC
    Particularly when working with large data sets, if the hash can be stored in memory, it really, really should be due to performance considerations.

    Secondly if you have to use a DBM file, I highly recommend not using dbmopen. That is because of the following gotcha. You wrote a program that stores data using dbmopen. Your program has been successfully running, storing, and accessing data, every night for months. Then it suddenly stops working one day, and nobody knows how to get at your data. How are you going to figure out that this is because a sysadmin installed DB_File on your machine? And once you do figure it out, how are you going to fix your program?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://727595]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2014-07-13 22:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (252 votes), past polls