Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re: Make my script faster and more efficient

by nagalenoj (Friar)
on Dec 03, 2008 at 06:24 UTC ( #727595=note: print w/replies, xml ) Need Help??

in reply to Make my script faster and more efficient


Your problem would have been solved by a reply posted(The code without using the inner loop).

As you have told that you are going to work with huge data, It's better to use the DBM hash instead of hash. Storing more records in the hash, will take more memory. Using DBM hash the data will stored in a database file, and at the same time it is as easy as working with hashes.

For clarity refer the following sample code to work with DBM hashes,

Example: #!/usr/bin/perl #===================================================================== +========== # FILE: # USAGE: ./ # DESCRIPTION: To test DBM. #===================================================================== +========== use strict; use warnings; my %hash; # To open a dbmfile and associate with hash dbmopen(%hash, "nagalenoj", '0666') || die "Can't open database bookdb +!"; $hash{'one'}="1"; $hash{'two'}="2"; $hash{'three'}="3"; #Retrieving values print $hash{'three'}; dbmclose %hash; #===================================================================== +==========

And I noted that you have opened all the files in the beginning. You can avoid that. open the second file after finishing the process with first file. This will help you to improve the efficiency and coding standard

Instead of using print and exit, you could use die.

Replies are listed 'Best First'.
Re^2: Make my script faster and more efficient
by tilly (Archbishop) on Dec 03, 2008 at 17:57 UTC
    Particularly when working with large data sets, if the hash can be stored in memory, it really, really should be due to performance considerations.

    Secondly if you have to use a DBM file, I highly recommend not using dbmopen. That is because of the following gotcha. You wrote a program that stores data using dbmopen. Your program has been successfully running, storing, and accessing data, every night for months. Then it suddenly stops working one day, and nobody knows how to get at your data. How are you going to figure out that this is because a sysadmin installed DB_File on your machine? And once you do figure it out, how are you going to fix your program?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://727595]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2016-10-24 01:22 GMT
Find Nodes?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?

    Results (302 votes). Check out past polls.