Beefy Boxes and Bandwidth Generously Provided by pair Networks DiBona
There's more than one way to do things
 
PerlMonks  

Re: How to save memory, parsing a big file.

by mirod (Canon)
on Mar 01, 2006 at 10:21 UTC ( [id://533670]=note: print w/replies, xml ) Need Help??

This is an archived low-energy page for bots and other anonmyous visitors. Please sign up if you are a human and want to interact.


in reply to How to save memory, parsing a big file.

If it is too big to fit in memory... then you have to use the disk! The easiest to use is probably GDBM_File, which will let you tie the hash to a disk file (but you will need to serialize the values of the hash). You could also go for a full DBMS, DBD::SQLite is very convenient, as the DB is a single file. it is pretty fast too.

In any case you will need to rewrite your code, and it will probably take much longer to run, there is no miracle here!

  • Comment on Re: How to save memory, parsing a big file.

Replies are listed 'Best First'.
Re^2: How to save memory, parsing a big file.
by dragonchild (Archbishop) on Mar 01, 2006 at 12:21 UTC
    DBM::Deep will avoid both those problems. See my reply above.
    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
[reply]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://533670]
help
Sections?
Information?
Find Nodes?
Leftovers?
    Notices?
    • hippo‥ epoptai's answer Re: how do I set a cookie and redirect was blessed by hippo!
    • erzuuli‥ Anonymous Monks are no longer allowed to use Super Search, due to an excessive use of this resource by robots.