Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Performance Trap - Opening/Closing Files Inside a Loop

by TedPride (Priest)
on Dec 10, 2004 at 10:03 UTC ( #413793=note: print w/ replies, xml ) Need Help??


in reply to Performance Trap - Opening/Closing Files Inside a Loop

tachyon is correct that printing once to each file is much more efficient then printing hundreds of thousands of times. The question is, do you have enough memory to do this? I'd personally store output in a hash, with a filehandle and output buffer for each file, and just print to the filehandle whenever the buffer reached allowable memory / number of files amount of data. Anything still in the buffers at the end of the run would also be printed to the files.

Allowing 10 MB of memory and 100 files, for instance, you could print whenever the buffer for an individual file reached 100K or so.


Comment on Re: Performance Trap - Opening/Closing Files Inside a Loop

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://413793]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (10)
As of 2014-09-17 20:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (99 votes), past polls