Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re: Performance Question

by dws (Chancellor)
on May 08, 2002 at 19:01 UTC ( #165137=note: print w/replies, xml ) Need Help??


in reply to Performance Question

I have some proof-of-concept code in Matching in Huge files that you might be able to adapt if your substitutions span lines and you want to do them large-chunk-at-a-time.

Reading an 81Gb file in 8K chunks requires about 10.7 million reads. You can reduce that number by reading the file in larger chunks, via sysread().

Another thing you might look at is whether part of the performance hit you're seeing has to do with disk. If you're writing to the same physical drive that you're reading from, the OS has to move the disk head a lot. This takes time that can add up. Doing writes in larger chunks (via syswrite()) should help, though writing to a separate disk is preferrable. Writing to a heavily fragmented drive will also add time.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://165137]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (5)
As of 2020-09-21 18:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If at first I donít succeed, I Ö










    Results (126 votes). Check out past polls.

    Notices?