Beefy Boxes and Bandwidth Generously Provided by pair Networks RobOMonk
Perl: the Markov chain saw
 
PerlMonks  

Re^2: Seeking through a large gzipped file

by Corion (Pope)
on May 14, 2013 at 17:34 UTC ( #1033529=note: print w/ replies, xml ) Need Help??


in reply to Re: Seeking through a large gzipped file
in thread Seeking through a large gzipped file

If you have at least one idle CPU, using open "zcat $file |" will allow you to spread the load across two CPUs, one for the perl process and one for the zcat process. That way, all the CPU cycles you need for decompressing the data get offloaded from perl. This should be at least as fast as using [mod://PerlIO::via::gzip</c> if not faster, as usually transferring the data between processes is fairly fast.


Comment on Re^2: Seeking through a large gzipped file
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1033529]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (9)
As of 2014-04-17 22:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (458 votes), past polls