Beefy Boxes and Bandwidth Generously Provided by pair Networks chromatic writing perl on a camel
The stupid question is the question not asked
 
PerlMonks  

Re: Seeking through a large gzipped file

by NetWallah (Monsignor)
on May 14, 2013 at 15:30 UTC ( #1033507=note: print w/ replies, xml ) Need Help??


in reply to Seeking through a large gzipped file

PerlIO::via::gzip will probably work as fast as "zcat", but without the need to shell out.

Since it gives you a file handle, "seek" should work.

             "I'm fairly sure if they took porn off the Internet, there'd only be one website left, and it'd be called 'Bring Back the Porn!'"
        -- Dr. Cox, Scrubs


Comment on Re: Seeking through a large gzipped file
Re^2: Seeking through a large gzipped file
by roysperlarnab (Initiate) on May 14, 2013 at 17:06 UTC
    I'll try that, thanks
Re^2: Seeking through a large gzipped file
by Corion (Pope) on May 14, 2013 at 17:34 UTC

    If you have at least one idle CPU, using open "zcat $file |" will allow you to spread the load across two CPUs, one for the perl process and one for the zcat process. That way, all the CPU cycles you need for decompressing the data get offloaded from perl. This should be at least as fast as using [mod://PerlIO::via::gzip</c> if not faster, as usually transferring the data between processes is fairly fast.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1033507]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (15)
As of 2014-04-16 20:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (433 votes), past polls