Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Re: Seeking through a large gzipped file

by NetWallah (Canon)
on May 14, 2013 at 15:30 UTC ( #1033507=note: print w/replies, xml ) Need Help??

in reply to Seeking through a large gzipped file

PerlIO::via::gzip will probably work as fast as "zcat", but without the need to shell out.

Since it gives you a file handle, "seek" should work.

             "I'm fairly sure if they took porn off the Internet, there'd only be one website left, and it'd be called 'Bring Back the Porn!'"
        -- Dr. Cox, Scrubs

  • Comment on Re: Seeking through a large gzipped file

Replies are listed 'Best First'.
Re^2: Seeking through a large gzipped file
by Corion (Pope) on May 14, 2013 at 17:34 UTC

    If you have at least one idle CPU, using open "zcat $file |" will allow you to spread the load across two CPUs, one for the perl process and one for the zcat process. That way, all the CPU cycles you need for decompressing the data get offloaded from perl. This should be at least as fast as using [mod://PerlIO::via::gzip</c> if not faster, as usually transferring the data between processes is fairly fast.

Re^2: Seeking through a large gzipped file
by roysperlarnab (Initiate) on May 14, 2013 at 17:06 UTC
    I'll try that, thanks

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1033507]
[thezip]: I can always tell when the dogs have been mousing, as there is a concentric ring of mud around the end of their snouts when they do.
[thezip]: As for more physical evidence, that always seems to be missing somehow
[Rabbi Bob]: We have a Jack Russell: off the leash she is a torpedo against mice, rats, chipmunks and squirrels
[Rabbi Bob]: Doesn't eat them, just snaps, kills, drops and moves on.
[thezip]: Efficient, that one.
[erix]: my dog could do the classic mouse jump (never caught a mouse :)
[erix]: he caught a hare once (must have been a young one...)

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (14)
As of 2017-11-17 20:01 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (272 votes). Check out past polls.