in reply to buffering zipped pipes

I'd skip all the seeks and tells and use a thread and a queue this way:

#! perl -slw use strict; use threads; use Thread::Queue; my $file = $ARGV[ 0 ]; my $Q = new Thread::Queue; async { open LIBBIN, "gunzip -c $file |" or die $!; while( <LIBBIN> ) { $Q->enqueue( $_ ); sleep 1 while $Q->pending > 100; ## Arbitrary + limit to stop it from running away with memory } $Q->enqueue( undef ); ## close queue }->detach; ## Main code swaps $Q->dequeue for <LIBBIN> ... while( my $line = $Q->dequeue ) { if( some circumstance ) { unshift @{ $Q->{queue} }, $line; ## 'push' a li +ne back to the queue. } ## do other stuff here.. }

The only slightly 'trick' thing here is that I'm reaching inside the Queue object to gain access to the underlying array reference with $Q->{queue}, which the OO purists would have a hissy fit about, but to my mind one of the big advantages of Perl's simple native OO mechanism, is that I as a user can easily extend -- or in this case regain access to existing -- functionality without having to go cap in hand to the authors of the module or complicate things with OO complexity.

If the basic mechanism of having a separate thread doing the IO and a queue to buffer the data fits with your mindset, then you should also take a look at the "ADVANCED METHODS" in the Thread::Queue docs. For the most part I consider their addition to the queue module an abomination, but for the first time ever, I can see your application might find them useful.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
In the absence of evidence, opinion is indistinguishable from prejudice.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.