Re: buffering zipped pipes

[BrowserUk]

I'd skip all the seeks and tells and use a thread and a queue this way:

#! perl -slw
use strict;
use threads;
use Thread::Queue;

my $file = $ARGV[ 0 ];

my $Q = new Thread::Queue;
async {
    open LIBBIN, "gunzip -c $file |" or die $!;
    while( <LIBBIN> ) {
        $Q->enqueue( $_ );
        sleep 1 while $Q->pending > 100;                  ## Arbitrary
+ limit to stop it from running away with memory
    }
    $Q->enqueue( undef ); ## close queue
}->detach;

## Main code swaps $Q->dequeue for <LIBBIN> ...

while( my $line = $Q->dequeue ) {

   if( some circumstance ) {
       unshift @{ $Q->{queue} }, $line;                 ## 'push' a li
+ne back to the queue.
   }
   
   ## do other stuff here..
   
}

The only slightly 'trick' thing here is that I'm reaching inside the Queue object to gain access to the underlying array reference with $Q->{queue}, which the OO purists would have a hissy fit about, but to my mind one of the big advantages of Perl's simple native OO mechanism, is that I as a user can easily extend -- or in this case regain access to existing -- functionality without having to go cap in hand to the authors of the module or complicate things with OO complexity.

If the basic mechanism of having a separate thread doing the IO and a queue to buffer the data fits with your mindset, then you should also take a look at the "ADVANCED METHODS" in the Thread::Queue docs. For the most part I consider their addition to the queue module an abomination, but for the first time ever, I can see your application might find them useful.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.