Re: How do I search this binary file?

[kschwab]

How about a find/rewind approach ? Go ahead and set $/ to your delimiter, but don't assign to a variable as you read. (for the sake of conserving memory on huge records)

Once you have the record boundaries, then you could seek() back and process it however you need to.

This is making the assumption that the bare <FILE> construct isn't using memory. I'm not clear on whether that's true or not.

#!/usr/bin/perl
my $delim="asdf";
$/=$delim;
open(FILE,"/some/big/file");
my $offset=0;
my $lastoffset=0;
for (;;) {
   <FILE>;
   my $offset=tell(FILE);
   last if eof(FILE); 
   print "this record spans [$lastoffset] to [$offset], including";
   print "the delimiter\n";
   print " this means you could seek to $lastoffset and read for ";
   printf "%d bytes to get the record\n",
       $offset-$lastoffset-length($delim);
   $lastoffset=$offset;
}

[John M. Dlugosz]

If <HANDLE> in void context functioned as a scan rather than a read, that would be terrific and elegant.

[kschwab]

It is a read, but in void context, it doesn't appear to be using excessive memory ( I suppose since it's not filling $_ or any other variable ).

I did try this on a fairly large file (with large "records"), and didn't notice a significant growth in memory footprint compared to a smaller file.

Did you have a different experience ? Update: Tried it on a really large file, and sure enough, it eats up VM :) Never mind...sounds like asking the gods to map <HANDLE> in a void context into a scan might be a good idea though...