Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re: How do I search this binary file?

by kschwab (Priest)
on Aug 20, 2002 at 19:42 UTC ( #191577=note: print w/replies, xml ) Need Help??

in reply to How do I search this binary file?

How about a find/rewind approach ? Go ahead and set $/ to your delimiter, but don't assign to a variable as you read. (for the sake of conserving memory on huge records)

Once you have the record boundaries, then you could seek() back and process it however you need to.

This is making the assumption that the bare <FILE> construct isn't using memory. I'm not clear on whether that's true or not.

#!/usr/bin/perl my $delim="asdf"; $/=$delim; open(FILE,"/some/big/file"); my $offset=0; my $lastoffset=0; for (;;) { <FILE>; my $offset=tell(FILE); last if eof(FILE); print "this record spans [$lastoffset] to [$offset], including"; print "the delimiter\n"; print " this means you could seek to $lastoffset and read for "; printf "%d bytes to get the record\n", $offset-$lastoffset-length($delim); $lastoffset=$offset; }

Replies are listed 'Best First'.
Re: Re: How do I search this binary file?
by John M. Dlugosz (Monsignor) on Aug 20, 2002 at 22:19 UTC
    If <HANDLE> in void context functioned as a scan rather than a read, that would be terrific and elegant.
      It is a read, but in void context, it doesn't appear to be using excessive memory ( I suppose since it's not filling $_ or any other variable ).

      I did try this on a fairly large file (with large "records"), and didn't notice a significant growth in memory footprint compared to a smaller file.

      Did you have a different experience ? Update: Tried it on a really large file, and sure enough, it eats up VM :) Never mind...sounds like asking the gods to map <HANDLE> in a void context into a scan might be a good idea though...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://191577]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (4)
As of 2018-05-20 14:42 GMT
Find Nodes?
    Voting Booth?