Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

Re: How do I search this binary file?

by kschwab (Priest)
on Aug 20, 2002 at 19:42 UTC ( #191577=note: print w/ replies, xml ) Need Help??

in reply to How do I search this binary file?

How about a find/rewind approach ? Go ahead and set $/ to your delimiter, but don't assign to a variable as you read. (for the sake of conserving memory on huge records)

Once you have the record boundaries, then you could seek() back and process it however you need to.

This is making the assumption that the bare <FILE> construct isn't using memory. I'm not clear on whether that's true or not.

#!/usr/bin/perl my $delim="asdf"; $/=$delim; open(FILE,"/some/big/file"); my $offset=0; my $lastoffset=0; for (;;) { <FILE>; my $offset=tell(FILE); last if eof(FILE); print "this record spans [$lastoffset] to [$offset], including"; print "the delimiter\n"; print " this means you could seek to $lastoffset and read for "; printf "%d bytes to get the record\n", $offset-$lastoffset-length($delim); $lastoffset=$offset; }

Comment on Re: How do I search this binary file?
Download Code
Replies are listed 'Best First'.
Re: Re: How do I search this binary file?
by John M. Dlugosz (Monsignor) on Aug 20, 2002 at 22:19 UTC
    If <HANDLE> in void context functioned as a scan rather than a read, that would be terrific and elegant.
      It is a read, but in void context, it doesn't appear to be using excessive memory ( I suppose since it's not filling $_ or any other variable ).

      I did try this on a fairly large file (with large "records"), and didn't notice a significant growth in memory footprint compared to a smaller file.

      Did you have a different experience ? Update: Tried it on a really large file, and sure enough, it eats up VM :) Never mind...sounds like asking the gods to map <HANDLE> in a void context into a scan might be a good idea though...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://191577]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (9)
As of 2015-10-09 18:24 GMT
Find Nodes?
    Voting Booth?

    Does Humor Belong in Programming?

    Results (249 votes), past polls