Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: How do I search this binary file?

by kschwab (Priest)
on Aug 20, 2002 at 19:42 UTC ( #191577=note: print w/ replies, xml ) Need Help??


in reply to How do I search this binary file?

How about a find/rewind approach ? Go ahead and set $/ to your delimiter, but don't assign to a variable as you read. (for the sake of conserving memory on huge records)

Once you have the record boundaries, then you could seek() back and process it however you need to.

This is making the assumption that the bare <FILE> construct isn't using memory. I'm not clear on whether that's true or not.

#!/usr/bin/perl my $delim="asdf"; $/=$delim; open(FILE,"/some/big/file"); my $offset=0; my $lastoffset=0; for (;;) { <FILE>; my $offset=tell(FILE); last if eof(FILE); print "this record spans [$lastoffset] to [$offset], including"; print "the delimiter\n"; print " this means you could seek to $lastoffset and read for "; printf "%d bytes to get the record\n", $offset-$lastoffset-length($delim); $lastoffset=$offset; }


Comment on Re: How do I search this binary file?
Download Code
Re: Re: How do I search this binary file?
by John M. Dlugosz (Monsignor) on Aug 20, 2002 at 22:19 UTC
    If <HANDLE> in void context functioned as a scan rather than a read, that would be terrific and elegant.
      It is a read, but in void context, it doesn't appear to be using excessive memory ( I suppose since it's not filling $_ or any other variable ).

      I did try this on a fairly large file (with large "records"), and didn't notice a significant growth in memory footprint compared to a smaller file.

      Did you have a different experience ? Update: Tried it on a really large file, and sure enough, it eats up VM :) Never mind...sounds like asking the gods to map <HANDLE> in a void context into a scan might be a good idea though...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://191577]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (7)
As of 2014-11-23 04:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (128 votes), past polls