Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^2: Matching in huge files

by dws (Chancellor)
on Dec 09, 2004 at 17:11 UTC ( #413620=note: print w/replies, xml ) Need Help??


in reply to Re: Matching in huge files
in thread Matching in huge files

I am having problems when the window happens to split the string you are searching for in two, so can never be found - how can this be fixed?

The algorithm uses a sliding window, and matches strings that fall within that (sliding) window. If you're trying to match a string that doesn't fit in the window, make the window larger. Or if you think you've found a problem, post a test case that demonstrates the failure.

Replies are listed 'Best First'.
Re^3: Matching in huge files
by Anonymous Monk on Jul 06, 2006 at 16:14 UTC
    Yep, it is very fast, but why is that better than this:
    open(F, "<", $file) or die "$file: $!"; binmode(F); undef $/; # switch off end-of-line separating # read file in large chunks while (<F>) { while ( m/$re/oigsm ) { print "$1\n"; } } $/ = '\n'; # switch back to line mode close(F);

    ?

    Thanks,
    Tamas

      but why is that better than this: ...

      My fragment doesn't assume that the huge file will fit in memory, and it matches across read boundaries. Your approach sets up for a single-read slurp.

      In addition to the answer that dws has already given, the original approach is better because it doesn't assume that the IRS was previously "\n", and it certainly doesn't put the IRS back as a literal \n (not a newline character, because of the single quotes). The usual idiom for changing $/ is to wrap any changes to it in a block, and then localise within the block.

      Also, maybe it's just my unfamiliarity with binmode, but I think that undef-ing the IRS means that the while loop only ever runs once.

Re^3: Matching in huge files
by Anonymous Monk on Oct 10, 2008 at 10:56 UTC
    Making the window large only limits the posibilities of having the string you are looking for cut in two, there is no real way to prevent this from happening unless you are looking for a fixed size string. In that case you could always keep that fixed size of the old window and append the new window to the old. This way if there was an intersection you have just undone it, make sure to move your position back accordingly.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://413620]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (6)
As of 2019-12-16 09:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?