Update: Wanted to mention that this will not work if the pattern crosses 2 or more chunks.
The following uses MCE which does not read the entire file into memory. The example is based on fastsearch.pl from File::Map.
use 5.010;
use strict;
use warnings;
use MCE::Flow;
die "Not enough arguments given\n" if @ARGV < 2;
my $regex = shift;
$regex = qr/$regex/;
sub user_func {
my ($mce, $slurp_ref, $chunk_id) = @_;
if (my $match = ${ $slurp_ref } =~ $regex ? 1 : 0) {
$mce->gather($match);
$mce->abort;
}
}
for my $filename (@ARGV) {
my @match = mce_flow_f { use_slurpio => 1 }, \&user_func, $filename;
say "File '$filename' does".( scalar @match ? "" : "n't" )." match";
}
Am providing a comparison in the event performance is a concern. Basically, MCE keeps up with File::Map.
$ time ./file_map.pl patternabc big_file
File 'big_file' doesn't match
real 0m0.327s
user 0m0.267s
sys 0m0.060s
$ time ./mce_slurp.pl patternabc big_file
File 'big_file' doesn't match
real 0m0.152s
user 0m0.263s
sys 0m0.118s
Also, see Re: Threads From Hell #2: How To Parse A Very Huge File.
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|