Beefy Boxes and Bandwidth Generously Provided by pair Networks Cowboy Neal with Hat
P is for Practical
 
PerlMonks  

Re: parallel reading

by Zaxo (Archbishop)
on May 09, 2006 at 13:38 UTC ( #548208=note: print w/ replies, xml ) Need Help??


in reply to parallel reading

Here's another,

open my $out, '>', 'ABC' or die $!; { local $_; open my $A, '<', 'A' or die $!; open my $B, '<', 'B' or die $!; open my $C, '<', 'C' or die $!; no warnings 'uninitialized'; while ($_ = <$A> . <$B> . <$C>) { s/\n//g; print $out $_, "\n"; } } close $out or warn $!;
That will let the files have different numbers of lines. Memory use is small, and independent of file size.

Update: Repaired the thinko blazar++ spotted. Empty lines are not a problem - we don't chomp, so they retain newlines until we s/// them gone. I like blazar's extension to different numbers of files.

After Compline,
Zaxo


Comment on Re: parallel reading
Download Code
Re^2: parallel reading
by blazar (Canon) on May 09, 2006 at 14:42 UTC

    Nice approach. And my be merged with mine, e.g.:

    #!/usr/bin/perl -l use strict; use warnings; my @fh=map { open my $fh, '<', $_ or die "Can't open `$_': $!\n"; $fh } @ARGV; no warnings 'uninitialized'; print while $_=join '', map { chomp(my $line=<$_>); $line } @fh, __END__

    However:

    • you should s/undefined/uninitialized/;
    • it may not be fully reliable if empty lines are to be expected in the files.

    Update: the second point was a thinko as Zaxo pointed out.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://548208]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (14)
As of 2014-04-17 20:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (455 votes), past polls