Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^2: Can Perl write multiple files at the same time?

by marioroy (Prior)
on Dec 13, 2014 at 02:39 UTC ( [id://1110235]=note: print w/replies, xml ) Need Help??


in reply to Re: Can Perl write multiple files at the same time?
in thread Can Perl write multiple files at the same time?

The MCE 1.515 was moved to back pan some time back. Here is the URL.

https://metacpan.org/pod/MCE

There is another example Re^4: Using MCE to write to multiple files..

The MCE->sendto method is available. A nice thing is that MCE caches the file handle, therefore only opening the file handle once during the run. The sendto method appends, thus the reason for unlink.

Below, using the Flow model as input_data is not required to run.

use MCE::Flow max_workers => 8; my $file1 = "/path/to/file1.txt"; my $file2 = "/path/to/file2.txt"; unlink $file1 if -e $file1; unlink $file2 if -e $file2; mce_flow sub { my $wid = MCE->wid; if (MCE->wid % 2 == 0) { MCE->sendto("file:$file1", "sunny day from $wid\n"); } else { MCE->sendto("file:$file2", "raining day from $wid\n"); } };

Another way is via MCE->print, MCE->printf, MCE->say. MCE serializes data from workers to the manager process. The output is done by the manager process. These methods are beneficial when many workers write to same file simultaneously.

use MCE::Loop chunk_size => 1, max_workers => 8; my @input = (100..199); open my $fh_1, '>', '/path/to/file1.txt'; open my $fh_2, '>', '/path/to/file2.txt'; mce_loop { my ($mce, $chunk_ref, $chunk_id) = @_; if ($chunk_id % 2 == 0) { MCE->say($fh_1, "id: $chunk_id: input $_"); } else { MCE->say($fh_2, "id: $chunk_id: input $_"); } } @input; close $fh_1; close $fh_2;

In case the reader missed it, the syntax for mce_flow requires the sub in front of the opening brace unlike mce_loop which takes a block. The reason is that mce_flow can take many anonymous blocks; e.g. mce_flow sub { ... }, sub { ... }.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1110235]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (6)
As of 2024-03-28 20:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found