Re^2: Parallel running the process

[ariesb2b]

thanks roboticus. yes definitely this is a way to do it. But This involves lot of manual work to be done. I have to run the script monthly and will be putting it inside cron job.

Is there any way that i can do this in a script itself. Like the process running in background and it writes the output in the desired file as and when it completes.

[roboticus]

ariesb2b:

It doesn't have to be hard. I was thinking that if that approach worked for you, that you could write a simple script to do the overhead for you. Something like (untested):

#!/usr/bin/perl
use strict;
use warnings;
use autodie;

my $num_jobs=4;

# Split the work up
open my $IFH, '<', 'data.inp';
my @OFH;
open $OFH[$_-1], '>', "data.$_" for 1 .. $num_jobs;
my $cnt=0;
while (<$IFH>) {
    ++$cnt;
    my $FH = $OFH[$cnt%$num_jobs];
    print $FH, $_;
}
close $OFH[$_-1] for 1 .. $num_jobs;

# Do the work
for my $j (1 .. $num_jobs) {
    `perl orig_do_job --infile=data.$_ --outfile=data.out.$_  & `;
}

# Collect the results
`cat data.out.* >data.out`;

...roboticus

When your only tool is a hammer, all problems look like your thumb.

[JavaFan]

Sure. You may want to read about fork. Or threads, if you feel more comfortable. Or the classic solution: a select loop. But I assume you're using the DBI to give you database access, and I don't think the DBI has support to give back control to the program while waiting for the results.

I would probably use fork, but that's because I understand the concept well, and a simple solution seems to be good enough to solve your problem.

[Anonymous Monk]

Or, just a batch file / shell script to launch the set of instances in the background.