Re: Multiprocessing on Windows (Cannot reproduce!)

[BrowserUk]

In summary, my questions are:

Are you running Windows in a VM? That's the only thing that comes to mind that might explain your results. I cannot reproduce them at all

I only have 4 cores, and the result from your script (slightly tweaked) show an almost perfect split of processing:

#! perl -slw
use strict;
use threads;
use Time::HiRes 'time';

our $T //= 4;

my @threads;
my $start = time;

foreach my $i (1 .. $T ) {
    $threads[$i] = threads->create(\&Work, $i);
}
foreach my $i ( 1 .. $T ) {
    $threads[$i]->join();
}

my $stop = time - $start;
printf "\nclock: %f sec user: %f\n", $stop, (times())[0];

exit;

#####

sub Work {
    my ($i) = @_;

    foreach ( 1 .. ( 20e5 / $T ) ) {
        my $acct_nrs = "abc\txyz\tdef\tabc\tghi\tghi";
        my @temp = split(m/\t/, $acct_nrs, -1);
        @temp = ( sort keys %{{ map { $_ => 1 } @temp }} );
        my $ans = join(', ', @temp);
    }
    printf " $i";
    return;
}

__END__
C:\test>for /l %i in (1,1,4) do @972137 -T=%i
 1
clock: 29.351637 sec user: 29.093000
 1 2
clock: 14.986346 sec user: 29.765000
 1 2 3
clock: 10.131188 sec user: 29.968000
 2 3 4 1
clock: 7.781729 sec user: 29.796000

Update: Ditto for 5.14:

C:\test>for /l %i in (1,1,4) do @\perl64-14\bin\perl.exe -slw 972137.p
+l -T=%i
 1
clock: 27.309776 sec user: 27.343000
 2 1
clock: 13.878018 sec user: 27.625000
 2 1 3
clock: 9.370494 sec user: 27.875000
 3 2 1 4
clock: 7.205594 sec user: 27.765000

[JohnRS]

No, I'm just running straight under Windows 7.

When I run your version I get the same results as I did originally. So perhaps there is something strange going on with my machine. I will tear into it and see what I can find.

I appreciate your help!

[JohnRS]

Followup: I figured it out. It wasn't anything wrong with my machine. It's just how it works.

I'm running a I7 840QM processor in a laptop. When you run just one or two CPU's, Turbo mode kicks in and boosts your speed by about 71%. So my "baseline" measurement of 25 seconds should really have been corrected to 43 seconds. Indeed, this matches almost perfectly with my 4 thread test results of 10.8 clock time and 42.1 user time.

Then the amount of CPU power changes. Hyper Threading doesn't really give you twice as much crunch power. It depends on what you are doing, but in this case it gave me about 50%. Again, this explains why the CPU time went from 42 to 62 (50% more) when running the test for 4 and 8 threads, rather than remaining constant.

Heat is definitely a bummer and it's worse in my laptop than it would be in a desktop or server. So overall I get about a 5.5x speed increase when running 8 threads instead of the full 8x increase.

Fortunately, I'll be running the actual job on a real server. I tested it and I see about a 7.8x speed increase when running the 8 thread test on it. So all is well!

Thank you again for your help. It put me on the right path to understand what was going on.

[BrowserUk]

Ah! The ol' when is a thread not a thread? When it's hyper :)

[BrowserUk]

A further thought. What that doesn't explain is why you got such different (more favourable) results when using linux?

Or were those results from runs on different hardware?