Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

Re: Sharing statics variable between Threads

by roboticus (Chancellor)
on Aug 30, 2013 at 11:54 UTC ( #1051613=note: print w/replies, xml ) Need Help??

in reply to Sharing statics variable between Threads


When I'm trying to make code faster, threading isn't the first tool I reach for. Usually you'll have the best results if you improve your algorithm(s). You don't show your code or algorithm, so I can't offer any suggestions there. However, you mention that you have five interlaced for loops. I think I'd start looking at those loops to see if you can tune things up there.

Verify that you're using data structures suited to your task. If you constantly find yourself looping through an array to find a data element, perhaps it would be better to use a hash so you can access data by name. Frequently that can remove some loops from your code, and can be a dramatic speed increase.

Once I'm happy with my algorithm and data structures, the next thing I try when I need to make something faster is to tune up the algorithm I'm currently using: First, I move calculations out of the most frequently executed loop(s) when possible. In the example below (first section), rather than compute sin($j) 100*100*100 times, we compute it only 100 times. Similarly we compute cos($k) only 100*100 times. The overall savings can add up quite a bit!

Next, if you have some expensive subroutines that you call frequently with the same parameters, you might find Memoize handy: It will store the results of a call and reuse the result when you call it again with the same value(s). As the second example shows, it can give a big win for only two lines of code.

There are many other optimization tricks you can use, too.

Now I must confess: I've never used perls code profiler, though I've been meaning to get around to it. But find and use a good profiler to find out what your code is doing that takes the most time. That will show you where to focus your attention. If you use a profiler rather than guessing, it can point you to some surprising places. (Most of my speed-critical code is in C/C++, where I *do* profile my code.) You might, for example, find that half your execution time is spent formatting your data for printing your progress to a log. In that case, you might log less information and save a lot of time. (As an example, see how loop3 is *so* much slower than loop1?)

Usually I don't reach for threading/forking until I've used the above techniques. If the problem is easy to split into independent pieces, then threading can be easy enough and give you some good results. But threading can often be difficult to apply to some problems. If you split your problem incorrectly, then you'll find that your program takes *longer* to run. If you need assistance with threading, you'll want to appeal to a monk who uses threading more than me (e.g., BrowserUk).

$ perl s/iter loop3 loop1 loop2 loop3 17.5 -- -88% -92% loop1 2.08 740% -- -36% loop2 1.33 1210% 56% -- no memoize: 30.9833 wallclock secs (30.98 usr + 0.00 sys = 30.98 CP +U) @ 1613.94/s (n=50000) with memoize: 0.816253 wallclock secs ( 0.82 usr + 0.00 sys = 0.82 C +PU) @ 60975.61/s (n=50000) $ cat #!/usr/bin/perl use strict; use warnings; use Benchmark qw(:all :hireswallclock); #use Memoize; my ($cnt1, $cnt2); sub loop1 { $cnt1=0; for my $j (0 .. 100) { for my $k (0 .. 100) { for my $l (0 .. 100) { $cnt1 += sin($j) + cos($k) + sqrt($l); } } } } sub loop2 { $cnt2=0; for my $j (0 .. 100) { my $sin_j = sin($j); for my $k (0 .. 100) { my $cos_k = cos($k); for my $l (0 .. 100) { $cnt2 += $sin_j + $cos_k + sqrt($l); } } } } open my $FH, '>', '/dev/null'; sub loop3 { $cnt1=0; for my $j (0 .. 100) { for my $k (0 .. 100) { for my $l (0 .. 100) { $cnt1 += sin($j) + cos($k) + sqrt($l); printf $FH "% 3u % 3u % 3u : %f\n", $j, $k, $l, $cnt1; } } } } cmpthese 5, { loop1 => q[ loop1( ) ], loop2 => q[ loop2( ) ], loop3 => q[ loop3( ) ], }; # double-check that the routines do the same thing! print "**** $cnt1 != $cnt2 ****\n" if abs($cnt1-$cnt2)>.0001; print "\n\n"; no warnings 'recursion'; # hideously expensive subroutine sub glarf { my $t = shift; return 1 if $t <2; return 2 if $t <3; return glarf($t-1)+glarf($t-2)+glarf($t-3); } my $iter=50000; my $fibsub = sub { glarf(10); #glarf(20); #glarf(30); glarf(40); glarf(50); glarf(100 +); }; timethis($iter, $fibsub, 'no memoize' ); # These two lines of code make it *much* faster use Memoize; memoize('glarf'); timethis($iter, $fibsub, 'with memoize' );

Update: It seems rodeo has posted the code while I was writing this. But I'm stopping for now because I've got to get to work...


When your only tool is a hammer, all problems look like your thumb.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1051613]
[choroba]: so you do kind of "integration testing" only

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (5)
As of 2017-06-25 23:04 GMT
Find Nodes?
    Voting Booth?
    How many monitors do you use while coding?

    Results (572 votes). Check out past polls.