Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re: using parallel processing to concatenate a string, where order of concatenation doesn't matter

by BrowserUk (Pope)
on Oct 18, 2006 at 17:04 UTC ( #579149=note: print w/ replies, xml ) Need Help??


in reply to using parallel processing to concatenate a string, where order of concatenation doesn't matter

Try this. I've used a randomly variable 'work pause', as if this is constant, the threads will obviously finish in the same order as they started.

Update: Applied locks per ikegami's post below.

#! perl -slw use strict; use threads; use threads::shared; my $content : shared = ''; sub concatenate_parallel { my $letter = shift; sleep 1 + rand 2; ## Do stuff that takes time { lock $content; $content .= $letter; } } print scalar localtime; my @threads = map{ threads->create( \&concatenate_parallel, $_ ) } 'a' + .. 'c'; $_->join for @threads; print $content; print scalar localtime; __END__ c:\test>579015 Wed Oct 18 18:01:20 2006 cab Wed Oct 18 18:01:22 2006 c:\test>579015 Wed Oct 18 18:01:24 2006 bca Wed Oct 18 18:01:26 2006 c:\test>579015 Wed Oct 18 18:01:27 2006 abc Wed Oct 18 18:01:29 2006 c:\test>579015 Wed Oct 18 18:01:30 2006 abc Wed Oct 18 18:01:32 2006

If you don't like the shared buffer being passed to the threads through closure--it smacks of globals--then you could use this version that passes a reference to the shared buffer to the threads as an argument and dereferences it when appending:

#! perl -slw use strict; use threads; use threads::shared; sub concatenate_parallel { my( $contentRef, $letter ) = @_; sleep 1 + rand 2; ## Do stuff that takes time { lock $contentRef; $$contentRef .= $letter for 1 .. 1e4; } } my $content : shared = ''; my $contentRef = \$content; print scalar localtime; my @threads = map{ threads->create( \&concatenate_parallel, $contentRef, $_ ); } 'a' .. 'z'; $_->join for @threads; print $content; print scalar localtime; __END__ c:\test>579015 Thu Oct 19 00:33:11 2006 cbefghlmqsvwadijknoprtuxyz Thu Oct 19 00:33:13 2006

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re: using parallel processing to concatenate a string, where order of concatenation doesn't matter
Select or Download Code
Re^2: using parallel processing to concatenate a string, where order of concatenation doesn't matter
by ikegami (Pope) on Oct 18, 2006 at 18:58 UTC
    Is .= atomic, or do you have a race condition?

      On my single cpu processor, only one thread runs at a time, so there is no race condition. On a multi-cpu machine, reading the docs on, and using threads::shared::lock() will be required.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        Race conditions can occur even if only one thread is running a time. What matters is when one thread can interrupt another thread.

        Consider two threads doing i+=1 in C.

        i+=1 CPU ----------+---------- thread 1 | thread 2 ==========+========== ... | T load i | i add 1 | m ----------+---------- e | ... | | load i | | add 1 v | save i | ... ----------+---------- save i | ... |

        Even though both threads have run, and even though only one thread executed at a time, i was only incremented by one.

        So my question remains. Is .= atomic (i.e. uninterruptable, safe) or does it introduce a race condition (like C's i+=1, requiring the use of locks)?

        Update: Answered in Threads: why locking is required when using shared variables.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://579149]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (4)
As of 2014-07-29 23:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (229 votes), past polls