Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^6: Threaded Code Not Faster Than Non-Threaded -- Why?

by Tommy (Chaplain)
on Jan 06, 2014 at 03:15 UTC ( #1069446=note: print w/ replies, xml ) Need Help??


in reply to Re^5: Threaded Code Not Faster Than Non-Threaded -- Why?
in thread Threaded Code Not Faster Than Non-Threaded -- Why?

The results are in. I forked and refactored the code to follow the pattern you laid out (BrowserUk). The threaded part of the code is over 2X faster now! Thanks again!

Tommy
A mistake can be valuable or costly, depending on how faithfully you pursue correction


Comment on Re^6: Threaded Code Not Faster Than Non-Threaded -- Why?
Re^7: Threaded Code Not Faster Than Non-Threaded -- Why?
by BrowserUk (Pope) on Jan 06, 2014 at 03:25 UTC

    Could you post (a link to) your latest code?


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      It isn't nearly finished and is undergoing a transformation since I typed my response, but at the time of this writing you can find the threading code here

      One of the first things you'll notice is that I didn't implement a "guard" or "doorman". I'm just stuffing everything in the queue for now. Even on directory trees larger than 200GB it barely takes more than 200MB RAM.

      Tommy
      A mistake can be valuable or costly, depending on how faithfully you pursue correction

        I'd suggest trying the following modification:

        my $thread_term :shared = 0; my $threads_init :shared = 0; my $file_read_sem : shared; + ### Add this ... WORKER: while ( !$thread_term && defined ( my $file = $work_queue-> +dequeue ) ) { { + ### Add this lock $file_read_sem; + ### Add this open my $fh, '<', $file or do { lock $d_counter; $d_counter++ +; next WORKER }; my $data = <$fh>; close $fh; } + ### Add this

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

      It isn't nearly finished and is undergoing a transformation since I typed my response, but at the time of this writing you can find the threading code here

      One of the first things you'll notice is that I didn't implement a "guard" or "doorman". I'm just stuffing everything in the queue for now. Even on directory trees larger than 200GB it barely takes more than 200MB RAM.

      Tommy
      A mistake can be valuable or costly, depending on how faithfully you pursue correction

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1069446]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (11)
As of 2014-07-23 08:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (137 votes), past polls