http://www.perlmonks.org?node_id=1069446


in reply to Re^5: Threaded Code Not Faster Than Non-Threaded -- Why?
in thread Threaded Code Not Faster Than Non-Threaded -- Why?

The results are in. I forked and refactored the code to follow the pattern you laid out (BrowserUk). The threaded part of the code is over 2X faster now! Thanks again!

Tommy
A mistake can be valuable or costly, depending on how faithfully you pursue correction
  • Comment on Re^6: Threaded Code Not Faster Than Non-Threaded -- Why?

Replies are listed 'Best First'.
Re^7: Threaded Code Not Faster Than Non-Threaded -- Why?
by BrowserUk (Patriarch) on Jan 06, 2014 at 03:25 UTC

    Could you post (a link to) your latest code?

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      It isn't nearly finished and is undergoing a transformation since I typed my response, but at the time of this writing you can find the threading code here

      One of the first things you'll notice is that I didn't implement a "guard" or "doorman". I'm just stuffing everything in the queue for now. Even on directory trees larger than 200GB it barely takes more than 200MB RAM.

      Tommy
      A mistake can be valuable or costly, depending on how faithfully you pursue correction
[reply]

        I'd suggest trying the following modification:

        my $thread_term   :shared = 0;
my $threads_init  :shared = 0;
my $file_read_sem : shared;                                           
+                   ### Add this

...

   WORKER: while ( !$thread_term && defined ( my $file = $work_queue->
+dequeue ) )
   {
     {                                                                
+                   ### Add this
         lock $file_read_sem;                                         
+                   ### Add this
         open my $fh, '<', $file or do { lock $d_counter; $d_counter++
+; next WORKER };

         my $data = <$fh>;

         close $fh;
      }                                                               
+                   ### Add this

        [download]
        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

      It isn't nearly finished and is undergoing a transformation since I typed my response, but at the time of this writing you can find the threading code here

      One of the first things you'll notice is that I didn't implement a "guard" or "doorman". I'm just stuffing everything in the queue for now. Even on directory trees larger than 200GB it barely takes more than 200MB RAM.

      Tommy
      A mistake can be valuable or costly, depending on how faithfully you pursue correction
[reply]

In Section Seekers of Perl Wisdom