sharing a hash of hashes between threads

[coontie]

Hi, everyone. I've got this setup: a thread that receives a udp multicast message and sticks it into a 3D hash ($foo{$bar}{$blah}). Then I've got main() that goes through the hash and does processing. The problem is sharing %foo between main and the thread. As I'm sure you know, ::shared is not allowed. What do I do? I tried using Thread::Queue but that doesn't work (prob because I'm using "use threads", as per perlthrtut. I need something simple that will let me enqueue a hash (or anything, really -- I could enqueue an unformatted udp msg and hash it upon dequeuing.) Multicast msgs arrive fast and furious and my receive thread can't do anything except pick it up off the wire, shove it some place and move onto the next msg, for fear of losing updates. Thank you all!

[BrowserUk]

It is entirely possible to share a multi-level hash between threads. There are a few rules to follow though.

1. Any value you store in a shared hash must itself be shared (or sharable). In the case of a hashref, this means that the reference must point to a shared hash. But, the share function will not accept an anonymous hash reference.
2. The hash must be shared before you populate it. If you use the share function on a populated hash, it will be silently emptied.
The combination of these two (strange) design decisions mean that you must declare any subhash as shared (:shared), then populate it. You can then assign a reference to it, as a value in another shared hash.

The following code is not intended as good coding technique, it simply serves to demonstrate that sharing nested hashes is possible.

• %hash is declared so that thread() closes over it.
• A thread is spawned and it loops 10 times adding a new HoHs each time around the (sleep delayed for demo purposes) loop.
• The main thread then goes on to dump the contents of the shared hash in a loop until the demo thread stop. You will see the hash growing in steps.

  #! perl -slw
  use strict;
  use threads;
  use threads::shared;
  use Data::Dumper;
  
  $Data::Dumper::Indent = 0;
  
  my %hash : shared;
  my $running : shared = 0;
  
  sub thread {
      $running++;
      for my $mainKey ( 1 .. 10 ) {
          my %subhash : shared = map {
              my %subsubhash : shared = map{;
                  ( "foo$_" , "bar$_" )
              } 1 .. 4;
              \%subsubhash;
          } 1 .. 4;
          {
              lock %hash;
              $hash{ $mainKey } = \%subhash;
          }
          sleep 1;
      }
      $running--;
  }
  
  my $thread = threads->new( \&thread );
  sleep 1 until $running;
  
  while( $running ) {
      print Dumper \%hash;
      sleep 1;
  }
  
  $thread->join;
  

[spurperl]

The solution proposed the sage BrowserUk looks good, but you can see that it's not entirely trivial to declare inner subhashes as shared. Moreover, the solution will be only become more and more complex the deeper your structure gets.

Maybe you can solve your problem by simplifying the specification ? Use Thread::Queue (it works great with "threads") to pass data between threads. On the receiving side, first shove the data into the hash and then print/process it.

This way, you also don't expose the internal representation of the processing thread to the receiving thread, thus keeping the interface cleaner and more flexible.

[coontie]

Thank you! I went back to threads:q. Sticking a hash onto the queue triggered an error msg from within Threads/Queue.pm. What I did instead, is to enqueue the raw data string and then I split it/hash it in main. That works perfectly! Thank you all for you help!

[zentara]

Maybe "stringify" the HoH before sending it to the threads as a shared scalar, then let the thread thaw it? Sort of the same technique used for sending HoH's thru sockets?