Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^5: Thread terminating abnormally COND_SIGNAL(6)

by BrowserUk (Pope)
on Jul 17, 2013 at 01:53 UTC ( #1044695=note: print w/ replies, xml ) Need Help??


in reply to Re^4: Thread terminating abnormally COND_SIGNAL(6)
in thread Thread terminating abnormally COND_SIGNAL(6)

I did have to add this at line 205, and am not sure why. I dont know if there is something wrong in the code or not, but it doesnt seem like it should be necessary. #next if(!defined($node));

It'll take me a while to digest the entirity of your code, but I think I can answer this one straight away.

The short answer is that you've fallen into the trap of using the so-called "Advanced Methods" of Thread::Queue that were added by that module's newest owner.

IMO these methods: peek(), insert() & extract() should never have been added to a Queue module as they break all the basic invariants (and thus expectations) of Queues. They effectively turn the Queue into an (shared)Array. Which is a nonsense because the underlying data structure is an array, and all this does is make it a very expensive to maintain array.

And your (perfectly understandable given the modules provision of these methods) usage of the module as an array is exactly what is giving you the problem.

Simplified, you are

  1. Querying the size of the array: (my $amtInQ = $queue->pending();).
  2. Then looping over the array by index: (for(my $i = 0; $i < $amtInQ; $i++){).
  3. Then peeking at array(i): (my $node = $queue->peek($i);).
  4. Then either:
    1. Doing nothing with it: (if(int($node->{canBeSet}) == 0){).
    2. Or: doing something with it then removing it from the array: ($queue->extract($i);).
  5. then looping back to process the next index.

But ... by extracting (splicing) an element from the array, there are now less items in it, than there were when you queried it (my $amtInQ = $queue->pending();), and so when you get towards the end of your loop, there are no ith items left to peek(). It is the very fact that you are using array semantics on the queue that creates the problem.

This is how I would code that same loop:

while( my $node = $Qjob->dequeue ) { if( $node->{canBeSet} ) { ## deal with this node } else { $node->{canBeSet} = 1; $Q->enqueue( $node ); ## push back for next time. } }

Queue semantics and no busy loops, nor any need to sleep to avoid burning cpu.

Now, one objection you might have to that is your ThreadDone check and processing:

But, if you used my queue processing loop above, your KillThread() method simply becomes:

sub killThread{ $queue->enqueue( undef ); }

When the undef is dequeued, the while loop ends and the thread self-terminates.

There's a slight wrinkle with that. If the undef has been queued and then you encounter a node with CanBeSet = 0; then my code would requeue that node after the undef and the loop will terminate before it gets reprocessed, which you seem to explicitly not want to do. So, I would then recast my version of the loop like this:

while( 1 ) { my $node = $queue->dequeue; if( !defined $node and $queue->pending ){ $queue->enqueue( undef ); next; } else { last; } if( $node->{canBeSet} ) { ## deal with this node } else { $node->{canBeSet} = 1; $Q->enqueue( $node ); ## push back for next time. } }

Again, it still retains the Queue semantics avoiding the busy loop; but ensures the queue gets cleared before terminating.

However ... I suspect that your entire design can be significantly further simplified but I'll need to think on that further and I'll save it for another post.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re^5: Thread terminating abnormally COND_SIGNAL(6)
Select or Download Code
Re^6: Thread terminating abnormally COND_SIGNAL(6)
by rmahin (Beadle) on Jul 17, 2013 at 19:20 UTC
    But ... by extracting (splicing) an element from the array, there are now less items in it, than there were when you queried it (my $amtInQ = $queue->pending();), and so when you get towards the end of your loop, there are no ith items left to peek(). It is the very fact that you are using array semantics on the queue that creates the problem.

    Ah cant believe i missed that one. So obvious, thank you.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1044695]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (11)
As of 2014-09-18 10:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (111 votes), past polls