note
BrowserUk
<blockquote><i>
I did have to add this at line 205, and am not sure why. I dont know if there is something wrong in the code or not, but it doesnt seem like it should be necessary. <c>#next if(!defined($node));</c>
</i></blockquote>
<p>It'll take me a while to digest the entirity of your code, but I think I can answer this one straight away.
<p>The short answer is that you've fallen into the trap of using the so-called "Advanced Methods" of [mod://Thread::Queue] that were added by that module's newest owner.
<p>IMO these methods: peek(), insert() & extract() should never have been added to a Queue module as they break all the basic invariants (and thus expectations) of Queues. They effectively turn the Queue into an (shared)Array. Which is a nonsense because the underlying data structure is an array, and all this does is make it a very expensive to maintain array.
<P>And your (perfectly understandable given the modules provision of these methods) usage of the module as an array is exactly what is giving you the problem.
<p>Simplified, you are
<ol><li>Querying the size of the array: (<c>my $amtInQ = $queue->pending();</c>).
</li><li>Then looping over the array by index: (<c>for(my $i = 0; $i < $amtInQ; $i++){</c>).
</li><li>Then peeking at array(i): (<c>my $node = $queue->peek($i);</c>).
</li><li>Then either:
<ol type=a><li>Doing nothing with it: (<c>if(int($node->{canBeSet}) == 0){</c>).
</li><li>Or: doing something with it then removing it from the array: (<c>$queue->extract($i);</c>).
</li></ol>
</li><li>then looping back to process the next index.
</li></ol>
<p>But ... by extracting ([splicing]) an element from the array, there are now less items in it, than there were when you queried it (<c>my $amtInQ = $queue->pending();</c>), and so when you get towards the end of your loop, there are no ith items left to peek(). It is the very fact that you are using array semantics on the queue that creates the problem.
<P>This is how I would code that same loop:<code>
while( my $node = $Qjob->dequeue ) {
if( $node->{canBeSet} ) {
## deal with this node
}
else {
$node->{canBeSet} = 1;
$Q->enqueue( $node ); ## push back for next time.
}
}
</code>
<P>Queue semantics and no busy loops, nor any need to sleep to avoid burning cpu.
<P>Now, one objection you might have to that is your ThreadDone check and processing:<spoiler><code>
sub killThread{
my $self = shift;
{ lock $self->{threadDone}; $self->{threadDone} = 1; }
return $self->{thread};
}
sub manageQueue{
my $self = shift;
while(1){
my $done;
{ lock $self->{threadDone}; $done = $self->{threadDone}; }
my $amtInQ = $queue->pending();
last if $done && $amtInQ == 0;
...
</code></spoiler>
<P>But, if you used my queue processing loop above, your KillThread() method simply becomes:<code>
sub killThread{
$queue->enqueue( undef );
}
</code>
<P>When the undef is dequeued, the while loop ends and the thread self-terminates.
<P>There's a slight wrinkle with that. If the undef has been queued and then you encounter a node with CanBeSet = 0; then my code would requeue that node after the undef and the loop will terminate before it gets reprocessed, which you seem to explicitly not want to do. So, I would then recast my version of the loop like this:<code>
while( 1 ) {
my $node = $queue->dequeue;
if( !defined $node and $queue->pending ){
$queue->enqueue( undef );
next;
}
else {
last;
}
if( $node->{canBeSet} ) {
## deal with this node
}
else {
$node->{canBeSet} = 1;
$Q->enqueue( $node ); ## push back for next time.
}
}
</code>
<p>Again, it still retains the Queue semantics avoiding the busy loop; but ensures the queue gets cleared before terminating.
<p><B>However ...</B> I suspect that your entire design can be significantly further simplified but I'll need to think on that further and I'll save it for another post.
<div class="pmsig"><div class="pmsig-171588">
<hr />
<font size=1 >
<div>With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'</div>
<div>Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.</div>
<div>"Science is about questioning the status quo. Questioning authority". </div>
<div>In the absence of evidence, opinion is indistinguishable from prejudice.
</div>
</font>
</div></div>
1044465
1044675