http://www.perlmonks.org?node_id=1081624

oobobe has asked for the wisdom of the Perl Monks concerning the following question:

I'm really struggling on how to get threads to work properly in my code. I've searched the web for days, but I still can't figure out a way to handle what I want. Maybe I'm just stupid. But, if anyone here can give me an idea no how to do this, I truly appreciate it.

I need to run threads to compare 2 versions of texts (thousands of them.) I also need to set a thread limit so that those thousands of threads won't eat away my CPU and memory.

However, there are a couple of things that can happens in the threads, but currently my code can't handle them.

1. Some threads may be terminated unexpectedly. How can I catch those threads to re-run them or handle them gracefully?

2. When some threads hang, how can I time out those threads?

The Thread::Queue code I have below is something I saw somewhere in the web:
use File::Slurp; use String::Diff; use threads; use threads::shared; use Thread::Queue; my $thread_limit = 15; my $worker = Thread::Queue->new(); # Worker Threads my $result = Thread::Queue->new(); # Result Threads my @pool = map { threads->create(sub { my $tid = threads->tid; my ($worker, $result) = @_; while( my $item = $worker->dequeue ) { // code that I have that might die unexpectedly, or // hang for a long time // Some Example code here: my $old_text = 'something I got from text version 1'; my $new_text = 'something I got from text version 2'; %String::Diff::DEFAULT_MARKS = (remove_open => '<span clas +s="old">', remove_close => '</span>', append_open => '<span class="ne +w">', append_close => '</span>'); my ($old_diff, $new_diff) = String::Diff::diff($old_text, +$new_text); my $diff = "<html>\n<head>\n<meta http-equiv=\"Content-Typ +e\" content=\"text/html; charset=utf-8\">\n"; $diff .= "<style type='text/css'>\nspan { font-weight: bol +d; }\n.new { color: blue; }\n.old { color: red; }\n</style>\n</head>\ +n<body>\n"; $diff .= "<h1>New Version</h1>\n<p>$new_diff</p>\n<h1>Old +Version</h1>\n<p>$old_diff</p>\n"; $diff .= "</body>\n</html>\n"; write_file("filename.html", {binmode => ':utf8'}, $diff); // More code ... $result->enqueue($tid); } $result->enqueue(undef); }, $worker, $result); } 1 .. $thread_limit; # Queue up all work items (Say, I have 10000 texts to compare) $worker->enqueue($_) for (0 .. 10000); # Tell all workers there are no work items $worker->enqueue((undef) x $thread_limit); # Process the results for ( 1 .. $thread_limit) { while (my $tid = $result->dequeue()) { // Can I handle terminated threads or // time out threads here? // If so, how? } } # Clean up the threads $_->join() for @pool;

So, when a worker thread die unexpectedly, where can I insert code to handle it?

And, when a worker thread hangs for too long, where in the code can I time it out?

Another somewhat related question, it seems that String::Diff (Text::Diff::FormattedHTML too) doesn't work well in threads. The program will die with "Bus Error: 10" when running String::Diff in threads (usually happens after hundreds of threads are executed.) If I use Text::Diff instead, I don't see "Bus Error: 10". But Text:Diff is too obscure for my targeted users, is there a way to run String::Diff safely in threads?

Thank you very much