http://www.perlmonks.org?node_id=415059


in reply to Re^2: question about running a system and a perl function background
in thread question about running a system and a perl function background

Actually, if I find 'serious.txt' only, I want the other threads to stop immediatly. If I find 'junk.htm', I want the other threads to continue and complete. Please advice.

Advice? Sure. Play with the code. Understand how it works and under what conditions the threads stop early. Then devise a way of conveying which of the looked for terms are stoppers and which are carryons. Then add logic to perform that function.

I seem to get "Scalar Leaked" error ...what could I be doing wrong?

Which version of Perl are you using?

That message used to come up a lot in versions 5.8.0-5.8.2. With 5.8.3 is became much less common but still happened under some circumstances--mostly avoidable if you knew how though.

I use 5.8.4, and I do not recall having ever seen that particular error with it.

I recommend 5.8.4 over 5.8.5 for threaded work simply because whilst I have 5.8.5, I have done very little with it.

I have not yet downloaded 5.8.6 and probably will not until AS make their version available, if they ever do. I took a quick look at the delta document and nothing leapt off the page as a "must have".

Certainly nothing that warrents me going through the process of downloading and building my own version. Followed by all the pain of downloading/ building/ installing all the extra stuff that AS add to their packages. Followed by working out all the stuff I've added to my installation through any of the half dozen different ways I install different packages from any of a dozen or so different sources.


Examine what is said, not who speaks.        The end of an era!
"But you should never overestimate the ingenuity of the sceptics to come up with a counter-argument." -Myles Allen
"Think for yourself!" - Abigail        "Time is a poor substitute for thought"--theorbtwo         "Efficiency is intelligent laziness." -David Dunham
"Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon

Replies are listed 'Best First'.
Re^4: question about running a system and a perl function background
by Anonymous Monk on Dec 15, 2004 at 18:25 UTC
    Hi,

    Thanks for the reply. I am using perl version 5.8.4 which you had suggested earlier.

    I have been playing with the code...but could not understand:

    last if keys %$lookfor == grep defined, values %$lookfor; $line =~ $_ and $lookfor->{ $_ } = $line for keys %{ $lookfor +};

    I understand it means that for keys %{ $lookFor }, if $line =~ $_, then $lookFor->{$_} = $line...but was not sure what if keys %$lookfor == grep defined, values %$lookfor meant.

    Also, had a little trouble understanding what the following means... (what does < do ?) :

    sleep 1 while grep( $_, @dones ) < @cmds;

    I understood that when the keywords are found, kill 1, $pid immediatly kills the pid for that particular thread, but does it also stop the other 3 threads that are still going on?

    Thank you.

      The shared hash %lookfor is initalised with the search terms as the arguments and undef as the values;

      my %lookfor: shared; @lookfor{ @ARGV } = ();

      In the thread, the keys are used to search the lines of input from the command, and if the key is found, then the line of text in which it is found is assigned to the hash as the value of that key:

      $line =~ $_ and $lookfor->{ $_ } = $line for keys %{ $lookfor };

      When one occurance of each of the search terms has been seen, all the lvalues of %lookfor will have been defined (because they will have had the line they where found in assigned to them), so the test for whether all teh search terms have been located once) is:

      last if keys %$lookfor == grep defined, values %$lookfor;

      That says, "We are done, when the number of defined values in this hash is equal to the number of keys".

      Also, had a little trouble understanding what the following means... (what does < do ?)

      The shared array @dones is initalised to zeros. One for each thread.

      my @dones : shared = ( 0 ) x @cmds;

      A reference to a different one of the array elements is passed to each thread on creation:

      \$dones[ $_ ],

      Each thread sets it's element of @dones to 1 as it exits.

      $$doneRef = 1;

      The condition of the while modifier to the sleep statement use grep to count the number of elements of @dones that have a true value (1) and compares that with the number of threads (@cmds) started, and sleeps until all the threads are done.

      sleep 1 while grep( $_, @dones ) < @cmds;
      I understood that when the keywords are found, kill 1, $pid immediatly kills the pid for that particular thread, but does it also stop the other 3 threads that are still going on?

      Sorry. Your confusion is understandable as that was my 'deliberate omission' :) (I wish!). Basically, I forgot to implement that part of the algorithm and my testcase (such as it is) was insufficuent for me to notice. By way of recompense, here's a modified version with the 'stop the other threads' part implemented, along with a coupe of changes to do the 'stop immediately if you fond this term' bit you wanted.

      The problem of getting the search to cease when a given term has been matched, regardless of whether the others have or not, becomes one of deciding how to indicate which of the terms is accorded this special treatment. First how do indicate this on the command line, and then how to indicate if in the hash. Though the former may not be a requirement for your application.

      For simplicity, I've opted to use a trailing pling '!' to indicated show stoppers from the command line.

      To convey the 'this is a stopper term' to the threads, I'm initialising the value associated with those term(s) in the hash to '1', and the others to 0. It then uses this information to decide whether to set the shared $quitNow flag and stop the other threads.

      ## Set the quitNow if the value indicates this $$quitNow = ( $lookfor->{ $_ } eq '1' );

      NOTE: As always, this isn't intended to totally solve the problem, or be fully tested code. Just an example of one way to approach the problem, that is tested in as much as it has appeared to do the right thing a few times :)

      #! perl -slw use strict; use threads qw[ async ]; use threads::shared; our $DRIVES ||= 'C'; die "Nothing to search for" unless $ARGV[ 0 ]; sub runAndCheck { my( $cmd, $doneRef, $quitNow, $lookfor ) = @_; ## Execute the command and retain the pid my $pid = open CMD, "$cmd |" or die "$cmd : $!"; ## While we haven't been told to quit ## And ther eis still output from teh commands while( not $$quitNow and my $line = <CMD> ) { chomp $line; ## If the number of keys == the number of true values ## We've found (1 of) everything, so we go home. last if keys %$lookfor == grep $_, values %$lookfor; ## For each of the search terms for ( keys %{ $lookfor } ) { ## Skip it if someone already found it next if $lookfor->{ $_ } and $lookfor->{ $_ } ne '1'; ## If this line contains it if( $line =~ $_ ) { ## Set the quitNow if the value indicates this $$quitNow = ( $lookfor->{ $_ } eq '1' ); ## And record the context. $lookfor->{ $_ } = $line; } } } close CMD; ##Shut the pipe kill 1, $pid; ## Kill the child process $$doneRef = 1; ## Flag we're done } my @cmds = map{ "attrib /s $_:\\* " } split '', $DRIVES; my @dones : shared = ( 0 ) x @cmds; ## Used to indicate when the thre +ads are done my %lookfor: shared; ## Used to pas teh search terms i +n. and the results out. my $quitNow: shared = 0; ## Used to signal early terminati +on for my $searchTerm ( @ARGV ) { ## look for and remove the trailing 'stopper' flag ## $1 will be defined if it is found. $searchTerm =~ s[(!)?$][]; ## Set the flag true if it was and false otherwise $lookfor{ $searchTerm } = defined $1; } ##start the threads async{ runAndCheck( $cmds[ $_ ], \$dones[ $_ ], \$quitNow, \%lookfor ); }->detach for 0 .. $#cmds; ## Wait until all the reads signal they are done ## Ie. count the number of true values in @done ## and continue waiting while this is less than the number of @cmds sleep 1 while grep( $_, @dones ) < @cmds; ## If we found anything, report it if( grep defined, %lookfor ) { printf "%20s : %s\n", $_, ( $lookfor{ $_ } and $lookfor{ $_ } ne '1' ) ? " was found at '$lookfor{ $_ }'" : " was not found." for keys %lookfor; } else { print "None of @ARGV were found"; } __END__ ## No 'stopper flag', everything is found P:\test>412014 -DRIVES=CDPR junk\.htm$ 412014 copyrght cmd\.exe CMD\ +.EXE$ CMD\.EXE$ : was found at 'A C:\WINDOWS\SYSTEM32\C +MD.EXE' cmd\.exe : was found at 'A C:\Program Files\Clas +sic PhoneTools\inssuitecmd.exe' 412014 : was found at 'A P:\test\412014.pl' junk\.htm$ : was found at 'A P:\test\junk.htm' copyrght : was found at ' R R:\cookbook\copyrght. +htm' ## A 'stopper flag' set on an item (that also hapens to be early in th +e search path) ## and the other thread quit before having found anything P:\test>412014 -DRIVES=CDPR junk\.htm$ 412014 copyrght! cmd\.exe CMD +\.EXE$ CMD\.EXE$ : was not found. cmd\.exe : was not found. 412014 : was not found. junk\.htm$ : was not found. copyrght : was found at ' R R:\cookbook\copyrght. +htm'

      HTH.


      Examine what is said, not who speaks.        The end of an era!
      "But you should never overestimate the ingenuity of the sceptics to come up with a counter-argument." -Myles Allen
      "Think for yourself!" - Abigail        "Time is a poor substitute for thought"--theorbtwo         "Efficiency is intelligent laziness." -David Dunham
      "Memory, processor, disk in that order on the hardware side. Algorithm, algorithm, algorithm on the code side." - tachyon
        Hello,

        Thank you so much for the detailed explanation. That really helps understand threads a lot more. I have been trying to implement it in my program and was getting a very strange result...when I directly run the perl file testThreads.pl from the command window...it runs fine.

        But, when I call testThreads.pl from another file...from the command

        system (" perl testThreads.pl");

        Only one of the threads starts...and then it hangs and does not complete. Is this expected?

        Hello,

        I am still working on this. Due to some complicated issues in my system, I have decided not to use threads...is there a way I can do the same thing using fork and exec?

        Please suggest. Thank you.