Re^5: question about running a system and a perl function background

[BrowserUk]

The shared hash %lookfor is initalised with the search terms as the arguments and undef as the values;

my %lookfor: shared; @lookfor{ @ARGV } = ();

In the thread, the keys are used to search the lines of input from the command, and if the key is found, then the line of text in which it is found is assigned to the hash as the value of that key:

$line =~ $_ and $lookfor->{ $_ } = $line for keys %{ $lookfor };

When one occurance of each of the search terms has been seen, all the lvalues of %lookfor will have been defined (because they will have had the line they where found in assigned to them), so the test for whether all teh search terms have been located once) is:

last if keys %$lookfor == grep defined, values %$lookfor;

That says, "We are done, when the number of defined values in this hash is equal to the number of keys".

Also, had a little trouble understanding what the following means... (what does < do ?)

The shared array @dones is initalised to zeros. One for each thread.

my @dones  : shared = ( 0 ) x @cmds;

A reference to a different one of the array elements is passed to each thread on creation:

    \$dones[ $_ ],

Each thread sets it's element of @dones to 1 as it exits.

$$doneRef = 1;

The condition of the while modifier to the sleep statement use grep to count the number of elements of @dones that have a true value (1) and compares that with the number of threads (@cmds) started, and sleeps until all the threads are done.

sleep 1 while grep( $_, @dones ) < @cmds;

I understood that when the keywords are found, kill 1, $pid immediatly kills the pid for that particular thread, but does it also stop the other 3 threads that are still going on?

Sorry. Your confusion is understandable as that was my 'deliberate omission' :) (I wish!). Basically, I forgot to implement that part of the algorithm and my testcase (such as it is) was insufficuent for me to notice. By way of recompense, here's a modified version with the 'stop the other threads' part implemented, along with a coupe of changes to do the 'stop immediately if you fond this term' bit you wanted.

The problem of getting the search to cease when a given term has been matched, regardless of whether the others have or not, becomes one of deciding how to indicate which of the terms is accorded this special treatment. First how do indicate this on the command line, and then how to indicate if in the hash. Though the former may not be a requirement for your application.

For simplicity, I've opted to use a trailing pling '!' to indicated show stoppers from the command line.

To convey the 'this is a stopper term' to the threads, I'm initialising the value associated with those term(s) in the hash to '1', and the others to 0. It then uses this information to decide whether to set the shared $quitNow flag and stop the other threads.

                ## Set the quitNow if the value indicates this
                $$quitNow = ( $lookfor->{ $_ } eq '1' );

NOTE: As always, this isn't intended to totally solve the problem, or be fully tested code. Just an example of one way to approach the problem, that is tested in as much as it has appeared to do the right thing a few times :)

#! perl -slw
use strict;
use threads qw[ async ];
use threads::shared;

our $DRIVES ||= 'C';

die "Nothing to search for" unless $ARGV[ 0 ];

sub runAndCheck {
    my( $cmd, $doneRef, $quitNow, $lookfor ) = @_;
    
    ## Execute the command and retain the pid 
    my $pid = open CMD, "$cmd |" or die "$cmd : $!";
    
    ## While we haven't been told to quit
    ## And ther eis still output from teh commands
    while( not $$quitNow and  my $line = <CMD> ) {
        chomp $line;
        
        ## If the number of keys == the number of true values 
        ## We've found (1 of) everything, so we go home.
        last if keys %$lookfor == grep $_, values %$lookfor;
        
        ## For each of the search terms
        for ( keys %{ $lookfor } ) {
            ## Skip it if someone already found it
            next if $lookfor->{ $_ } and $lookfor->{ $_ } ne '1';
            
            ## If this line contains it
            if( $line =~ $_ ) {
                ## Set the quitNow if the value indicates this
                $$quitNow = ( $lookfor->{ $_ } eq '1' );
                
                ## And record the context.
                $lookfor->{ $_ } = $line;
            }
        }
    }
    close CMD;         ##Shut the pipe
    kill 1, $pid;    ## Kill the child process
    $$doneRef = 1;  ## Flag we're done
}

my @cmds = map{ "attrib /s $_:\\* " } split '', $DRIVES;
my @dones  : shared = ( 0 ) x @cmds; ## Used to indicate when the thre
+ads are done
my %lookfor: shared;                 ## Used to pas teh search terms i
+n. and the results out.
my $quitNow: shared = 0;             ## Used to signal early terminati
+on

for my $searchTerm ( @ARGV ) {
    ## look for and remove the trailing 'stopper' flag
    ## $1 will be defined if it is found.
    $searchTerm =~ s[(!)?$][];
    
    ## Set the flag true if it was and false otherwise
    $lookfor{ $searchTerm } = defined $1;
}

##start the threads
async{ 
    runAndCheck( 
        $cmds[ $_ ],
        \$dones[ $_ ],
        \$quitNow,
        \%lookfor        
    );
}->detach for 0 .. $#cmds;

## Wait until all the reads signal they are done
## Ie. count the number of true values in @done
## and continue waiting while this is less than the number of @cmds
sleep 1 while grep( $_, @dones ) < @cmds;

## If we found anything, report it
if( grep defined, %lookfor ) {
    printf "%20s : %s\n", $_,
        ( $lookfor{ $_ } and $lookfor{ $_ } ne '1' )
        ? " was found at '$lookfor{ $_ }'"
        : " was not found."
            for keys %lookfor;
}
else {
    print "None of @ARGV were found";
}

__END__
## No 'stopper flag', everything is found
P:\test>412014 -DRIVES=CDPR junk\.htm$ 412014 copyrght  cmd\.exe  CMD\
+.EXE$
           CMD\.EXE$ :  was found at 'A          C:\WINDOWS\SYSTEM32\C
+MD.EXE'
            cmd\.exe :  was found at 'A          C:\Program Files\Clas
+sic PhoneTools\inssuitecmd.exe'
              412014 :  was found at 'A          P:\test\412014.pl'
          junk\.htm$ :  was found at 'A          P:\test\junk.htm'
            copyrght :  was found at '     R     R:\cookbook\copyrght.
+htm'

## A 'stopper flag' set on an item (that also hapens to be early in th
+e search path)
## and the other thread quit before having found anything
P:\test>412014 -DRIVES=CDPR junk\.htm$ 412014 copyrght!  cmd\.exe  CMD
+\.EXE$
           CMD\.EXE$ :  was not found.
            cmd\.exe :  was not found.
              412014 :  was not found.
          junk\.htm$ :  was not found.
            copyrght :  was found at '     R     R:\cookbook\copyrght.
+htm'

HTH.

[Anonymous Monk]

Hello,

I am still working on this. Due to some complicated issues in my system, I have decided not to use threads...is there a way I can do the same thing using fork and exec?

Please suggest. Thank you.

[BrowserUk]

is there a way I can do the same thing using fork and exec?

As forking uses threads on my system, I've never explored it's use to any great extent.

It is probably possible using some arrangement whereby the original process sets up a separate STDOUT/STDIN prior to forking each command and then uses select to read the ouput from the child processes back into the parent for further processesing, but I am the wrong person to be advising on that.

Hopefully someone with more experience of the fork & exec style of coding will step up and provide an example solution.

I would also learn a lot from a worked example of this.

[Anonymous Monk]

Hello,

Thank you so much for the detailed explanation. That really helps understand threads a lot more. I have been trying to implement it in my program and was getting a very strange result...when I directly run the perl file testThreads.pl from the command window...it runs fine.

But, when I call testThreads.pl from another file...from the command

system (" perl testThreads.pl");

Only one of the threads starts...and then it hangs and does not complete. Is this expected?

[Anonymous Monk]

Hello,

Actually I got it to work...problem was that it did not install completly...so after it did...the thread works.

Thank you.