Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re^2: forkManager PID 1 less than top

by MAR99 (Initiate)
on Oct 02, 2013 at 13:08 UTC ( [id://1056607]=note: print w/replies, xml ) Need Help??


in reply to Re: forkManager PID 1 less than top
in thread forkManager PID 1 less than top

Thanks for looking, it helps level set my next phase of investigation. I will attempt to simplify and retest in my env.

Replies are listed 'Best First'.
Re^3: forkManager PID 1 less than top
by MAR99 (Initiate) on Oct 02, 2013 at 15:01 UTC

    Where did you find the code example you posted? I looked at the written example documentation on cpam and didnt see that version of the code. At any rate I typed in your code and ran it. The PID returned from the fork manager and the PID listed by the system 'top' cmd matched. So I started looking of issues with my specific code. So after investigating my code, I think I have located the issue. I am not sure I have the knowledge to fully explain the technical cause of the issue, but hopefully someone else reading can and will explain.

    In your code example, if you replace the line... while () {}; # ie the work item for the child process which in this case is just a forever loop.

    replace with .... system("foreverLoop.pl"); # here the foreverLoop.pl file contains the same while() {} code.

    and run... you will see the +1 offset in PIDs.

    In general my code is attempting to use the forkManager to submit jobs by running external scripts(pl, ksh, etc), so I am using the perl 'system' cmd to start those task. The introduction of the system cmd is causing the PID reported by the forkManager to be 1 less than the PID tracked by the top cmd.

    I should have expected this since system probably starts it's own child process. However having said all that I am really not sure how to properly work around this. Is it safe for me to just change the reported PID by +1 or is there a better way? I am just not sure if the +1 offset is a for sure thing or if the actual offset is subject to system timing?

      Where did you find the code example you posted?
      I edited first example (in SYNOPSIS) here.
      I should have expected this since system probably starts it's own child process.
      Exactly!
      or if the actual offset is subject to system timing?
      yep.
      having said all that I am really not sure how to properly work around this.
      Why do you need exactly the PID which do the work?

      in general this problem is hard to solve..
      You can use something like this:
      use Parallel::ForkManager; $pm = new Parallel::ForkManager(100); my @all_data = qw/1/; foreach $data (@all_data) { # Forks and returns the pid for the child: my $pid = $pm->start; print "PID $pid\n"; $pid and next; print "$data", "\n"; my $pid2 = fork(); unless ($pid2) { exec $^X, '-e', 'while(){}'; } print "Real PID $pid2\n"; waitpid $pid2, 0; $pm->finish; # Terminates the child process } $pm->wait_all_children;
      i.e. use "exec", "fork" and "wait/waitpid" calls.

      Note that I don't recommend putting it into production until you fully understand how those functions works.
      Also this example won't work if you replace
      exec $^X, '-e', 'while(){}';
      with
      exec "perl -e 'while(){}'";
      those calls are equivalent, but former calls PERL, and later one calls SHELL, which will call PERL.
      so with later one you'll get wrong pid again (PID of shell)

      Also, you really cannot control if scripts, that you call are calling other scripts or other processes or no

      Thus this task is hardly solvable.

      I propose explain you business requirements (exactly why you need that PID who do works) and someone might suggest something.

        Thanks for the detailed response. I was worried that this was going to turn into a subtle and problematic situation per your response. I will study what you have written on exec, but it does sound like the whole thing is a little touchy

        Basically the reason I was hoping for correlation between the forkManger & top cmd PIDs is as follows. Lets say I submit 3 jobs(call them j1,j2,j3) using my forkManager code. During the submission process my code prints to stnd out log info related to the submission flow. ie j1:PID=123, j2:PID=130, j3:PID=140, etc. It also keeps track of when the jobs complete via the forkManager. These jobs dont all run at the same rate, some run for hrs or even days. At some point while the jobs are still running I may decide that the j2 is no longer needed. So I would like to go into my log and find the PID for the the thing I symbolically know an love as j2. I would like to be able to use that logged PID to issue a kill on said PID. The problem is that the PID as reported in 'top' are not what is stored in the log. Hence the log PIDs can't be used to issue the kill cmd.

        Frankly correlation is not an absolute requirement. But never the less, it would have been nice. So unless someone comes up with an easy, bullet proof fix for this. I most likly wont be killing myself to make it happen. I'm not sure I am versed enough to work around all the potential land mines to which you eluded.

      or if the actual offset is subject to system timing?

      On more secure systems, the pid is picked randomly, not sequentially. I didn't think anyone still did sequential pids! What system are you using?

        Where did you see random PIDs ? I think all (most?) Linux distros use sequental. http://en.wikipedia.org/wiki/Process_identifier#Unix-like
        Process IDs are usually allocated on a sequential basis

        Also:
        http://www.redhat.com/archives/redhat-list/2004-August/msg00004.html

        http://lists.freebsd.org/pipermail/freebsd-security/2010-February/005549.html

        I am running Ubuntu with 3.8 kernal, PIDs are sequental.
        Also, I think there can be problems with some software, when recently-used PID re-used too fast.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1056607]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (2)
As of 2024-04-20 16:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found