Debugging Perl scripts which use fork()

[IlyaM]

Intro

Some scripts in test suite for module HTTP::WebTest which I maintain use fork for creation of child proccess - test HTTP servers. I'm used to debug problems in my code with perl debugger so I was interested how can I use it with scripts which use fork. I found that perl debugger does provides support for such scripts but this feature is completely undocumented.

Note that techniques described in this article probably work only on UNIX and UNIX-like systems. This article is probably is not very easy reading because its topic is quite complex itself.

Debugging Several Proccesses at Same Time

I need TTY

After fork there are more than one process and more than one instance of perl debugger. The problem is that each instance of perl debugger requires its own TTY. Right after fork instance of perl debugger which runs in child process tries to switch TTY. If it can't do that it tries to use same TTY as perl debugger which runs in parent process. This can result sometimes with either screwed TTY or runaway child process which takes 100% of CPU.

How can debugger in child process find free TTY? Perl debugger can use undocumented variables $DB::fork_TTY or undocumented subroutine $DB::get_fork_TTY to find avialable TTY or it can try to start xterm process and use xterm's TTY.

How it works?

• First of all if there exists subroutine $DB::get_fork_TTY debugger calls it. Debugger ignores value returned by this subroutine but this subroutine can set value of variable $DB::fork_TTY.
• Second of all if defined variable $DB::fork_TTY debugger will treat it's value as name of TTY device and will try to use this device. Note that debugger doesn't check if it is really TTY device. If it doesn't then child process gets into infinite loop taking 100% of CPU.
• If $DB::fork_TTY is not defined but debugger detects that it runs inside xterm it tries to launch another xterm and it tries to use new xterm's TTY.

Easiest option is run your script with debugger in xterm. On fork debugger will create additional xterm window. Another option - define subroutine $DB::get_fork_TTY. It should somehow find new TTYs and set $DB::fork_TTY value accordantly. Example:

sub DB::get_fork_TTY {
    open XT, q[3>&1 xterm -title 'Forked Perl debugger' -e sh -c 'tty 
+1>&3;\ sleep 10000000' |];
    $DB::fork_TTY = <XT>;
    chomp $DB::fork_TTY;
}

And the last option - just set manually $DB::fork_TTY.

Some Useful Tricks (Kinda Q&A)

• How can be created additional free TTY? Start terminal emulator like xterm, type 'tty' to get TTY device name and 'sleep 10000000'.
• How can debugger related code (like definition of DB::get_fork_TTY or $DB::fork_TTY) be put outside scripts? Create file MyDebug.pm and put it where. When you are using perl debugger use

  
  perl -MMyDebug -d script.pl
  

  instead of just

  
  perl -d script.pl
  

  or even better just set environment variable PERL5OPT to '-MMyDebug'.
• Is where any way to ignore a child process (that is don't give it it's own TTY) if there is no need to debug the child process? It is possible. Actually every process running with the debugger requires it's own TTY only when it breaks into the debugger. If you don't have any break points in the child process it will break into the debugger only on an exit. Since the debugger in the child process doesn't have it's own TTY it should not be allowed to break on the exit of the child process. There is exist a variable $DB::inhibit_exit which forbids the debugger to break on the process exit if it is set to false.

  You can add

  
  $DB::inhibit_exit = 0;
  

  in your script after fork so the debugger in the child process will not try to break the child process on its exit.

See Also

perldoc perldebug, Using the Perl Debugger and sources of perl debugger (perl5db.pl).

P.S.

I'm gladly accepting correction for this article. Including corrections for my poor English.

Update: Thanks for correction cybear.

[dash2]

Ilya,

Great article, really useful. I think cybear was joking btw.

One thing that tripped me up when trying to send multiple processes to a series of ttys: DB::get_fork_TTY is called from the child, after the fork. So, this won't work:

{ 
my $ttyno = 1;
sub DB::get_fork_TTY{
    $DB::fork_TTY = "/dev/pts/" . $ttyno++;
}
}

because the $ttyno is in separate processes. Instead, redefine your $ttyno after the fork:

use vars $ttyno;

for my $fork (0 .. $forks) {
    if (fork) {

    } else {
        # in child
        $ttyno = $fork;
    }
}

sub DB::get_fork_TTY {
    $DB::fork_TTY = "/dev/pts/" . $main::tty;
}

dave hj~

[kcorcam]

Has anyone tried debugging fork process usings emacs perldb debugger? I tried using the above xterm method discussed in the article, I was able to successfully create an xterm for each child process but I can't view the source code from the child window with debugger commands, such as 'l'ine command. I can issue 'n'ext and 's'tep from the child and see its output in the parent window. But its not helpful if I can't easily tell where i am in the source code for the child. Any idea what is wrong? What I would really like is to be able to fork another emacs debugger process with debugger command and source code window for each child similar to the parent. Any help would be greatly appreciated Kelly

[pemungkah]

You should be able to define the subs you mention in .perldb (either in the current directory, or in ~/.perldb on Unix-like systems). Quoting the debugger's internal docs (perldoc perl5db.pl):

When perl5db.pl starts, it reads an rcfile (perl5db.ini for non-interactive sessions, .perldb for interactive ones) that can set a number of options. In addition, this file may define a subroutine afterinit that will be executed (in the debugger's context) after the debugger has initialized itself. This can be set in the options, or by setting it in .perldb.

afterinit() runs a little too late - after TTY allocation - so you can't do the TTY mojo there. However, .perldb gets control right after option processing, and just before the get_fork_TTY subroutine is checked for and called, so it's an optimum point to set this up if you don't want to use another module to do it.

You can of course use a module that defines DB::get_fork_TTY in .perldb as well, if you prefer to have all your magic in one place, so to speak.

Another item of note is the $DB::CreateTTY variable, which controls when the debugger decides to try grabbing a new TTY (this is a bit-field scalar, so if you want more than one option, you sum them):

• 1 - on fork()
• 2 - when debugger is started inside the debugger
• 4 - on startup
This lets you decide when you want (and when you don't want) the debugger to try grabbing another TTY. The default is 3 (on fork, and on nest).

[petr999]

Hello,

I've just released a server-tolerant solution on the problem discussed.

This involves using the Tmux terminal multiplexer (console window manager) instead of xterm.

Thus you guys shouldn't ever apparently need the X11 while debugging forks on a server.

Thank you for attention.

[jimpudar]

In case anyone else is looking for this, it's here:

http://search.cpan.org/~veresc/Debug-Fork-Tmux-1.000012/README.pod

[tom93]

Thanks for the info re: $DB::inhibit_exit ; makes debugging a simple fork much easier.

[cybear]

P.S.

I'm gladly accepting correction for this article. Including corrections for my bad English.

You should say POOR english

- cybear