Reverse Engineering Perl Tool?

[Anonymous Monk]

I've inherited some perl code that traverses several source code files with very little documentation. I was hoping that there might be a tool that would reverse engineer the code and would create a flow diagram of the system. Anyone heard of such a beast? Any recommendations short of walking through the code line by line? The code doesn't currently work and they want me to get it working.

[mikfire]

You may also want to try a nice GUI debugger, like DDD which has some nice features for drilling into data structures, etc. Beyond that, it is likely worth the effort to liberally sprinkle the code with print statements. When attempting to learn code like this, I find it useful ( although sometimes painful :) to add lines like this

sub foo {
  print STDERR "Entering sub foo(@_)\n";
 # Do something
  print STDERR "Exiting sub foo: return value $bar\n";
  return $bar;
}

Failing all of that, I will offer the following two toys. They are not perfect - they may not catch everything - but they have worked quite well for me.

First Toy

This one is a bit strange. It was originally designed to help me update my POD. It crawls through a bit of source and attempts to extract every time a return is called and the context in which it is called. I have modified it slightly so it would not bother parsing the pod in the second loop and instead print to screen. It is a little raw - I am the only one who uses it and I know how it is to be called :)

#!/usr/bin/perl -w
# usage: $0 filename [optional output file]
# If the output file is not specified, it will print to STDOUT

use strict;

my $outp;

open FILE, $ARGV[0] or die "A grim and horrible death: $!";
if ( defined( $ARGV[1] ) ) {
    open OUTP, ">$ARGV[1]" or die "Couldn't write: $!";
    $outp = \*OUTP;
}
else {
    $outp = \*STDOUT;
}

my ( %calls, $name, @context );
#---
# First parsing phase is to try to gather all the return codes togethe
+r,
# keyed by the function name
#---
while ( <FILE> ) {
     next if /^\s*#/;
    next if /^\s*$/;
    last if /__END__/;

    $name = $1 if /^sub (\w+)/;

    #---
    # Push the context and get next line if possible
    #---
    if ( /^\s*return (\$?[\w->:]+) (if .+)[;{]$/ ||
        /^\s*return (\$?[\w->:]+) (unless .+)[;{]$/ ) {
        my ( $code, $context ) = ( $1, $2 );

        chomp $context;
        $context =~ s/^\s+//;

        push @{$calls{$name}{$code}}, $context;
        next;
    }

    if ( /(.+){$/ ) {
        chomp;
        s/^\s+//;
        push @context, $_ 
    }

    pop  @context if ( /}$/ );

    push(@{$calls{$name}{$1}}, $context[-1]),next if ( /\s*return (\$?
+[\w->:{}]+
)/ );
}
close FILE;

for my $name ( sort keys %calls ) {
    for my $rc ( sort keys %{$calls{$name}} ) {
        print $outp "$rc when:\n\t";
        local $" = "\n\t";
        print $outp "@{$calls{$name}{$rc}}\n";
    }
}

Second Toy

This next one needs at least two parameters. The first arg is the source file. This file will be searched for all subroutine declarations. It will then search through all the remaining files on the command line, looking for those subroutines. When it finds one, it will print the name of the file, the line number and the call itself. It is heavily oriented towards OO perl and it does require perl 5.5 or better - I am something of a compiled regex junky.

#!/usr/bin/perl -w
use strict;

die "$0 <source> <target> ..." unless @ARGV > 1;
my %keyword = ();

my ( $source, @targets ) = @ARGV;

my $package = '';
#---
# Try to extract the keywords, precompile the regex and store
#---
open SRC, $source or die "Couldn't open $source : $!";
while( <SRC> ) {
    $package = $1 if ( ! $package && /^package\s+(.+);/ );
    next unless ( /^sub\s+(.+){\s*$/ );

    my $name;
    $name = $1;
    next if ( substr($name,0,1) eq "_" );

    $name =~ s/\s+$//;
    $keyword{$name} = qr/([\w:{}]+)->$name/;
}
close SRC;

for my $file ( @targets ) {
    my ( $line );

    open FILE, $file or die "Couldn't open $file : $!";

    LINE:
        while( $line = <FILE> ) {
            next LINE if ( $line =~ /^\s*#/ || $line =~ /^\s*$/ );
            last LINE if $line =~ /__END__/;


            for ( keys %keyword ) {
                my $regex = $keyword{$_};
                if ( $line =~ /$regex/ ) {
                    my $starts_at = $.;
                    my $lpack = $1 || '';

                    next LINE if ( $file eq $source ) && ( $line =~ /^
+sub/ );
                    next LINE if ( $line =~ /Usage/ );
                    next LINE if ( $lpack =~ /::/ && $lpack ne $packag
+e );

                    while( $line !~ /;/ && ! eof(FILE) ) {
                        $line .= <FILE>;
                        next LINE if $line =~ /^EOF/m;
                    }
                    $line =~ s/^\s*//;
                    printf "%s (%d) %s", uc $file, $starts_at, $line;
                    next LINE;
                }
            }
        }
    #LINE
    close FILE;
}

< mikfire

[clemburg]

Hm. This might be too low-level, but you might want to have a look a the following:

• Devel::GraphVizProf
• Devel::Trace
• Devel::TraceFuncs
• Devel::DProf
• B::Graph
• B::Tree
• GraphViz

Christian Lemburg
Brainbench MVP for Perl
http://www.brainbench.com

[BlueLines]

</code>The easiest way to reverse engineer a perl program is:

xemacs your.program.here.pl

Seriously, if the code baffles you that much, you should use the perl debugger. There's a node discussing its usage here.

Also make sure that you are using the -w switch and that you are using strict. If the code is this sloppy, then there's bound to be errors in it, and these options will help you debug the errors more quickly.

Text highlighting and indentation make code more understandable. Use an editor like Xemacs or VIM to reformat the code into more readable pieces. Both of these editors are available for all *nix and Win32 systems.

And if there's C/C++/XS stuff involved, check out strace(1), which exists for most *nix systems, as well as NT.

BlueLines

Disclaimer: This post may contain inaccurate information, be habit forming, cause atomic warfare between peaceful countries, speed up male pattern baldness, interfere with your cable reception, exile you from certain third world countries, ruin your marriage, and generally spoil your day. No batteries included, no strings attached, your mileage may vary.

[quidity]

You might want to use Deparse to show you the reduced syntax, this will produce slightly prettier output, which might help you a little.

# bad perl
print $foo; s/dog/cat/gm; m%foo%;
FOO: for ($foo) { /dog/ && do {last FOO}; /cat/ && do {$dog = 'rabbit'
+};}
exit(0) or die "wooot?";

When run through Deparse:

bash-2.03$ perl -MO=Deparse badperl.pl
badperl.pl syntax OK
print $foo;
s/dog/cat/gm;
/foo/;
FOO: foreach $_ ($foo) {
    if (/dog/) {
        last FOO;
    }
    if (/cat/) {
        $dog = 'rabbit';
    }
}
die 'wooot?' unless exit 0;

As you can see, some bits get clearer, while others get a little more murky, but it's a useful tool.

[BBQ]

It sounds like what you really need is a Perl IDE... This has been discussed in the past and there are a few options out there ranging from perl's own comand line debugger and vi, to Win9x visual tools with bells and whistles!

I am particular to the bells and whistles for debugging, but that's just me. :)

#!/home/bbq/bin/perl
# Trust no1!

[snellm]

http://sourceforge.net/projects/ctags -- Michael Snell -- [e-mail]