Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Becoming familiar with a too-big codebase?

by aboyd (Sexton)
on Jun 15, 2005 at 06:41 UTC ( #466813=perlquestion: print w/replies, xml ) Need Help??
aboyd has asked for the wisdom of the Perl Monks concerning the following question:

I am for the first time taking over a Perl codebase that is too big for me to keep conceptually within my head. It's a CGI CMS, and the code isn't commented. It contains hundreds of modules and I cannot deciper half of it. It's time for me to leave behind my childlike habits of burying tons of print statements everywhere to see what comes up.

I need to find automated discovery systems, ways to watch what the CGI does. I tried an old CPAN module, Devel::Trace. I got it installed but couldn't get any output from it. I see that Perl has a debugger, which I've never used. Can the debugger log each sub & object as they are called? How would I use the debugger on CGI scripts, or can I simulate a 500k text file upload via the command line? Are there other systems/modules/tools/ideas that can help a developer to learn his way around a too-huge codebase? Thanks for any advice.


  • Comment on Becoming familiar with a too-big codebase?

Replies are listed 'Best First'.
Re: Becoming familiar with a too-big codebase?
by PodMaster (Abbot) on Jun 15, 2005 at 07:05 UTC
    Devel::Trace usually does the trick for me. When it doesn't, I open up Devel::TraceCalls and GraphViz::ISA. If dealing with apache, I use Apache::Status.

    Devel::ModInfo (and the tools it comes with) might also be useful (though I've not used them).

    There are also useful B:: modules like B::Xref, but but they're more useful for developing tools like the forementioned modules (which might be what you're after).

    MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
    I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
    ** The third rule of perl club is a statement of fact: pod is sexy.

Re: Becoming familiar with a too-big codebase?
by planetscape (Chancellor) on Jun 15, 2005 at 09:43 UTC

    I have found several utilities to be helpful for just this sort of thing.

    When I asked something similar in the FullPage Chat, castaway pointed me in the direction of podgen, which will jump-start the process of commenting your monolith.

    DoxyFilt* is a filter than allows the source-to-documentation tool Doxygen to understand Perl. Once you have your source commented, documentation becomes absurdly easy.

    Perl::Tidy helps clean up your source and make it more readable. Perl Builder and ActiveState's Komodo may also have some helpful options or utilities.

    Update: Seeking advice about learning another's code and Reverse Engineering Perl Tool? also contain excellent suggestions.


    * Update: 2005-12-28 Kudos to both john_oshea and tfrayner for alerting me to the fact that my link above has been rendered usless by the foul creatures known as spammers... I have found what appears to be a good link to obtain DoxyFilt; the most recent version seems to be from August 24, 2005: Doxygen-0.84.tar.gz. Thanks again, guys!

Re: Becoming familiar with a too-big codebase?
by eyepopslikeamosquito (Chancellor) on Jun 15, 2005 at 10:57 UTC

    Find out who wrote the code and be nice to them. Offer to buy them lunch. Pick their brains. Learn all about them. Were they accomplished programmers or newbies when they wrote the system? Are they careful or slapdash? Are they domain experts? What was the political climate when they wrote the system? No matter how well documented a system, there is always extra important information and ideas available only inside the original designer's head.

    As you understand a section of code, comment it. If you see something that looks dubious, mark it with XXX, say. You can come back to these XXXs when you better understand the system as a whole and have more time.

    Whenever you find a bug in the system, write a test for it. Grow the test suite over time. I assume (like me) you will not be given time to write a comprehensive test suite before taking over maintenance of the code base.

    See also Analyzing large Perl code base. and the book Perl Medic by Peter Scott.

Re: Becoming familiar with a too-big codebase?
by samtregar (Abbot) on Jun 15, 2005 at 18:29 UTC
    Drop everything. Run to the bookstore. Buy:

    Perl Medic

    It's a book about rescuing bad Perl code from itself. It has all kinds of good advice for you. Read it.


      Drop everything. Run to the bookstore. Buy: Perl Medic It's a book about rescuing bad Perl code from itself. It has all kinds of good advice for you. Read it.

      Seconded (I'd mentally begun a reply with "Run, don't walk, ..." just before i read this node :-)

Re: Becoming familiar with a too-big codebase?
by dragonchild (Archbishop) on Jun 15, 2005 at 14:07 UTC
    Write a test suite. Slowly, over time, write a good test suite that provides 90%+ coverage. Then, refactor your ball of mud. That is the only way.

    • In general, if you think something isn't in Perl, try it out, because it usually is. :-)
    • "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"
Re: Becoming familiar with a too-big codebase?
by Fletch (Chancellor) on Jun 15, 2005 at 11:14 UTC

    See also Refactoring: Improving the Design of Existing Code (ISBN 0201485672). It may not be exactly applicable, but it outlines a very good mindset and process for improving existing code (e.g. Test, test, test; make small incremental changes; test, test, test; lather, rinse, repeat).

    We're looking for people in ATL

Re: Becoming familiar with a too-big codebase?
by Anonymous Monk on Jun 15, 2005 at 15:34 UTC
    To figure out what programs are doing, I used B::Xref and some wrapper scripts to generate a call tree for code written in a procedural manner.

    You might developing something along those lines yourself (since I foolishly wrote my program at work, it is,of course, copyright by my company, so I can't post code for you).

    Learn to use the perl debugger: it's handy. It will let you execute arbitrary perl code to see what it does, single step through functions, stop at certain lines, and so forth. While you just can't "go back" to a previously executed line in the debugger, you can often just copy the line, and single step through it. This is most useful for function calls that have gone awry.

    eg. Suppose the line $x = foo($y) returns the wrong thing. Just type s foo($y) into the debugger, and single step through foo() until you find the call in foo() that's wrong, say bar(). Then single step through bar(), and so on. Keep chasing your way down the call stack until you find the bug.

    The debugger certainly doesn't default to printing each subroutine that's called, but it can be made to do it by writing some custom debugging code. There may be CPAN modules that will do this for you, as well.

    Do a 'perldoc perldebug' for the standard debugger documentation; do 'perldoc perldebguts' to learn how to reprogram the internals of the perl debugger to make it do other useful things.
    Ytrew Q. Uiop

Re: Becoming familiar with a too-big codebase?
by eXile (Priest) on Jun 15, 2005 at 14:04 UTC
    Since you specifically asked for CGI debugging, i googled for 'CGI debug' and found CGI::Debug, never used it, but it seems to be able to give you quite some output on what happens at the CGI-level.

    I think I'm going to give it a try in my own day-to-day CGI debugging.

      CGI::Debug just dumps information to the browser.

      What you probably _really_ want (esp for the file upload stuff) is CGI::Capture, which is the anally-retentively-accurate record and playback module for CGI calls.

      Add the CGI::Capture line to the script pointing at a file, and after you do the first call to the CGI, it will replay the same request over and over, and if you run the CGI script from the console it will do a whole bunch of checks to make sure you are doing the debugging as accurately as possible.

      And because it works at the filehandle level, it isn't even dependant on CGI. You should be able to use it with anything.
Re: Becoming familiar with a too-big codebase?
by mkirank (Chaplain) on Jun 16, 2005 at 05:02 UTC
    I also work on a big code base (which is in mod_perl) and to watch the CGI, I use Data::Dumper and Enabling GUI Debugging under Mod_Perl , The debugging helps me watch each line and what is the actual result that is stored in variables and the different subroutines the program accesses
Re: Becoming familiar with a too-big codebase?
by aboyd (Sexton) on Jun 16, 2005 at 19:54 UTC
    I just wanted to take a moment to say "thank you" to the 9 people who offered replies. You guys have been a big help. Thanks so much. -Tony

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://466813]
Approved by Corion
Front-paged by broquaint
[Lady_Aleena]: shmem, that is understandable! The two examples in File::Find don't make sense to me on a quick glance.
[marioroy]: LA if the find worked from Unix command line, or does it not. Likely a quoting issue inside Perl qx.
[Lady_Aleena]: marioroy, the find worked fine at the command line.
[marioroy]: LA, yeah. than there's no reason why it cannot work inside qx. But chatting is hard in PM. I cannot see the code now.
[shmem]: Lady_Aleena: sometimes a quick glance isn't enough.

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (10)
As of 2017-04-23 20:56 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (432 votes). Check out past polls.