### Documenting Perl Scripts

by ajcannon48 (Acolyte)
 on Nov 09, 2005 at 15:31 UTC Need Help??
ajcannon48 has asked for the wisdom of the Perl Monks concerning the following question:

I have been tasked with documenting some perl scripts which I did not write - currently my perl is not that strong. Does anyone know of an analysis tool/script/utility which will interrogate a perl script and produce a report of the subroutines/flow/required arguments/ etc etc?? I think I may be asking too much but any advice or guidance would be most appreciated BTW - *many* apologies if this is posted in the wrong place..

Replies are listed 'Best First'.
Re: Documenting Perl Scripts
by radiantmatrix (Parson) on Nov 09, 2005 at 15:48 UTC

That would be really nice, yes. Unfortunately for would-be authors of such tools, Perl isn't well-suited to such interrogation. This is partially because the syntactic flexibility allows people to write really bad code: the kind which only perl can really parse.

So, you have to start with the assumption that the code follows some kind of guidelines or best practices. That's a big assumption -- after all, if the programmer were disciplined enough to follow good coding practices, she would have already documented well, and this issue wouldn't exist.

Also, consider that things like required arguments are not language-native. Subroutines are not declared in a way that says 'arguments foo, bar, and baz are required, and aleph and beth are optional'. Sometimes there are prototypes, but they are generally avoided (and for good reason). Some scripts will not even have subroutines, since it isn't required by Perl.

All of these things are positive from a developer standpoint: they make Perl a very powerful and flexible tool. From the standpoint of trying to automatically determine how given code functions, it's a thorn.

A collection of thoughts and links from the minds of geeks
The Code that can be seen is not the true Code
"In any sufficiently large group of people, most are idiots" - Kaa's Law
Re: Documenting Perl Scripts
by rinceWind (Monsignor) on Nov 09, 2005 at 16:49 UTC

I have several thoughts about this subject. In particular, when writing documentation, you need to pitch it at an intended audience. Perl has two methods of documenting inline - pod and comments. These serve two different purposes, and have two different audiences. Pod is aimed at users, whereas comments are aimed at code maintainers.

When writing pod for scripts, I consider the pod to be a user guide. As such, this will include full details of command line options, environment variables, configuration options, etc. I am a fan of Pod::Usage, especially in conjunction with Getopt::Long. For Tk GUI apps, I have used Tk::Pod::Text to launch a help window, browsing the rendered pod.

Your requirements are for documenting subroutine calls and dataflows. These fall into the category of commentary rather than pod. But, it's probably better as separate standalone files rather than in-line, in which case pod is just as acceptable a format as anything - probably more so.

I don't know any existing module for spitting out such documentation automatically, but I can suggest several places to start:

• The stash %:: (or %<namespace>::) contains entries for all subs. You can get this information from the perl debugger, thus:

  DB<1> x %::
[download]
• The module Pod::Coverage contains some code for parsing out all sub declarations from a source file.

• You might be able to catch all calls to a given sub with Hook::LexWrap

--

Oh Lord, won’t you burn me a Knoppix CD ?
My friends all rate Windows, I must disagree.
Your powers of persuasion will set them all free,
So oh Lord, won’t you burn me a Knoppix CD ?
(Missquoting Janis Joplin)

But what does this have to do with his problem?
Re: Documenting Perl Scripts
by swampyankee (Parson) on Nov 09, 2005 at 15:53 UTC

I'd break the problem into a couple of pieces.

2. Determine what each Perl program is supposed to be doing. This may require interrogating users (don't forget the thumbscrews to apply pressure).
3. If they are available, talk to the people who wrote them. This may be more difficult; the live ones may be incommunicado. You'll still need the thumbscrews; I hope you've kept them.
4. Get one of the Perl books, large quantities of highlighters, and oceanic quantities of the caffeine source of choice (I prefer Starbucks) and become the human Perl parsing machine for a while. While this is less than fun, you will soon know more about how to write (and not write) Perl than you've ever wanted to know.

emc

Except that he doesn't know Perl very well, so parsing it would be very very hard for him.

I didn't say it would be easy &snarky_grin;. It's also why I suggested the OP invest in some thumbscrews.

When the OP starts, s?he will have trouble with the vernacular; as the task progresses, the skill will improve. This is pretty much the way I learned CLISTS

emc

Re: Documenting Perl Scripts
by samtregar (Abbot) on Nov 09, 2005 at 18:51 UTC
Pick up a copy of the book "Perl Medic". It's all about doing maintainance on Perl code you didn't write.

-sam

Re: Documenting Perl Scripts
by cbrandtbuffalo (Deacon) on Nov 09, 2005 at 17:09 UTC
It might be useful to run the scripts through perltidy first. This will help you examine the code because it will make sure the indentation is correct and clean things up in a few other ways.
Re: Documenting Perl Scripts
by philcrow (Priest) on Nov 09, 2005 at 15:42 UTC
My first thought is to grep for lines in the script that start with sub.

There is also UML::Sequence <warning>I wrote it and I don't use it much</warning> which can run a perl program and generate a sequence diagram of what it did.

Baring those, a pencil and paper are good for your understanding. Building that will help you write good docs.

Phil

Re: Documenting Perl Scripts
by snowhare (Friar) on Nov 09, 2005 at 19:51 UTC
Re: Documenting Perl Scripts
by Anonymous Monk on Nov 09, 2005 at 22:34 UTC
I have been tasked with documenting some perl scripts which I did not write - currently my perl is not that strong.

Let me get this straight: you've been assigned to document a system you don't already understand, in a language you're not proficient in? :-( :-(

With no disrepect to you, this sounds like a bad idea: it's like asking a man who speaks only Japanese to write English documentation using a Japanese-English dictionary. The end results are unlikely to be great; and it's not a fair task to ask for the documentation writer. You might asking telling your boss, politely, to find a better use for your time and energy.

That said, if your boss is dead set on making you do this work, there's no one single tool that will help out. Perl has many, many ways to accomplish any given task (by design, perl's motto is "There's More Than One Way To Do It"). This means it's typically very hard to pin down what's actually being done; you have to check every single way someone might have done something to find out whether or not that thing has happened.

There's no good way to document the list of arguments to a perl function, because perl lets you pass nearly anything into a function. For example, perl functions can take a list of arguments (of any length), and return a list of arguments (of any length). The arguments may or may not even have names. Some functions even do different things depending on the number or types of the arguments you give them. The best way to document arguments, is, unfortunately, to read the code, and find all the places @_, \$_ <something> , or "shift" is used.

As for tracing program flow, again, it's complicated. There are a *LOT* of ways to transfer program flow control, and a lot of valid syntaxes for calling functions. There are normal function calls, method calls, calling code references, AUTOLOADED functions, functions created using the "*FOO{thing}" syntax, gotos and strings evaluated at run time. Then there's perl XS, which links Perl to other languages. There's die() which terminates a program, except if you're calling it from an 'eval' statement, or if you've got the __DIE__ signal handler set. There's the tie() command, which calls certain subroutines every time a tied variable is accessed. And so on...

Perl sometimes can't even figure out what some of these calls will do until it encounters them; the "eval" function, for example, will attempt to execute the contents of a given variable as if it contained valid perl code.

That said, if you *just* have a list of subroutines that call other subroutines, you could try taking the output from B::Xref, and organizing it into a call tree. That's what I did: but the copyright for that code belongs to my company, so I can't post it for you. :-( *sigh* ( Lesson: Never do cool stuff at work, no matter how handy the tool may seem. :-() Otherwise, someone else will own the cool stuff, and it will be illegal for you to recreate it :-().

If that sounds too hard, then just grit your teeth, and read the code as best you can. :-( Good luck! It sounds like you've been saddled with quite an unfair job; I've got several years of perl experience, and I'm still getting headaches untangling my ex-boss's hairy code. 8,000 lines of code without a subroutine in sight! :-( In your case, pray that the person writing the code was good at it, (unlikely, because good perl is documented perl), or that they were very basic, so that they're not trying to be "tricky" all the time. (*grumble* Stupid nested map/grep/conditional/postscript if debug statements with *no* indentation! ) :-(

--
Ytrew Q. Uiop

You poor fella... As said above you really shouldn't be tasked with something like this, but stick around here and learn Perl.

I've had to document\comment some shifty code that someone else wrote and they were never heard from again so I was on my own... (Thanks Paco!!)

Anyway, what I did was print the code out using an editor that uses syntax colors.(The original code was on Linux and it needed to be changed to Windoze). I use editplus http://www.editplus.com/ tape all the pages together and bust out the crayons and start pointing which subs are going where in different colors etc.. Then document using a flow chart. If you see spots that aren't commented and you know what it's doing, add in proper comments.

Hang around here long enough, and before long you'll bump into that really cool script you first wrote, and say... Ahh what was I thinking... This could be better!

Good Luck!

Whether or not it's a bad idea on the part of the boss (the OP seems to be a victim in this), it's got to be done. In the best case the person (or people) who wrote the various Perl programs is|are available, have an accurate idea of what they did, and are willing to talk. In the worst case, the original programmer has had an unfortunate incident involving a train. Somewhere in between is the realm where various forms of persuasion may be useful (sweet talk, beer, food, origami swans, pictures of the original programmer in compromising positions, thumbscrews)

emc

Re: Documenting Perl Scripts
by saberworks (Curate) on Nov 09, 2005 at 15:42 UTC
It's impossible to figure out which arguments to a function are required in perl. It's also impossible to even figure out all the subroutines, due to autoload and other types of dynamic subroutine declaration.

That said, there have been many threads here about documentation generators. They generally require you to format your comments in a specific manner, and then they parse your code and spit out pretty HTML documents. In general, these solutions are okay, but most people prefer to use pod (plain old documentation).

pod is nice because there are a zillion scripts out there to do stuff with it, and all the CPAN modules use it, so there's no need to learn multiple documentation syntaxes.
Re: Documenting Perl Scripts
by mikeock (Hermit) on Nov 09, 2005 at 15:56 UTC
When creating perl scripts/programs, I always end up creating the POD while writing it. If I do it while the program is still developing, I get the correct info.

Items I include are:

### Usage

- What you need to input to proggie

### Requirements

- what modules are required

### Known bugs

- Issues I need to fix.

Edit: Realised I did not answer the above ?. These are just my personal steps.

If you are having to figure them out... choose a function, learn it then document. Repeat for all section of the script/program.

What does this have to do with the problem though?
Re: Documenting Perl Scripts
by graff (Chancellor) on Nov 10, 2005 at 05:32 UTC
This is kinda old, but I think it still pretty much works for "run-of-the-mill", vanilla style perl coding: Tabulate sub defs, sub calls in Perl code

It was originally motivated by a question about probing apps written in perl as multiple, inter-related source files; I'm sure there are many coding styles where it doesn't work so well -- for example, I didn't give much consideration to OO-style modules and method calls -- but there is a chance it might help you a little. Just save it and run it on a set of perl scripts and see what you get.

(update: I just tried it now on a module I'm working on this week, and decided I should change two occurrences of  /^sub ... to  /^\s*sub ... (should be okay for work when sub definitions to be intended are indented) -- that old node has been updated accordingly. I'm sure other tweaks would be worthwhile too.)

One other update -- something that needs to be said to the boss: The code should have been documented when it was written. An important correlary to that point is that, if people are writing code for this person at the present time, those people should be writing at least some documentation -- and someone like you or the boss should be checking it -- before they write code. The code itself should be written to follow the specs as laid out in the documentation, and if that turns out to be a problem, the docs need to be fixed and rechecked.

Re: Documenting Perl Scripts
by perrin (Chancellor) on Nov 09, 2005 at 21:14 UTC
I recommend getting a copy of the book Perl Medic, by Peter Scott.
Re: Documenting Perl Scripts
by ghenry (Vicar) on Nov 10, 2005 at 08:48 UTC

You could write some tests that will fail for the scripts and then use Devel::Cover when you run them, it will then list all subs/functinos etc. in a nice html web page. You could then reference that quicker than parsing lots of lines of code, visually.

HTH.

Walking the road to enlightenment... I found a penguin and a camel on the way.....
Re: Documenting Perl Scripts
by vishi83 (Pilgrim) on Nov 10, 2005 at 05:41 UTC

Sorry if am late on this .......

As far as i know bout Perl utilities.. you have perltidy which cleans up your perl code and may it mite not do wat u hav asked for ( checking for arguments etc ..)

Also refer to perl checkstyle ... it will help you to produce codes with better style.. Also pods can be written to document your perl script..

All these stuffs available in CPAN.. so do an extensive search on this..

A perl Script without 'strict' is like a House without Roof; Both are not Safe;
Re: Documenting Perl Scripts
by planetscape (Chancellor) on Nov 10, 2005 at 17:02 UTC
Re: Documenting Perl Scripts
by chrism01 (Friar) on Nov 10, 2005 at 04:11 UTC
Having a had a quick look, the perltidy prog/site looks like a good place to start.
Properly formatted code is a lot easier to understand.
As mentioned above, the flexibility of Perl can make your task tricky.
If it's fairly straightforward code, then grepping for 'sub ' (ie with trailing space) to find the start of subroutines will help, as you can then use the results to look for sub invocations.
If you also want to lok for the more obvious flaws, (if you have time/have been asked to do so eg you will be maintainer), then using
perl -wc yourfile.pl
and inserting
use strict;
at the beginning of each prog will warn of a lot of potential run-time problems.
Good luck
Chris
PS Perl Medic book idea sounds good, but I've never read it. I can recommend the usual suspects ie Programming Perl (aka Camel), Perl Cookbook. Possibly even Learning Perl (aka Llama book) if you are very new to Perl. All 3 books pub by O'Reilly.
Re: Documenting Perl Scripts
by mkirank (Chaplain) on Nov 10, 2005 at 15:58 UTC
Re: Documenting Perl Scripts
by singingfish (Novice) on Nov 11, 2005 at 02:04 UTC
Looks to me like you're going to be:
1. Learning how to use the perl debugger. Emacs has a nice interactive debugging mode.
2. Writing lots of test cases.
3. Learning how to write pod (it's easy).
4. Suggesting to your boss that someone else does it.
Re: Documenting Perl Scripts
by Anonymous Monk on Nov 10, 2005 at 03:49 UTC
You should tell your boss to stuff it ... or maybe he is trying to make you quit or something. You must of done something that pissed him off. Either that or your boss has a sick sense of humor.

"You should tell your boss to stuff it."

This tends be be a career limiting move.

In my years of programming, I've been given maintenance problems in Fortran, PL/1 (and some of its subsets), COBOL, wacky special purpose languages for simulation, SQL, awk, C, C++, VB, various *ix shell-scripting languages, JCL, CLISTS, DCL, VBS, VBA, sed, InstallShield's scripting language, html, RAMIS II, and Speakeasy (this list may be incomplete). I haven't had any in Perl, because I'm the only Perl programmer at my current employer.

It's a learning experience; I really want to be indispensible, as in when I leave my employer frequently calls me with offers of large sums of money

emc

I really want to be indispensible
I've been there - on both sides of the fence. It doesn't work. Noone is indispensible. The world doesn't end if you're hit by a bus - let alone if you leave. No matter what mess you create, someone will figure it out, or rewrite it. The only one knowing Perl at the company certainly doesn't make you indispensible. Someone else can always learn. Or hired. Or your work can be replaced by a Java program.
Perl --((8:>*

Create A New User
Node Status?
node history
Node Type: perlquestion [id://507094]
Approved by saberworks
Front-paged by rinceWind
help
Chatterbox?
 [Corion]: Do you want to launch a script and keep the command prompt/console window open? [Corion]: Do you want to wait for a key press before closing the window? [LanX]: I want the command line in the history [tye]: -Mouse [Corion]: Option a) would mean launching cmd.exe /k c:\path\to\ batchfile- launching-perl- script.cmd. Option b) would be to add pause as the last line of said batch file. [LanX]: First day after holidays ... and already stressed by the fact that colleagues changed stuff without communication ... apparently I'm the only one trying to fight entropy [Corion]: LanX: The command is always in the history if you typed it in before. If you didn't type the command into the command line, it will not be there. I think there is doskey which can stuff command lines into the history LanX damns the cult of CB ;-) LanX WTF WTF WTF [LanX]: please forget my last 3 posts

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (13)
As of 2017-03-27 15:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
Should Pluto Get Its Planethood Back?

Results (320 votes). Check out past polls.