Thoughts on pod.

[ehdonhon]

Appologies in advance, very lengthy post

Introduction

Hello Monks,

I've been giving some thought lately to the plain old documentation (pod) system in perl. I must say that I really like a language that lends itself to self documentation so easily. The idea that I can create man pages and utilize cool utilities like pod2usage directly from the documentation within my code is really great, and a step above other languages I have used in the past. But, I find that the more I take advantage of all the great things Perl has to offer, the more spoiled and lazy I get. :)

With that in mind, there are two issues where I think pod could stand to grow.

Embedded documentation writers

I think any good programmer knows that any time you write code that relies on being updated in two different places any time you need to make a change, you are destined for some kind of catastrophe some time in the future. I think it is a rather minor extension to also say that failure to update your documentation whenever you make a change in your module's functionality is also a formula for trouble, or at least great inconvenience. Take the following code for example:

## Set FORKING_CHILDREN to zero to turn off forking
use constant FORKING_CHILDREN => 0;

...  ## Pages and pages of un-related code here.

=item C<some_function>

Here is my pod for some function.  In this function,
you are allowed to specify one parameter, the number of
forking processes.  You may turn parallel mode off by
setting the number of processes to zero.  If you neglect
to provide a value, then the default value of 0 will be
used.

=cut

That's a very basic example, but imagine the potential grief that you could cause somebody that is using your code if you later decide to set FORKING_CHILDREN to 1, or 5, or 200... and neglect to update the pod.

I suppose somebody might say, "Well, that's the programmers fault, because they should always update their documentation every time they make a change." But I would argue that while the burden of documentation is definitely upon the programmer, there is a greater burden of coding responsibly such that changes in one place of code do not necessitate changes in other places of the code. What I would rather see in the above example is a way to modify my constant value and have my pod automatically updated.

People who have experience using Concurrent Versioning System (CVS), are familiar with embedding tags that get substituted at commit time. For example, if you add the code $Revision$ into your text and commit it to CVS, then your code will be modified to include the present revision number of your code, like this: $Revision: 1.3. I wonder how easily such a system could be embedded into pod?

And while I'm on a roll, I would suggest that rather than invent a new language that can handle variable substitution and any other tricks we want to be able to do, why not just re-use a language that has already been written? Why not just be able to embed Perl code to write our documentation?

What I would love to see is something where I can write this:

=pod

=item C<my_function>

Here is some documentation, it is nice documentation.

=begin perl

print "The default value is ", ($default) ? 'ON' : 'OFF';

=end

=begin generated_pod

The default value is ON

=end generated_pod

In the above example, I would expect to be able to write the the perl text, and then have perl automatically insert the generated_pod text the next time I ran perl with the correct switch. When I run perldoc, the perl text would be ignored, and the generated_pod text would be considered equal to normal pod. The next time I run perl with the appropriate switch, all generated_pod sections would be removed, and then re-generated.

Re-Using pod

The other thing that I encounter quite a bit in object-oriented programming is a situation where I am sub-classing some existing module or joining multiple modules so that I can add new functionality, or create an interface for some specific task at hand. The problem is that I run into a 'lesser of two evils' situation.

I usually want to be able to provide a complete API for somebody to use, meaning that they don't need to think about how I'm implementing it or where the functionality is coming from, and that they don't have to read four different perldocs at the same time to understand how to use it. But on the other hand, I don't want to import somebody else's pod into my code, because that would mean always updating my code every time somebody else updates their code. That would be a maintenance nightmare and lead to out of date pod very quickly.

A much better solution, in my humble opinion would again to be able to embed some tag that means "Insert some block of pod from a different file/module here". The idea being that as one file's pod is updated, your pod would be updated dynamically.

I see two approaches that could be taken for this. It could be implemented exactly the same as my above suggestion, where you have some substitution and text manipulation that is actually done by the compiler. The problem with this one is that your module might sit compiled on somebody's machine for a year without being updated, while somebody else's module the provides pod for your module could have been updated on the same machine monthly.

The second option that I see in this case would be to allow perldoc to implement the substitution on the fly without actually updating the source. In a situation where we have dynamic content, this is probably better, but it is at the expense of spending the cpu cycles every time you load the page, which could potentially take a while for pages which use pod from pages which use pod, etc.. Perhaps some switch could be added to perldoc to toggle the pod inclusion on and off.

Conclusion

These are just two observations / desires that I have about pod. I'm mainly throwing them out because I really would like to know what other people think about these ideas. I'm sure I'm not the only person to have ever thought about this, so I'm fully prepared to insert foot in mouth if it turns out that the things I'm looking for are already possible. On the other hand, I really can't find much talk about pod anywhere, save a few brief mentions here and there in various perl books. Am I alone in my quest for easier and better documentation?

UPDATED: Added readmore tag

[zengargoyle]

Ooops... back button got me...

Check out Simple Document Format. It's Perl/POD friendly.

4.3. Embedded Perl Scripting

As systems built by embedding scripts within HTML are
typically easier to customise than systems build by
generating HTML from scripts, embedded scripting is now
a well established web-publishing solution.

Likewise, Perl embedded in SDF can be a powerful
combination providing this same flexibility with some
additional benefits:
SDF is arguably easier to read than HTML
the information can be published in several formats.

To embed a block of Perl code, the script filter is
used.

For example:

    !block script
    for $i ('a' .. 'z') {
        print "$i";
    }
    !endblock

To embed an expression within paragraph text,
the [[..]] syntax is used.

For example:

  Hello [["wor" . "ld"]]

Note: If the expression is a single word, it is assumed
to be a variable name, otherwise the expression is
treated as a Perl expression.

For single line scripts, the script macro can be used.

For example:

  !script $next_version = $var{'VERSION'} + 1

[PodMaster]

First I want to emphasize that pod is plain old documentation, so no changes are welcome and will be systematically rejected (yours particularly and by ten orders of magnitude).

I love pod, and I will fight with all my might to keep pod the way pod is because pod is great. What you do with pod, and how you choose to manipulate pod, should never affect pod itself, as perl views it.

CVS ~ is just that, Concurrent Versions System, and perl it self shall not be CVS, nor shall it be altered to do magical dynamic pod.

As for pod maintenance, well, you will run into the same issue even if you have some magical software do programmatic changes to the pod, you still gotta write it ... basic documentation trap

perldoc may potentially only be modified to support such a wild feature only through the use of a non-standard switch, such as -magic, as to not interfere with the perldoc we've al known so far. Stability is key.

What you speak of (reusing and "dynamic" pod) can be easily accomplished. One can use Pod::Tree, and perhaps you ought to do just that, write a module that when fed a source file would do just what you want it to do, as perl shall never indulge such a wild, but interesting, idea.

If you ever decide to implement this, I would like to play with it, so please share if you do.

As of now, the term "Dynamic Pod" is trademarked by podmaster, and can be freely and widely used as long as podmaster is the only one making money.

These have been my views on perl pod, and should be highly regarded (i'm not smiling).
 

Look ma', I'm on CPAN.

---

** The Third rule of perl club is a statement of fact: pod is sexy.

[ehdonhon]

Thank you for your input, I think you have a couple valid points nestled in there.

What you do with pod, and how you choose to manipulate pod, should never affect pod itself, as perl views it.


Good point. I agree that whatever changes occur, they should only suppliment the present system, not alter it. Also, any new additions should not break systems that correctly use present standards.

As for pod maintenance, well, you will run into the same issue even if you have some magical software do programmatic changes to the pod, you still gotta write it ... basic documentation trap


Well.. I agree. You do still have to write it. But writing pod and maintenance of pod are not the same thing. It is easy to remember to write pod as you are creating something new. Remembering all the places you need to update your pod when you make a change 9 months later is harder. While self-updating pod doesn't remove the responsibility of checking the pod from the programmer, it does make it more difficult for something to be missed.

Perhaps you ought to do just that, write a module that when fed a source file would do just what you want it to do


Good point. I suppose the best way to go about making changes is to provide a proof of concept and let people kick the tires around before proposing it as a standard.

[demerphq]

Hi.

While I havent read your post in detail (will later,) it struck a chord with me because I have been doing some interesting work with POD lately.

The basic scenario was that we have a variety of applications with fairly complicated command line and configuration components. The problem is these tools are hard to hand over to production teams due to their cryptic nature. The solution we started looking at was self documenting config files. So over a weekend with no cash I threw together the following:

• Config::Pod::Writer -- A module for dumping out arbitrary data structures encoded as pod, with documentation if it is available.
• Config::Pod::Reader -- A module for reading pod files generated by the writer
• Config:Pod::ParamRead -- A module for parsing the command line in the way required for the write reader.
• TODO Config::Pod -- A wrapper for all of the above to make it an easy to use tool both for the programmer and customer.
Ultimately all of these will be wrapped in a module Config::Pod which will supply all configuration and command line parsing and validation. Since the config file _is_ pos then pod2html (in module form or cmd line form) can be used to generate pretty HTML versions of the config files, as well POD::usage can be used to display the rules.

When I get it all completed I plan to release it here then if its thought to be usefull to CPAN.

BTW more related to your post, have a look at the =for directive and the =begin =end directives. Using these and a POD::Parser subclass you could accomplish most of what you are interested in.

UPDATE

A clarification. ehdonhon /msgd me with a comment suggesting that I wasnt clear enough. What i meant by the above was in regard to ehdonhons idea of embedded values in pod. Specifically
then your code will be modified to include the present revision number of your code, like this: $Revision: 1.3. I wonder how easily such a system could be embedded into pod?
My point with for directive is this (and Ive basically done something similer in my code)

=for config VERSION

=cut

our $VERSION=0.1;

=for config END_VERSION

The for directive tells the preprocessor that the following chunk of code should serve a double purpose, code _and_ documentation. The subtleties involved are left as an exercise for the preprocessor author.

Yves / DeMerphq
--
When to use Prototypes?

[Amoe]

Although pod is designed for writing documentation of Perl scripts and modules, it's not restricted to use solely for that purpose. I write school projects in pod. I write my web pages in pod, then use pod2html. At the moment I'm working on a system that turns pod to html and delivers it to a browser on the fly.

I think specializing pod is a danger, as it could damage all these perfectly legitimate uses. Pod is supposed to be a very flexible documentation system - "plain" as in you make it what it is. Dynamic pod (*royalties at podmaster*) takes away from the aim of it.

Having said that, though, one thing I would like is for it to be easier to convert pod to html. Although that's really a weakness of HTML rather than pod. Damned ennumerations.



--
my one true love

[Stegalex]

When I was a Java programmer, there was a neat utility called Javadocs which publishes documentation about module interfaces automatically. It's pretty cool and very automagic. Is there such a thing for Perl? I like chicken.

[robot_tourist]

Not having used Javadoc or pod 'in anger' yet (I am but a university student and students only doc their code if they get marks for it), I can say that the plain old mark-up system of pod looks very easy to use. Javadoc, on the other hand, makes much prettier documentation, but has a more difficult syntax and you need an html viewer to look at it. The Javadoc convention of putting the documentation above a method and explaining all your params and return values is good.

I have a wiry brain/each eye a camera. Robot Tourist, by Ten Benson

[Fengor]

a very interesting writeup. It's clearly structred and well written so I ++ it.

Although you could have used the readmore-tag ;-)

--
"WHAT CAN THE HARVEST HOPE FOR IF NOT THE CARE OF THE REAPER MAN"
-- Terry Pratchett, "Reaper Man"

[ehdonhon]

Thanks for the suggestion. I had forgotten that such a feature existed. I've updated my post.

[erikharrison]

You've not been the first (Or the last I would guess) who has suggested amplifying POD. Though I have strong objections to altering POD (many of which have been already cogently made. Thank you podmaster.) there are some interesting ideas to be discussed. POD Is Not Literate Programming at perl.com is an excellent discussion of some of these ideas.

The best solution to the problem, as we seem to be converging upon it, is to have a "POD preprocessor" that parses a superset of POD in a program and alters it for inclusion in a distributed module or the like. This would be a useful tool for the programmer, and make documenting the code easier, but wouldn't break POD's unique uses as a markup or even alter it at all.

Cheers,
Erik