Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

The problem of documenting complex modules.

by BrowserUk (Pope)
on Jul 05, 2015 at 08:41 UTC ( #1133234=perlmeditation: print w/replies, xml ) Need Help??

This is meditation; but I also hope that it might start a discussion that will come up with (an) answers to what I see as an ongoing and prevalent problem.

This has been triggered at this time by my experience of trying to wrap my brain around a particular complex module; but I don't want to get into discussion particular to that module, so I won't be naming it.

Suffice to say that CPAN is replete with modules that are technically brilliant and very powerful solutions to the problems they address; and that deserve far wider usage than they get.

In many cases the problem is not that they lack documentation -- often quite the opposite -- but more that they don't have a simple in; a clearly defined and obvious starting point that gives a universal starting point on which the new user can build.

And example of (IMO) good documentation is Parallel::ForkManager. It's synopsis (I've tweaked it slightly to remove a piece of unnecessary fluff):

use Parallel::ForkManager; my $pm = Parallel::ForkManager->new($MAX_PROCESSES); foreach my $data (@all_data) { # Forks and returns the pid for the child: my $pid = $pm->start and next; ... do some work with $data in the child process ... $pm->finish; # Terminates the child process }

is sufficient to allow almost anyone needing to use it, for almost any purpose, to put together a reasonable working prototype in a dozen lines of code without reading further into the documentation.

It allows the programmer to get started and move forward almost immediately on solving his problem -- which isn't "How to use P::FM" -- and only refer back to and utilise the more sophisticated elements of P::FM, as and when he encounters the limitations of that simple starting point.

As such, the module is successful in hiding the nitty-gritty details of using fork correctly; whilst imposing the minimum of either up-front learning curve or infrastructural boiler-plate upon the programmer; who has other more important (to him) things on his mind.

Contrast that with something like POE which requires a month of reading through the synopsis of the 800+ modules in that namespace POE::*, and then another month of planning, before the new user could put together his first line of code. As powerful as that module, suite of modules; dynasty of modules is, unless you have the author's help, and lots of time, getting started is an extremely daunting process. In that respect (alone perhaps), POE fails to enable a 'simple in'.

And before anyone says that it is unfair to compare those two modules -- which maybe true -- the purpose was to pick extremes to make a point; not to promote or denigrate either.

Another module that I know I should have made much more use of in the type of code I frequently find myself writing, is PDL. I've tried at least a dozen times to use PDL as a part of one of my programs; and (almost) every time I've abandoned the attempt before ever writing a single line of PDL, because I get frustrated by the total lack of a clear entry point in to the surfeit of documentation.

There's the FAQ, and the Core; and the Index; and the QuickStart; and the Doc; and the Basic; and the Lite; and the Course; and the Philosophy; and the pdldoc; and the Tips; and ... I'm outta here. I'm trying to write my program, which does a little math on some biggish datasets that would benefit from being vectored, but life's too short...

Again; the underlying code is brilliant (I am assured), and it isn't a case of a lack of documentation; just a mindset that says: "this is PDL in all its glory, power and nuance. bathe yourself in its wonderfulness and wallow in its depth". Oh, and then when you've immersed yourself in its glory, understood its philosophy, and acclimated its nuance, then you can get back to working out how to use it to solve your problem.

And that's a real shame; and a waste.

I'm not sure what the solution is. I do know that the modules I use most List::Util, Data::Dump, threads etc. I have rarely ever had to look at the documentation; their functionality has (for me) become an almost invisible extension of Perl itself, and only the occasional (perhaps you forgot to load "sum"?) reminds me that they aren't.

Of course, what they do is essentially pretty simple; but that in itself is a perhaps a clue.

I do know that (for me) the single most important thing in encouraging my use of a module is being able to C&P the synopsis into my existing program, tweak the variable names, and have it do something useful immediately.

In part, that comes down to a well designed API; in part, to well-chosen defaults; and part having a well-chosen, well-written synopsis that addresses the common case; with variable names and structure that make it obvious how to adapt that synopsis for the common case. Once I have something that compiles and runs -- even if it doesn't do exactly what I need it to do; or even what I thought it would do from first reading -- it gives me a starting point and something to build on. And that encourages me to persist. To read the documentation on a as-I-need-to basis to solve particular problems as I encounter them.

Over a decade ago, I posted My number 1 tip for developers.; and this is the other side of that same philosophy. Start simple and build.

And that I think has to be the correct approach to documenting complex modules. They need to:

  1. Offer a single, obvious, starting point. The in.
  2. That needs to be very light on history, philosophy, jargon, technical and social commentary and background. And choice.
  3. It needs to offer a single, simple, well-chosen, starting point, that requires minimal reading to adapt to the users code, for the common case.
  4. It then needs to offer them a quick, clear, simple path to solving their problem.

What it must not do:

  • It mustn't present them with 'a bloody great big list of entrypoints/methods'.
  • It mustn't offer them a myriad of choices and configuration options.
  • It mustn't take them on a deep immersion in the details of either algorithms or implementation.
  • It mustn't present them with either "Ain't this amazing" nor "Ain't I clever" advert.
  • It mustn't waste their time with details of your personal preferences, prejudices, philosophies and theologies.

If you want programmers to use your modules, you need to tell them what (the minimum) they *NEED TO KNOW* to get started. And then give a clear index to the variations, configurations and extensions to that basic starting point.

Achieve that, give them their 'in', with the minimum of words, fuss or choice, and they'll come back for all the rest as they need it.


This is ill-thought through and incomplete, so what (beyond risking offending half the authors on CPAN) am I trying to achieve with this meditation?

I'd like to hear if you agree with me? Or how you differ. What you look for in module documentation. Examples that you find particularly good; or bad.

It'd be nice to be able to derive from the thread, a set of consensus guidelines to documenting moderate to complex modules -- that almost certainly won't happen -- but if we managed to get a good cross section of opinions on what makes for good and bad documentation; and a variety of opinions of the right way to go about it; it might provide a starting point for people needing to do this in the future.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

Replies are listed 'Best First'.
Re: The problem of documenting complex modules.
by toolic (Bishop) on Jul 05, 2015 at 12:46 UTC
    Here is my experience with XML:

    The first time I had to parse an XML file, I started out using regular expressions. A while later, I discovered this thing called CPAN, and it had XML::Parser. I stumbled along with that until I started hanging out at PerlMonks, where people were using other things like XML::LibXML. I could easily see the clear advantage of this over the lower-level XML::Parser, but I struggled mightily with the docs. I gave up on it and tried XML::Twig. Twig had its own tutorial website which walked me through the baby steps I needed to get going. It had short XML example files and equally short code examples that I could run. Because of that tutorial, I've stuck with Twig because it does everything I need. Other modules may be better, but I have confidence that I can figure out any new things I need from the Twig docs.

    XML is probably in the middle as far as complexity goes. Not as trivial as List::Util, but not as complex as PDL. Whenever I answer someone's XML question here or elsewhere, I typically warn XML new-comers that any parser requires an investment in time.

    My experience with PDL mirrors yours. I've given a couple half-hearted attempts at it, but I easily give up. It looks cool, but I guess I really don't need it. I see this more as a personal flaw of mine than a criticism of PDL. I agree that I wish it had an easier jump-in point, if that is even possible. Is it possible for this and other complex modules? Should there be smaller stepping-stone modules that have a reduced function set? Once you need more than the basics, there is an obvious migration path to the real deal? The internet is full of free trial software that offers reduced functionality. Before you commit the big dollars, you can give it a trial run with the freebie. I realize this isn't the most direct analogy, but it is along the same lines.

    Another experience: The first time I saw Devel::Cover, I knew I wanted to use it. I struggled a little bit, but finally got it. As penance for posting a worthless reply to Devel::Cover for Dummies, I put together a quick-start for the module: Re: Devel::Cover for Dummies (quick start). PerlMonks is hardly the best place for this; it would be better as part of the module's distribution.

    Some suggestions:

    • Baby-step tutorials packaged with the module's files.
    • Small example scripts with small inputs that are runnable.
    • Companion starter modules with reduced functionality (Tiny, Simple, etc.).

    I agree that many CPAN modules lack sufficient documentation. My initial reaction is disappointment, until I remember the adage: You get what you pay for. CPAN is free, after all. Perhaps the author just didn't have time, or is ignorant in the ways of documentation, or just doesn't care. I do have the option of trying to improve this situation: I could contribute and upload a patch to the POD (or example code files).

      I agree that many CPAN modules lack sufficient documentation. ... You get what you pay for. CPAN is free,

      Thanks for your response. I may come back to your points once we get a few more, but at this stage I just want to make clear, the OP isn't a complaint about the documentation of any of the existing modules on CPAN. Those I mentioned or any others.

      It is about trying to come up with ideas for what authors of new complex modules -- or those contributing new or amended documentation for existing ones -- can do to improve the situation.

      It's not enough to just provide lots of documentation -- if it was, we wouldn't be mentioning PDL which has tons.

      So, the purpose of the OP is not to complain about existing modules, but rather to brainstorm about how new modules can avoid their mistakes.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
        Your intention is clear to me, and I think it will be to others. By no means did I intend to steer the conversation in the direction of complaining about existing docs. I think my stream-of-consciousness point was that I can (and should) make the module easier to use for myself and others by contributing patches/examples.

      If the intro would be better as part of the module's distribution, did you submit it to the module maintainer? Programmers in general hate to write documentation so if someone does the job for them, they may be happy to accept the results.

      Jenda
      Enoch was right!
      Enjoy the last years of Rome.

Re: The problem of documenting complex modules.
by marinersk (Priest) on Jul 05, 2015 at 17:17 UTC

    Spot on.

    I'm not as fast as I used to be, but I'm a fairly talented and reasonably accomplished software architect and designer. I find it fun to design and code the solution, but when the time comes that I surrender to using CPAN, it is almost always because I am interested in saving time. The lengthy list of other benefits of using CPAN are more or less incidental for me, most of the time.

    I have had many modules I've simply ceased trying to use because of the time investment required to use them -- I could be writing the code to accomplish the task and thus having fun, and have an API that makes sense to me; or I can suck down a metric ton of coffee and Mt. Dew in order to stay awake reading documentation.

    I'm currently in a holding pattern on one of my personal projects because I have a module that does what I need, and the algorithm is sufficiently complex that even I can tell it will be faster to learn to use the module.

    The working snippet I have from another Monk's post showed me which subroutine/method is likely to yield what I want, and preliminary tests show it truly does the heavy lifting -- I just need to adjust its results.

    The PerlMonks node was my "in".

    Unfortunately, the documentation chooses to teach how the algorithm works, and wisely approaches it by degrees, showing how to use smaller, easier-to-understand functions first, and the method I need to use is naturally dead last on the list.

    The old hacker in me wants to just reverse-engineer the sample code, figure out how the method works, apply that knowledge to the problem I'm trying to solve, and just get on with it.

    However, I'm an algorithm junkie; going through the motions as presented in the module would be juicy reading and would likely expand my knowledge base on problem-solving algorithms very nicely. But that will take time. So I'm resisting temptation now to take my in and run with it -- but make no mistake, that is precisely what I am motivated to do.

    In this case I'm keeping myself at bay, waiting for the time to research this one properly, for those tangential benefits. But I will need to make time for it, which reinforces your point.

    Get the programmer programming, and he or she will program. Make them stop to read, and many will fall away (or fall asleep).

    As to solution, the only thing outside of the obvious would be perhaps to encourage the creation of a CPAN equivalent of a Tech Writer team; folks whose other CPAN documentation meets with widespread appeal and approval, and, like a Guild, could provide their services upon request of the module's maintainer.

    I have no idea if this is even feasible; but it's the only thing I can think of at the moment.

      the only thing outside of the obvious would be perhaps to encourage the creation of a CPAN equivalent of a Tech Writer team;

      About a decade ago I was part of a huge project that was responsible for overhauling, consolidating and unifying the IT systems of an entire country. 100,000 desktops and 5000+ servers across 23 different government depts.

      As Senior System Architect for the server and desktop deployment; the single best decision I made was to bring a Technical Writer on board. With almost zero programming knowledge, but several years of taking programmers notes, comments and diagrams and turning them into properly structured documents; he paid for his not expensive salary every week in the time he saved the far more expensive programmers from expending their time trying to reprogram Word (the customers choice) to work like emacs or textpad or whatever other editor that particular programmer favored.

      The second best decision was to give him the authority to demand full and timely answers from programmers, to any questions he had arising from their notes or code. That included me. Only by giving him the authority, to go with the responsibility we were all only to glad to pass to him, did we enable him to do the job we needed of him.

      Programmers are nearly always too close to the code they write, to also do a good job of documenting it. Because they know the internal workings so well, they are apt to simply forget to mention stuff that is ingrained in their brains, but that will be completely opaque to users. On the other hand, they have the habit of going into excruciating detail about stuff that users simple have no need to know. However, you do need the input of someone with a seriously good knowledge of how to use the code. And that's where the non-programing technical author comes into his own.

      By having them able to directly pick the brains of the programmers, but also stand apart from the code, they are able to act on behalf of the users to extract the salient information. And their authorship skills mean that they can structure the information in the way and form that the users need it; rather than the way the programmers prioritise it.

      Finally, they need to be dogged, persistent, and thick-skinned. Programmers collectively have a tendency to prize their time above that of other people; and to be intolerant of those that don't get what they get. Tech.Auth is a tough gig, that requires a tough mindset in addition to all the other skills. The best ones are few and far between; but very worth seeking out and retaining.

      All of that said, I do think there is some mileage in coming up with a template for module documentation. Not just a set of section names and keywords; but also a set of concrete suggestions and guidelines for what should appear in those sections.

      Also, as unpopular a suggestion as it is likely to be, I happen to think that POD is just too limited and limiting to allow the creation of good documentation for complex (and even moderately complex) modules; much less frameworks and toolkits. IMO; html is far better suited to the task. People will probably point out that POD can be converted to html (amongst many other formats), but the only reason it can do that, is because it is the lowest common denominator amongst those other formats; and that makes it very limiting.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

        Excellent thread, BrowserUK ... very thought-provoking throughout, and showered with up-votes.

        I definitely agree with you that POD is too limiting ... except for the fact that it can be, and to the extent that it is, embedded within or alongside the Perl code that it refers to, and therefore also subject to change control.   It is, therefore, a very special purpose form of documentation, and not good enough for the entire system.   (But, one might suggest, really not intended so to be.)

        For most documentation purposes, I use the DocBook format (or, in some cases, another proprietary format such as the one you encountered Intel using ...), and the XMLMind XML Editor (XXE) as the writing tool of choice.   (I’ve never purchased the commercial version.)   The documentation is written in XML format, but the tool formats it on-the-fly so that you are seeing it, and writing it, in a WYSIWYG-formatted view.

        These are “semantic markup” techniques:   for instance, the definition of a <function> includes subtags (and subtags of these) which describe what the various elements are ... arguments, return values, description, errors thrown ... not how they will look on the page.   From this XML-database, which is always known to conform to the specified schema, information can be selected, extracted, and then formatted into a variety of deliverables, either using (commercial ...) XXE facilities, or third-party XML tools like Saxon.

        There are also converters which can scan for and extract POD documentation, although they oblige the POD-writer to adhere to certain “markup” conventions in order to convey usable semantic meaning.

Re: The problem of documenting complex modules.
by bitingduck (Chaplain) on Jul 05, 2015 at 21:00 UTC

    I'm probably as much a "CPAN programmer" as Perl programmer, since there's so much stuff out there that's already done that helps me solve my problems.

    The single most valuable thing to me in module documentation is a few little snippets of working code that I can paste in and run as a starting point. It removes ambiguity over what really needs to be done in setup and what needs to be passed where. Ideally for a complex module there will be a few examples that address likely use cases of people coming to the module for the first time. They don't have to be complicated, but if they address in a few lines what I might have spent a bunch of time trying to sort through without the module, then they a) show me that the module can address my problem, and b) give me an entry point where I can start building my bigger solution without having to spend a bunch of time just trying to get started.

      Sounds like we are pretty much in accordance. Do you have any specific examples of (somewhat complex) modules you use regularly that either were:

      1. really easy to get started with despite their complexity.
      2. Or really tough to get started with; but you persisted because you needed their functionality.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
Re: The problem of documenting complex modules.(Summation.)
by BrowserUk (Pope) on Jul 07, 2015 at 14:00 UTC

    Thanks to everyone who took the time to contribute.

    The consensus, such as it is, it that the synopsis should contain a short, clear, complete & correct sample usage, that acts as a working starting point on which to build.

    The other idea that (for me) stood out was toolic's suggestion that there could be a Tiny/Simple version of the package that acts as an 'easy in' for the full blown all-singing & dancing version.

    Beyond that; the only conclusion I can draw is that good documentation is hard, and not fun, and programmer's are apt to get away with the bare minimum. And often less than that.

    My own personal view, unsupported by anyone else who contributed here, is that too much documentation is both a real and prevalent factor; and that automated documentation al la doxygen, javadoc etc. is almost worse than none; as it gives the appearance of 'well documented' with none of the true benefits.

    Volume does not equate to good; and comprehensive is perfect for Reference; but it needs a Guide to make it usable.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
    I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

      How can a module author express gratitude for this wonderful thread other than saying thank you? This is my favorite thread on PM BrowserUk.

      I will merge MCE::Flow and MCE::Loop into MCE::Simple and leave small wrappers to not break existing scripts. I will add MCE::Tutorial and MCE::Cookbook both of which were planned for the upcoming 1.7 release.

      By myself, I am clueless on how to solve the duplicity seen in MCE::Flow and MCE::Loop or worst what to call it. Thank you toolic. MCE::Simple is the perfect answer and will help resolve the indecisive storm hovering in my brain.

      Thank you all for this.

      Mario

      ... and that automated documentation al la doxygen, javadoc etc. is almost worse than none; as it gives the appearance of 'well documented' with none of the true benefits.

      I could not agree more with this assertion. While such systems have one or two plus points (consistency of presentation, docs always in sync with code) this is usually far outweighed by the complete lack of any semantic meaning to the resulting documentation. There are no reasons given, no caveats, no SSCCEs, no algorithm references. And if/when the code author uses less than meaningful symbol names the whole exercise becomes pointless.

      For the particularly complex dists which this thread addresses the resulting volume of brutalist formal data churned out by such auto-documenters is enough to send anyone scurrying for sanctuary. Fine for a reference no doubt, especially to those already well familiar with the dist in question, but not for the neophytes who are only after a gentle concept-based introduction.

        far outweighed by the complete lack of any semantic meaning to the resulting documentation.
        the resulting volume of brutalist formal data churned out by such auto-documenters is enough to send anyone scurrying for sanctuary.

        The worst part in my experience is that their use serves as a ready justifiction for not providing anything else.

        It's like a car manufacturer supplying dealers with a comprehensive parts list, in place of Workshop manuals.

        No other industry would get away with the practice.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!

      I've found even if the docs are overwhelming, or autogenerated, there is still hope most of the time

      I skip the documentation and I look for "examples/" and "t/"ests as its code that actually supposed to run :) (also "etc/")

      I look for a FizzBuzz HelloWorld type example, clear simple purpose, easy to follow varnames..

      If it takes more than three steps its magic ie useless as a foothold

        I skip the documentation and I look for "examples/" and "t/"ests as its code that actually supposed to run :)

        I'd have to need what that module did very, very badly before I'd try to piece together how to use its functionality from a few dozen or hundred ok()/nok() tests.

        Reminds me of another module I recently looked at:

        • The module pod tells you nothing; just points you at a readme file in a source code tree.
        • The readme contains:
          1. An advert.

            I'm already looking, why are you advertising to me?

          2. Some thankyous to the developers.

            If the author works with the devlopers; can't he thank them in person?

          3. Copyright and license information.

            Aren't they traditionally in files called COPYRIGHT and LICENCE?

          4. A link to a website.

            At last; I'm getting somewhere

          Buuuuuuut no.

        • The website, a triumph of pointlessly oversized graphics and fonts and worshiping at the alter of social media. conceals all the links behind oversized buttons and meaningless text such that you have to click randomly in the hope that you find something that is active.

          And when you get to what might be (very) loosely termed, "the documentation" you read:

          Software Developers

          See step 3 of the INSTALL notes for simple examples, and the `xxxx` command documentation for more information.

          Great! A whole website and I've got to go off, download a .tar.gz, unpack it, scrabble around inside to locate a text file called INSTALL; then scrabble around inside that for a subsection of it in order to find the documentation.

        This is how they want to encourage me to use their code? Could they have made working out what it is and how to use it more obscure? Perhaps they could have encrypted it all and posted a web treasure hunt to track down the decryption keys al la GCHQ."Starting from www.canyoufind.it.co.uk entrants must hunt down four codes hidden around the world-wide web."

        I should coco. I don't care if it is free code, my time is more valuable than that.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
Re: The problem of documenting complex modules.
by talexb (Canon) on Jul 08, 2015 at 10:30 UTC

    This is very timely for me -- I'm looking into a way of developing an application that will run on both Windows and OS X (and ideally I can develop on Linux), and as part of my search I found Wx. Great, I thought, the "Hello, World!" example works -- what else ya got?

    Ah, Wx::Demo. Great. Hmm.

    Well, the code's installed, but there's no way to run the demo. Anyway, after a couple of dead ends, I finally figured out

    #!/usr/bin/perl use Wx; use Wx::Demo; my $app = Wx::SimpleApp->new; my $frame = Wx::Demo->new; $frame->Show; $app->MainLoop;
    was the answer. Not terribly difficult, but I sure wish that code fragment had been added to the README. (Yes, I know I can file a bug on CPAN, and perhaps even make a pull request for the change -- I just figured this out yesterday.)

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

      Not terribly difficult, but I sure wish that code fragment had been added to the README.

      That sums the problem up in a nutshell. Often the response from authors is something along the lines of: "I've been so busy with the code that I haven't time to do 'documentation'".

      But most times we aren't asking for, and don't need, reams and reams of heavily structured and typeset words; just a simple piece of code that the authors of these modules can write in a few minutes, from memory.

      10 or 20 lines of code that a) compiles clean; b) runs; c) does something simple but correct.

      And that isn't hard for the author(s) of packages to write; but without it, it can be days (or weeks; or never) for the new user to work out.

      Authors: Eshew the 'bloody great big list of repetitively formatted apis; ditch the diatribes of personal/professional/project philosophies; don't bother with fancy graphics, cute mascots, pretty logos; don't refer me to files buried in tarballs; and don't bore me with adverts, hyperbole and marketing spiel; GIVE ME SOME CODE!

      A small, complete, correct, (standalone), working example that does something simple, but vaguely realistic and useful.

      After that, I can try things, read the errors, make changes, see what happens, and if I get really stuck, come back to you (or a forum) with a specific question that probably won't take anyone familiar with module more than a few seconds to answer.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
        'That sums the problem up in a nutshell.'

        You should head your own advice when you write nodes here at Perlmonks. When people ask you genuine questions about your motivation, you berate them with a nasty rhetorical question that makes them sound stupid and refuse to answer their question. You are living up to the name BullyUk. You are a bully.

        Wow -- that's neat. It didn't occur to me to dig through the MANIFEST file, but I'll remember that for the next time. However, I think my point is still valid -- it's absolutely worthwhile to add something like my ten lines of Perl code to the documentation, so there's at least one example of how to get the code going.

        Alex / talexb / Toronto

        Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

      Your demo code is excellent and I urge you to go file a PR with it today :-)

      In my experience the more concise and clear the change you are proposing, the more likely it is to be applied.

      The module authors are no doubt busy, and if they can scan and understand your proposed change in two seconds, and especially if it's to the docs, they are very likely to hit the "merge" button. If they have to squint, or spend more than ~15 seconds understanding your change, the button is not gonna get pressed. Not at that moment when they get the email about your PR. And after that, who knows when.


      Remember: Ne dederis in spiritu molere illegitimi!
Re: The problem of documenting complex modules.
by planetscape (Chancellor) on Jul 07, 2015 at 05:06 UTC
    I do know that (for me) the single most important thing in encouraging my use of a module is being able to C&P the synopsis into my existing program, tweak the variable names, and have it do something useful immediately.

    I wholeheartedly agree. A personal pet peeve is the - thankfully rare - bit of synopsis code that doesn't work, or worse, errors. I immediately dump that module and start looking for another.

    HTH,

    planetscape
Re: The problem of documenting complex modules.
by ksublondie (Friar) on Jul 07, 2015 at 17:45 UTC
    ...slow clap with misty eyes...

    FINALLY, someone said it. I don't know how many times I can read documentation and still not have a flipping clue how to start. And documentation without any example code at all? I'm sure the author is extremely brilliant, but if I don't know how to use the module, it doesn't matter. It ain't getting installed.

    Next pet peeve would be modules requiring an extensive installation process without any documentation besides the copy/pasted ./configure; make; make install in the README. You know the ones. The ones that require jumping on your left foot backwards 7 steps under a harvest moon at midnight in a cemetery before they'll install correctly. I've been coding in perl for about 10 years and I DREAD a new module.

    ...Finally in desperation, pleading to the blessed holy monks for help. Only to receive the answer: "It's in the documentation."

    ...sigh...

Re: The problem of documenting complex modules.
by saberworks (Curate) on Jul 06, 2015 at 20:02 UTC

    I run into this all the time and agree with you on most counts. XML::Compile is one I had a problem getting into. There were tons of docs but nothing that walked through how to do it from start to finish. This was back in 2011 and (with help from the author of the module) I was finally able to get things working, and I did a blog post about it: Using XML::Compile to Output XSD-Compliant XML.

    Most of the time, if the documentation is too confusing I just pass and find another module. I had your exact experience with Poe and PDL and never spent sufficient time to learn them. Sometimes I feel like I might be missing out.

Re: The problem of documenting complex modules.
by karlgoethebier (Prior) on Jul 10, 2015 at 19:06 UTC
    "...they don't have a simple in; a clearly defined and obvious starting point that gives a universal starting point on which the new user can build."

    This is nothing but the truth.

    It seems like there is a tendency on CPAN that the documentation of some modules is written in a disastrous style. Just good for some initiated who know what to do.

    But compared to the documentation of less or more complex modules/classes in other languages, especially a certain language i think that we still live in the Garden of Eden.

    I don't know if you program in Java (unfortunately i'm forced to do it now). If not please try to figure out some stuff in this language. A very special experience. I guess you will land on some websites that just show that crap from javadoc. Next step: go to Oracle's Java Tutorial Site. Then go to Stackoverflow and continue. Now buy a book. Steps are interchangeable. No JavaMonks. That was it. I wish you well ;-)

    Best regards, Karl

    P.S.: IMHO another example of good documentation of complex stuff at MCE by marioroy.

    «The Crux of the Biscuit is the Apostrophe»

      I don't know if you program in Java

      I do. Or at least I did once. It is the only language I ever got certified in. Not that I consider that any recommendation.

      It took me just 3 weeks to go from zero Java to passing the three levels of Java certification that BrainBench offered back then (something like: Beginners, Advanced and Enterprise).

      I was angling for a (senior) position in a startup my brother was getting involved with, that required Java.

      (I kick it in to touch after my first telephone conference with the head honcho of the founding trio; he was a salesman, selling something that didn't exist, he didn't understand, and which he had no idea of how to create. But he talked up a storm and had secured $5,000,000 round 1 VC cash on the back of it. When pressed, he admitted that his main goal was a couple of years of fast cars and high living before folding the company.)

      If not please try to figure out some stuff in this language. A very special experience.

      I had to do so very recently in order to produce a Perl equivalent to a Java library.

      Not a pleasant experience, hence why my aversion to javadoc and its ilk is so fresh in my mind.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
        "He was a salesman, selling something that didn't exist, he didn't understand, and which he had no idea of how to create."

        A cool lyric. Perhaps it is a bit too long to print it on a T-shirt to wear in office.

        Best regards, Karl

        «The Crux of the Biscuit is the Apostrophe»

Re: The problem of documenting complex modules.
by einhverfr (Friar) on Jul 17, 2015 at 01:04 UTC

    I think one problem is actually complexity itself. One important goal of software engineering is to minimize complexity the irreducible complexity in a system and replace it with reducible complexity. The former makes it very hard to change things in predictable ways. The latter makes maintenance a lot easier too.

    This isn't to say that sometimes complexity is unavoidable. I have never seen a simple ORM for example (and ORMs are by nature very complex, which is why I prefer not to use them). In these cases it isn't clear that complexity can be avoided, or that a simple entry point in the docs will help.

    One thing I have found immensely helpful in dealing with complex modules is the idea of cookbooks. Cookbooks give you a series of contrived examples you can build stuff on. In this sense, perhaps, the synposis should focus on what a style of code might look like. The Moose documentation is a great example here. You have a simple example, sufficient for someone to read and get a sense of what the module kind of does. Then for real examples, you go to the cookbook.

Re: The problem of documenting complex modules.
by ribasushi (Monk) on Jul 13, 2015 at 05:47 UTC
    "This is meditation; but I also hope that it might start a discussion that will come up with (an) answers to what I see as an ongoing and prevalent problem."

    Not sure if this is exactly what you had in mind, but I kicked off a proto-proposal based on your musings 😈

    Cheers and thanks for the writeup!

      That first "Ideal candidate criteria": "You have general familiarity with Perl conventions and “perlish documentation practices”"; could be a mistake.

      Imagine if instead of this, you had something like:

      my $mce = MCE->new( max_workers => 8, chunk_size => 2000, tmp_dir => $tmp_dir, freeze => \&encode_sereal, thaw => \&decode_sereal, gather => \@a, init_relay => 0, input_data => $input_file, RS => "\n>", use_slurpio => 1, parallel_io => 1, use_threads => 1, spawn_delay => 0.035, submit_delay => 0.002, job_delay => 0.150, on_post_exit => \&on_post_exit, on_post_run => \&on_post_run, user_args => { env => 'test' }, user_begin => \&user_begin, user_func => \&user_func, user_end => \&user_end, user_error => \&user_error, user_output => \&user_output, stderr_file => 'err_file', stdout_file => 'out_file', flush_file => 1, flush_stderr => 1, flush_stdout => 1, interval => { delay => 0.007 [, max_nodes => 4, node_id => 1 ] + }, sequence => { begin => -1, end => 1 [, step => 0.1 [, format => +"%4.1f" ] ] }, bounds_only => 1, task_end => \&task_end, task_name => 'string', user_tasks => [ { ... }, { ... }, { ... },], );

      And when you mousever the keys, you get a brief explanation of the parameter; mouseover the value and the default is displayed; click the key and the embedded description comments from the original above are displayed in a popup.

      Suddenly, those five screens worth of dense, complicated, hard-to-read details are all available on one screen, with a clear overview and a quick way to compare/contrast/lookup the details.

      Apply that thinking through from the top level of the module.

      Doesn't that seem a better way to both provide and explore the details of a complex module?

      Is it too late to change “perlish documentation practices” for the better?


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!
        "Imagine if instead of this, you had something like:"
        The beginning of your reply precisely illustrates why this isn't a great idea for POD - your anchor got mangled entirely and does not (or at least originally didn't) point anywhere.
        "And when you mousever the keys, you get a brief explanation of the parameter"

        I am not seeing this mouseover effect under SCO... all I get is the "code snippet" gray thing. Can you elaborate what are you referring to?

        In any case by "perlish documentation practices" I meant "knows how POD works", not any particular specifics. For example this is how some of the APIs within DBIC are documented (this is the most prevalent, but not consistent throughout style, the project is definitely open to suggestions how to do it better).

        TL;DR: I don't think the wording perlish documentation practices is really that far off.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://1133234]
Front-paged by Athanasius
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (4)
As of 2017-12-16 15:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What programming language do you hate the most?




















    Results (455 votes). Check out past polls.

    Notices?