Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Perl Monks += TMTOWTDI

by hacker (Priest)
on Apr 14, 2003 at 13:17 UTC ( #250279=perlmeditation: print w/replies, xml ) Need Help??

How many times have you seen or heard a conversation like this?

<foo> "I'm trying to use Module::Thing, and I'm running into trouble with the constructor. Can someone help?"

<bar> "Have you tried using Other::Module instead? It has a more complete series of objects you can use, and it does the same thing as Module::Thing, but in less code."

<blort> "What about using IO::Thing instead?"

I've noticed this type of evolution many times over the last several years, where one (still useful) module has been deprecated by another module with similar or additional functionality. How do you know which module best fits the job, unless you can talk to people who walk around the halls of CPAN on a daily basis and poke at module innards?

Without the human interaction and experience provided by a site like Perl Monks, this really wouldn't be as possible as it is with. IRC is a good medium for "real-time" exchange of information, but IRC doesn't have Super Search!

I ran into this very recently with a script I wrote that used LWP::UserAgent, HTTP::Cookies, and HTML::TableExtract to log into PayPal and extract some payment statistics for me. The code worked, but was a bit longer and more convoluted than I would have liked. Enter WWW::Mechanize, which allowed me to reduce the modules used from 5 to 3, and cut the amount of code down by 2/3, and add a bit of additional functionality, in less than 75 lines.

If I hadn't talked to bart and Corion and some others who had used these kinds of modules before, I would still be rolling around with the stock LWP modules and doing hand-parsing of the content itself.

Another example which just came up in the ChatterBox was an example of validating a URI through the use of Regexp::Common and Regexp::Common::URI::http. I've been using the stock URI module to do this, splitting out the scheme and authority section, but now I see where Regexp::Common could be a better use of it. I would never have known about it unless bart or broquaint had mentioned it.

Another one from yesterday was Pod::Usage. I never knew you could dump the pod from within a file itself, without having the user interactively run 'perldoc ./' on the code. Thanks to the experience and insight of Enlil and tye, I've learned a "better way", and I shortened my code in the process.

Yet another, HTML::LinkExtor, which works well for extracting links from HTML pages, but can also be done (with much more robust functionality) using HTML::TreeBuilder. Granted, most of the modules I'm mentioning are HTML or "web" related, but that's the space I play in right now. I know this pardox exists across all of perl, however.

What do other monks do when there is the possibility of a "better path to walk" with code and modules?

How do you know which module is "right" for the task? (or wrong for the task)

How do you search for something by name, which can only be described by concept?

Replies are listed 'Best First'.
Re: Perl Monks += TMTOWTDI
by jmcnamara (Monsignor) on Apr 14, 2003 at 13:38 UTC
Re: Perl Monks += TMTOWTDI
by l2kashe (Deacon) on Apr 14, 2003 at 13:43 UTC
    Nice node..

    On the note of the questions at the end, I have to say thats what this community is all about. I firmly agree with that.

    On a different note I think part of the issue is CPAN's organization of modules, and what information can be garnished about modules quickly. Imagine if you did a search for say URI on CPAN, and instead of getting all of the URI:: modules, and anything that has URI in its name or description, you get a list of modules starting from most frequent downloads to least frequent downloads.. Then you could quickly see what modules are being used more by the community at large, as well as close runner-ups. This way your search space has been cut down drastically and you can compare similar modules functionality as you see fit, without wasting time working with modules whose names sound close, but don't provide what you thought they did..

    But that leads to newer modules possibly being totally ignored as people see the most frequent d/l and simply go with that as opposed to a newer module which may fix "features" in other similar modules.. So I'm not sure what to say aside from I think CPAN could have better sorting and searching capabilities

    MMMMM... Chocolaty Perl Goodness.....
      First, this wouldn't be possible at all, as there is no central all-controling CPAN site. There are many mirrors, and not all mirrors are known. Furthermore, people put CPAN on CDROMs, and distribute them.

      Second, download figures don't mean anything. I may download two modules, ditch one of them because it doesn't even want to build, and just use the second. But I still have downloaded both modules the same amount of times.

      Third, it would actually "favour" buggy, unstable, modules. They (hopefully) get new releases, and hence, new downloads, because people hope the bugs get fixed. Stable modules don't get updated often, so there's less need for an upgrade (and even if there's a new release, there's often no need to bother with an upgrade). So, they would get downloaded less often.


        Good points. About the lack of an "all-controling CPAN site" - do you think this is a good thing? Aside from the increased resource benefits and (possibly) reduced data loss from creating mirrors, are there any other positives you can think of? Do you think it outweighs problems of Mirrors being set up that could distribute (intentionally or not) buggy/trojaned modules? Also, does anyone know what measures are currently in place to prevent this? thanks.

      Great thread hacker. The sentiment of what you're saying certainly rings true with this Perl journeyman. ++

      I agree with what you're saying too l2kashe. However, I think the problem is more general than you suggest; perhaps a more holistic approach is needed for CPAN metadata?

      I would like if the functionality of were more integrated in to the page for each distribution (or at least a link to the relevant page, so I don't have to dig it up), and to make it easier for people to post their sample code and other comments.

      I note that there are a bunch of great ideas in schwern's CPANTS talk.

      One just has to look at how the PHP online manual is done (look at the URL manual page as an example) to see another potential direction that this could be taken in.

      Then you could quickly see what modules are being used more by the community at large, as well as close runner-ups.

      You might want to take a look at CPANSTATS.

        This is EXACTLY the kind of problem I think hacker is describing. How many other people here knew that there was a module called CPANSTATS, and that it did what the previous monk was describing?
Re: Perl Monks += TMTOWTDI
by artist (Parson) on Apr 14, 2003 at 15:34 UTC
    ++hacker for making an excellent point.

    My suggestions:

    1. Associate keywords with the modules.
    2. Voting mechanism for module by perlmonks.
    3. Search CPAN with google like mechanism for keywords.
    4. Apply the sort orders. (Latest, Depend on, VoteRank, RefereceRank)

    Keywords will provide a very good search mechanism. Voting is not absolutely necessary. Sorting would keep latest/useful/obscure module within reach. It will also take care for the definition of better module for a given task as the time passes.

    New Section - "I use this new module for":
    We say that 90% code is written in terms of modules. May be we should establish a formal mechanism to introduce useful modules here. Perlmonk can say that I use such and such modules for this purpose(s) which will require lot less effort than module review section, but would be immediately helpful to us. Practical examples would serve an added value for modules to the documentation, tutorials and reviews.


      I really like the 2) It would be cool if there was a section on Perlmonks were you'd see the listing of modules on CPAN and you could vote and attach notes to them. Whether the notes would be "I use this module for" or "Don't use this module, use that one" would be up to us monks. Maybe that would get more attention than the Reviews.

      I'll go on dreaming ... the module's page would be linked from, it would contain some activity data (date of first version, dates of last N updates, ...), the module nodes would belong to the module authors if they are perlmonks so that they are notified of new notes, the voting on the modules look like the polls do, ...

      Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
         -- Rick Osborne

      Edit by castaway: Closed small tag in signature

      May be we should establish a formal mechanism to introduce useful modules here.

      I guess that is what the reviews section was meant for? I think "review" does not mean that the module has to be new, or extraordinary. A thorough review, with pros/cons, information about related modules, some code samples etc. is always useful, whether you already knew the module or not.

        Current --->: Modules for which reviews are written:
        Of course, Module Reviews are there and helpful. That is a mechanism to fully describe the module, pros and cons in great depth. It's done by author or by the users who understand the module very well enough to explain others. We do have around 92 modules reviews currently.

        New --->:Recently Useful Modules:
        I am sure that people use much more than these 92 modules. I am proposing a section in addition to module reviews, about using the modules. So if you download a module from CPAN and found it useful in your code than you can post that information here as how it was useful to you. You may not have time to review it, but others can pickup your pointer and explore it further for the purposes similar to yours and other. This will be useful to perlmonks who want to keep up with the latest work without too much searching or analyzing.

        May be we can associate keywords with the module-node. It is to see which modules are used via perlmonks. Discussion on the same would provide valuable information for which 90% code (modules) should be used.


Re: Perl Monks += TMTOWTDI
by zby (Vicar) on Apr 14, 2003 at 13:53 UTC
    A common case is when one module is built on another. This can be strictly defined by the use or require constructs. A catalog of CPAN modules categorized by that should be usefull. You could easilly check if there are some new modules built on the module you know (or you have just found by some other means). This should be quite effective way of searching since new modules add to the older ones - so if you know a module you would use for the job there is much chance that a module based on it does much of the work you planned to do yourself.

    Update: Added some argumentation.

      That sounds like the only remotely possible way to automatically classify modules. Other than that, all I can think of are social solutions such as asking people about their favourite modules in a place like this. The first mention I saw of WWW::Mechanize, f.ex, which hacker mentions, was the Perl Advent Calendar for 2002, which is basically Mark Fowler's "favourite modules this Xmas" site.

      Makeshifts last the longest.

      This won't work. In Excel::Template, I am forced to load at runtime the rendering module I'm using, based on a flag passed in to the constructor. Spreadsheet::WriteExcel has a 7MB limit. Spreadsheet::WriteExcel::Big doesn't, but both depends on a number of other modules and (currently) has not been made to work with mod_perl. Your method will miss the fact that Excel::Template is an interface to Spreadsheet::WriteExcel and Spreadsheet::WriteExcel::Big.

      It may be a good start, but would have to be modified by people knowledgable about the guts of various modules (like, say, the current maintainers).

      We are the carpenters and bricklayers of the Information Age.

      Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Re: Perl Monks += TMTOWTDI
by Mr. Muskrat (Canon) on Apr 14, 2003 at 15:41 UTC

    What do other monks do when there is the possibility of a "better path to walk" with code and modules?
    Try out the available paths and see which one has less thorns.

    How do you know which module is "right" for the task? (or wrong for the task)
    Trial and error. You can only learn so much about a module by reading the docs. To really know a module you have to use it.

    How do you search for something by name, which can only be described by concept?
    Once again, trial and error. Concepts are hard to search for because different people express concepts differently.

Re: Perl Monks += TMTOWTDI
by Desdinova (Friar) on Apr 14, 2003 at 20:18 UTC
    Most of the time i come to perlmonks with a question it is usually "whats the best module for X..." The great thing here is that i can a couple of recomendations from people, I don't think you can automaticlly say which is better unles syou know the project and the direction it is going to go in as well as the person writing the code.

    Take for example the ::Simple subset of modules, if I am trying to perfomr a common task with little fuss than the *::Simple version is better, but if I am attempting something odd than I will need the find grain control offered by the base module. When looking at things like the XML::* modules it gets even more specific, with the different models for looking at that data. My point is that "better" is subjective based on what YOU are doing with the code you write.

    As for searching for other modules when searching here i try to look for posts with a mention of the task I am trying to accomplish like "sending email" this will usually bring a few discussions with different modules listed and what situations they fit well in. This has worked well for me so far.

    Another thing I find useful is the See also section in the CPAN docs for the modules. Oftem times the modules writer realize that thier solution might not be the best fit for everyone and point to to other options, I feel this feeling of cooperation between modules writers rather than compition is one of the reason PERL is so useful..
    Anyway thats my 1/50th of dollar on this topic.
Re: Perl Monks += TMTOWTDI
by tjh (Curate) on Apr 14, 2003 at 23:43 UTC
    A voting system would only be marginally useful or indicative of much. It may only be a slight indicator.

    I like the search tool concept with keywords, etc.

    It could be that the most universally useful system would be a Voting system combined with an Amazon-like user reviews, and/or maybe a bbs attached to the module display page.

    As CPAN continues to grow, it becomes increasingly onerous to move through the modules looking for something that I may not know common (or at least community-accepted) terminology for. Perlmonks is a wonderful knowledge base for this kind of meta-data or contextual data, but I've often still missed a module when not knowing a splinter of jargon that others might more easily understand.

    This is a classic content or knowledge base organizational discussion. I wonder where it'll end up? This is surely an appropriate community to have such a discussion in.

    If there is already a site that presents such ratings, data or evaluation discussion about a continuingly growing list of modules, I'd love to know where it's at.


Re: Perl Monks += TMTOWTDI
by Wally Hartshorn (Friar) on Apr 14, 2003 at 19:29 UTC

    Although some kind of voting or rating system would be quite useful, I think it would also be useful to have something similar to Amazon's system where people can recommend another item "in addition to" or "instead of" the current item.

    Wally Hartshorn

    (Plug: Visit JavaJunkies, PerlMonks for Java)

      I can see it now:

      "... People who use this module also use ..."

      Or as the collection of "related modules" grows:

      "... Friends don't let friends use Bone::Easy ..."



      perl -le 'print+unpack("N",pack("B32","00000000000000000000001001001010"))'

Re: Perl Monks += TMTOWTDI
by traveler (Parson) on Apr 14, 2003 at 22:08 UTC
    I think it would be way cool to have a graph with the nodes being the modules and the paths being "refers to in the POD" and "uses/requires". Then one could travel the graph looking for related modules. A "See Also" section in the PODs would also help with the information and in building the graph.

    Just musing, --traveler

Re: Perl Monks += TMTOWTDI
by hossman (Prior) on Apr 15, 2003 at 05:56 UTC

    I find that the best way to be aware of new modules is to subscribe to the CPAN Updates mailing list, or to check the various places where the lists are posted.

    Just being subscribed to the daily mail list for a while now, and skimming it on a regular basis has kept me informed of all sorts of new modules, and given me a good feel for which modules are continuesly being updated on a regular basis.

Re: Perl Monks += TMTOWTDI
by toma (Vicar) on Apr 16, 2003 at 05:57 UTC
    This is how I think about it: it is a matter of taste, much like choosing music. You make personal decisions based on your own weighting factors. Here is an example of a set of choices that someone might use.

    How do you choose your music?

    • Other listeners
    • Reviews
    • Artist
      • Brian Setzer can do no wrong
    • Publisher
      • Barking Pumpkin can do no wrong
    • What is on the radio
      • XFM 104.9
    • What you found all by yourself
      • Go to Backdoor Records in Cotati, California, find interesting looking discs, put them in the player and listen.
    How do you choose your perl modules?
    • Someone tells you about them
    • Reviews
      • Magazine articles by Randall Schwartz
      • PerlMonks module reviews
    • Author
      • Damian Conway can do no wrong
    • Publisher
      • O'Reilly can do no wrong (module in book)
    • What is on PerlMonks
      • PerlMonks questions, answers, and commentary
    • Browse CPAN
      • Downloads, test and evaluate the modules yourself
      • Search results order is a good first guess
    This is from a talk that I recently presented about perl.

    It should work perfectly the first time! - toma

      Damian is fallible like every else. O'Reilly has released several very bad books. Randal has made mistakes in articles. Search results don't tell much, especially for more complex queries. Perlmonks reviews are hardly ever in-depth, and tend to be regurgitating the API docs.

      Trust nothing. Analyze the code for yourself.

        The question is: they certainly make mistakes, too, but are you less fallible than TheDamian, Abigail or merlyn?

        Makeshifts last the longest.

        The music that you are willing to give a listen might not follow a strict set of rules. Code is analogous to music. My idea is that you might have a favorite artist or a favorite art critic, and this is analogous to having a favorite module author or module reviewer.

        You might invest a few dollars to buy the newest release from your favorite musician without first listening to it. This is analogous to spending a few hours evaluating a module from your favorite CPAN author without knowing why you need the module or if it is any good.

        It should work perfectly the first time! - toma

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://250279]
Approved by broquaint
Front-paged by nothingmuch
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (11)
As of 2017-01-23 15:17 GMT
Find Nodes?
    Voting Booth?
    Do you watch meteor showers?

    Results (192 votes). Check out past polls.