Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Challenge: CPAN Golf

by shmem (Canon)
on Jan 08, 2008 at 12:43 UTC ( #661071=perlmeditation: print w/ replies, xml ) Need Help??

<rant>

Sometimes I just want to try out a module; sometimes I just want to help somebody facing trouble using it. So I fire up cpan and install it, automatic dependency installs turned on. WHOA! follow, follow, follow installing dependencies, and after an hour or so and several fails resolved, I have the top level module ready to use.

Four dozens of intertwined dependency modules installed, another 10 megs of disk space gone. After all, disk space is cheap, no? And those modules are really needed by the top level one, aren't they? (But then there are all those Test::Tester, Test::Tester::Bestertester, Test::Tester::Bestertester::Helper modules and their dependencies as well.)

Then I install the otherwise fine CGI::Ajax module and note it uses Class::Accessor to generate setters/getters for a list of seven object attributes, trading 29 lines of trivial hand-written code against 874 lines or 48 kBytes. That can easily be shrugged off, because other fine modules use Class::Accessor as well - and 900 lines of code don't really count. But it sums up.

This kind of laziness places the burden of installing and updating dependencies, which are not crucial for the functionality of the module in question, on the module user.

And on the long run, we might end up with CPAN modules being blobs that pull in a multitude of modules of which only a minuscle part will ever be used at runtime.

IMHO, hubris beats laziness for module authors, and modules should not depend on other modules if the latter only bring small benefits.

</rant>

So here is the challenge (draft :-)

  • download and install the most part / all of CPAN with the least cpan runs possible
  • install 50 modules along with their dependencies; most diskspace allocated wins

Rules:

  • all modules in a dependency chain have to exercise code of each other, just useing it is not enough (the import() sub doesn't count)
  • package managers are not allowed

So you can't just make up a module that uses all CPAN modules and upload that to CPAN; likewise, building a meta-package (e.g. perl-CPAN-all.deb) and installing that doesn't count.

Have fun!

--shmem

_($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                              /\_¯/(q    /
----------------------------  \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

Comment on Challenge: CPAN Golf
Re: Challenge: CPAN Golf
by Corion (Pope) on Jan 08, 2008 at 12:52 UTC

    I think some of the part of your rant is adressed by the Tiny subclasses that adamk writes.

    But I have a question - does exercise code of each other also include code generated by the used module?

    For example, Class::Accessor injects code into my module/namespace - does that count as "exercise", at least if the accessor is used within my code?

    As counter example, parent.pm does all its work in the ->import() subroutine and hence will not count as "exercised", right?

      For example, Class::Accessor injects code into my module/namespace - does that count as "exercise", at least if the accessor is used within my code?

      Of course yes - but as you say, only if your code actually uses those accessors.

      As counter example, parent.pm does all its work in the ->import() subroutine and hence will not count as "exercised", right?
      This is an edge case; parent.pm works like a pragma (at compile time), so I would count that exercised, too. If you just use parent qw(Catalyst Jifty) without milling any code of those, then you are cheating not only with Catalyst and Jifty, but with parent.pm also.

      --shmem

      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
Re: Challenge: CPAN Golf
by dragonchild (Archbishop) on Jan 08, 2008 at 14:17 UTC
    First off, yes. :-) I feel your pain, particularly when I have to start installing something in Crypt::.

    But, I think the problem is overstated and not just by you. Let's take Class::Accessor. I just updated to the latest (0.31) and here's what I find:

    • 675 total lines.
    • Total space taken - 17k.
    • After removing POD, 196 total lines of real code (reduced to 4K).
    • ~140 lines really needed if we applied a couple space reducers.
    And, that code goes ahead and provides an extensible way of safely building classes. Personally, I'm ok with that, particularly if I use it across multiple projects. Frankly, writing an OO system used to be almost de rigeur for becoming a solid Perl developer. Pointing people at existing projects and having them do something else to cut their teeth on is a great thing.

    Now, I think what you (and everyone else) are complaining about boils down to a lack of standards. There are so many different versions of the wheel that when you install a non-trivial amount of CPAN, you end up with 17 different OO systems. And, on its face, this is something that sucks. But, I think it's a necessary price to pay to have the necessary pot au feu that Perl provides. What other language has the same kind of joyous abandon leading to the amazing things one can find on CPAN? As I've said before, 90% of every Perl application is already written. You just have to find it. I'm willing to pay with a little bit of pain in dependencies.


    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: Challenge: CPAN Golf
by stvn (Monsignor) on Jan 08, 2008 at 14:39 UTC

    To start with, I don't understand your rant. What is wrong with code-reuse? Is CPAN's automated dependency resolution a problem? Sure, long dependency chains and resolving failed installs can be annoying, but an hour or so of work on your part and you now have, at your fingertips, what probably amounts to hundreds of hours of work by others.

    But then there are all those Test::Tester, Test::Tester::Bestertester, Test::Tester::Bestertester::Helper modules and their dependencies as well.

    Testing code is tedious, writing complex code to test your code is even more tedious (and error prone). One of the great strengths of the Perl community is its dedication to testing as evidenced by the many high-quality Test:: modules available on CPAN. I don't know about you, but I don't want to have to re-invent the wheel that modules like Test::JSON and Test::LongString have already invented.

    Then I install the otherwise fine CGI::Ajax module and note it uses Class::Accessor to generate setters/getters for a list of seven object attributes, trading 29 lines of trivial hand-written code against 874 lines or 48 kBytes.

    Well you say "29 lines of trivial hand-written code", I would like to change that slightly to "29 lines of tedious, repetative (and therefore possibly error prone) hand-written code". Modules like Class::Accessor (which is much less then 874 lines of code, not sure where you got that number) help increase the quality of modules that use them.

    Just stop and think for a moment about this.

    If I write my own accessor code, then I should test that code. So for 9 object attributes, I write 9 accessors, each which should be tested in a couple of ways. First I need to test that they store the values properly, then that they return stored values, if I want to be diligent I should test edge cases as well like passing in undef and such. Sure you could say "but those are so trivial, why even bother to test them", and actually I would agree with you. If you use Class::Accessor you don't have to test them, because Class::Accessor has already tested it's accessor generation code and you can be assured it works as expected.

    In short, building your module from other well tested modules automatically ups the quality of your module. Standing on the shoulders, etc etc blah blah blah.

    This kind of laziness places the burden of installing and updating dependencies, which are not crucial for the functionality of the module in question, on the module user.

    Okay, this was the line that really kinda bothered me the most. I will point to what I said in the first paragraph in regard to installing dependencies. You are getting software which took someone else many hours to write FOR FREE. I think it is worth a few moments of your time to install it.

    As to the "not crucial to the functionality of the module" argument, this is ridiculous. Why should the module author have to write, debug, test and maintain extra code so that you don't have to sit though a longer automated install? Trading their time for yours, how kind of you.

    IMHO, hubris beats laziness for module authors, and modules should not depend on other modules if the latter only bring small benefits.

    Well then I look forward to seeing all your "dependency free" modules. Please let us all know how much fun you have re-inventing, debugging, testing and maintaining innumerable small wheels.

    -stvn
      Thank you for this good retort. If I have come across as a ungrateful fuck, I'm really sorry about it. Maybe I should have written <rant mode="not serious">.

      Of course the time and work I have to spend to install modules stands in no relation to the time spent and effort put into writing, testing and maintaining just one of the modules I install along with the top-level target, and I am not wanting to be ungrateful; giving back is maybe the strongest reason for hanging out here on PerlMonks. I'd write useful modules myself if I were more brillant and faced problems for which there isn't a module yet on CPAN; or if I could significantly improve existing solutions.

      That said, I still hold up the statement

      IMHO, hubris beats laziness for module authors, and modules should not depend on other modules if the latter only bring small benefits.

      because there are problems with module dependencies; dragonchild has named one - lack of standards - and pointed out the benefit that drawback brings. Off my head (there might be more)

      • "you don't have to test them" - part of a sane module install (and of automatic install) is testing, so that pulls in the entire module-author's testing infrastructure as applied on the module
      • a chain is only as strong as its weakest part; cpandeps provides a nice view of how and where automatic install might fail
      • longer inclusion chains have a hit on performance

      Of course, each of those arguments can be wiped away with "Trading their time for yours, how kind of you" without being really countered. For the CGI::Ajax / Class::Accessor example see Re^3: AJAX popup windows - an example.

      Well then I look forward to seeing all your "dependency free" modules. Please let us all know how much fun you have re-inventing, debugging, testing and maintaining innumerable small wheels.

      You did note the "small benefit" part? I am not arguing for dependency-free modules, but for as little dependencies as possible, which of course is no absolute, but something worth thinking of; and "as possible" has many constraints.

      --shmem

      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
        Of course the time and work I have to spend to install modules stands in no relation to the time spent and effort put into writing, testing and maintaining just one of the modules I install along with the top-level target, and I am not wanting to be ungrateful; giving back is maybe the strongest reason for hanging out here on PerlMonks. I'd write useful modules myself if I were more brillant and faced problems for which there isn't a module yet on CPAN; or if I could significantly improve existing solutions.

        If you want something to work on, I'd love to give you either PDF::Template and/or Excel::Template. Or, I could use help working on DBM::Deep on Win32. Or, any of the other 20 really cool module ideas I have rattling around that I simply don't have time to work on. Like Presto. Or PerlGems.

        You did note the "small benefit" part? I am not arguing for dependency-free modules, but for as little dependencies as possible, which of course is no absolute, but something worth thinking of; and "as possible" has many constraints.

        So, I go ahead and put in a dep on ABC because it saves me half the project. It has a dep on DEF cause that saves it half the project. DEF's author is stupid and has 30 deps. Should I avoid ABC because of that?


        My criteria for good software:
        1. Does it work?
        2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
        because there are problems with module dependencies; dragonchild has named one - lack of standards - and pointed out the benefit that drawback brings.

        Yes, I agree with dragonchild on that, however TIMTOWTDI is core to the Perl Way of Life and standards are counter to that. Of course TIMTOWTDI is a double edged sword here especially.

        You did note the "small benefit" part? I am not arguing for dependency-free modules, but for as little dependencies as possible, which of course is no absolute, but something worth thinking of; and "as possible" has many constraints.

        I will say, and I suspect you will agree, that choosing to add dependencies is something that should not be taken lightly. For instance, if the module looks abandoned I will sometimes opt to write it myself because I don't want to depend on something which is unmaintained and might break and mess up my module with it. But a good solid module (like Class::Accessor) can be depended and it used widely enough that I would not even think twice about using it, knowing full well that many people will probably already have it installed.

        One of the really great things about Perl is CPAN, and one of the really great things about CPAN (IMO at least) is how much interdependency there is in it. If you take a look at the state of Python, Ruby or even Java modules you will find many more monolithic codebases (although ruby and python are getting better now last I checked). I would submit that this is directly related to the level of tool support to automatically resolve dependencies such as CPAN has. Anyway, just my 2 cents :)

        -stvn
Re: Challenge: CPAN Golf
by rhesa (Vicar) on Jan 08, 2008 at 16:38 UTC
    Just taking a guess at challenge 1 (I have no intention to install anything just for fun ;-):
    cpan Jifty Plagger Catalyst
    Also, does that count as one run or three? :-p

    And challenge 2 is just no fun, thanks to merlyn's column The Big Modules in the Mini-CPAN.

      Last I remember, Angerwhale held the record for "Most of CPAN installed"

      -stvn
        Good example of software that shouldn't be on CPAN. It's a "blogging software that reads posts from the filesystem" etc and definitely not a module despite its disguise as such ( = it meets the formal criteria for perl modules).

        --shmem

        _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                      /\_¯/(q    /
        ----------------------------  \__(m.====·.(_("always off the crowd"))."·
        ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
Re: Challenge: CPAN Golf
by sundialsvc4 (Monsignor) on Jan 08, 2008 at 16:59 UTC

    I, too, generally agree that some CPAN module writers use other CPAN features “gratuitously,” and that this does cause an increased headache for installers. CPAN contributors ought to be duly mindful of this. Try to strike a sensible balance between “self sufficiency” and “leverage.” I don't feel that this is a blanket statement, merely something to take in advisement...

Re: Challenge: CPAN Golf
by xdg (Monsignor) on Jan 08, 2008 at 17:44 UTC
    • download and install the most part / all of CPAN with the least cpan runs possible
    • install 50 modules along with their dependencies; most diskspace allocated wins

    When I want to stress a new perl installation (e.g. 5.10.0) or some new platform, I usually try installing Plagger or Jifty and automate saying yes to all the recommended modules.

    $ PERL_MM_USE_DEFAULT=1 cpan Jifty

    Assuming you can get it all to install, the result will be huge.

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

      That can work sometimes, but it can also put you into the dreaded “dependency hell.” I guess that a good rule of thumb for module-writers might be... if what you want to do is actually substantial and is already-done by a CPAN module, then go for it. But do try to keep the vines pruned:   don't let them just grow all over the place. Let there be a reason for each one.

      A module that is reasonably self-contained, both in purpose and in implementation, will always bring smiles to your “customers.” And they are, in a certain very-real sense, your customers. The module that you are contributing to The Matrix is, in a very real sense, a “commercial product.” Try to make your customers' experience with your product a pleasant one, by anticipating their needs.

Re: Challenge: CPAN Golf
by explorer (Chaplain) on Jan 09, 2008 at 11:20 UTC
Re: Challenge: CPAN Golf
by DrHyde (Prior) on Jan 10, 2008 at 11:23 UTC

    I've written modules that have dependencies. I've also written modules that have no dependencies, such as XML::Tiny.

    Writing a good enough test suite for XML::Tiny without using any Test::* modules was a gigantic pain in the arse. But if I hadn't done it, I wouldn't have found a fair number of bugs in my code.

    So I'm afraid that if you want to use any of my other modules, then in the interests of me providing you for free with well-tested, reasonably bug-free code, then you have to put up with me using all kinds of other modules. In particular for CPAN::FindDependencies I'm going to use the well-tested libwww-perl rather than write my own shonky networking code, and I'm going to use various bits of Test::* rather than write my own is_deeply() and ok() functions yet again. If you don't like this, then you are welcome to submit patches which, if I think they're of acceptable quality, I will apply to the code.

    That's what's so great about open source. If you don't like what I've done, then you can contribute a fix. And if you don't like how I treat your fix, you can fork my code. If you do neither of those, then you can stick your whinging where the sun don't shine.

      Hasn't all you write already been addressed by stvn in Re: Challenge: CPAN Golf and answered in Re^2: Challenge: CPAN Golf?
      If you do neither of those, then you can stick your whinging where the sun don't shine.
      No call for abusive remarks, even if they're somewhat smoothened.

      --shmem

      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://661071]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (11)
As of 2014-07-30 03:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (229 votes), past polls