Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Help module authors by testing them on older versions of perl

by mirod (Canon)
on Aug 31, 2005 at 16:43 UTC ( #488147=perlmeditation: print w/ replies, xml ) Need Help??

I mean testing the modules, not the authors ;--)

You might or might not know it, but modules released on CPAN are tested, and the results are reported on CPAN testers. The tests results are linked from the module page on search.cpan.org. Most of the tests are done automatically by a great group ofvolunteers.

There is one problem though: those volunteers are usually very up-to-date in their configurations, and understandably so. So you get a lot of results for various architectures and OSs, but mostly for recent versions of perl. To take a (completely random!) example, on August 31rst 2005 the XML::Twig result page shows 26 tests, 24 of those being on perl 5.8.4+.

I posted myself one of the 2 results for 5.6 BTW.

This leaves some big holes in the testing of the module, as for example Unicode processing changed quite a bit between 5.5, 5.6, 5.8.0 and 5.8.1+.

So if you are using modules, here is a simple thing you can do to help the authors: get Test::Reporter, install it, and run cpantest for the modules you have installed that pass the tests.

If you wonder what is the format of the package option, it is the "normalized" name of the distribution: Test-Reporter-1.27, XML-Twig-3.21... so cpantest -g pass -auto -p Test-Reporter-1.27 will report that indeed Test::Reporter passed the tests on your machine (it is even more important to report failures, but I assume that in most cases the author ends up being informed of those, through CPAN testers, rt.cpan.org or direct email).

This will help other users with the same version and architecture, as they can then check that the module runs for at least one person. It will also help the author of the module: when a failure is reported but there is a previous pass on the architecture, it eliminates a whole class of errors.

An other option is to use cpanplus (I don't), and to activate the test reporting option.

BTW, I test modules on quite a few versions of perl before a release. So I would have posted more results, if cpantestet had the option to report tests for other versions of perl than the default one. Saddly this is not available (yet?), and I don't like editing the emails it generates by hand, I don't trust my typing skills.

Comment on Help module authors by testing them on older versions of perl
Re: Help module authors by testing them on older versions of perl
by Anonymous Monk on Aug 31, 2005 at 17:36 UTC
    <flame-on>

    Does it really help module authors though? I can see people having a legacy production system where the code works fine, so there's a risk in upgrading perl versions. But, if you're starting development using new perl modules, shouldn't you also be able to upgrade your perl version? (looks like 5.6.1 came out in April of 2001). Wouldn't a module author have better things to do than tracking down bugs in obselete versions of perl? (adding new features, fixing bugs for a recent version of perl, etc.) And even if it was easy to track down a bug in previous versions, do you really want to clutter up the module code with crufty one-off patches? Shouldn't we (as a community) be encouraging people to use the latest-greatest version of perl (for the bug-fixes as well as the new features)?

      Its absolutely useful. Just recently, I found that I have to work with 5.005 and 5.6.0 when I didn't have to previously. Similarly, it was on a platform I don't normally use. Just because my linux box at home uses the latest version doesn't mean there's plenty of potential value in being backwards compatible.

      Getting accurate reports from old versions is essential in that task.

        Just recently, I found that I have to work with 5.005 and 5.6.0 when I didn't have to previously.
        I guess I'm suffering from a failure of imagination, but I'd be interested in knowing the reason that you have to work with such ancient versions. Most of the reasons I can come up with are compeletely frivolous. Like: "we're running low on disk space", "two installed versions of perl is complicated", "my ISP is lazy and won't upgrade", "our machine only has 8MB of memory and can't handle newer versions", etc.
      Howdy!

      Some of us are stuck with a Perl environment we can't upgrade (5.6.0, in my case). It's not even so much of a "risk of upgrading" as a "have no control and no influence" problem.

      I've just been trying to install the modules Perl Best Practices recommends, and finding some that won't install properly. Some of these are called out in existing test reports, but others aren't.

      yours,
      Michael
      Turn that flame off. Forget about the authors for a second and help CPAN. CPAN needs more testers. CPAN needs more testers with different platforms. CPAN needs more testers with different perls. So what if becoming a cpan tester and testing with older perls may or may not help some AUTHOR, it always helps somebody, and always helps CPAN.

      Keep CPAN healthy, become a tester today!

      MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
      I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
      ** The third rule of perl club is a statement of fact: pod is sexy.

      even if it was easy to track down a bug in previous versions, do you really want to clutter up the module code with crufty one-off patches

      Depends what those patches are.

      Testing on an older version on Perl might uncover simple things that are easy to change. With a minimum of effort, you can write code that works just as well on 5.6 as it does with 5.8.

      Case in point: I am having trouble building Compress::Bzip2 for perl 5.6.1 on an ageing Solaris box. I tried to upgrade it using cpanplus, it failed, and sent a failure report off. Now I'm in the middle of a discussion with the author and we are fixing it.

      What it looks like at the moment is that the only reason it fails to build is that the author used the 5.8-ish constant hash syntax:

      use constant { waste => 2, godolphin => 3, mantissa => 5, sferics => 7, };

      When the above is changed to:

      use constant waste => 2; use constant godolphin => 3; use constant mantissa => 5; use constant sferics => 7;

      ...the module works on 5.6 (there are still a couple of other issues to sort out, but this looks like the biggie). Depending on the changes required, the author may, or may not, choose to support 5.6. But at least she has the knowledge available in order to make such a decision.

      I know that I write code differently depending on whether it's for internal consumption or for CPAN. In the latter case I'll use more archaic forms that I would otherwise. The feedback from testing lets you know when you've stumbled across something that may be a more or less gratuitous bug that stops it from running on prior versions.

      And if you really don't give a damn, you can always add a require 5.8; at the top of your module and then it is clear to everyone what the minimum requirements are to use it.

      - another intruder with the mooring in the heart of the Perl

Re: Help module authors by testing them on older versions of perl
by BrowserUk (Pope) on Aug 31, 2005 at 18:56 UTC

    This is crazy. If people "have their reasons" for sticking with old versions of Perl, that's their choice, but presumably they have the modules they need for those reasons already working.

    If they want to upgrade those systems to use new modules, they should upgrade their Perl, or at least take the responsibility for testing those modules on their version of Perl.

    Their expecting module authors, or others, to test every module on every old version of Perl just so they can have the luxury of avoiding the pain of upgrading is a complete nonsense.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

      Did you notice that I focused primarily on reporting modules that pass the tests? It gives me a warm and fuzzy feeling to know that a module I wrote still runs on 5.005.

      But even when the tests fail, I'd rather know it. Then I can figure out why, and fix it if I can, or if I can't be bothered just upgrade the required version of perl. Or point people to CPAN testers and tell them that they should not expect the module to run in their environment.

      An other aspect is that in XML::Twig's case, the module doesn't change that much, but it the last year test coverage has improved quite a bit. And I found the hard way that writing tests that are portable accross platforms and versions of Perl is every bit as difficult as writing a portable module. Actually what I found that it was much harder to write portable tests. So if the module code doesn't change but new tests fail, I think it's only fair for me to fix the tests.

      On a philosophicallevel, in any case, this is Open Source, the more test results we get, the more information users get about the software, the better it is. Users know what they are getting, and developers get feedback and can do what they want with it, from screaming "Upgrade or Go To Hell" to "ok, fixed, new version on its way to CPAN". And asking (politely, no one is forced to do anything here) users to share their test results, in exchange for a module doesn't seem to bad of a deal.

      I agree. I like lexical filehandles so much that I don't care if my code doesn't work before Perl 5.6.0 (released five and almost a half years ago, with nine stable releases of Perl since then).

      I don't break backwards compatibility gratiutously, but I certainly don't fix bugs, write comprehensive test suites, and add new features so that someday in the future I might possibly be able to take advantage of new code.

      >If people "have their reasons" for sticking with old versions of Perl, that's their choice

      Or the choice of their employer, or the choice of their ISP. or the choice of their client.

        Whomever makes the choice is the "people" referred to.

        If it is the employer, then the employee discovering that the module they wish to use doesn't work, or isn't certified to work on the level the employer has chosen, must bring that to the employer's attention. They must make the case for how the use of that module would benefit the project and that must be balanced against the cost of upgrading to a Perl that supports it. This is bread & butter legacy project management.

        If the problem is the ISP, then the user of that ISPs service must make his calculations on the cost of moving to a different ISP, against the benefits he might accrue from using the module.

        If it is the client's choice, the programmer must make the case for the benefits of using the module. The client must balance those against the costs of upgrading.

        I'm not suggesting "upgrade or die". Just that anybody making a deliberate choice to stick with an older version of Perl is doing so knowingly--most likely on economic grounds. Upgrading costs money.

        But putting the onus on those already giving their code away for free, to perform time consuming and laborious extra testing of their modules in down-level environments, so that the users can avoid the costs of upgrading is a nonsense.

        For one thing, if the users are sufficiently aware to have made the choice to remain back-level, they would also need to thoroughly test any new modules before adding it to their existing setups. The risks of a new module breaking existing systems is just to high to take someone else's word for it.

        In reality, the most usual reason for people failing to upgrade is laziness. They have a working system and they cannot be bothered to test the existing code in a new environment. And the more modules pander to their desires by ensuring backward compatibility, the less likely they are to bite the bullet and test.

        In effect, the module authors that provide this service are making a rod for their own back by doing so.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
      I disagree: knowing (as in "the fact being documented") that a module won't work with some older version of perl, is not an unnecessary luxury.

        So, if you write a mdoule tomorrow and place it on CPAN, how many versions of Perl are you going to install and test it on?

        5.8.0?, 5.6.1?, 5.005? 4.x,? 3.x?

        How about also documenting whether it will run on 64-bit Windows or Cray, or Atari or Psion? Aren't these equally important?


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
      Howdy!

      Their expecting module authors, or others, to test every module on every old version of Perl just so they can have the luxury of avoiding the pain of upgrading is a complete nonsense.

      I think that grossly overstates and distorts mirod's request.

      If people are in a position to test modules against older versions of Perl (as well as on a variety of platforms), and they are willing to see those results posted to CPAN, it is a Good Thing.

      Jumping mirod's case like that seems petty and small.

      yours,
      Michael
      Their expecting module authors, or others, to test every module on every old version of Perl just so they can have the luxury of avoiding the pain of upgrading is a complete nonsense.

      Tosh.

      The OP wasn't requesting module authors to test on old perls. The OP wasn't asking module authors to patch their modules to work on old perls.

      The OP was asking for more people who are using modules to contribute to cpan-testers. As the OP says:

      This will help other users with the same version and architecture, as they can then check that the module runs for at least one person. It will also help the author of the module: when a failure is reported but there is a previous pass on the architecture, it eliminates a whole class of errors.

      As somebody who has to sometimes deal with people using old Perl installs I appreciate having accurate test results on these platforms.

      As somebody who writes modules I appreciate having information on what platforms they work on.

      I'm unlikely to write patches for old Perl's (unless I actually need it to run), but who cares. I can add an appropriate minimal perl to my Build.PL and leave it at that.

      More public test results makes life better for everybody.

      • Vendor A supports no later than 5.6.1 in their API module
      • Module author B has only tested on 5.8 and foward on their API

      Write code to work as glue between the two. Do you write it as two different applications and have them do some sort of IPC, or do you test Vendor B's code to see if the test suite passes under 5.6.1?

      --MidLifeXis

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://488147]
Front-paged by TStanley
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (10)
As of 2014-09-19 19:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (144 votes), past polls