Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Strange CPANTS report

by rvosa (Curate)
on Oct 03, 2005 at 08:45 UTC ( #496839=perlquestion: print w/replies, xml ) Need Help??

rvosa has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,

I received an odd cpants report: FAIL Bio-Phylo-0.07 i686-linux-thread-multi-64int-ld 2.4.30.

An earlier release passed on the same machine, so I'm trying to track down what's been going on.

It's a bit problematic though, because the error messages seem to be in Chinese, if FireFox is not fooling me (the email address is in Taiwan, I think). Unfortunately I can't read Chinese.

It looks like the errors occur in a part of the tests that deals with parsing files. I included some text files in the t/* directory to test if parsing works. Perhaps CPAN testing on this machine now runs in a process that can't access these files? How might I be able to figure out that's what it is?

The distribution is Bio::Phylo, so if anyone has any hints what's wrong with it I'd be grateful.

Thanks!

Replies are listed 'Best First'.
Re: Strange CPANTS report
by LTjake (Prior) on Oct 03, 2005 at 14:13 UTC

    First, a minor nit, but "CPANTS" ne "CPAN Testers" (i have no idea why they chose to include "testing" in the former other than the fact that CPANTS sort of rolls off the tongue, doesn't it? :).

    As for the test result, sometimes they can be misleading. CPANPLUS might be having troubles with dependencies, or something in the tester's perl setup could be messed up. Hard to say! Your best bet might be to contact the author and ask them to run it manually.

    FWIW, i downloaded your distro and ran the test suite -- everything passed (Win32, AS perl 5.8.4)

    --
    "Go up to the next female stranger you see and tell her that her "body is a wonderland."
    My hypothesis is that she’ll be too busy laughing at you to even bother slapping you.
    " (src)

      Ah, sorry about the cpants / cpan testers mixup. I emailed the address in the failure message header. Fingers crossed.

      Thanks for trying out the test suite! It's got a tiny niche so I don't get a lot of tests. Every failed test hurts, of course. :)
Re: Strange CPANTS report
by jkeenan1 (Deacon) on Oct 04, 2005 at 00:51 UTC
    First, as another monk has noted, you have confused the perl.cpan.testers newsgroup/mailing list with CPANTS. The latter is an attempt (and a very fallible one, IMHO) to measure certain characteristics of distributions uploaded to CPAN. The former gives you important feedback on how your distribution tests in different operating environments -- environments being defined as a combination of the OS and the perl version being used for testing.

    Second, I recently spent several weeks fine-tuning a new version of the ExtUtils::ModuleMaker distribution. I knew the code was valid, but I was trying to be much more rigorous in my testing than in my earlier CPAN distributions and, as a result, I had considerably greater problems getting the tests right. When I got FAILs from the cpan.testers, it forced me to think very carefully with respect to how my tests were interacting with the tester's environment.

    Third, of all the CPAN testers who posted my reports on ExtUtils::ModuleMaker (and its new companion, ExtUtils::ModuleMaker:PBP), the most helpful of all of them was imacat -- the tester in Taiwan who sent you a fail. Why was she the most helpful? (A) In the immediate sense, because, upon my request, she sent me the output of 'prove -vb t/mytest.t' several times. (B) Because her testing environment (at least on her linux box) is more pristine than that of other cpan.testers, so it gives you a more accurate reading as to whether you have loaded all the prerequisites. Example: On my own boxes and those of many testers, Module::Build is installed even though its not (yet) core. Only a testing box without Module::Build will tell you when your code fails a 'require Module::Build' test.

    All of which is to say: Treat a FAIL from imacat seriously, even if that test is receiving PASSes from other testers.

    Fourth, while I would recommend asking a cpan.tester for the output of 'prove -vb t/myscript.t' for any failing test, it's not going to be very helpful in your case, because the messages (or 'names' or 'labels' or whatever they're calling them this week) that you have included with your tests are overly terse and not self-documenting (IMO). For example, opening up your first failing test file, t/02-newick.t, I see that the third test is:

    ok( Bio::Phylo::IO->parse( -file => 'tree.dnd', -format => 'newick' ), + '3 parse' );

    A message such as 3 parse is probably meaningful to you, the module's author. But it gives me no clue as to what was supposed to happen with this test. What is supposed to be the result of the parse call? Creation of a file? Verification of a new file's existence? Verification that a new file contains particular content?

    Similarly, in the next file in which you are encountering problems, t/14-nexus.t, your test labels simply read test 1, test 2 and so on. No diagnostic help there.

    Okay, enough ranting. Let me advance a hypothesis as to what's going wrong. Let's take t/02-newick.t as an example. In this test file, you print to a file called tree.dnd, which, since no path information is provided, is presumably created in the same directory from which the tests are being run -- most likely Bio-Phylo-0.07. More to the point, you are not creating this file in a temporary directory such as would be provided by File::Temp and would be disposed of once the test file has finished.

    I note also that while your version 0.04 of Bio::Phylo passed on this box, your version 0.05 did not. In fact, it failed at the same point as 0.07. Based on my own experience with testing on that box, I would explore the possibility that your FAIL on 0.05 left a file on that box which is continuing to "pollute" your testing environment.

    Granted, you don't have access to the tester's box. But if, after running the tests on your own box, you notice that files created by the testing process are lying around, then you may be leaving such files on the tester's box as well, and that could be the source of the FAILs.

    Whether or not that hypothesis is confirmed, I recommend more descriptive test messages and use of File::Temp to create directories which hold files created during the testing process.

    Jim Keenan

    Update an hour later:

    I'm less confident than earlier about my hypothesis. Presumably, any files created during the process of testing Bio-Phylo-0.04 are distinct from those created while testing 0.05 or 0.07.

    So let me suggest another approach. Since 0.04 passed on imacat's box while 0.05 and 0.07 failed, you should look at the differences between 0.04 and 0.05, taking care to distinguish between changes in the modules and changes in the test suite. A quick application of the CPAN grep tool (http://search.cpan.org/diff?from=Bio-Phylo-0.04&to=Bio-Phylo-0.05) suggests that you did some heavy surgery on your modules between those two versions. In particular, lib/Bio/Phylo/IO.pm, the package that exports the problematic parse function, was not present in 0.04 but does appear in 0.05. Perhaps that's where you should focus.

    jimk

      imacat has replied that the garbled error messages were:
      "Inappropriate ioctl for device"
      I'm pretty sure Jim is right about the dodgy way test input files are read and written, so I'll look into that, but can someone tell me how and why "Inappropriate ioctl for device" might crop up during a test? I'm normally on Win32, so I need to do some reading op on ioctl.
        Googling for that phrase I got a lot of entries on many different systems, few or none of them Perl-related. So I think it's just a message that prints out as part of the FAIL and doesn't say anything in particular about your problem.

        And this is supported by the fact that the phrase is present in all my FAIL reports from imacat's Linux box. Example: http://www.nntp.perl.org/group/perl.cpan.testers/254139. In my case, this message had nothing to do with the problems in my test suite; those lay in the test suite itself. My hunch is that it's the same for you.

        jimk

      Wow, good points. Thanks!
Re: Strange CPANTS report
by ErichTheRead (Initiate) on Oct 03, 2005 at 22:54 UTC
    It's not Chinese. It's garbage. It just garbage that accidently has a few characters that are the same values as Chinese characters. We can tell it's not Chinese because most of the characters don't map to ANYTHING.

    It isn't unusual that garbage would look like Chinese. Think about it: How many latin characters are there? Now how many Chinese characters are there? Tens of thousands. So garbage will accidently map to Chinese much more frequently than to something else.

Re: Strange CPANTS report
by jkeenan1 (Deacon) on Oct 04, 2005 at 14:47 UTC
    Bio::Phylo::IO::parse() is the subroutine that is FAILing. There's a lot going on in this subroutine. For example, I counted 8 different error messages that could be thrown in the course of calling the function:

    Your documentation says,

    The parse method makes assumptions about the capabilities of Bio::Phylo::Parsers::* modules .... Exceptions are thrown if either assumption is violated.

    If so, then the approach you should consider in testing parse() is to relax each of those eight assumptions in turn, allow the function to die inside an eval block, capture the error message and compare it with Test::More::like() to the error message you expect to receive.

    Your documentation further states that parse() returns an object of the appropriate Bio::Phylo::* class. If so, then the final test called on the function should probably be done with Test::More::isa_ok() rather than merely ok().

    But before writing all these tests, I'd recommend thinking about simplifying the structure of Bio::Phylo::IO::parse(). If it were simpler, it would be easier to test and it would be easier for you to isolate the cause of the failure on the cpan.tester's box.

    Jim Keenan

      Yes, you're right, that method could definitely use some simplification. It sort of grew organically, so now it's time for some pruning. I'm not sure if it's necessarily the cause of the errors on imacat's machine, but some refactoring will certainly aid in tracking down whatever is causing build fails in the next release.

      Incidentally, thanks a lot for taking this so seriously, digging into the code, reading the docs - I really appreciate it.
Re: Strange CPANTS report
by imacat (Initiate) on Oct 09, 2005 at 20:19 UTC

    Hi. This is imacat.

    First of all, it is not garbage. It is definitely Chinese. The first error message reads "No such file or directory" (mei2 you3 ci3 yi1 dang3 an4 huo4 mu4 lu4), and the second reads "Inappropriate ioctl for device" (bu4 xi1 wang4 de5 zhuang1 zhi4 shu1 chu1 ru4 kong4 zhi4). I don't know why ErichTheRead thinks it's garbage. Maybe you are using Simplified Chinese encoding system (GB2312)? We people in Taiwan are using Traditiona Chinese encoding system (Big5), and, as a Chinese computer encoding system Big5 is a lot ager than all the rest.

    And for the rest, I think jkeenan1 had stated very clear on many points. So I won't repeat them. If you still need help on this, feel free to ask me.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://496839]
Approved by greenFox
Front-paged by Tanktalus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (7)
As of 2020-01-24 08:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?