Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

How do track down underlying reason for smoke test failure

by nysus (Vicar)
on Sep 12, 2019 at 09:22 UTC ( #11106052=perlquestion: print w/replies, xml ) Need Help??

nysus has asked for the wisdom of the Perl Monks concerning the following question:

I submitted a CPAN module which is passing most tests. However, there is one particular tester that is failing my tests with the same failure message.

I took a cursory look at the module versions the failing tests were using. Nothing jumped out at me. I'm not sure what the best way is to begin isolating this problem. Do I set up a perl distro with the same exact module config as the tester or is there a better way to try to track down the problem?

$PM = "Perl Monk's";
$MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
$nysus = $PM . ' ' . $MCF;
Click here if you love Perl Monks

  • Comment on How do track down underlying reason for smoke test failure

Replies are listed 'Best First'.
Re: How do track down underlying reason for smoke test failure
by marto (Archbishop) on Sep 12, 2019 at 09:38 UTC

    "Do I set up a perl distro with the same exact module config..."

    Since your module is on github you can add Continuous_integration via Travis_CI with very little effort, and no financial cost, have it test your code against several versions of perl, on different OS etc. Create your .travis.yml (or see Travis-CI Helpers for Perl), setup Travis CI, activate the repos and it'll do it automatically from then on.

Re: How do track down underlying reason for smoke test failure
by bliako (Vicar) on Sep 12, 2019 at 10:35 UTC

    I have managed to reproduce the problem (Perl 5.26.2, fedora linux) when I omitted ./ from INC dir - as Test::Classifier requires t::TestMods::Test::Processor :

    prove -Iblib/lib -It/TestMods t/01-file-collector.t
    t/01-file-collector.t .. # Running my tests t/01-file-collector.t .. 1/13 # Failed test 'creates an object' # at t/01-file-collector.t line 30. # died: Test::Classifier is not a Role::Tiny at /home/andreas/Download +s/File-Collector-0.037/blib/lib/File/Collector.pm line 391. # Failed test 'has files property' # at t/01-file-collector.t line 34. # Failed test 'has common_dir property' # at t/01-file-collector.t line 38. Can't call method "get_count" on unblessed reference at t/01-file-coll +ector.t line 41. # Looks like your test exited with 2 just after 4.

    This works fine on my system:

    prove -Iblib/lib -It/TestMods -I. t/01-file-collector.t

    Could it be that some of the modules in t/TestMods do not load correctly under certain circumstances?


    A comment on your code if I may (File-Collector-0.037 / lib / File / Collector.pm):

    # eval class code foreach my $class ( @$classes ) { eval "require $class"; }

    Adding a check on eval will tell you if requireed class was not found or failed to load. e.g. from eval, e.g. eval "xyz"; die $@ if $@;

    I make this comment because the test fails when constructor sets @classes to contain Test::Classifier. Does this module load OK?

    Same goes for AUTOLOAD in said file:

    my $obj = $class->new($s->{_files}{all}, \($s->{selected}), $s->{_files}{"${cat}_files"}{_files}); return $obj;

    You may want to check whether $obj materialised before returning instead of relying on caller checking if got undef.

    bw, bliako

      Nice find. Thanks. That led me to see that the failed tester has PERL_USE_UNSAFE_INC = 0 in the reports. In one of the reports that passed, it is set to "1".

      I'm not completely clear why it fails, though. My test file has use lib 't/TestMods'; in it. So I would think the File::Collector would be able to find my module in there. I don't quite follow how the non-existence of '.' in @INC causes the test to fail.

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
      $nysus = $PM . ' ' . $MCF;
      Click here if you love Perl Monks

        I'm not completely clear why it fails, though. My test file has use lib 't/TestMods'; in it. So I would think the File::Collector would be able to find my module in there. I don't quite follow how existence of '.' in @INC plays into this.

        If my theory is right, loading Test::Classifier fails under certain INC. Because requiring these:

        use t::TestMods::Test::Processor; use t::TestMods::Test::TestObj;

        in t/TestMods/Test/Classifier.pm, requires an INC of ./

        bw, bliako

        edit: changed pre to blockquote

Re: How do track down underlying reason for smoke test failure
by davido (Cardinal) on Sep 12, 2019 at 17:25 UTC

    You have the ability to add environment exploration to your tests, of course. Your tests run on the smoker. So add a diag to your tests that produces output about some of the hunches you want to chase down -- things that don't already show up in the smoker's output but that you think would be helpful to know.

    Also when your tests encounter failures, there will be an entry in http://analysis.cpantesters.org/ for your module. Sometimes it takes 24 hours or more to show up, and it only shows up when the distribution has test failures. As a hint, you can create a development version number -- see perlmodstyle for how to do that, and that will allow you to iterate without impacting your typical users.


    Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11106052]
Approved by marto
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (6)
As of 2019-09-17 10:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    The room is dark, and your next move is ...












    Results (207 votes). Check out past polls.

    Notices?