http://www.perlmonks.org?node_id=594914

What's your favorite impossible bug? You know, the kind of behavior that defies the obvious logic rules of the universe, like the exception being triggered in the following code:

if ($some_condition) { return $X; } else { return $Y; } die "not reached"; # but see comment below

Often, when these things happen we blame the compiler out of frustration — but in the vast majority of cases, this is unwarranted: we overlooked something, and the behavior is perfectly reasonable (if sometimes quite subtle) when all the facts are adequately considered. The fear of looking like a fool once the Hidden Reason is revealed only makes the revelation harder to achieve :-)

Here's something that's biting me right now; I'm writing up my experience mostly to distract myself from the horror, and also to solicit from you your own "can't happen" sories, as they tend to be pretty amusing... in retrospect.

I'm installing PAR::Packer on a machine, and am getting the error:

is only avaliable with the XS version at /usr/lib/perl5/site_perl/5.8 +.5/Compress/Zlib.pm line 9

This is one of the cases where Carp::croak shows its evil side: the error is actually coming from Scalar::Util, which has this code:

if (grep { /^(dualvar|set_prototype)$/ } @_ ) { require Carp; Carp::croak("$1 is only avaliable with the XS version"); }

Apart from the typo in "available", this code is wrong because it uses $1 not immediately after obtaining it. require Carp is probably overwriting it (it's a global var, after all, overwritten in every capturing regexp), and indeed the error message I was getting (" is only avaliable") can be explained by a failed match or an empty capture.

Impossible #1: I do have the XS version of Scalar::Util installed! But okay, let's hack up the code and get diagnostics. Since this is the system-installed version of the module, I copy it over to /tmp/Scalar/Util.pm and hack that, then rerun make with PERL5LIB pointing at it.

Impossible #2: I don't see my changes! Could I have typoed PERL5LIB? I rerun make, this time with strace, and see that my file actually is being found. Can't be! But it is. Adding a die at the top of the script doesn't change things. Can't be! Putting the die inside a BEGIN block also impossibly doesn't change anything!

Okay, here I am now. To be sure, I'll sleep on this and find the solution — or maybe one of you monks will have found it for me — but for now, I just revel in the phenomenology of this condition. Does gravity still work? Ouch. Yes. Good.




Update: The apparent dishonor of PERL5LIB can probably explained by PAR's myldr, which has taint checks on, at least in part of its flow. But this leaves us with Impossible #3, which is, why does Scalar::Util think it hasn't got the XS stuff available? I installed it myself and saw the c code go in.

Update II: make install in Scalar-List-Utils doesn't seem to work! At least Util.pm is never updated. Edit: no, I must have been on crack.

Update III - resolution: The 2.x line of Compress::Zlib relies on dualvar, but PAR's myldr does something that cannot load dynamic code during build time. Downgrading to version 1.42 has fixed the problem! Hurrah! Am now corresponding with tsee and Paul Marquess to bring this to a close. Wow, this bug that couldn't happen has involved no fewer than three important distributions!

Replies are listed 'Best First'.
Re: "Can't happen" bugs
by ysth (Canon) on Jan 16, 2007 at 18:36 UTC
    if (grep { /^(dualvar|set_prototype)$/ } @_ ) { require Carp; Carp::croak("$1 is only avaliable with the XS version"); }
    this code is wrong because it uses $1 not immediately after obtaining it. require Carp is probably overwriting it (it's a global var, after all, overwritten in every capturing regexp), and indeed the error message I was getting (" is only avaliable") can be explained by a failed match or an empty capture.
    Nope. require Carp can't overwrite a different scope's $1. Even if (grep { /(...)/ } "abc") {print ":$1:"} has $1 undefined.

    $1 is not "global", it is automatically dynamically scoped.

    The problem is that the code is using $1 outside the scope in which it was set (the grep block). The code as written will never use the $1 from the match.

      You're right; I've since realized that what this code should do is

      if (my @bad = grep { /^(dualvar|set_prototype)$/ } @_ ) { ... }

      And then give @bad in the error message. (This'll give a bogusly redundant list if the import spec of Scalar::Util contains bogus redundancies, but I doubt that's a problem.)

      Anyway there's now a bug in the RT queue for this distro with the correct fix in it.

Re: "Can't happen" bugs
by almut (Canon) on Jan 16, 2007 at 21:06 UTC

    Not really related to gaal's problem - except for the "Can't be!" part - but I thought I'd share this little episode which happened to me some time ago.

    I'd been hacking at the Perl interpreter to create a modified version for a special purpose (details irrelevant here). When I was done, I thought it would be a good idea to rename the binary from "perl" to "plc", to make it clear to me and anyone else that this is not a normal perl. Consequently, I installed this version under /usr/local/plc/..., together with the usual symlink /usr/local/bin/plc pointing to the binary.

    To play with this new toy, I had put the shebang line #!/usr/local/bin/plc in my little test script, and the first tests looked good so far. But as my script grew larger, weird things started to happen. Of course, my first thought was "Ok, Almut, you've messed it up!", or at least - with less detrimental effects on my self-esteem - "there's still room for improvement in your initial hacking attempts".

    To verify this hypothesis, I quickly went to check whether my test script would still be working as expected when run with the normal (unchanged) Perl. In cases like these, I usually call the desired version explicitly, to avoid having to edit the shebang line. So, I typed - full path to be absolutely sure - /usr/local/bin/perl test_script.pl   Nothing special really. I had done this hundreds of times before, as I usually have several versions of Perl to play with. (Unless you mess with other things, the respective perl binary has no problem finding everything else that belongs to it...)

    Somewhat to my surprise, my script's behaviour didn't change. Same weirdness ... Can't be! But after a couple of "Hmm, I could've sworn I'd written code like this before which worked", I then went on to rewrite the problematic code, which wasn't all that difficult, thanks to Perl's TIMTOWTDI. And things were fine again. Until strangeness kept creeping in again and again while doing further tests. At this point, I would've had to seriously question my general Perl coding abilities, so I figured there must be some other explanation... ;)

    Well, to make it short, after lots of strace-ing, losing trust in my sanity, etc., I finally found this little paragraph while perusing the output of perldoc perlrun

    If the #! line does not contain the word "perl", the program named after the #! is executed instead of the Perl interpreter. This is slightly bizarre, but it helps people on machines that don't do #!, because they can tell a program that their SHELL is /usr/bin/perl, and Perl will then dispatch the program to the correct interpreter for them.

    And my almost infinite ignorance had suddenly been reduced to almost_infinite - 1.

    The moral of the story: Thou shalt not dispossess Perl of its proper name, or its wrath and black magic will come down on thee like a ton of bricks :)

Re: "Can't happen" bugs
by gaal (Parson) on Jan 16, 2007 at 15:08 UTC
    (Incidentally, the leading example was inspired by this snippet, seen on one of those Code WTF sites:

    if (myVar == true) return true; else if (myVar == false) return false; else return !true && !false;

    Perhaps this was written by someone who is too into intuitionist logic.)

      actually constructive mathematics (and tertium non datur) is one of my favourite subjects ;) there are lots of systems with non-binary logics, like SQL, OS or... perl ;)

      cheers --stephan
        A meditation on non-binary logics in Perl (or indeed other computer systems) will be received with applause, I think :-)
Re: "Can't happen" bugs
by hardburn (Abbot) on Jan 16, 2007 at 17:28 UTC

    Nearly any encoding problem makes me go bonkers.

    Recently, I had a function that took a WWW::Curl object and some parameters, which then sets the parameters on the Curl object. Something like this:

    set_opts({ curl => $curl, url => 'http://www.example.com/path/', . . . });

    When $curl->perform method was called to run the query, no query was actually being sent. After doing some debugging, I found out that the URL within the object was gibberish.

    I fixed it by changing the set_opts() parameter like this:

    set_opts( 'http://www.example.com/path/', { curl => $curl, . . . });

    Which shouldn't have any effect in a sane world, but in a world where encoding issues come up, it does.

    I swear that encoding problems are non-deterministic, and should be used as a cryptographically-secure source of random numbers.


    "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

Re: "Can't happen" bugs
by xdg (Monsignor) on Jan 16, 2007 at 15:20 UTC

    Is there another Perl that comes earlier in your $ENV{PATH}? If so, maybe you aren't running the Perl you think you are.

    As a more general point about tracking error messages, I find that App::Ack or Google Code Search are often quick ways to find out where some strange error message is being generated.

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

      Shouldn't another perl also honor my PERL5LIB, which after all is in the environment?

      Since posting I've found that there are, in fact, two loadings of Scalar::Util in the strace dump. The first one finds my version as desired. The second does not! Since this was an strace -fF on make, not perl, it may very well be that a second interpreter (whether another installation or another instance of the same one) is being invoked. But the question is now: under what circumstances is the environment override (which prepends to @INC, by design of course) ignored? Or is there someone who is weeding my @INC?

      Hmmm. Maybe taint mode does this! *dives in again*

        Par::Packer uses Module::Install and it looks like it is doing some auto installation (ick) -- and that might be invoking another interpreter. You might want to take this up on #par or with the Module::Install people.

        -xdg

        Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

      This is probably a bit silly, but I've occasionally found Acme::JavaTrace to be useful for finding where stuff actually happens.
Re: "Can't happen" bugs
by sfink (Deacon) on Jan 17, 2007 at 06:04 UTC
    I ran into this one not long ago:
    if ($x = 0) { return "zero"; } elsif ($x < 0) { return "negative"; } elsif ($x > 0) { return "positive"; } # Cannot reach here
      Beautiful! That's why you should always put the constant on the left:

      if(0 = $x) # Can't modify constant item in scalar assignment.

        Or use warnings:

        use warnings; my $x; if ($x = 0) { 1 } __END__ Found = in conditional, should be == at - line 4.

        lodin

Re: "Can't happen" bugs
by tsee (Curate) on Jan 16, 2007 at 21:53 UTC

    If you find the solution for this problem and think it's PAR(::Packer) related (but not caused by PAR), could you please add it to the PAR FAQ at par.perl.org so others can benefit from your detective work? That would be great!

    Naturally, if it's a PAR bug, I'm all open for trouble tickets, suggestions, and most importantly patches.

    Thanks!

    Steffen

      I'm not sure it's a bug! See, myldr has a pretty critical block between TAINT and NOTAINT macros. This means that people who rely on the environment for their Perl setup might have trouble running static during installation... but I suspect that they're sort of screwed anyway, because any other thing running in taint mode may similarly break!

      Update: on second thought no doubt we can at least issue a warning at build time. Patch coming up...

      Update II: Patch follows. Not 100% sure about the wording.

      --- Makefile.PL.orig 2007-01-17 09:28:20.000000000 +0200 +++ Makefile.PL 2007-01-17 10:02:15.000000000 +0200 @@ -76,6 +76,16 @@ } } + if ($ENV{PERL5LIB} || $ENV{PERLLIB}) { + warn <<'.'; +*** You have extra Perl library paths set in your environment. Please + note + that these paths (set with PERL5LIB or PERLLIB) are not honored b +y perl + when running under taint mode, which may lead to problems. This i +s a + limitation (by design) of Perl, not of PAR::Packer; but some of t +he + problems may manifest here during installation. +. + } + my @bin = ("script/parl$exe", "myldr/par$exe"); push @bin, ("script/parldyn$exe", "myldr/static$exe") if $dynperl +;
      I've added the workaround of downgrading Compress::Zlib to the front page, seeing as it's an immediate installation issue. Feel free to move it to the FAQ if needed :)

      Also, please see email w/ Paul the maintainer of Compress::Zlib about avoiding this problem.

Re: "Can't happen" bugs
by fireartist (Chaplain) on Jan 17, 2007 at 15:30 UTC
    The most confusing (non-perl) bug I've have to deal with was when someone tried using Dreamweaver to ftp a file to a remote webserver, and the file contents were stomped on (zero file size).
    I then realised that the local file had also been stomped on.
    I eventually realised that they'd mapped a drive (windows) to the remote webserver, and were editing the remote file directly in Dreamweaver...
    ...which had tried to upload over itself...
Re: "Can't happen" bugs
by ikegami (Patriarch) on Jul 04, 2008 at 18:01 UTC

    the error message I was getting (" is only avaliable") can be explained by a failed match or an empty capture.

    A failed match doesn't clear $1

    >perl -le"'a'=~/(.)/; 'b'=~/c/; print $1" a

    $1 is cleared by a successful match with no captures, or capturing a zero-length string.

    >perl -le"'a'=~/(.)/; 'b'=~/./; print $1" >perl -le"'a'=~/(.)/; 'b'=~/(.*?)/; print $1"

    Apart from the typo in "available"

    It's a feature to make the error message easily findable in the source. *wink*

Re: "Can't happen" bugs
by ambrus (Abbot) on Jan 23, 2007 at 09:02 UTC

    I'm not sure it's the require Carp; that overwrites $1. Coudln't it also be later, failing regex matches in the grep statement?

      Yes yes, as I said to ysth I realized that was a bogus claim when I actually put up the code.
Re: "Can't happen" bugs
by chrisdolan (Beadle) on May 30, 2007 at 02:18 UTC

    Thank you for this post! I have encountered the EXACT same problem (CentOS 4.5, perl 5.8.5) and took your downgrade advice. I had to delete the original Compress/Zlib.pm too since the new version installed in a different place with the latest CPAN.pm.

    Chris

Re: "Can't happen" bugs
by ambrus (Abbot) on Jan 04, 2011 at 11:46 UTC