Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Chasing Shadows

by samtregar (Abbot)
on Sep 05, 2003 at 05:17 UTC ( #289111=perlmeditation: print w/replies, xml ) Need Help??

Large software systems sometimes develop very strange bugs. Just recently I found one, and here is its tale. My current project is around 15,000 SLOC across 70 or so modules. The test suite runs over 4,000 tests. One of our test scripts started dying around halfway through with this mysterious error:

  DBD::mysql::st execute failed: Lost connection to MySQL server during query

It didn't happen every time and it didn't happen on the same database call every time. The test itself didn't give me any ideas. Its job is to run through all the modules, find the ones that implement a standardized interface and test that interface. About half our modules implement this interface and they all passed the tests, when the test script got that far.

I searched the web and the docs, but found no other mention of the error message. I posted to the DBI and MySQL mailing-lists, but got nothing back. Finally I found something that made the problem go away; I put in a call to $dbh->ping to check the DBI object before each time the $dbh is returned. I didn't think this was much of a fix, but it did avoid the error.

Weeks later a co-worker mentioned seeing a similar problem report on the DBI mailing-list. I rushed to check it out and found a reply mentioning fork() as possible culprit. As every good DBI geek knows you can't use a connected DBI object across a fork(), and the error message I got was a common symptom.

Eureka! The way the test works is by loading every module, calling $module->can('foo') and then proceeding to test the $module->foo() call. But in loading every module it happens to load one that does:

  BEGIN { Proc::Daemon::Init }

That call fork()s! Sure enough, as soon as I changed the test to skip loading this module the error stopped happening, even without the $dbh->ping() check.

I find this episode interesting for two reasons:

1. Sometimes the bug is the test itself. I might have found it earlier if I'd thought carefully about what made this test different from all other tests.

2. The proverb "only perl can parse Perl" is an important one. It's more than just a slander of Perl syntax. It also reminds us that parsing a Perl script or module also means executing it. You can't just go loading Perl code unless you're certain you know it won't do something while loading.


PS: Props to Kalin Kozhuharov for the post to dbi-users that put me on the right track.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://289111]
Approved by dbwiz
Front-paged by cchampion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (5)
As of 2023-03-27 22:27 GMT
Find Nodes?
    Voting Booth?
    Which type of climate do you prefer to live in?

    Results (66 votes). Check out past polls.