Bugs. Don't you hate 'em? Pesky typos, documentation mis-reading, late-night algorithm confusion...

The ones I hate most of all though are all those bugs in perl (or Linux, or Windows, or MySQL). You know the ones. When you've tried everything to get your code to work, but to no avail. You know *your* code is right - heck - you were there when you wrote it - it's not like the other guy has messed up *your* source...so it must be a problem with the OS/language/DBMS/CPAN module/Sunspot-activity/whatever.

Having recently attempted to show a fellow monk this was *not* the case, it reminded me of (a) just how long it took for me to learn that it was almost always my code that had the problem, and (b) how many times I'd tried to teach others what I'd learnt.

Fair enough - there *are* bugs in just about every OS/DMBS/etc. you could point to...most of them well documented, or so obscure that the chance of you hitting them is minute. (Occasionally, in earlier times, these could even be a 'feature'...C64 sprites-in-the-border sprint to mind) What many programmers forget though is that the baseline code they're sitting on top of has probably been run billions of times before without a problem, while *their* code hasn't.

I suspect it's a rites-of-passage thing. After $x years learning to perform the task, it takes $y years to learn that $x was just the beginning, and it then takes $z years... etc. (I was going to attempt some contrived '$x += $y' recursive thing, but you get the idea <g>). My favourite definition of 'getting old' is 'stopping wanting to learn' - I hope never to get old myself, but unfortunately I know some people who got old at 14.

Most mystical tradition teaches that ego-loss is a precursor to illumination, and a certain degree of this is always necessary in order to grow, as a programmer or otherwise. It's always hard to admit that one makes mistakes, but until you get into the habit of looking on your own doorstep first, it'll take a lot longer to find them.

Ben - ever an initiate, regardless of XP :)

Replies are listed 'Best First'.
Re: It's your code...
by TVSET (Chaplain) on Apr 26, 2003 at 12:16 UTC
    Updated: Added few headers (thanks for the tip goes to artist). Added P.P.S. :)

    Well, debugging techniques are vital for any developer. Bugs are everywhere and everyone comes across them in all different peices of software. You cannot do anything about it. What you can do though, is debug, fix and submit.

    Intro

    Unfortunately, I haven't seen much of the examples of good debugging sessions and had to learn that the hard way. Still learning... Whatever info I found was oversimplified and general. Let me share my knowledge in hope that everyone else will correct, add and beautify it and share their own. :)

    First rule to follow is: "Work from top to bottom". Here, I consider the top of the system to be your program, which runs on top of something, which runs on top of something, which runs on top of something. Since that is the new program you are writing, it is obviously less tested then all those parts on which it runs, therefore, the chances are that your program contains bugs, that make your life misarable, and not the parts on which it runs. (That was one long sentense, wasn't it?).

    Identify the problem

    How to debug the problem? Well, that's a deep question. First, I'd say you need to identify the problem. Until you don't know exactly WHAT is wrong, you cannot fix it properly. Do some testing. Feed all sorts of different data onto the problematic part and see if you can find any patterns. That will help you to localize the problem.

    Localize the problem

    Problem localization is important since it narrows down your search and increases your success chances. Here, print statements and Data::Dumper module are your best friends. Usually, the problematic area is small, few lines of code. If it is all inside the same part of your program, you are a lucky guy. If the problem is between two or more components - the world is hard on you. :)

    Understand the problematic piece of code

    When you know where you problem is, reread and reunderstand the piece of code. Make sure that you know precisely what's going on. Still cannot see the bug? Go on... Take the piece of code completely outside of your program and work on it separately, without any context. You are lucky again if you can do that. :)

    Is it time to learn perl's debugger?

    If you still don't have it, maybe it's a time to use a debugger. Perl's built-in debugger is very nice. Go through a perldebug manual. That should help. With debugger, you can do lots of fancy staff like going through the program step by step, printing out variable values at each step, setting up breakpoints, etc. That should help.

    Dive deep down

    If you still have a problem and you still cannot see it, possible there is a problem in someone else's code, not yours. (surprise-surprise). First thing to check is the next level on which your program runs. Maybe that's a module from CPAN or perl itself. If that's a module, go through and check it out. Read modules documentation. Possibly, you are not using the module correctly. Whatever.

    Reimplement in other language, maybe?

    In case of perl, though it gets more complicated. One of the things I can suggest is try to translate your perl code into some other language (PHP/C/Java/whatever) and execute it. It should as close translation as possible. If you don't see the problem with other language, maybe that's perl. If you still see it, possibly, it's on the lower leve, like an operating system. And yes, that happens too.

    Try another operating system..

    Does your code produces the same problem on the other system? If it still does, the problem is above the operating system, obviously. :) Operating system specific debugging is a deep topic on its own. I'll just say that if you use Linux, then you can have a handful of tools like strace(1) and ltrace(1) which will help you get really down and below.

    That's a long node already, isn't it? I'll cut right here. Almost here...

    Few words on submitting bug fixes/reports

    Few words on fixing and submitting. You cannot always fix the problem that you encounter. Ask for help. Ask other people to look into it. Each and every project, has it's own developer and user community. Find it with Goggle and ask those people. At least they might point you to the right direction.

    If you can fix the bug you found, then I wish you to have a long and happy life. :) Check project documentation for tips on how to submit bugs and/or bug reports. You did a lot of work by now, don't let it be lost. Let people know.

    If you get ignored

    Sometimes, though, you might get into akward situation, when developers will refuse to accept your fix and will not aknowledge the problem. This, unfortunately, happens too. You still can add some good onto the world. Publish the problem description, solution description, and the patch onto YOUR OWN web site. People will find it, when it's needed.

    Share the experience

    Now, really last words. Share your debugging experience. Once in a while, document the problem you encountered and the way you found a solution for it. Let the others learn from your experience. Mention the tools you've been using. Suggest testing data. Whatever. There are not that many examples of these on the web, unfortunately. I don't have a good example for developer's debugging, but here is a nice page with Admin Horror Stories. It lays out things nicely.

    P.S.: Ok, next time I'll use the "Read More" thingy and a spell-checker. :)

    P.P.S: I haven't mentioned any of #!/usr/bin/perl -w and use strict; since I assume you already had them from the very beginning. :)

    Leonid Mamtchenkov

Re: It's your code...
by halley (Prior) on Apr 26, 2003 at 11:27 UTC
    You mentioned a sort of hump where someone realizes they have a long way to go. Some other thinkers have described "the path to mastery" with four levels.
    • novice: unaware of his incompetence
    • apprentice: aware of her incompetence
    • journeyman: aware of his competence
    • master: unaware of her competence

    The first rite of passage you're referring is the novice-apprentice transition. Instead of playing around on your own tasks, you decide to really use someone else's methods, and are coming to grips with your need to apply some discipline.

    Another "enlightenment" is when you develop a sense of confidence, realizing you're pretty good at what you set out to do. You still learn and you still want to learn, but you're adept at taking new knowledge and filing it in your own way.

    The final stage, mastery, is just another stage, not better or worse than the others, but has its own characteristics. If most of what you learn is from how others approach their problems, and not from how you approach your own projects, you may be a master. Competence becomes somewhat irrelevant, it's not the results that really matter, but instead, the opportunities in the path taken.

    --
    [ e d @ h a l l e y . c c ]

Re: It's your code...
by Abigail-II (Bishop) on Apr 26, 2003 at 18:48 UTC
    You know *your* code is right - heck - you were there when you wrote it - it's not like the other guy has messed up *your* source...so it must be a problem with the OS/language/DBMS/CPAN module/Sunspot-activity/whatever

    Whenever I think "it's not my code that's buggy - it's perl/the OS/the database", I write a program from scratch that's uses this supposedly bugged feature. Most of the time, it result is very sobering, the new program runs correctly, and the bug is in my code afterall.

    Abigail

      ...for me, this was what I call the "found a bug in the library" lesson.

      I had a programmer working for me who would come in occasionally and report that he'd found a bug in the database library we'd purchased for our application development project. Being new as a team leader and having always believed for my own individual work that a "help desk" was really just a nice way to spend some time on the phone with a stranger instead of working on my project, I was unprepared for the sinkhole I stepped in next.

      I took Enoch at his word and called the library vendor to report the problem. Of course they asked me to supply a small section of code the demonstrated the bug. They didn't want to see my whole application, just a tiny program that demonstrated the bug. No help from my telephone call, but their request seemed a reasonable one to me.

      So I had Enoch work up such a program to submit to the vendor. He never came back with that little program, and when I questioned him a couple of days later, he told me that he'd discovered that there wasn't a bug in the library after all. Come to think of it, my phone call with the vendor wasn't really a "no help at all" call. I learned a lesson that I've passed on to students for years now. 19 out of 20 "bugs in the library" are really bugs in the client code. Or they result from a misunderstanding about how to use the library.

      So I tell people to test new functions in a small demo program before using them in the full-scale application. And if there is a bug that needs to be reported to a vendor, then create a very bare demonstration program to illustrate the misbehavior. The vendor will ask you for this anyway, so you have to do it sooner or later. And many times your demo program will uncover the misconception that is keeping your project from working.

      ...All the world looks like -well- all the world, when your hammer is Perl.
      ---v

      Indeed, it's amazing how OS-based bugs disappear when they're not in the middle of 500 other lines of code - probably something to do with dodgy RAM, or a bug in the CPU ... <grin>