In Defense of Smart::Comments

[Xiong]

I am a huge fan of Smart::Comments, to the point where it astonishes me that few Monks are. I think it is absolutely the bee's knees. It does have its limitations and it has one particular shortcoming, which I'll discuss briefly at the end of this node.

My basic thesis is that although Smart::Comments is a source filter, this is not an issue, since it is easily and completely disabled in production. It does a difficult job well and I haven't seen an alternative I like.

Demo

Do this:

use Smart::Comments;    # Enable special comments for debugging

my $data_structure  = {
    a => [ 1, 2, 3 ],
};
my $scalar          = 42;

### This is a comment.
### <now> I'm happy.
### $data_structure
### $scalar

#my $max        = 2**4;     # slow
 my $max        = 2**2;     # fast

for (0..$max) {         ### for1...         done
    sleep(1);
};

for (0..$max) {         ### for2 |===[%]    |
    sleep(1);
};

The first smart comment is printed literally. The second is timestamped. The third prints the name of $data_structure and dumps its value; the fourth does the same for a simple $scalar. The fifth and sixth smart comments demo progress bars. (You might wish to play around with $max. The trouble with demoing progress bars is that, well, you only see them if you are forced to wait.)

Smart::Comments does more than this but I think the dumping feature alone is worth the price. Others don't agree and this is my defense.

My View of the Task

I'm a "print statement debugger"; I've used interactive debuggers and don't much like them. In general, I'm wary of anything that introduces another level of meta-operation. When I run my stuff, I just run it; I make executable scripts. I don't run perl with command line options. I like to hope that my code is well enough structured that its bugs aren't too deep.

Actually writing print-for-debug statements is a chore -- a minor one but this is not a place where complexity is desired. I frequently want to dump something, somewhere; so I don't want to type one more character of code than necessary. I'm debugging something in my target code; the last thing I want is a buggy print-for-debug statement. A small simplification is a big win here.

After a short time, my code is littered with such statements, which must all be cleaned up for production. Of course, I often just comment them out, in case I want them again later. The cleanup process is liable to short-between-the-headphones error, as I either fail to comment out or remove all the debugging statements; or I make the contrary mistake, destroying production code.

I don't use the other kinds of smart comments much. I may in future. But I'll defend S::C on its dumping.

To dump a scalar with a regular print statement:

    print '$var: ', $var, "\n";    # DEBUG

Dumping arrays, hashes, and more complex structures is much worse. I have old code with entire dumping subroutines (using Data::Dumper, of course); then the calling syntax is something like:

    _dump( '%var', \%var );        # DEBUG

Leave off a backslash and I'm toast. I'd rather debug my target code than my debugging code. I don't even like print, really; the four keystrokes needed to append a newline annoys me, especially since two are shifted. I pretty much insist on say in all my code. This fails, of course, when for whatever reason, feature 'say' is not available.

But the real gorilla, for me, is twice typing the name of the thing to be dumped. If I do this in a regular print/say statement, then I'm likely to leave off some niceties. If I'm in a hurry, I tend to (in increasing disorder):

1:    say "var: $var";     # DEBUG
2:    say '$var', $var;    # DEBUG
2:    say $var;            # DEBUG
3:    say $var;

In line 1, I've left off the sigil from the label so it doesn't interpolate, which is fine unless there's also a @var in scope. In line 2, I've omitted any delimiter between the label and the value, which may lead to confusion. In line 3, I'm impatient enough not to label the dump at all. In line 4, I'm so concentrated on the underlying task that I omit the flag that lets me search for my litter later on cleanup. I've done all these bad things.

The probability of writing a funked-up print-for-debug statement is a function, with positive correlation, of the code length and number of variables to be dumped. Consider:

    print '$very_long_identifier: ', $very_long_identifier, "\n";    # DEBUG

This is still only a dump of a simple scalar! Complex structures really do require resort to Data::Dumper.

A Solution?

I tried for some time to roll my own dump-for-debug subroutine, which preferably I would wrap in a module and call this way:

    mydump( $var );

Presumably, this bit of vaporware would also accept a simple flag to silence such dumps in production. Of course, the calling overhead would still be there. Or I could manually go through and comment out the little guys.

Another hangup is getting the label, based on the variable name as written in the code. Pad::Walker provides a partial solution. But between lexicals and package variables, this and that, I hit a wall in that direction. The plain fact is that there is only one lexical scope that properly knows what every variable in that scope is named: any called routine, even in the same package, is going to have to break-and-enter to do its dirty work.

I don't say that a Saint couldn't deal with all these issues. I do say I haven't seen it done yet.

Enter Smart::Comments

Since you need to be in scope to get easily the right names for everything in that scope, a source filter is the natural approach. The 'statement' is then:

    ### $var

I don't see any way to golf a char off that. It expands into a call to Smart::Comments::_Dump, which does the dirty work -- and does it very cleanly. Package variables, lexicals, arrays, hashes, and nested structures are all dealt with reasonably. You can try to dump stuff that won't; coderefs don't play well. But mostly, you get sensible output. At worst, caller needs to set up a temporary variable to dump:

my $tmp    = scalar @foobar;
### Elements count: $tmp

Another useful feature is that smart comments need not always be introduced with exactly three octothorpes:

use Smart::Comments '###', '####';
###   $foo          # $foo will be dumped
####  $foo          # $foo will be dumped
##### $foo          # nothing happens

Any given smart comment can be disabled selectively by prepending another octothorpe. You can re-enable them as a group by adding another string to the use line.

There's still another disabling method available:

use Smart::Comments;
###   $foo          # $foo will be dumped
no Smart::Comments;
###   $foo          # nothing happens

Objections Answered

It's a source filter.

Yes. I don't use any other source filters for any purpose, so there's nothing for Smart::Comments to conflict with head-to-head. I don't see why a source filter is an outright deal-killer. Like any tool, it can be misused. I need to see a more concrete objection.

It's a source filter, so it should never be used in production.

Agreed. Release production code after it's been debugged. You comment out the use Smart::Comments line itself and all the formerly smart comments become dumb comments, which can of course be re-enabled next development cycle. If you object to too many dumb comments littering the code, it's easy to find and delete them; if you miss one, no matter.

It's a TheDamian module.

Is that merely an ad hominem attack? Or is his track record so bad that one presumes his stuff will eventually blow up?

I have a better way of doing this.

I don't. Please show me.

The Big Fat Shortcoming

Smart::Comments does have a shortcoming which I find serious. This is, it prints all its output to the screen via STDERR. This is hardwired; there's no way to change it, except perhaps by redirecting STDERR elsewhere.

This conflicts with my current testing framework, which captures both STDOUT and STDERR using IO::Capture. I want my debugging output to go to a disk file, so I can pore over it at my leisure, while output from the module under test be undisturbed, so that it tests correctly.

I'm working on a fix. Please hold any comments on the fix to another node. Thank you.