Bad news for IO::Handle magic

[Stevie-O]

People familiar with IO::Handle, please skip the following paragraph.

For the uninitiated, there's an OO interface to the filehandles in Perl. All you do is use IO::Handle and at the expense of a mere *cough* few thousand lines of Perl code, you can replace non-OO access to filehandles:

open(my $foo,'<+ /dev/null');
#old
select($foo); $|=1; select(STDOUT);
print $foo "|<3wl! 31337 d00d!";
printf $foo "%.02f", $money;
$fd = fileno $foo;
read($foo, $input, 1024) unless eof $foo;
@stats = stat $foo;
close $foo;
#new
$foo->autoflush(1);
$foo->print("|<3w1! 0bj3ct 0r13nt3d f1l3h4ndl3z! l33t!");
$foo->printf("%.02f", $money);
$fd = $foo->fileno;
$foo->read($input, 1024) unless $foo->eof;
@stats = $foo->stat;
$foo->close;

OK, lesson over.

Everybody back with me? Good.

Now, some people may be familiar with this nifty oneliner, that demonstrates the nature of the magic by which Perl associates filehandles with the IO::Handle package:

$ perl -le 'STDIN->close()'
Can't locate object method "close" via package "IO::Handle" at -e line
+ 1.

This shows that Perl is magically understanding that STDIN is a filehandle being used as an object, and trying to invoke IO::Handle::close, and failing (because IO::Handle wasn't required).

I've known about this problem, but something occurred to me today that reveals a serious issue with this magic. Please read on, I'd like to know if anyone agrees with me.

• perl -le '{package STDIN;sub close{print "case closed"}} STDIN->close(
  +)'
  

  According the normal rules of engagement, this should invoke STDIN::close() as a method. People understanding the problem in 347618 will know his right away.

  Instead, the error "can't find method 'close' in IO::Handle" occurs, indicating that Perl is actually trying to treat the bareword STDIN as a bareword filehandle first, and as a package name second.

  While that may not be 100% optimal, the fact that filehandles have no identifying sigil makes this a necessary evil. So, I took it one step further.

• perl -le '{package STDIN;sub close{print "case closed"}} "STDIN"->clos
  +e()'
  

  Again, I get "can't find method 'close' via IO::Handle", instead of "case closed". I guessed that Perl was trying harder to do the filehandle thing than I thought.

• perl -le '{package STDIN;sub close{print "case closed"}}my $foo="STDIN
  +"; $foo->close()'
  

  Still IO::Handle.

  Perl is apparently trying to treat $foo as a symbolic reference to *STDIN, and doing the filehandle thing.

  I finally get to the real kicker:

• perl -Mstrict -Wle '{package STDIN;sub close{print "case closed"}}my $
  +foo="STDIN"; $foo->close()'
  

  Lo and behold:

  Can't locate object method "close" via package "IO::Handle" at -e line 1.
  

Now, the only way for Perl to go looking for IO::Handle is for it to treat the 'STDIN' as a symbolic reference. Ignoring the fact that in no other case does the $obj in $obj->foo() attempt to treat $obj as a symbolic reference, but Perl is treating 'STDIN' as a symbolic reference in violation of strict refs (that's the -Mstrict) without so much as a warning.

Comments?

[tye]

Don't use single-word all-upper-case package names.

Your example isn't the only way that you'll run into problems with such poor package names. Don't follow the bad examples of CGI.pm and CPAN.pm (which are badly named for more than just this reason, as anyone who ever tried to talk about "CGI" or "CPAN" knows; Net::CGI and Net::CPAN would be much better).

For that matter, don't use bareword file handles. Your example isn't the only way you'll run into problems using those.

Now, you can make an exception for STDIN, STDOUT, STDERR, and DATA. But if you really write "package STDIN;", then you deserve all of the grief you get.

Below are some other things you shouldn't do. Perl has a lot of DWIM features and DWIM always implies ambiguity and the possibility that Perl will misinterpet what you thought you meant.

Don't use single-word, all-lower-case subroutine names. If you do, then you can reduce your risk by either calling them like &mysub(...) or by putting them into a module (and then import them, use the full name with package name in front, or use OO).

Note that writing &mysub(...) will likely get you yelled at by people who think it is "bad style" or "looks like Perl 4" or some other bogus reason [such as confusing &mysub(...) with &mysub;].

And if you make the mistake of using a prototype, then you probably shouldn't be writing &myprototypedsub(...) but that's okay because using a prototype means you should put the function into a module anyway (but OO isn't an option in this case) -- and you should include an apology in the module's POD.

Similarly, don't use single-word, all-upper-case subroutine names or single-word all-lower-case package names.

In fact, don't use single-word package names. Make sure all of your package names contain :: (with exceptions for "main" or other package names that you don't get to pick).

Don't leave off parentheses when calling a function as part of an expression. If you use an assignment as part of an expression, put parens around it. If you use bit-wise operators, remember that their precedence just doesn't make sense and so put in explicit parens inside and out. Don't use here-docs; they break on invisible spaces (trailing spaces) or if you reindent. Don't use unless, it is confusing (even though it doesn't seem like it at first) and isn't any clearer (even if it seems so at first). use vars not our (as discussed in other threads). Don't use source filter modules. Use strict and (at least during development) enable warnings. etc. (:

Updated: Moved a misplaced sentence.

[Stevie-O]

While my examples always used STDIN, that was for brevity. Any code can play tricks on you if you use the Module->sub() OO syntax.

$ perl -MCGI -le 'select(select(CGI)); print CGI->new'
Can't locate object method "new" via package "IO::Handle" at -e line 1
+.

This isn't limited to all-caps names, either:

perl -MFile::Spec -le 'select(select(File::Spec)); print File::Spec->r
+ootdir'
Can't locate object method "rootdir" via package "IO::Handle" at -e li
+ne 1.

[tye]

I said not to use bareword file handles. Using mixed-case bareword file handles is an even worse idea and you deserve any grief you cause yourself if you do that.

Though, with other problems, I'm starting to think "File::Spec"->method() is the better syntax. But more than that I prefer putting the module's factory (which for many modules is simply the module's name) into a scalar, even better if this can be done as part of the use or require for the module.

[ysth]

I'm curious to know how you found the select select trick.

[Joost]

Hi tye, good points, but I have a question about this remark:

Don't use single-word, all-lower-case subroutine names. If you do, then you can reduce your risk by either calling them like &mysub(...) or by putting them into a module (and then import them, use the full name with package name in front, or use OO).

I can see there is a potential problem in that calling single word subroutines without parentheses can be confusing to the parser, but what's the difference in this regard between all-lower-case subroutines and (for example) CamelCase subroutines? Or are you warning against redefining future keywords/build-ins?

Joost.

[tye]

#!/usr/bin/perl
use strict;

# For debugging:
sub dump {
    my( $name, $ref ) = @_;
    require Data::Dumper;
    my $dumper = Data::Dumper->new( [$ref], [$name] );
    $dumper->Indent(1)->Useqq(1)->Terse(1);
    print STDERR $dumper->Dump();
}

my $foo = "Test value";
dump( foo => $foo );
# Should instead used:
#&dump( foo => $foo );
# ...

produces:

abnormal program termination

Helpful, isn't it? I suspect quite a few people don't immediately see what the problem is. I also suspect that no small number of people could waste quite a bit of time trying to figure this problem out before the solution comes to them and they snack themselves in the forehead.

Adding the & or renaming the sub to debugDump or moving the sub to a module will solve the problem.

The problem is that dump is a built-in. I don't care to memorize the entire list of Perl built-ins nor to put my code at risk when new built-ins are added (rarely).

[broquaint]

This probably does deserve a better warning but you can find this behaviour documented in the Technical Note on the Syntax of Variable Names section in perlvar, where it is noted that ...

Perl identifiers that begin with digits, control characters, or punctuation characters are exempt from the effects of the "package" declaration and are always forced to be in package "main"; they are also exempt from "strict 'vars'" errors. A few other names are also exempt in these ways:

               ENV             STDIN
               INC             STDOUT
               ARGV            STDERR
               ARGVOUT         _
               SIG

[bunnyman]

But strict 'vars' is different than strict 'refs'.

[graff]

Well, it seems like Perl treats STDIN as a file handle, always (kinda like "stdin" and "stdout" and "stderr" are always pre-defined as "FILE *" thingies in C, and you shouldn't try to use them for other things). Using STDIN as the name/namespace for a package/object of your own design won't work, "so don't do that."

I guess I'm not particularly sensitive towards this issue (i.e. I don't have a problem with it); if you get by with calling your package "Stdin" or "MYSTDIN" or some such, where's the trouble? What would a package named "STDIN" get you that you can't get some other way? (I really would be curious to know...)

[gmpassos]

The question is that if you have a handler opened Perl will treat it as a IO::Handle object:

package ALIAS ;

sub close { print "CLOSE>> @_\n" ; }

package main ;

 open(ALIAS , ">&STDIN") ;

 ALIAS->close() ;

This will produce the IO::Handle error, but if you remove the open() line, ALIAS will work as a package name. So, this is not a reserved word problem!