Re^6: The Most Essential Perl Development Tools Today (static analysis)

[pemungkah]

Since what you say, what I remembered, and what many of my Google searches say conflict, I ran an experiment. I created this package in the debugger:

package bar; 
sub new { my $self = {}; bless $self, shift } 
sub openme { open BLODGETT }
sub closeme { close BLODGETT }
1;

and then did this:

$x = bar->new()
$x->open
x %main::
x %bar::

*main::BLODGETT does not exist, and *bar::BLODGETT does exist.

This confirms that, despite many webpages to the contrary, bareword filehandles in different packages do not conflict, and I am therefore pleased to have fixed a bug in my internal inside-my-head Perl documentation, which was wrong.

However, it's of interest that the scenario at the CERT secure coding site depends on bareword filehandles; specifically, if someone defines a sub with the same name as your bareword filehandle, that will take precedence, and can lead to the problems noted there.

A programmer probably wouldn't do that to herself, but the potential exists for someone else to do so; lexical file handles actually prevent this vulnerability from working.

As far as the "did I reuse this filehandle before I ought to have?" test, I definitely agree: that *is* hard to write. You could prevent it by having a table of filehandles and currently-open-for-output files, and throw an error if I try to re-open one that is currently open, or do simulated runtime analysis that verifies that there is no path that could lead to the file being reopened before it is closed.

But now we're outside the scope of the question I asked, which was not about whether I could formulate this particular check right, but what "dynamic analysis" meant to BrowserUK. I'm guessing you're both talking about "simulated execution", which puts us into a much more sophisticated kind of analysis. Not impossible to do, but way more sophisticated than Perl::Critic. If that is the case, I'll agree: Perl::Critic can't do that.

It can do some things that are appropriate in large development projects, like verifying you remembered strict. You can certainly do that yourself if you want. I prefer to have mechanical tests handled in a mechanical way, just like I prefer automated tests, even though Test::More can't test every possible situation. We just need to remember and emphasize to more naive programmers that tests don't guarantee the code is right, only that what we've tested passes the tests as they were defined, and is therefore probably not wrong.

I think that a tool that lets me know about stuff that is possibly going to mess me up is useful. Others don't. I think we'll have to agree that we don't agree about this, as I have the feeling that perhaps this isn't about the tool, but instead about approaches to development, and I already know BrowserUK and I have wildly different preferences on that! And that since those are matters of taste rather than fact, arguing about them isn't productive, so I shall stop.

[tye]

I'm guessing you're both talking about "simulated execution"

That would be guessing at more agreement between us than is warranted. On that point, I'm not completely sure what BrowserUk is talking about (though I certainly agree with many things he said). What I'm thinking of is fairly simple analysis by a programmer.

And I'm pretty doubtful that a software program would be able to do that analysis well. For example, I see little problem with sometimes writing code that intentionally makes use of the fact that open FOO silently implies close FOO if FOO was still open. And that code can be written where that intent is quite clear (to a human, and not even using comments).

A lot of what often makes this type of analysis so simple for (at least some) programmers, is being able to discern the purpose of parts of the code. That isn't an easy trick to teach to a piece of software.

specifically, if someone defines a sub with the same name as your bareword filehandle

Yeah, somehow that just doesn't worry me much at all. I tried to go read the page you linked to but the code examples were simply empty, brightly-colored boxes. Perhaps that would be remedied if I told my browser to run the unknown code that that web page offered up. Rather ironic from a site espousing "secure coding".

From the text description, it sounds like the "problem" was demonstrated by somebody introducing code very similar to "sub BAD() { 'GOOD' }". Yeah, that's a very believable example. I write code like that all of the time. I especially do that using ALL-CAPS names that match filehandles that I use in the same source file (required in a sane coding environment because separate source files have distinct 'package' declarations).

But it does suggest a much better idea for a Perl::Critic policy. Tell me when I have a conflicting pair of same-named items, such as a package name ("use CGI"), a subroutine name ("sub CGI"), or a filehandle name ("open CGI"). That'd actually be something worth pointing out.

And not that I'm encouraging everybody to run off and use bareword filehandles. I am aware of several down sides to them. But I also appreciate at least in some situations the value of a filehandle that looks nothing like my other lexical variables.

And I'd probably discourage inexperienced Perl programmers from using bareword filehandles. But I don't think I'd recommend that those same programmers try to use Perl::Critic to aid them in avoiding that practice. Though, I can certainly see it sounding like a really smart idea if you don't really think it through thoroughly. And it could've been a much less bad idea if several things about Perl::Critic hadn't been done so very badly.

But my experiences of seeing some results of people having tried to use Perl::Critic have really only greatly strengthened my prior assessment that using it would usually end up being a bad idea. Re^2: Hello World for module writers (stumper) is an excellent example. Re^2: Symbolic reference with strict "refs" (it stinks!) even better.

Telling me I forgot to "use strict" at the top of some module would be another fine use for something like Perl::Critic. Though, even that use is, IMHO, best done rather differently than most people appear to do it.

The fact that ProhibitNoStrict actually got published is a pretty clear demonstration that the people maintaining Perl::Critic are not thinking clearly and thoroughly about the implications of their work. But it also isn't the only indicator of that.

So I'm not surprised to find smart people having pretty strong reactions against using something like Perl::Critic.

Things that would make Perl::Critic at least significantly less bad:

• The default policy set should err on the side of only including things that are clearly serious problems
• It should complain every time you run it if the set of policies to be included hasn't been customized yet (just make a policy for that and include it in any pre-defined set of "not just clearly serious errors" policies)
• 'ProhibitNoStrict' is a great example of a policy that should never be easily enabled, not even by some "--be-extra-picky" mode, and not included in any pre-defined set of policies
• It should default to complaining if you have added "#nocritic" markers to your code
• Instead, if you ask it to tell you about suggestions other than about clearly serious problems, then it should provide ways for you to record "yeah, none of those really need to be addressed" without polluting your code
• It should encourage being run in a mode where it only complains about non-serious problems if they are associated with lines of source code that have changed (run it against a commit or against a branch)
• It should encourage that the complaints should always be evaluated by a qualified human, not as part of some automated step that has any consequences other than to provide details to a qualified human
• Policies against things other than clearly serious problems should be careful to be clear in the description that this is not always a problem

I gave up trying to find a sample report of "violations" (a poor word choice for use with this module). But I did find this in some documentation:

    --severity gentle
    --severity stern
    --severity harsh
    --severity cruel
    --severity brutal

Yeah, that is really the wrong way to go. The levels should describe how likely a variance from the policy is to be a real problem or to risk becoming a real problem. The last level shouldn't be "brutal", it should be something more like "nit-picky", "of questionable value", "probably worth ignoring". And "lint" was a much better name choice than "Critic".

[Anonymous Monk]

'ProhibitNoStrict' is a great example of a policy that should never be easily enabled, not even by some "--be-extra-picky" mode, and not included in any pre-defined set of policies

This makes it sound like they'll add anything :) http://search.cpan.org/perldoc/Perl::Critic#EXTENDING_THE_CRITIC

If you develop any new Policy modules, feel free to send them to EMAIL and I'll be happy to put them into the Perl::Critic distribution

[pemungkah]

One comment: in the context of the sub GOOD { BAD } example: remember that this is about creating vulnerabilities in code, not, "DOY I CAN'T PROGRAM'. I think from that perspective, they have a point. Remember that creating vulnerabilities isn't about good programming practice, but of taking advantage of possible weaknesses.

[BrowserUk]

in the context of the sub GOOD { BAD } example: remember that this is about creating vulnerabilities ... [which] isn't about good programming practice, but of taking advantage of possible weaknesses.

"Vulnerability refers to the inability to withstand the effects of a hostile environment."

So, the hostiles somehow detect that I'm using two bareword filehandles in my script and then devise a mechanism by which the succeed in injecting a constant subroutine that effectively redirects one as the other into my scripts namespace.

The only way I can see for that to be possible, is that they modify the script itself; or, they modify one of the modules my script uses.

If they have access to my filesystem sufficiently to be able to exploit that "vulnerability"; don't you think that they might find easier, more direct ways of achieving their nefarious goals? Like maybe just writing whatever they damn please into whatever file they want to corrupt.

There's this vague memory running around my head. Something about shutting doors and horses bolting.

I'm also reminded of a company I sub'd at for a few months; they suddenly got paranoid about the number of people using one of the development labs and at some considerable expense added keycard locks and cameras to the entry points. One day the system that controlled the keycards crashed. That system lived inside the lab and no one could get in. They were about to get some guys into remove teh door from its hinges when one of the operators lifted a floor panel in the UPS room next door to the lab, crawled through the cable space and popped up in the lab and unlocked the doors from the inside.