Clear questions and runnable code
get the best and fastest answer
The SituationOver the past few years, I’ve come to be the “go-to guy” for an existing code base of several million lines of legacy Perl code running in a non-stop (365×24) environment across scores and indeed hundreds of machines spread all over the globe.
Whenever anything anywhere goes wrong in this code base, or if it needs enhancement, or if it simply needs explaining to people who don’t understand Perl very well, I’m the wizard-in-residence who gets tapped to explain what’s really going on and how one might go about enchancing, revising, fixing, or refactoring the old code.
Some of this code is as robust and sophisticated as the best of CPAN, written with a modular, testable, and often object-oriented design. But most of it is not. Much of it is written by very recent college graduates with no formal Perl training who just copied existing code without understanding it, making tiny tweaks along the way. So you see the same bad memes spread like a virus. Think of the bad old “script-kiddy” days of the very start of all the Perl CGI code people were writing back in the early 90s, and you’ll have a pretty good idea of what this code looks like. It is . . . astounding.
One enormous problem is that the code is written very un-idiomatically — even anti-idiomatically — as far as Perl norms are concerned. Often it looks like they’re writing a gigantic shell script or sometimes an ugly C program. They aren’t always very experienced with Unix, so something like the proper care and feeding of errno with all its vicissitudes and caveats completely eludes them, and process-management with fork, waitpid, pipes, and the correct 16-bit handling of $? is often missed.
The worst problem of all seems to be that they don’t have any experience in being able to recognize what natural, idiomatic, and maintainable Perl code even looks like in the first place. They are just not familiar with standard Perl idioms. So you see them doing strange things like foreach(<INPUT>), or resorting to character-by-character processing of strings using index and substr instead of using regexes. Almost all their data structures are flat arrays or hashes, nothing multilevel, and there are no structure/records to speak of. It’s at best Perl4ish.
The — or at least a — Solution(?)So here is what I want to do, and my question for my fellow monks. I want to develop a highly tuned set of perlcritic policies to help provide automated advice for when they write new Perl code. This would consist of only a subset of the available Perl::Critic::Policies::* on CPAN, and even those would be tuned. That’s because I disagree with the severity levels of many of them, and with the very premises of others.
But some are really very helpful. I especially like the ones involving complexity metrics, both the McCabe stuff and the rest of it.
Question 1: Are there useful P::C Policies that are not on CPAN that I should be aware of?It seems like other people must have made their own tweaked policies. Where are these? Are they hidden off on DarkPAN somewhere?
Question 2: Where are the policy sets that correspond not to Perl Best Practices, but rather to Programming Perl?In Chapter 21 of the fourth edition of Programming Perl, “Common Practices”, there are three sections that very much lend themselves to conversion into P::C policies. These are:
Some of those are of course conflicting, especially in the Efficiency section. But many are quite straightforward (and uncontroversial), and so should be easily enough coded up into separate policies.
But I can’t find any Camel policies for P::C. Do these not exist? Or are they hidden on DarkPAN? Surely someone somewhere must have begun work on this particular project of turning the relevant parts of Chapter 21 in perlcritic policies.
Or just maybe, since the book has only been out a couple of years in its current incarnation, this hasn’t been started yet. Would anyone out there be interested in such a set of policies being available?
I recognize that some of those “Style” matters fall more within the realm of perltidy than of perlcritic. That’s ok: I already have a perltidy config to help establish a standardized look for our new code base. It’s perlcritic policies that are giving me trouble right now, in that there just aren’t enough of them, and not the right kinds.
Question 3: Where are the Unicode P::C policies?I notice that the triadic open policy completely ignores any advice to include the Encoding layer. That seems like a mistake. Specifying the encoding is so important that we included it as early as page 19 in the current Camel. It’s something that people really need to pay attention to.
That does remind me of something else. Chapter 21 doesn’t include any of the advice from my famous Stack Overflow Unicode advice list. There really needs to be a P::C policy set for all that stuff, too.
Question 4: How best to set up a project-wide set of defaults for perlcritic?I have not yet read perlcritic’s documentation in full, but I don’t quite see how to have a per-project set of defaults. I can see how to have per-user overrides, but I’m trying to figure out the best “big project” approach. Is the best way to set the PERLCRITIC envariable to the project-specific config file? But what happens if individual users want to override those settings?
I guess what I would like is for it to do the project one first, the ~user one second, and the per-directory one third, with later values overriding earlier ones. But it looks to me like it can only read at most one config file, not a chain of them with priority. Am I misunderstanding or missing something?
Let’s Start a Project!
I would really like to see a set of Camel policies, plus various others. I also recognize that coding up all those policies is a lot of work, stuff I myself don’t have time for. That’s why I am hoping that someone else has already done so, or at least begun that work.
If not, then let’s please get that project going. Is this something that volunteers alone would be able to tackle? Or should we try to secure a funding grant to help pay for the development work?
I never, ever thought I would want anything like this, but if you were stuck with a few million lines of Perl Worst Practices code, I think you too might start thinking along these similar lines.
Thanks for all your ideas and help, suggestions and criticisms alike.
In reply to Where are the Perl::Critic policies for the Camel? by tchrist