Re^2: When is it better to NOT release a new module?

[BUU]

Your module appears to be a source code filter operating on general purpose perl. Such things should never be released. They're evil, nasty things and cause problems for other people.

Ya know, I too once had this opinion. I even asked Ovid about it. I felt his response was enlightening enough to reproduce here:

Now that I have the lying out of the way, I can understand the hesitation about them. I worked for Rentrak for a year and they did something very similar (but on a much smaller scope) to what I'm trying to do, but they did it with Filter::Util::Call. Naturally, when I first started working with this I complained heartily to my programming friends about using a source filter in production code. And not just any old production code, either. We're talking about code that is rightfully called "enterprise class." And you know what? In the entire time I used it, I only stumbled across one bug (and admittedly it was a bugger to track down.) I know what the bug is and it's not going to be in my version :)

So after working with a source filter for over a year, what did I discover? Source filters, when done properly, can make programming much, much easier. I didn't realize how much I missed signatures until I had them again. I also didn't realize how much I missed having an entire class of bugs pretty much disappear because I had signatures. For the one bug that was caused by this source filter, programming was much more productive and bug free.

I was wrong about source filters. I used to think they were terribly evil and should not be used, but now I realize (as with all dogmatism) that this is not universally the case. Admittedly, source filters generally need to be some of the most heavily tested code, but the payoff can be huge.

But serious programmers don't use source filters, right? Don't tell that to Damian and Ingy, two well-known and respected programmers who are quite happy to reach for source filters when that's the solution to a thorny problem. Those, naturally, are two that come to mind but there are plenty of others. Source filters are to be used sparingly, but it's not the case that they should never be used. They are not evil, just dangerous.

This was taken from his journal at: http://use.perl.org/~Ovid/journal/22152. The following discussion also links to this node: Re: Use method/function signatures with Perl which discusses it further.

[diotalevi]

[Edited Its wrong of me to characterize everything done by a particular author this way. There are a small set of examples of bugs in his published work and some things like years passing while bug fixes submitted to rt.cpan.org aren't applied.] Ingy is no example. Bugs in his work are unsurprising and regular. I don't know whether this is related to his use of source filters or not.

I checked Damian's list of categorized modules for stuff that used something from Filter. Here's how it breaks down for his source filtering modules. It works out that while Damian has written several things using source filters, he has only one thing that isn't marked experimental that actually uses source filters that would be available in production. [Edited He writes some fabulous stuff but there isn't any evidence that he's written anything using source filters that he considers worthy of running in production.]

Modules you aren't supposed to use anyway

• Lingua-Romana-Perligata
• Perl6-Currying
• Perl6-Export
• Perl6-Placeholders
• Perl6-Rules
• Switch

Attribute-Handlers-Prospective is the only module that was superceded by other things. You shouldn't use it.

He said Inline-Files is production worthy but its marked as experimental in its pod?

He uses Smart-Comments and Toolkit as development aides. It turns out that you aren't supposed to use Smart-Comments in production anyway - its a design feature that they're easy to remove. So production code doesn't even touch a source filter.

Regarding Ovid's code. Its much easier to have a source filter when you're running it against code you control and with some known code standards. Its much less sane to do that when you're releasing things onto CPAN where it will have to deal with anything that's potentially available.

---

I assert that Ovid's anecdotal evidence does not indicate that source filters are ok for general use on CPAN.

[Perl Mouse]

Its much less sane to do that when you're releasing things onto CPAN where it will have to deal with anything that's potentially available.

Excuse me? Since when is it a requirement that anything released on CPAN needs "to deal with anything that's potentially available"? Answer: never.

CPAN never has, and never will put any requirements on the code stored there, other than its license should allow distribution. If there were such requirements, we wouldn't have a gazillion Class::* modules, many of which don't play nice with each other.

If you don't want to use source filters - don't. Just don't go on a crusade claiming they are universally evil and people shouldn't put modules on CPAN that use source filters. CPAN is open for everyone - and that includes open for code that uses techniques you don't like.

[diotalevi]

I would tend to wish, vainly, that authors would put good faith efforts into doing things that were generally useful and that don't have mysterious failuremodes, that work equally well on multiple platforms (or at least with that intention), that don't fail just because my other perl code didn't look enough like what the author expected. Its a wish. A vain wish. I know. It is unhelpful to have you point out that it is nothing more than a wish and I also wish you hadn't done that. Actually, I think its more than a wish but a bit of a community expectation that authors should at least try for that. I expect that one way of reaching those goals is to avoid such obviously perilous and known impossible jobs. Impossible in the general sense anyway, I'm quite sure that the author's code works more often than it doesn't mostly because I have faith that the author put at least a little bit of effort into it.

People who don't even try to make reasonable stuff for CPAN ought to have something anatomically improbable done to them. I wish them ill and hope their efforts fail everywhere they go.

The only non-parse related method I know to rewrite perl code is to treat perl like lisp and rewrite it at the optree level. At least then there are no ambiguities about how something is parsed - its all been handled by the perl parser. But then, perl is an exceptionally ugly lisp when viewed from that direction.

[diotalevi]

That first response was all pretty irrational as I was just flummoxed and upset that you'd actually make that point to me. Yes, source filters can work. Apparently there's one that POE uses internally. I just resent the proliferation of known buggy techniques and would prefer to give no quarter to them. This happens to be something that's well known as impossible to get right 100% of the time. Maybe its actually pretty high now that stuff like Filter::Util exists. I dunno. I'll likely never reach for it as I don't know of any circumstance where it would be the right enough tool to allow introducing its quantum weirdness into my code.