Re: The cost of unchecked best practices

[kyle]

Let me say first that I find this kind of discussion of internals and performance impact interesting. A character class with one item is not the same as a literal one item. Fascinating!

That said, I find this kind of statement irritating:

Don't apply "Best Practices" without fully understanding the underlying mechanisms.

The "underlying mechanisms" go pretty deep. You seem to be making reference to the implementation of Perl, in C, but there are mechanisms underlying that too. Should one also understand the underlying mechanisms of the C compiler? There's the OS. There's the hardware. There's physical science that explains how the hardware works.

These are all underlying mechanisms. There are performance differences in simple arithmetic from architecture to architecture. There are differences from version to version of the compiler. I don't think that it's worth my time to study all these things to figure out the very best way to implement something. I don't feel obligated to plumb the depths of computing to justify my decision to use one practice or another.

What I think is important is readability and maintainability (after, of course, functionality). The best way to write code is the way that makes it easiest for the programmer and the programmers who come after. At some point, performance becomes a functionality issue. At that point, it's time to do some profiling and optimize where it counts. Before that point, time spent optimizing is time wasted.

Part of the point of best practices is precisely so that one does not have to learn every detail of what underlies the advice given. Call me selfish and lazy if you like, but I want the benefit of others' experiences without paying the price of those experiences myself. I admit those experiences have value that I might perhaps miss sometimes, but they also have a cost in time and energy. Here again, I don't feel obligated to go exploring a place if someone has already made me a map.

[moritz]

The "underlying mechanisms" go pretty deep. You seem to be making reference to the implementation of Perl, in C, but there are mechanisms underlying that too. Should one also understand the underlying mechanisms of the C compiler? There's the OS. There's the hardware. There's physical science that explains how the hardware works.

That's one of the things that makes programming a science and art: you have to know which abstraction levels you care about.

To decide that, you have to know them, at least most of them a little bit.

Let's stay at the regex example: if you do more than just very simple regexes, you have to know that perl's regex engine is a NFA (which means the order of alternations is important), and that it matches greedy by default.

It also helps to know that all kind of anchoring results in a performance benefit.

For some programmers that might be enough to know, for others it isn't.

Suppose you decide for a larger team which of the regex best practices you apply. In that case I think that you should know more about implementation and optimzations.

If you just decide for yourself, then you don't need to know more, but you shouldn't complain about slow regexes when you don't.

Every attempt to say "I'll inform myself until I reach abstraction level $x, and then no more" is bound to fail at some point, and your curiosity should prevent you from stopping at one level ;-)

I agree that most micro-optimizations are not worth having, but runtime differences of two orders of magnitudes are worth noting nonetheless.

Readability shouldn't suffer, so perhaps I should write my regexes as my $space = " "; m/... $space/; or use the \N notation.

IMHO sticking to best practices without understanding them makes coding something mechanical, and poses the risk of cargo cult programming.

My view might be influenced by the fact that programming is mostly a hobby for me, and I want to be fun above all else, and apart from this forum most code that I maintain is my own.