The reason is that most of your time and energy in programming is going to be spent in maintainance. Most of the time spent by the program will be in a small section of code. Making the code easy to read/understand will assist with getting it written and written correctly up front. It assists with maintainance. This is a good thing.

If the code winds up fast enough, then you never have to think about performance. If it winds up too slow, then you can profile it, figure out what small section is slow (frequently not what you thought - consider how the naive sort method performed in the above benchmarks), and then optimize only that to fix that problem. If it still is too slow, then it is time for a more aggressive strategy - like converting a chunk of your program to C.

In no case is the proper solution to performance to micro-optimize every step of the way.

For further reference read what Code Complete says about code tuning. Sure, the examples are in C and statistics are quoted from FORTRAN days. But that is knowledge that ages well and applies for Perl as well as any other language.