I hate that some stuff just ISN'T there:
The OO system. It can do pretty much everything you want from a class based OO system, but it's too verbose to the point of discouraging good practices (i.e. no good, fast, standard, portable way doing private instance variables and methods sucks). There are good modules that help, but too late - they have tons of dependencies and most code you're working with won't use them. Class::MOP and related modules should have been in the core since 5.0 and perl's built-in collections should have been based upon the standard OO model.
No *real* lisp-style macros. Personally, I think lisp macros are probably not workable in perl anyway, but once you've used them, they're absolutely amazing.
No good inter-thread communication system. Why can't I pass real objects from one thread to another? IMO this makes threads in pure-perl code completely useless. You have to do all the work you have to do in forking code, and the forking code is probably more stable.
Auto-vification has it's issues too, especially with strict/warnings enabled.