Perl features that make Perl a good first language:

• marking of variables with symbols (% @ $)
• automatic garbage collection
• simple set of types -- only array, hash, and scalar
• expressive syntax (well, generally)
• availability of modules via CPAN

Perl features that make Perl a poor first language:

• strict and warnings are off by default
• "tacked on" object support
• confusing "config" variable names (e.g. $/)

Of course, good instruction can take care of all the negatives.

Personally, as much as its blasphemy here, I think Python is one of the best languages for learning about programming -- that's a good bit of what it was designed for, after all. That said, though, one could certainly do much worse than Perl.

And don't get me started on introducing programming by using Java -- I think that idea is perpetuated by instructors that want to keep people out of the field.