Often enough, the right tool is the one you know because you know it. It doesn't much matter whether you use a slide rule or a TI-89 Titanium to solve an arithmetic problem so long as you get the right answer.
Of course, there are problems where the tool does matter (which you are well aware of as evidenced by your assertion that you wouldn't use Perl to write an OS kernel.) But they usually aren't that hard to recognize. In the event that they are, and you use Perl but you get slaughtered on performance, all is not necessarily lost anyway. You might have a good prototype.
The things you listed that you've used Perl for seem to me to be fine uses of Perl, though. Perl is great because it's like a Swiss Army Power Tool. It not only saves you time and effort with the software you write but, it saves you the time and effort of learning those 3000 languages.
If it was Prolog you knew instead of Perl, what would you do in vim? Wait for someone to write prologdo? Ever seen QT bindings for awk? Think you could create an Excel report from data in an MS SQL-Server DB using csh?
I'm sure you get the point.
If you really feel the need to learn another language, by all means, go do it! The broad base of development experience is certainly a good thing. Not so much because it helps to know a bunch of languages, but because you tend to learn different ways of looking at problems. But don't be too concerned about relying on what you know. Depth of development experience can be a good thing too. The more things you use Perl for and the better you get with it, the easier it will be for you to recognize its limitations and know when it really is the wrong tool.¹
 Which, of course, is almost never. ;-)
-sauoq "My two cents aren't worth a dime.";