I assume those non-perl programs are binaries? In some recent testing I did in image manipulation, a pure C solution linked to libgd was about 50 times faster than a perl solution linked to GD.pm. Perl scripts, by nature, are slower than binary applications. Perl scripts need to be interpreted. There are entire layers of logic between how you program it and how the system eventually executes the code eventually generated by the underlying endgine. It all depends on how your code is written :) Maybe it's time to grab Devel::DProf
and have a field day tweaking :)
... Quidquid perl dictum sit, altum viditur.