Still, even with these precautions, the make install step has the possibility of wrecking havoc to your existing Perl installation, and even if your Perl installation is not your system Perl installation, it's not really safe to blindly install modules from CPAN.
True, but at least with modules with a SIGNATURE we have some vague notion of accountability. Modules that download other code from third parties are when it gets scary for me.
Maybe a nice approach with the Alien:: modules would be for the user to specify trusted sources (sunfreeware.com, fink, etc.) for the third party software. So you would have a CPANesque configuration mode that would allow me to say something like "I'll trust anything available in the stable fink section, otherwise ask me".
Of course - much more work for you :-)