There is a package 'libtest-simple-perl' (v1.001) for Ubuntu.
@Weebo: Why don't you simply 'aptitude install libtest-simple-perl' instead of manually building the CPAN-version? Do you ~need~ the most recent CPAN version? Why don't you use Test::More right from the start? It's quite likely you will end up using it at some point anyway...
@toolic: The package libtest-simple-perl is a bit strange because usually core modules are not packaged separately.(At least the ones i found useful. Probably someone is going to point out at least 6 other core-packages a microsecond after posting this. ;) ) For the record: Test::More is NOT packaged separately.