I did not find a solution to this problem, but I guess that the reason is that there are too many modules which
requires too much interaction (as they are asking questions, don't compile cleanly on Solaris, requires GNU-tar or
I did end up with a long list of modules to install, a perl-wrapper setting up the environment and doing iterative cpanp installs, with some helper-methods (e.g. Net::SSLeay asks Do you want to run external tests?, the helper answers yes\n ).
I will elaborate on this script, as I hope for automatic installation of the same bunch of modules on several different platforms, but for now I will have to settle with cpanp -o followed by a number of cpanp -i.