Well, given that so much of CPAN.pm is still based on globals, you can do this:
perl -MCPAN -e "$CPAN::Suppress_readline=1; shell"
Eventually, that should probably get moved to (yet another) configuration option. Please consider opening up an RT ticket about it if there isn't one already so we have it on the list of things to do.
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.