Possible. Just give it the appropriate configuration options when compiling Perl. I'd guess the appropriate options are mentioned and described in the INSTALL that comes with the core tarball.
If you get your Perl from a third-party vendor (ActiveState, RedHat, ...), you should talk to them and see if they'll compile one for you.
Having said that, what ought to work (but I won't make any promises) is:
$ rm -rf $LIBDIR/site_perl/<old-perl-version>/<archdir>
$ mv $LIBDIR/site_perl/<old-perl-version> $LIBDIR/site_perl/<new-perl-
+version>
