"I don't use perlbrew, but I expected it would be possible to configure it such that *all* perls it builds use the same .cpan location."
Nope! But this should not be a problem ... why? Because why are you installing multiple Perls? I install them so i can test the installation of my CPAN releases and as such, NOT having the same modules shared across multiple perls actually is a benefit to me, because i see what a completely fresh install of my module entails. (Did i really get that dependency correct?)
As far as provisioning your installed perls via perlbrew ... just keep a record of the CPAN modules you use and install them all with one fell swoop: cpanm -n Dancer WWW::Mechanize YAML JSON etc. (and yes, i recommend skipping CPAN module tests because many authors do not care how long their tests take to complete.) This is where your Puppet/Chef/cfEngine3 manifest comes into play ... you do have one, don't you? :)
(the triplet paradiddle with high-hat)