IIRC perl 5.8 changed the hash function... It may be that it's performing differently for your key set in such a way that more buckets are needed to store the values (arguably a good thing), to the point where Perl cannot allocate a contiguous chunk of memory for the bucket array (it probably needs to do that but I'm not sure).
in reply to Hashing Memory Usage
Another problem could be that the new machine is shipping with soft/hard ulimits set differently by default. Check ulimit on the command line before running. Tweaking the hard limits may require a kernel recompilation.