I definitely want to suffix this post with all of the extremely-valid points that BrowserUK made in his comment concerning “all the implications of ‘adding an extra stick.’” As his Voice Of Experience™ says, there are a lot of compelling reasons why my “point #1” might not apply to “your case, whatever it is.”
My essential point ... which was intended to be more generic ... is basically this: “either you must buy the hardware necessary to make [most of ...] the data really be “in memory,” or you must redesign your approach to demand less RAM. The one thing that you cannot do, IMHO, is to remain where you are. The operational characteristics of both a hash-table and Perl’s memory-manager work cripplingly against you if physical memory is actually “grossly overcommitted,” as is the case here.
Effectively, the storage-pool that is being accessed by your application is “disk” ... not “memory.” And the operating system’s virtual memory manager is being asked to “graciously do” ... the impossible. It simply isn’t credible to pretend that the memory resource is “actually ‘real,’” when it is more than 200% over-committed.
The application’s fundamental design, unfortunately, is flawed. However, if you know that the problem-to-be-solved never will “grow ten-fold,” it certainly will be fast! Hence, my suggestion to seriously explore “throwing silicon at it.” (Even $10,000 worth of hardware is less expensive than $10,000 worth of developer-time.) However, if the problem might grow, you are ... ahh ... scr00d.™ You’re going to have to re-design this thing, maybe from the ground up.