in reply to memory issues
Why is this happening?
You wrote it that way, you're storing that much data in memory
Observe
$ perl -MDevel::Size=:all -le " @F = 1 .. (5 * 30 * 1024 ); @F{@F}=@F +; print total_size($_) for \@F, \%F " 6758436 13225503
5 files, 30k lines each, stored in hash, and stored in array, 6.5MiB and 13MiB respectively ( 19.5MiB combined)
You actually store three times as much data, mostly duplicated
The solution, store less data, store data on disk, get a better system, or lift ulimits on your account
In Section
Seekers of Perl Wisdom