well, i don't know if you are familiar with SQLite (or MySQL), i often get stuck with processing large amount of data, at first i also tried to do it through hashes and arrays, but then i just gave up on it. my advise would be to use one of the freely available databases like SQLite or MySQL. just import data, sort it (order by), and when processing it retrieve line by line of data and do what ever you want to do with it.
this is memory efficient, but certainly slower way to do it
SQLite
example on how to use it through DBI and it would be usfull to check out DBD::SQLite
plus there are tons of concrete examples on perlmonks !!!