in reply to How to deal with Huge data
You could use the Perl Data Language (PDL). PDL compactly stores and speedily manipulates large N-dimensional data arrays. For information on PDL, you could the book: PDL -- Scientific Programming in Perl. For a list of resources related to PDL, you could check The Perl Data Language (PDL): A Quick Reference Guide. Finally, some examples for statistics are available in: Using Perl for Statistics: Data Processing and Statistical Computing
I hope this information helps.