|Perl Monk, Perl Meditation|
Issue on covariance calculationby Mandrake (Chaplain)
|on Apr 13, 2007 at 08:03 UTC||Need Help??|
Mandrake has asked for the wisdom of the Perl Monks concerning the following question:
I am in the process of building a huge covariance matrix( around 10000 x 10000 elements). Since covariance matrix is a symmetric matrix, I attempt to create only one half of the matrix.
The input the script is an array of comma separated values
The relevant part of my script :
The issues with my script.
It took almost an hour for the first 2500 (out of 10000) records to get generated / written to output file. I badly want to optimize the performance. Can you wise ones give some suggestions?
2. Storage space.
The file size of the output file would be almost a GB (or even more). Sorry if this does not make sense but I was given a suggestion that instead of writing the data as text, if we write it as binary it would save space. I mean, 0.09992020202 would take 14 bytes of space. Instead of character if we write it as float, it should take less space. Is this idea possible to be implemented in Perl ?
Many thanks for your time.