Re: Reducing the memory usage of Spreadsheet::ParseExcel

[Anonymous Monk]

The 45MB file is compressed on-disk. Why are you particularly alarmed that it might take 300MB to represent in memory .. and, why does such a "small" amount concern you?

[Neighbour]

I'm not sure if .xls files are compressed as well, but I like to keep the memory-footprint of my applications small. If the same amount of data requires (much) less memory when loaded from other types of file (fixed length files, csv files, mysql database), then I'm going to complain when .xls needs more :)

[Anonymous Monk]

then I'm going to complain when .xls needs more :)

MS-Excel itself probably needs that much

[Neighbour]

I've used Spreadsheet::ParseExcel::Stream instead of Spreadsheet::ParseExcel and it uses 1.0GB memory after parsing an xls-file with a single worksheet and 64772 rows and 120 columns. This is before I go through the parsed worksheets and rows.
Opening this in Excel (2007) seems to use only 230MB.
So even with this module, reading a somewhat large Excel file uses excessively large amounts of memory. Edit: The xls-file is 122MB.

print("Before creating new Spreadsheet::ParseExcel::Stream:\n" . `free
+ -m`);
    my $ExcelParser = Spreadsheet::ParseExcel::Stream->new($FileName);
print("After creating new Spreadsheet::ParseExcel::Stream:\n" . `free 
+-m`);

results in

Before creating new Spreadsheet::ParseExcel::Stream:
             total       used       free     shared    buffers     cac
+hed
Mem:          3011       1250       1761          0          2        
+177
-/+ buffers/cache:       1071       1940
Swap:          956        177        779
After creating new Spreadsheet::ParseExcel::Stream:
             total       used       free     shared    buffers     cac
+hed
Mem:          3011       2257        753          0          2        
+177
-/+ buffers/cache:       2077        933
Swap:          956        177        779