Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Reducing the memory usage of Spreadsheet::ParseExcel

by Anonymous Monk
on Jan 21, 2013 at 20:03 UTC ( #1014511=note: print w/ replies, xml ) Need Help??


in reply to Reducing the memory usage of Spreadsheet::ParseExcel

The 45MB file is compressed on-disk. Why are you particularly alarmed that it might take 300MB to represent in memory .. and, why does such a "small" amount concern you?


Comment on Re: Reducing the memory usage of Spreadsheet::ParseExcel
Re^2: Reducing the memory usage of Spreadsheet::ParseExcel
by Neighbour (Friar) on Jan 23, 2013 at 07:39 UTC
    I'm not sure if .xls files are compressed as well, but I like to keep the memory-footprint of my applications small. If the same amount of data requires (much) less memory when loaded from other types of file (fixed length files, csv files, mysql database), then I'm going to complain when .xls needs more :)

      then I'm going to complain when .xls needs more :)

      MS-Excel itself probably needs that much

        I've used Spreadsheet::ParseExcel::Stream instead of Spreadsheet::ParseExcel and it uses 1.0GB memory after parsing an xls-file with a single worksheet and 64772 rows and 120 columns. This is before I go through the parsed worksheets and rows.
        Opening this in Excel (2007) seems to use only 230MB.
        So even with this module, reading a somewhat large Excel file uses excessively large amounts of memory. Edit: The xls-file is 122MB.
        print("Before creating new Spreadsheet::ParseExcel::Stream:\n" . `free + -m`); my $ExcelParser = Spreadsheet::ParseExcel::Stream->new($FileName); print("After creating new Spreadsheet::ParseExcel::Stream:\n" . `free +-m`);
        results in
        Before creating new Spreadsheet::ParseExcel::Stream: total used free shared buffers cac +hed Mem: 3011 1250 1761 0 2 +177 -/+ buffers/cache: 1071 1940 Swap: 956 177 779 After creating new Spreadsheet::ParseExcel::Stream: total used free shared buffers cac +hed Mem: 3011 2257 753 0 2 +177 -/+ buffers/cache: 2077 933 Swap: 956 177 779

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1014511]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (7)
As of 2014-10-31 06:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (215 votes), past polls