Beefy Boxes and Bandwidth Generously Provided by pair Networks Cowboy Neal with Hat
Perl: the Markov chain saw
 
PerlMonks  

Delete parse data spreadsheet::parseexcel

by cibien (Novice)
on Feb 05, 2013 at 18:43 UTC ( #1017249=perlquestion: print w/ replies, xml ) Need Help??
cibien has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, sorry for my precedent wrong question....

Is possible delete parsed data (with spreadsheet::parseexcel) from ram memory?

Because my a script parses a series of Excel files, and write data I need with XML::LibXML in a xml file. The logic works fine but my problem is that I keep running out of ram memory (4gb). I'm processing the files sequentially and I'm losing a lot of memory with parsing each new file. Now I have over 30 big excel files and after parsed 5 excel file the ram memory is full and process very slow. I need delete the parsed data stored in ram memory just before start parsing the second excel file, and the same for each excel file. Or I need alternative cell handling techniques that store only one excel file at once to limit memory usage..

Thankyou very much for your help

Andrea

Comment on Delete parse data spreadsheet::parseexcel
Re: Delete parse data spreadsheet::parseexcel
by Kenosis (Priest) on Feb 05, 2013 at 18:51 UTC
      Thanks, I try Spreadsheet::ParseExcel::Stream, but is complicated for me... http://www.perlmonks.org/?node_id=1015095
        but is complicated for me

        I don't get it...it is simpler to use...if you have trouble using something simpler, what hope is there?

Re: Delete parse data spreadsheet::parseexcel
by moritz (Cardinal) on Feb 05, 2013 at 18:58 UTC

      Sorry but I'am novice in perl....

      how I do "simply stop keep a reference to the parser object" in spreadsheet::parseexcel?

      Thanks

      simply stop keep a reference to the parser object. Then the memory allocated for that object is freed.

      No, not for Spreadsheet::ParseExcel objects...they contain circular references (workbooks reference worksheets and vice versa, etc.), so they don't go away until the end of your program. Best thing to do is to parse each spreadsheet in a separate process (using Parallel::ForkManager or just plain fork or just launch a separate system command or something).

        they contain circular references (workbooks reference worksheets and vice versa, etc.), so they don't go away until the end of your program

        If that's really the case, please open a bug report.

        However the source does use Scalar::Util::weaken, and my very basic testing shows that the objects do get released when I set the reference to the workbook to undef.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1017249]
Approved by Kenosis
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (6)
As of 2014-04-16 11:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (424 votes), past polls