Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Memory errors while processing 2GB XML file with XML:Twig on Windows 2000

by mirod (Canon)
on May 15, 2005 at 20:07 UTC ( [id://457284]=note: print w/replies, xml ) Need Help??


in reply to Memory errors while processing 2GB XML file with XML:Twig on Windows 2000

You don't give us much to work with. Some code and perhaps the amount of RAM on your system would help.

If you do a straight XML::Twig->new->parsefile( 'my_big_fat_xml_file.xml');, then the resulting data structure should need somewhere around 20GB. That's why XML::Twig let's you process a file one chunk at a time, and purge the memory when you're done with it.

The README for the module (at least for the latest version) includes links to lots of resources about the module. You could start by looking at xmltwig.com.

  • Comment on Re: Memory errors while processing 2GB XML file with XML:Twig on Windows 2000
  • Download Code

Replies are listed 'Best First'.
Re^2: Memory errors while processing 2GB XML file with XML:Twig on Windows 2000
by nan (Novice) on May 16, 2005 at 11:03 UTC
    Hi,

    If you don't mind, please refer to the codes and XML segment in my replies to other people. My RAM is 1GB and my perl version is the latest. Initially, I have a virtual memory error too but it's solved after I changed virtual memory to the maximum (4GB), now I only have experienced a memory writen error. Do you think it could be better if I try linux?

    Thanks again,

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://457284]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (5)
As of 2025-01-22 19:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Which URL do you most often use to access this site?












    Results (63 votes). Check out past polls.