Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

XML::Simple is there a maximum xml file size?

by webchalkboard (Scribe)
on Jan 12, 2006 at 15:42 UTC ( #522716=perlquestion: print w/replies, xml ) Need Help??

webchalkboard has asked for the wisdom of the Perl Monks concerning the following question:

Hi,
Just wondering if anyone knows if XML::Simple has a maxiumum file size it works with? I have a little script which seems to work fine when I take the first few records out of it, but when I try and run the script over the whole 5.1mb XML file it just does nothing... I can't even make it give me an error message, i'm running CGI::Carp 'fatalsToBrowser' and this doesn't print any help at all. It just doesn't work and dies without error on the my $xml = XMLin($xmlfile); line.

Any help would be appreciated,
Thanks
Tom

Learning without thought is labor lost; thought without learning is perilous. - Confucius
British Art

Replies are listed 'Best First'.
Re: XML::Simple is there a maximum xml file size?
by monkey_boy (Priest) on Jan 12, 2006 at 17:05 UTC
    The maximum file size depends on your RAM size, XML::Simple creates a complex data structure, which in perl terms, will be much larger than the xml files size.
    The other thing to note is that XML::Simple can be very slow on complex/large xml documents, so it may be that your browser is timing out, before the parsing is finished,

    Id try running the script on the command-line to see if you get an "out of memory error" (from the OS), or to time how long the parsing takes.



    This is not a Signature...

      I'd like to second what monkey boy says in his second paragraph.

      Write a three line script that does nothing but open that file with XML::Simple, then you know it's not something else causing trouble. Make it work in the simplest possible environment, then put the solution into your working CGI.

Re: XML::Simple is there a maximum xml file size?
by aufflick (Deacon) on Jan 12, 2006 at 23:33 UTC
    I've always found XML::Twig to be excellent for parsing large XML files - it also works on streams (ie. infinte sized XML files :)
Re: XML::Simple is there a maximum xml file size?
by johnnywang (Priest) on Jan 12, 2006 at 19:24 UTC
    Just want to second what's been said, my experience with XML::Simple is that it's perfect for parsing such things as xml configuration files which are usually very small and not complex, try other parsers or regex if yours is either big or complex. In the two cases where I started with XML::Simple, both ended up with simple regex.
Re: XML::Simple is there a maximum xml file size?
by holli (Abbot) on Jan 12, 2006 at 17:41 UTC
    over the whole 5.1mb XML
    Lucky you ;-) At work I have to process XML files that are five Gigabytes (!) and bigger. I can only praise the lord that they are program generated and have tight schemes, so I can do lots of stuff (safely) using regexes and state machines.


    holli, /regexed monk/
      You must be working for the government. The only place I've seen that thinks 5GB of data is best formatted in a single XML file.
Re: XML::Simple is there a maximum xml file size?
by etm117 (Pilgrim) on Jan 13, 2006 at 15:55 UTC
    The parser XML::Simple underneath the hood uses makes a difference also. If you leave it at the default, it will use XML::SAX. XML::SAX by default will use XML::SAX::PurePerl. As implied by the name, it is an XML parser written in Perl.

    For faster results, try installing XML::LibXML. You will need to install the library that it uses underneath the hood (libxml2) and it can be found here.

Re: XML::Simple is there a maximum xml file size?
by webchalkboard (Scribe) on Jan 13, 2006 at 12:56 UTC

    Hi,
    Thanks for the responses, i've run the script on the command line and get an error now.

    End tag mismatch (venue != primaryAct) [Ln: 2438, Col: 64]

    There must be something wrong with the structure of the XML file, i'm not sure. I will try and use the other module suggested and see if that works any better.

    Learning without thought is labor lost; thought without learning is perilous. - Confucius
    British Artists

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://522716]
Front-paged by gargle
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2021-10-21 21:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (85 votes). Check out past polls.

    Notices?