Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

XML::Twig size problem

by slugger415 (Scribe)
on Jun 23, 2012 at 00:21 UTC ( #977909=perlquestion: print w/replies, xml ) Need Help??
slugger415 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, I have a CGI script using XML::Twig to parse <row> elements an XML file. I've discovered through a lot of testing that I get a "500 Internal Server Error" in the browser (Firefox) if the number of rows is too big (more than 4689 to be exact). I know it's not a problem with row #4690 in the file because I've tested it on different files and it crashes at the same number of rows.

I checked the HTTP server's error log and saw this:

Premature end of script headers:

I'm using both the CGI and XML::Twig Perl modules. Is it running out of memory?

Is there a way to capture the error (other than what's in the server log) when Perl fails? The "500" error doesn't tell me much.

Here's the little bit of code that parses the file, and prints HTML when done:

use CGI; use CGI::Pretty; use CGI::Carp qw( fatalsToBrowser ); use XML::Twig; use strict; # set up the parser: my $twig= XML::Twig->new( twig_handlers => { row => \&row_processing }, ); $twig->parsefile("$outputDir/my-zos-shorter.xml"); print $q->header, $q->start_html(-title=>$TITLE, -style=>{'src'=>$stylesheet}), "<a name='top'></a>\n", $q->h2('Here are your results!'), $q->end_html;

BTW as a test I also had the row_processing subroutine/handler print every row it finds into a file as it goes along, and it makes it all the way through to the last one, so it's actually parsing everything before it barfs. I am clueless as to what's going on.

Any suggestions on how to troubleshoot this or about what the problem is would be most welcome.

thanks, Scott

Replies are listed 'Best First'.
Re: XML::Twig size problem
by mirod (Canon) on Jun 23, 2012 at 05:07 UTC

    Which version of perl are you using? Up until 5.16.0 there was a bug in the interpretor that made XML::Twig crash when it hit a large number of children. See a description at Re: XML::Twig too many children?.

    Edit: it's early and I haven't had my coffee, so I am not sure I read the post right, so just to be sure: in the handler you do </t>purge</t> the tree right? Otherwise it could be just a case of the memory filling up.

      Hi mirod, I added purge after I posted and am still getting the same result.

      I'm using ActivePerl 5.14.2; I don't see any later downloads on the ActiveState site, so I guess I'm out of luck?

      thanks, Scott

Re: XML::Twig size problem
by Anonymous Monk on Jun 24, 2012 at 07:49 UTC

    Do you declare and create $q anywhere? I don't see it in the code you posted. Did you try testing without any of the XML related code?

    You could also try running the code from the command line? That may make it easier to see what's going on.

      Yes I do declare $q, sorry I didn't include that.

      my $q = new CGI;

      BUT! I was able to install the latest XML::Twig on my ActivePerl 5.14 installation (I unzipped the file with 7-zip and just copied the two .pm files to my site\lib directory) and that seems to have fixed the problem. Thank you all for the suggestions!


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://977909]
Approved by toolic
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2017-03-26 08:23 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (313 votes). Check out past polls.