|Perl: the Markov chain saw|
XML-Twig: more efficient tree processingby zuma53 (Beadle)
|on Aug 07, 2012 at 16:29 UTC||Need Help??|
zuma53 has asked for the
wisdom of the Perl Monks concerning the following question:
I've written a script using XML-Twig and it works as planned. But as I am running the script I see a huge amount of memory being used and the processing time taking longer and longer. This is particularly true when the XML tree gets larger and larger.
I'm processing the tree via TwigRoots (no manipulations) by performing a top-down scan using find_nodes, then foreach'ing each of the returned array items. Once I find <typeC> in the tree, everything above the node is irrelevant. Within <typeC>, as I loop through and process it's children, I can purge each child when I proceed to the next one.
Is there a way to do this during the foreach loop?
I've tried purging/disposing of the tree when I am done using it, but none of the memory gets freed.
I guess what I'm looking for is a way to treat the XML as a file, where I can read a line at a time and process it without gulping the whole into memory first (which is what I think Twig is doing).