Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

I do hope you meant XML::LibXML::SAX. The thing is that what's normally meant under XML::LibXML is a DOM style parser, that is something that slurps the whole XML into memory and creates a maze of objects. In case of XML::LibXML the objects reside in the C land so they do not waste as much space as they would if they were plain Perl objects, but still with a huge XML this is not a good candidate. Even if the docs make some sense to you.

If perl_gog can convince some HTTP library to give him a filehandle from which he can read the decoded data of the response, he could use XML::Rules in the filter mode and print the transformed XML directly into a file with just some buffers and a twig from the XML kept in memory. Of course he'd have to make sure he doesn't add a rule for the root tag as that would force the module to attempt to build a datastructure for the whole document before writing anything! Feeding chunks of the file to XML::Rules is not (yet) supported. Seems it would not be hard to do though, XML::Parser::Expat has support for that.

Update 2012-09-27: Right, adding the chunk processing support was not hard. I did not release the new version yet as I did not have time to write proper tests for this and one more change but if you are interested you can find the new version in the CPAN RT tracker. The code would then look something like this:

... $parser->filter_chunk('', "the_filtered.xml"); $ua->get($url, ':content_cb' => sub { my($data, $response, $protocol) += @_; $parser->filter_chunk($data); return 1 }); $parser->last_chunk();

Enoch was right!
Enjoy the last years of Rome.

In reply to Re^2: Best way to Download and Process a XML file by Jenda
in thread Best way to Download and Process a XML file by perl_gog

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others wandering the Monastery: (8)
    As of 2018-07-20 16:33 GMT
    Find Nodes?
      Voting Booth?
      It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?

      Results (438 votes). Check out past polls.