note
tobyink
<p>150 GB? Ouch.</p>
<p>[mod://AnyEvent::HTTP] should allow you to issue an HTTP request, and process it a chunk at a time, while it arrives, without having to save it anywhere.</p>
<p>And [mod://XML::Twig] can parse XML chunk by chunk.</p>
<p>Pairing the two you ought to be able to do this without temporary files. Exactly how to do it, I can't help you. I have limited experience with AnyEvent::HTTP; and virtually none with XML::Twig.</p>
<!-- Node text goes above. Div tags should contain sig only -->
<div class="pmsig"><div class="pmsig-757127">
<small><small>
<tt>perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
</tt></small></small>
</div></div>
995446
995446