note BrowserUk <blockquote><i></i></blockquote> <p>The first thing to say is that that is not valid XML. (A valid XML document must contain a single top level tag.) <P>That said, for the purposes of processing, that (arbitrary) XML rule works in our favour and makes writing a program that processes the large file in smallish chunks very simple:<code> #! perl -slw use strict; use XML::Simple; use Data::Dump qw[ pp ]; $/ = '</object>'; while( <DATA> ) { last if /^\n+$/; my $xml = XMLin( $_ ); pp $xml; } __DATA__ <object some_param="abc" other_param="def"> <attrib1>val1</attrib1> <attrib5>val3</attrib5> </object> <object some_param="xxx"> <attrib3>valx</attrib3> <attrib7>valy</attrib7> </object> <object some_param="xyz"> <attrib1>valx</attrib1> <attrib2>valy</attrib2> <attrib3>valx</attrib3> <attrib4>valy</attrib4> <attrib5>valx</attrib5> <attrib6>valy</attrib6> <attrib7>valx</attrib7> <attrib8>valy</attrib8> </object> </code> <P>That produces:<code> C:\test>t-XML.pl { attrib1 => "val1", attrib5 => "val3", other_param => "def", some_param => "abc", } { attrib3 => "valx", attrib7 => "valy", some_param => "xxx" } { attrib1 => "valx", attrib2 => "valy", attrib3 => "valx", attrib4 => "valy", attrib5 => "valx", attrib6 => "valy", attrib7 => "valx", attrib8 => "valy", some_param => "xyz", } </code> <p>In addition to that allowing the huge file to be processed very quickly in minimal memory, it would -- were the processing requirements of the individual chunks sufficiently taxing to warrant it -- enable multiple individual chunks to be processed in parallel with threading very easily. <P>But, if the example is anything like representative of the actual data, that above code will probably allow the entire file to be processed sufficiently quickly -- in a very casual test; less that 2 minutes -- that the need for considering threading disappears completely. The saving coming simply from processing the file in small chunks rather than en masse. <div class="pmsig"><div class="pmsig-171588"> <hr /> <font size=1 > <div>With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'</div> <div>Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.</div> <div>"Science is about questioning the status quo. Questioning authority". </div> <div>In the absence of evidence, opinion is indistinguishable from prejudice. <p align=right> [http://thebottomline.cpaaustralia.com.au/|RIP Neil Armstrong]</p></div> </font> </div></div> 1005623 1005707