Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re^3: Processing LARGE text files

by thedoe (Monk)
on Mar 07, 2006 at 21:19 UTC ( [id://535035]=note: print w/replies, xml ) Need Help??


in reply to Re^2: Processing LARGE text files
in thread Processing LARGE text files

I notice you put in your example: <DELIMITER>(.*?)<\/DELIMITER>. Is this because you are working with very large XML files? Or is this simply your way of seperation?

The reason I ask is because I have recently dealt with very large XML files, and found XML::Twig to be very helpful. You can read in smaller chunks of XML data at a time. You can then process it with the same ease as a tree based parser, such as XML::Simple. Once you are done processing that chunk, simply either flush (which prints the chunk) or purge (does not print) the data, freeing the memory.

Replies are listed 'Best First'.
Re^4: Processing LARGE text files
by Craig720 (Initiate) on Mar 08, 2006 at 14:55 UTC
    The delimiters are words in angle braces such as <BOUNDARY> and </BOUNDARY>.

    Can the XML modules you mentioned be rigged to operate on very large text files containing non-standard XML? My experience with XML is minimal.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://535035]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (5)
As of 2024-04-23 16:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found