in reply to
Re: XML::Twig and threads
in thread XML::Twig and threads [solved]
XML is very simple. I cannot share it as it is company confidential, but it is just like:
<object some_param="abc" other_param="def">
Many objects (1752 in 93MB file) and each object has list of attributes (up to 700 in 93MB file).
He further clarified that his concern is yet something else, namely he reads the file into memory, does alterations to some params and writes back to another file. This altered data is used to test the system - e.g. 150 different versions of 10MB file written to one file which is then 1.5GB -> so if we can manage inserting threads into managedObject => \&handle_fasade function it may be really of some help while producing output.
Simple program reading 100MB XML file took 2 minutes and 3.5GB RAM, I think his 30 hours may be out-of-physical memory problem. I'll add more details tomorrow.