in reply to XML::Twig and threads [solved]
he wants anyway to add threads to this script and speed it up.
Tell him it simply will not work. XML::Twig uses an OO interface, and sharing objects between threads, whilst possible, will never speed things up. This is because the time cost of accessing shared memory (in Perl) are far higher than accessing private memory.
If you (he) would care to share a (small) sample of the XML in question that shows the repetitive structure, then there is probably an an effective way to process it in parallel, by breaking up the top level using non-XML parser techniques and then using several non-shared XML parser instances withing threads. But it will be necessary to see a realistic sample to advise further.
The real problem here is that the design of XML requires that an XML document be treated as an indivisible entity, which -- if you stick to the XML parsing rules -- makes parallel processing of XML not just difficult, but impossible. By design. It is lamentable that XML has become so ingrained in peoples psyches.
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: XML::Twig and threads
by grizzley (Chaplain) on Nov 26, 2012 at 16:15 UTC | |
by BrowserUk (Patriarch) on Nov 26, 2012 at 16:36 UTC | |
by remiah (Hermit) on Nov 27, 2012 at 09:01 UTC |