Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

XPath searching too slow

by dietbuddha (Sexton)
on Jul 31, 2008 at 17:08 UTC ( #701474=perlquestion: print w/replies, xml ) Need Help??

dietbuddha has asked for the wisdom of the Perl Monks concerning the following question:

Is there some modules we can swap out HTML::TreeBuilder and XML::XPathEngine which are either faster or will at least push more of the cost into the parsing and creation of the tree instead of the searching?

We've recently started testing our dynamically generated HTML. However it's taking much longer now for the test suite to complete. We did some profiling on our slowest test file and found our 95% percent of the time spent in XML::XPathEngine which is the searching we do. However, less than half the tests are HTML related.

Replies are listed 'Best First'.
Re: XPath searching too slow
by mirod (Canon) on Jul 31, 2008 at 17:52 UTC

    First, are your queries optimized? It is very easy with XPath to write a very expensive query that needs to traverse the entire tree (think //p). Then are your tests separate processes? XML::XPathEngine caches the parse tree for queries, but only within the same process of course. That might be one reason why the tests run slower.

    That said it the XPath engine _is_ rather slow. Patches or replacement engines are welcome ;--) One way I can think of, to speed it up, might be to allow placeholders instead of values in the queries.

Re: XPath searching too slow
by moritz (Cardinal) on Jul 31, 2008 at 17:17 UTC
    XML::LibXML can search for XPaths, and since it's mostly a wrapper (a very good one, that is) around a C library it could be quite fast. But of course you'd have to benchmark that first.

    If you need to make many queries, you can also build relational indexes for xpath, but that's likely to require quite a lot of work.

    Or maybe you could just use a different templating technique.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://701474]
Approved by moritz
Front-paged by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2020-07-04 05:16 GMT
Find Nodes?
    Voting Booth?

    No recent polls found