|laziness, impatience, and hubris|
WWW::Scripter eval of JQueryby ait (Friar)
|on Mar 05, 2012 at 21:52 UTC||Need Help??|
ait has asked for the
wisdom of the Perl Monks concerning the following question:
I am trying to learn WWW::Scripter to scrape some JS intensive pages. I tried evaling the JQuery.js from a page received with $mech->get. Since that didn't work I tried a simple example and this doesn't work either. Maybe JQuery needs a loaded DOM for this to work?
Any help greatly appreciated...
To further expand on the comment below by tobyink, the fact that it can't parse JQuery is by no means a limitation of WWW::Scripter, it just means that I was trying to use a great tool for the wrong job.
In my particular case I need a full-blown rendering engine so there is not getting around the use of something like Gecko, whether it's using Xvfb, Crowbar, offscreen or something along those lines. I would definitively try to keep everything within WWW::Mechanize, after all, the final scraping is very effective with the tools provided in that namespace, so probably WWW::Mechanize::Firefox and Xvfb is the way to go. Expect another update when I get one of these working...