Hello,

I am trying to learn WWW::Scripter to scrape some JS intensive pages. I tried evaling the JQuery.js from a page received with $mech->get. Since that didn't work I tried a simple example and this doesn't work either. Maybe JQuery needs a loaded DOM for this to work?

#!/usr/bin/env perl

use warnings;
use strict;
use HTTP::Cookies;
use WWW::Scripter;

my $cookie_jar = HTTP::Cookies->new();

my $mech = WWW::Scripter->new(
  agent => 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident
+/5.0',
  cookie_jar => $cookie_jar,
);

$mech->get("https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.
+min.js");
my $jquery = $mech->content;

$mech->use_plugin(
  JavaScript =>
    engine  => 'SpiderMonkey',
);

$mech->eval($jquery);

[download]

I get "c has no properties at WWW::Scripter::Plugin::JavaScript::SpiderMonkey line 165"

Any help greatly appreciated...

Update 20120306

To further expand on the comment below by tobyink, the fact that it can't parse JQuery is by no means a limitation of WWW::Scripter, it just means that I was trying to use a great tool for the wrong job.

In my particular case I need a full-blown rendering engine so there is not getting around the use of something like Gecko, whether it's using Xvfb, Crowbar, offscreen or something along those lines. I would definitively try to keep everything within WWW::Mechanize, after all, the final scraping is very effective with the tools provided in that namespace, so probably WWW::Mechanize::Firefox and Xvfb is the way to go. Expect another update when I get one of these working...