http://www.perlmonks.org?node_id=946090

docster has asked for the wisdom of the Perl Monks concerning the following question:

Hello all. I am attempting to write a web crawler in Scrappy. All I can find are posts on how easy it is to do, but not really how to... To me, it is like swimming through mud. I seem to have one issue after another with it. I thought maybe I had some corrupt perl modules so I tested it on a couple of different machines, Mac and Ubuntu Linux. They act the same. In this script it is the user_agent. This is pretty much directly from cpan. What am I missing? ( Posts to stop breathing and die will be ignored. ):
#!/opt/local/bin/perl use strict; use warnings; use Scrappy qw/:syntax/; user_agent random_ua; my $url = 'http://google.com'; my $scraper = Scrappy->new; $scraper->get("$url"); print $scraper->domain; # print www.google.com __END__ This script returns: Can't locate object method "user_agent" via package "random_ua" (perha +ps you forgot to load "random_ua"?) at ./scrappy.pl line 5.