|Perl: the Markov chain saw|
Re^5: Scrappy user_agent errorby jethro (Monsignor)
|on Jan 04, 2012 at 10:16 UTC||Need Help??|
Look at the source https://metacpan.org/source/AWNCORP/Scrappy-0.94112090/lib/Scrappy/Scraper/UserAgent.pm, first parameter is browser, second parameter the operating system, i.e. you would use something like scraper->user_agent("opera","Macintosh")
PS: As the others have said, if the sites doesn't want robots, you shouldn't just ignore that.