Re: LWP::UserAgent Bad and Forbidden requests

[Anonymous Monk]

(my browser displays them without a problem):

pretend to be your browser

This is the solution to every such problem, see Re^5: can't get WWW::Mechanize to sign in on JustAnswer or http://www.perl.com/pub/2004/06/04/recorder.html#Web Testing with HTTP::Recorder or WWW::Mechanize::Firefox

[1arryb]

Hi Anonymous,

In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround. LWP::UserAgent is kind of a grey area, since you could use it to build interactive, as well as automated web applications. However, the folks at sciencedirect.com are clearly taking a hard line against it. If taioma chooses to spoof his user agent from a spider, and it is detected, he could get himself (and his whole company) banned.

Cheers,

Larry

[tobyink]

I agree that spoofing another user agent is an ugly hack, however setting an appropriate user agent is always a good idea:

my $ua = LWP::UserAgent->new(agent => 'MyApp/0.1');

[Anonymous Monk]

In general, changing your user agent string to spoof Firefox, for example, is not a legitimate workaround.

says who?

and his whole company) banned.

I am not a lawyer, I don't give legal advice