Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re: Any idea why LWP:Simple doesn't like this particular website?

by epoptai (Curate)
on Jul 02, 2001 at 03:39 UTC ( #93080=note: print w/replies, xml ) Need Help??


in reply to Any idea why LWP:Simple doesn't like this particular website?

Update: Tried lwp on that site and it works.
perl -e "use LWP::Simple; getprint 'http://www.theredkitchen.net/'"

It's possible for a site to block lwp transfers via $ENV{'HTTP_USER_AGENT'} on the server side. You could try using LWP::UserAgent to specify your own agent:

$ua = LWP::UserAgent->new; $ua->agent('Odyssey/2001');

--
Check out my Perlmonks Related Scripts like framechat, reputer, and xNN.

Replies are listed 'Best First'.
Re: Any idea why LWP:Simple doesn't like this particular website?
by Cody Pendant (Prior) on Jul 02, 2001 at 06:02 UTC
    OK I'm using the following script to do it, and it returns absolutely nothing except "The script has successfully started up.":
    #!/usr/bin/perl -w $| = 1; use diagnostics; use CGI::Carp qw(fatalsToBrowser); use LWP::Simple; print "Content-type: text/html\n\n"; print "The script has successfully started up.\n\n<BR><BR>"; # just to + prove there's nothing wrong $theURL = 'http://www.theredkitchen.net/'; unless($doc=get($theURL)){ print "can't get the URL"; die "$!"; } print $doc; exit;
      As long as you've got -w and diagnostics you might as well use strict and put them to good use. Also declare your variables with my so typos will be apparent. When that is done the only error left is this enigmatic line:
      die "$!";
      which should just be exit so it won't try to print the empty $doc and get that nasty uninitialized value warning. Try this:
      #!/usr/bin/perl -w use strict; # added use diagnostics; use CGI::Carp qw(fatalsToBrowser); use LWP::Simple; my$doc; my$theURL = 'http://www.theredkitchen.net/'; print "Content-type: text/html\n\n"; print "The script has successfully started up.\n\n<BR><BR>"; unless($doc=get($theURL)){ print "can't get the URL"; exit; # added } print $doc; exit;
      This code is tested and works. If it doesn't work for you the problem may lie elsewhere...
        OK, thanks for your help, but it still behaves exactly the same way. I'm launching this script by going to/reloading its URL with a browser by the way.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://93080]
help
Chatterbox?
[1nickt]: marto would like to hear that. Of course he speaks very deliberately and dramatically, allowing pauses between words, so if anybody could pull it off it would be him, or someone impersonating him ...
[Eily]: Discipulus that's a chess joke BTW right? Because bishops don't walk straight :D
[choroba]: stop making chessy jokes about bishops!
[choroba]: I mean cheesy
[LanX]: lol
[ambrus]: no, the chess bishop itself is a joke on real bi-shops
[ambrus]: or at least on the stereotype of bi-shops
[choroba]: we call chess bishops "archers"
[Eily]: choroba sorry, I had to get that off my chess
[ambrus]: chess certainly hasn't started those stereotypes, like how kings aren't that powerful but has convinced their whole country to work for them, etc. it's just poured them to a nice clean form that would easily get propagated.

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (13)
As of 2017-09-26 12:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    During the recent solar eclipse, I:









    Results (294 votes). Check out past polls.

    Notices?