Beefy Boxes and Bandwidth Generously Provided by pair Networks vroom
There's more than one way to do things
 
PerlMonks  

Re^5: LWP::Simple get function...

by maverick (Curate)
on Dec 17, 2007 at 16:15 UTC ( #657447=note: print w/ replies, xml ) Need Help??


in reply to Re^4: LWP::Simple get function...
in thread LWP::Simple get function...

Wow, a reply to a seven year old post. Is that a record?

Your 400 error message is probably because you're using a short url:

/index.html
as opposed to a full url
http://www.somehost.com/index.html
As for saving the output in a given directory; you'll need to make a couple of changes. First you'll need to make the program take two parameters, the URL you want to get and the save location. For the sake of simplicity, let's not only specifiy the directory to save in, but also the filename to save the page into.
use LWP::UserAgent; use HTTP::Request::Common; if ($#ARGV != 1) { print STDERR "Usage: $0 \"URL to fetch\" \"save location\"\n"; exit; } my $agent = new LWP::UserAgent; $agent->proxy(['http','ftp'],'http://192.168.1.1:8080'); my $req = GET($ARGV[0]); my $res = $agent->request($req); if ($res->is_success) { open(OUTPUT,">".$ARGV[1]) || die "Can't open outfile: $!"; print OUTPUT $res->content; close(OUTPUT); } else { print $res->status_line,"\n"; }
Note the differences between the two. The first "if" has changed, as well as the usage message. Then there are the added "open" and "close" statements, and the alteration to the "print" statement.

Then you can use it like this:

program.pl http://www.somehost.com/index.html /tmp/somehost_index.ht +ml
The output directory will have to exist first and the program doesn't do any sanity or security checks on the filename you give it. There are lots of references here on how to do add that sort of bulletproofing.

/\/\averick


Comment on Re^5: LWP::Simple get function...
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://657447]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (8)
As of 2014-04-16 11:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (424 votes), past polls