Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

LWP::Simple - cookies?

by Anonymous Monk
on Sep 27, 2001 at 21:38 UTC ( #115179=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello. I want to retrieve a whole bunch of pages from one site, the problem is that it requires a cookie, from a windows browser, u enter the username/passwd & proceed, never being asked again (until the cookie expires). How can I surf to the www page from a perl script and make it retrieve the page as though the cookie has been set? I'll have to set the cookie first time? save it somewhere? any particular modules I should look at?

Comment on LWP::Simple - cookies?
Replies are listed 'Best First'.
Re: LWP::Simple - cookies?
by cacharbe (Curate) on Sep 27, 2001 at 21:59 UTC
    Yes, you should be looking at:
    It isn't actually that hard. Here is something I use with frequency lately.
    ##Assumes: use LWP::UserAgent; use HTTP::Request::Common qw(POST); use HTTP::Cookies; sub CreateSession{ ##Takes Ref to user agent object my ($ua) = @_; $$ua->cookie_jar(HTTP::Cookies->new(file => "<FILEPATH>", autosave => 1)); $$ua->proxy(http => '<PROXYINFO - OPTIONAL>'); $$ua->timeout(300); #Use some basic Authentication my $req = POST 'https://<SOME SITE>/login.cgi', [cdsid=>'<USERID>', pw=>'<PASSWORD>',back=>'<BACK>']; $req->proxy_authorization_basic('<PROXY USERID>, <PROXY PASS>); my $res = $$ua->request($req); unless ($res->is_success) { print "Login Failed: : ". $res->status_line . "\n"; } #get the next page $req = POST 'https://<NEXT PAGE>/main1.cgi'; $res = $$ua->request($req); unless ($res->is_success) { print "Main Page Failed: : ". $res->status_line . "\n"; } return 1; }

    C-.

Re: LWP::Simple - cookies?
by merlyn (Sage) on Sep 27, 2001 at 21:50 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://115179]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (15)
As of 2015-07-28 22:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (260 votes), past polls