LWP::Simple - cookies?

[Anonymous Monk]

Hello. I want to retrieve a whole bunch of pages from one site, the problem is that it requires a cookie, from a windows browser, u enter the username/passwd & proceed, never being asked again (until the cookie expires). How can I surf to the www page from a perl script and make it retrieve the page as though the cookie has been set? I'll have to set the cookie first time? save it somewhere? any particular modules I should look at?

[merlyn]

See HTTP::Cookies and note that LWP::UserAgent can be told to use a "cookie jar" for its fetches.

I really need to do a column on that. {grin}

-- Randal L. Schwartz, Perl hacker

[cacharbe]

Yes, you should be looking at:

• LWP::UserAgent
• HTTP::Request::Common
• HTTP::Cookies
It isn't actually that hard. Here is something I use with frequency lately.

##Assumes:
use LWP::UserAgent; 
use HTTP::Request::Common qw(POST); 
use HTTP::Cookies;

sub CreateSession{ ##Takes Ref to user agent object
  my ($ua) = @_;

    $$ua->cookie_jar(HTTP::Cookies->new(file => "<FILEPATH>",
                                   autosave => 1));
    $$ua->proxy(http => '<PROXYINFO - OPTIONAL>');
    $$ua->timeout(300);
   #Use some basic Authentication
    my $req = POST 'https://<SOME SITE>/login.cgi',
              [cdsid=>'<USERID>', pw=>'<PASSWORD>',back=>'<BACK>'];

    $req->proxy_authorization_basic('<PROXY USERID>, <PROXY PASS>);
    my $res = $$ua->request($req);
 
    unless ($res->is_success) {
        print "Login Failed: : ". $res->status_line .  "\n";
    }

   #get the next page
   $req = POST 'https://<NEXT PAGE>/main1.cgi';
   $res = $$ua->request($req); 

   unless ($res->is_success) {
      print "Main Page Failed: : ". $res->status_line .  "\n";
   }
   return 1;

}

C-.