Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

LWP cookie mystery

by Anonymous Monk
on Aug 03, 2002 at 02:34 UTC ( #187280=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have been trying to GET a page which requires a cookie to be received properly. What I eventually want to do is to POST some info to a form on this page and do a login with the help of LWP.

I know there are some other nodes on this subject, like this one LWP, extract_cookies, etc. , but I can't figure out what is wrong in my case.. The cookie which I receive does not seem to be "picked up" and is not saved to file (I save cookies in file for debugging purpose of this script).

What I bascily want to do is just to GET the url where the form, that I'm going to POST to later, is located. If I can receive the url with correct cookie handling, I'm sure POST'ing will not be difficult either. Here is my code:

#!c:/perl/bin/perl use strict; use LWP::UserAgent; use HTTP::Request; use HTTP::Cookies; use HTTP::Response; use HTTP::Headers; use HTTP::Message; use HTTP::Request::Common; use LWP::Debug; use CGI; #STORE COOKIES IN FILE my $cookie_file = qq(c:/tmp3/mycookie.txt); #HOW MANY TIMES HAVE $cookie_jar BEEN CHANGED local $main::called_cookie = 0; my $query = new CGI; print $query->header; #NEW USER-AGENT OBJEKT my $ua = LWP::UserAgent->new; $ua->agent('Mozilla/4.0'); #NEW COOKIE_JAR my $cookie_jar=HTTP::Cookies->new ('file' => "$cookie_file", 'autosave' => '1'); #SUB TO BE CALLED WHEN $cookie_jar CHANGES _ FOR DEBUGGING $cookie_jar->scan(\&cookiesub); $cookie_jar->clear(); #VARIOUS TEST-SITES WHERE WE RECEIVE COOKIES #my $req = GET 'http://www.cdnow.com'; #my $req = GET 'http://www.msn.com' ; #my $req = GET 'http://www.amazon.com'; #WANTED !! my $req = GET 'http://mother.netcom.no/Login/login_page.html'; #GO GET INFO my $resp=$ua->request($req); #UPDATE $cookie_jar; $cookie_jar->extract_cookies($resp); #PRINT RETURNED HEADERS print $resp->headers_as_string(); #SHOW VALUE OF $cookie_jar NOW print "\n\nMy Cookie_jar is now:".$cookie_jar->as_string."\n\n"; #SHOW RESPONSE #print "\n\n".$resp->as_string; #SAVE $cookie_jar TO FILE; $cookie_jar->save(); exit; sub cookiesub { print "\n\n----------------\n\Cookie has been altered"; print "\nCookie_counter:$main::called_cookie \n"; $main::called_cookie++; print "@_"; print "\n----------------\n"; }
I receive reply that I need to "accept cookies" when I GET the wanted url. No cookie-values are saved to file, but when I print the cookie value with
print "\n\nMy Cookie_jar is now:".$cookie_jar->as_string."\n\n";
I get:
Set-Cookie3: NETSCAPE_LIVEWIRE_ID=305A68095915248EF4A4BD81DABFD4F90021 +A1F2; path="/Login"; domain="mother.netcom.no"; path_spec; discard; v +ersion=0
What is wrong? Should this not have been handled automatic by LWP::UserAgent? What Can be done to receive the wanted URL with correct cookie handling?
I've tried doing several GET's after updating $cookie_jar with
$cookie_jar->extract_cookies($resp);
but that does not seem to help.

Also when I try to GET the amazon url, I see 3 Set-Cookies Headers in the HTTP::Response object ($res), but only 2 is saved to file? What is that all about?

I've been banging my head in the wall for too long about this. I need some help!

I'd REALLY appreciate any tips.
Thanks.
Regards, Tom

Comment on LWP cookie mystery
Select or Download Code
Re: LWP cookie mystery
by dws (Chancellor) on Aug 03, 2002 at 04:20 UTC
    What I bascily want to do is just to GET the url where the form, that I'm going to POST to later, is located. If I can receive the url with correct cookie handling, I'm sure POST'ing will not be difficult either.

    This example script solves exactly that problem.

Re: LWP cookie mystery
by Cine (Friar) on Aug 03, 2002 at 09:13 UTC
    Try using $ua->cookie_jar($cookie_jar) before the request, instead of the $cookie_jar->extract_cookies($resp); after the request. The latter wont pick up cookies recieved on redirects in the original request (I've hit plenty of sites with this problem).

    T I M T O W T D I
      Hi,
      I just want to say thanks to you guys for helping me out with this script (I got the script working earlier today). Once I started to ignore output to file, everything got much easier.Just let the force of LWP do the work and ignore cookie-fileoutput..

      Special thanks to dws for his similar script as the one I was trying to make.

      I hope to become a better coder so that I can come back and offer help too.
      Regards,Tom

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://187280]
Approved by dws
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (14)
As of 2014-09-18 16:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (117 votes), past polls