Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: I am trying to access data from a site that needs login and password

by tinman (Curate)
on Apr 20, 2001 at 11:09 UTC ( #74109=note: print w/ replies, xml ) Need Help??


in reply to I am trying to access data from a site that needs login and password

Try the code in Answer: How can I login to get data from a webpage? as a starting point...

I took a look at the source code for the page that you gave... in your case, the "GET" method should be replaced with the "POST" method... and the login URL is http://login.yahoo.com/config/login

What you need to do is to construct the values for the variables within the login form (ie: login, password) and POST it into the form URL... Thereafter, if the login is successful, a simple web page fetch should work for you..
HTH


Comment on Re: I am trying to access data from a site that needs login and password
Re: Re: I am trying to access data from a site that needs login and password
by aykes (Initiate) on Apr 22, 2001 at 00:55 UTC
    I tried using this code. It saved the cookies to a file, but I don't know how to use them or if I have to use them. But I think I am posting properly, but at the end of the code where I GET the data, it just gives me 'guest' data as if I didn't log in.
    use HTTP::Response; use HTTP::Request; use LWP::UserAgent; use HTTP::Request::Common; use HTTP::Cookies; $login = "username"; $password = "password"; $action = "http://login.yahoo.com/config/login"; $cookie_file = "d:/perl/yahoo_cookie.txt"; $cookie_jar = HTTP::Cookies->new(file=>"$cookie_file"); $ua = LWP::UserAgent->new(); $ua->agent("Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)"); $ua->timeout(600); $ua->cookie_jar(HTTP::Cookies->new(file => "$cookie_file", autosave => + 1)); $response= $ua->request(POST $action,[ '.tries'=>"1", '.done'=>"http://baseball.fantasysports.yahoo.com/baseball", '.src'=>"", 'lg'=>"us", '.intl'=>"us", 'login'=>$login, 'passwd'=>$password, '.chkp'=>"Y"]); $cookie_jar->extract_cookies($response); $content = $response->content(); print $response->as_string; $team1 = "http://baseball.fantasysports.yahoo.com/baseball/show?pag +e=roster&lid=23413&mid=2"; use LWP::Simple; $content2 = get($team1); print $content2;
    Is there something wrong with this? Am I GETing the data of the next page properly? By the way I am trying to do all of this in a DOS window. Can I do it that way? By the questions I am asking, I'm sure you can tell that I am new to this. I hope you can help. Thanks.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://74109]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (11)
As of 2014-09-01 15:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (14 votes), past polls