I am trying to access data from a site that needs login and password

[aykes]

I am very new to Perl and really need some help. The webpage where I need to enter my login and password is: http://baseball.fantasysports.yahoo.com/baseball The page I want to view after submitting the form is: http://baseball.fantasysports.yahoo.com/baseball/show?page=roster&lid=23413&mid=2 How can I submit the form, then view the second webpage source code at a command prompt? Can someone give me code for this, or point me in the right direction? I've tried some of the suggestions that other have written, but have been unsuccessful. Is it possible to do with this website?

[tinman]

Try the code in Re: How can I login to get data from a webpage? as a starting point...

I took a look at the source code for the page that you gave... in your case, the "GET" method should be replaced with the "POST" method... and the login URL is http://login.yahoo.com/config/login

What you need to do is to construct the values for the variables within the login form (ie: login, password) and POST it into the form URL... Thereafter, if the login is successful, a simple web page fetch should work for you..
HTH

[aykes]

I tried using this code. It saved the cookies to a file, but I don't know how to use them or if I have to use them. But I think I am posting properly, but at the end of the code where I GET the data, it just gives me 'guest' data as if I didn't log in.

use HTTP::Response;
use HTTP::Request;
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Cookies; 
$login = "username";
$password = "password";
$action = "http://login.yahoo.com/config/login";
$cookie_file = "d:/perl/yahoo_cookie.txt";
$cookie_jar = HTTP::Cookies->new(file=>"$cookie_file");
$ua = LWP::UserAgent->new();
$ua->agent("Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)");
$ua->timeout(600);
$ua->cookie_jar(HTTP::Cookies->new(file => "$cookie_file", autosave =>
+ 1));
$response= $ua->request(POST $action,[
'.tries'=>"1",
'.done'=>"http://baseball.fantasysports.yahoo.com/baseball",
'.src'=>"",
'lg'=>"us",
'.intl'=>"us",
'login'=>$login,
'passwd'=>$password,
'.chkp'=>"Y"]);
$cookie_jar->extract_cookies($response);
$content = $response->content();

print $response->as_string;

$team1 =    "http://baseball.fantasysports.yahoo.com/baseball/show?pag
+e=roster&lid=23413&mid=2";
use LWP::Simple;
$content2 = get($team1);
print $content2;

Is there something wrong with this? Am I GETing the data of the next page properly? By the way I am trying to do all of this in a DOS window. Can I do it that way? By the questions I am asking, I'm sure you can tell that I am new to this. I hope you can help. Thanks.

[plazm]

On your pages use CGI.pm to parse out CGI input. Do perldoc on CGI.pm.

To view the output of the second page you have simulate form input. To do this, start the second page script from command line and then give it name=value pairs. ^D end the input. You will get your output back along with HTTP headers.