Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

connecting to a secure password authenticated site

by dannoura (Pilgrim)
on May 06, 2004 at 12:50 UTC ( #351093=perlquestion: print w/ replies, xml ) Need Help??
dannoura has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,

I've recently been trying to automate my login to this site. I've asked before about this but I feel that now I'm pretty close to the solution. Using the code:

#! c:\perl\bin -w use strict; use LWP::UserAgent; use HTTP::Request::Common; use HTTP::Cookies; use LWP::Debug qw(+); use LWP::Protocol::https; use crypt::SSLeay; my $ua = LWP::UserAgent->new; $ua->cookie_jar(HTTP::Cookies->new(file => 'cookie_jar', autosave =>1) +); # configure LWP::UserAgent to follow redirects after POST push @{ $ua->requests_redirectable }, 'POST'; my $request=$ua->request( POST "https://www.saxobank.com/Default.aspx/?id=2&Lan=EN&Au=1&Grp=5" +, { __VIEWSTATE => 'dDwxOTk0Mzg2NjQzOzs+/+MSvllHQREYBFP5zZXrPV/rhdM=', txtUID => '666666', txtPWD => 'secret', submit => 'log in' }); print $request->is_success ? $request->content : "failed\n";

I get the reply:

<SCRIPT LANGUAGE="javascript"> top.location.replace('http://www.saxobank.com/?ID=101')

Which is a redirect to the page I get to after logging in with a browser. When I add

$request= $ua->request(GET "http://www.saxobank.com/?ID=101"); print $request->is_success ? $request->content : $request->header;
after the previous code I get
LWP::UserAgent::request: Simple response: Internal Server Error </SCRIPT>

I'm guessing there is some sort of redirect going on here that I can't follow (notwithstanding the fact that redirects are supposed to be followed automatically) because I'm not posting enough data in my GET request. But I don't know what it is. Maybe I should add the session ID somehow? Below is a capture of HTTP headers generated during my browser session with the site, in case it has any additional useful information.

https://www.saxobank.com/Default.aspx/?id=2&Lan=EN&Au=1&Grp=5 POST /Default.aspx/?id=2&Lan=EN&Au=1&Grp=5 HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9 +,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,*/ +*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://www.saxobank.com/ Cookie: ASP.NET_SessionId=aaaaaaaaaaaaaaaaaaaaaaaa; SAXOBANK%5FMEMBER= +UserNumber=666666&AccessLevel=2&MemberGroupID=6&CookieID=xxxxxxxxxxxx +xxxxxxxxxxxxxxx Content-Type: application/x-www-form-urlencoded Content-Length: 114 __VIEWSTATE=dDwxOTk0Mzg2NjQzOzs%2B%2F%2BMSvllHQREYBFP5zZXrPV%2FrhdM%3D +&txtUID=666666&txtPWD=secret&submit=log+in HTTP/1.x 200 OK Server: Microsoft-IIS/5.0 Date: Tue, 04 May 2004 12:57:58 GMT X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" X-AspNet-Version: 1.1.4322 Set-Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGr +oupID=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; expires=Wed, 04-Aug-20 +04 00:00:00 GMT; path=/ Cache-Control: no-cache Pragma: no-cache Expires: -1 Content-Type: text/html; charset=utf-8 Content-Length: 136 ---------------------------------------------------------- http://www.saxobank.com/?ID=101 GET /?ID=101 HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9 +,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,*/ +*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 HTTP/1.x 200 OK Date: Tue, 04 May 2004 12:57:59 GMT Content-Length: 65219 Content-Type: text/html; charset=utf-8 Expires: Tue, 04 May 2004 12:57:59 GMT Cache-Control: no-cache Server: Microsoft-IIS/5.0 X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" X-AspNet-Version: 1.1.4322 Pragma: no-cache Via: 1.1 netcache (NetCache NetApp/5.5R3) ---------------------------------------------------------- http://www.saxobank.com/inc/js/listmenu.js GET /inc/js/listmenu.js HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: */* Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Referer: http://www.saxobank.com/?ID=101 Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 HTTP/1.x 200 OK Age: 111 Accept-Ranges: bytes Date: Tue, 04 May 2004 12:57:59 GMT Content-Length: 3347 Content-Type: application/x-javascript Server: Microsoft-IIS/5.0 X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" Last-Modified: Wed, 14 Apr 2004 14:15:02 GMT Etag: "0f747e02a22c41:d99" Via: 1.1 netcache (NetCache NetApp/5.5R3) ---------------------------------------------------------- http://www.saxobank.com/__DotNet/Menu/WebMenuGetChildren.js.aspx?Lan=E +N&Au=2&Grp=6&GetLangList=True&RootNodeList=forex-1244_cfds-1267_futur +es-1352_funds-1366_about-1233 GET /__DotNet/Menu/WebMenuGetChildren.js.aspx?Lan=EN&Au=2&Grp=6&GetLan +gList=True&RootNodeList=forex-1244_cfds-1267_futures-1352_funds-1366_ +about-1233 HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: */* Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Referer: http://www.saxobank.com/?ID=101 Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 HTTP/1.x 200 OK Date: Tue, 04 May 2004 12:57:59 GMT Content-Length: 13008 Content-Type: text/javascript; charset=utf-8 Expires: Tue, 04 May 2004 13:58:00 GMT Cache-Control: private Server: Microsoft-IIS/5.0 X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" X-AspNet-Version: 1.1.4322 Via: 1.1 netcache (NetCache NetApp/5.5R3) ---------------------------------------------------------- https://www.saxobank.com/favicon.ico GET /favicon.ico HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 HTTP/1.x 200 OK Server: Microsoft-IIS/5.0 Date: Tue, 04 May 2004 12:57:59 GMT X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" X-AspNet-Version: 1.1.4322 Cache-Control: private Content-Type: text/html; charset=utf-8 Content-Length: 83 ---------------------------------------------------------- http://www.saxobank.com/inc/css/style.css GET /inc/css/style.css HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: text/css,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Referer: http://www.saxobank.com/?ID=101 Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 If-Modified-Since: Thu, 22 Apr 2004 10:43:08 GMT If-None-Match: "08673995628c41:d99" HTTP/1.x 304 Not Modified Etag: "08673995628c41:d99" Date: Sun, 02 May 2004 13:16:01 GMT Warning: 113 netcache "Heuristic expiration" ---------------------------------------------------------- http://www.saxobank.com/_Components/StyleSheets/FOREX_screen.css GET /_Components/StyleSheets/FOREX_screen.css HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: text/css,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Referer: http://www.saxobank.com/?ID=101 Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 HTTP/1.x 200 OK Age: 111 Accept-Ranges: bytes Date: Tue, 04 May 2004 12:57:59 GMT Content-Length: 9784 Content-Type: text/css Server: Microsoft-IIS/5.0 X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" Last-Modified: Thu, 15 Apr 2004 07:51:55 GMT Etag: "80df5f85be22c41:d99" Via: 1.1 netcache (NetCache NetApp/5.5R3) ---------------------------------------------------------- http://www.saxobank.com/_Components/StyleSheets/FOREX_print.css GET /_Components/StyleSheets/FOREX_print.css HTTP/1.1 Host: www.saxobank.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Ge +cko/20040206 Firefox/0.8 Accept: text/css,*/*;q=0.1 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Proxy-Connection: keep-alive Referer: http://www.saxobank.com/?ID=101 Cookie: SAXOBANK%5FMEMBER=UserNumber=666666&AccessLevel=2&MemberGroupI +D=6&CookieID=xxxxxxxxxxxxxxxxxxxxxxa59+PM; ASP.NET_SessionId=nqq1fd55 +hk2c4x45izubtm45 HTTP/1.x 200 OK Age: 112 Accept-Ranges: bytes Date: Tue, 04 May 2004 12:57:59 GMT Content-Length: 2510 Content-Type: text/css Server: Microsoft-IIS/5.0 X-Powered-By: ASP.NET P3P: CP="CAO DSP CURa ADMa DEVa TAIa IVAa IVDa CONa TELa OUR SAMa IND +UNI" Last-Modified: Tue, 06 Apr 2004 09:57:36 GMT Etag: "0687196bd1bc41:d99" Via: 1.1 netcache (NetCache NetApp/5.5R3) ----------------------------------------------------------

-----------------------------------

Any comments about coding style are welcome.

20040506 Edit by Corion: Obscured account number and cookies

Comment on connecting to a secure password authenticated site
Select or Download Code
Replies are listed 'Best First'.
Re: connecting to a secure password authenticated site
by jdtoronto (Prior) on May 06, 2004 at 15:03 UTC
    I looked at the site and the VIEWSTATE changes. I wonder if you need to scrape that from the form so that you can send it back correctly?

    jdtoronto

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://351093]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (19)
As of 2015-07-28 14:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (256 votes), past polls