Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize

by OfficeLinebacker (Chaplain)
on Sep 30, 2011 at 22:17 UTC ( #928943=perlquestion: print w/ replies, xml ) Need Help??
OfficeLinebacker has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, esteemed monks!

I'm running into a wall with trying to have my Mech program log into the following site: http://www.nyscr.org/Login.aspx .

I think it's because it's doing something tricky with the User Type field. It's not a typical select, it's something weird. The response content I get back is what I would get if I did not specify a User Type, ie the red '* required' text is visible next to the User Type box.

Here is my code:
#!/usr/bin/perl -- use strict; use warnings; use LWP::UserAgent; use WWW::Mechanize; use Readonly; use HTML::TreeBuilder; #use HTML::Element qw(Table); use HTML::TableExtract;# qw(tree); use HTML::Encoding 'encoding_from_http_message'; use Encode; use File::Slurp; use Date::Manip; Readonly::Scalar my $url => 'http://www.nyscr.org/Login.aspx'; Readonly::Scalar my $un => 'email@firm.com'; Readonly::Scalar my $pw =>'IheartPERL'; my $mech = WWW::Mechanize->new( agent => 'Mozilla/5.0 (Windows NT 6.1; + WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2' ); $mech->get($url); #The user name input is ctl00$ContentPlaceHolder1$ctl00$txtUserName #PW field is ctl00$ContentPlaceHolder1$ctl00$txtPassword #"User Type field (actually a select but think I can treat as a regula +r input) my $response = $mech->submit_form( form_number => 1, fields => { 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, } ); if ($response->is_success) { print $response->decoded_content; # or whatever $mech->save_content('resp.html'); } else { die $response->status_line; }
I have also tried
$mech->set_fields( 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, ); my $response = $mech->click_button( name => 'ctl00$ContentPlaceHolder1 +$ctl00$LoginButton', x=>26, y=>10);
(the 26,10 x,y coords are from an examination of the headers using livehttpheaders) I've tried including the ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState parameter as well as some other tomfoolery.

I'm hoping the presence of js on the page isn't going to preclude me from using Mech, just that I have to figure out what's going on, and, if necessary, cut out the middleman.

Thanks!


I like computer programming because it's like Legos for the mind.

Comment on Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize
Select or Download Code
Re: Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize
by Corion (Pope) on Sep 30, 2011 at 22:26 UTC

    Have you looked at what your browser sends as data?

      Yes, if you read my post again, I'm sure you'll see I used "live http headers," which is a Firefox extension that allows me to examine what POST data my browser sends.

      I like computer programming because it's like Legos for the mind.

        You will need to replicate all data you see in the HTTP Live Headers, at least as long as submitting the form fails otherwise. You should also include ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState in your request, and also the __EVENTTARGET and other fields that the form has. The Live HTTP Headers should show you all values submitted, and you should also send all those values unless you know better.

      Hey you know what? I did go ahead and look more closely at the headers after your post and I had been using a non-URL-decoded value for a parameter called ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState. The encoded value was %7B%22logEntries%22%3A%5B%5D%2C%22value%22%3A%22Subscriber%22%2C%22text%22%3A%22Subscriber%22%2C%22enabled%22%3Atrue%7D and I used http://meyerweb.com/eric/tools/dencoder/ to decode it so now the following works:
      $mech->set_fields( 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState => '{"logEntries":[],"value":"Subscriber","text":"Subscriber" +,"enabled":true}' ); my $response = $mech->click_button( name => 'ctl00$ContentPlaceHolder1$ctl00$L +oginButton', x=>26, y=>10 );
      !

      I like computer programming because it's like Legos for the mind.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://928943]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (7)
As of 2014-08-01 02:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (256 votes), past polls