Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize

by OfficeLinebacker (Chaplain)
on Sep 30, 2011 at 22:17 UTC ( #928943=perlquestion: print w/replies, xml ) Need Help??
OfficeLinebacker has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, esteemed monks!

I'm running into a wall with trying to have my Mech program log into the following site: .

I think it's because it's doing something tricky with the User Type field. It's not a typical select, it's something weird. The response content I get back is what I would get if I did not specify a User Type, ie the red '* required' text is visible next to the User Type box.

Here is my code:
#!/usr/bin/perl -- use strict; use warnings; use LWP::UserAgent; use WWW::Mechanize; use Readonly; use HTML::TreeBuilder; #use HTML::Element qw(Table); use HTML::TableExtract;# qw(tree); use HTML::Encoding 'encoding_from_http_message'; use Encode; use File::Slurp; use Date::Manip; Readonly::Scalar my $url => ''; Readonly::Scalar my $un => ''; Readonly::Scalar my $pw =>'IheartPERL'; my $mech = WWW::Mechanize->new( agent => 'Mozilla/5.0 (Windows NT 6.1; + WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2' ); $mech->get($url); #The user name input is ctl00$ContentPlaceHolder1$ctl00$txtUserName #PW field is ctl00$ContentPlaceHolder1$ctl00$txtPassword #"User Type field (actually a select but think I can treat as a regula +r input) my $response = $mech->submit_form( form_number => 1, fields => { 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, } ); if ($response->is_success) { print $response->decoded_content; # or whatever $mech->save_content('resp.html'); } else { die $response->status_line; }
I have also tried
$mech->set_fields( 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, ); my $response = $mech->click_button( name => 'ctl00$ContentPlaceHolder1 +$ctl00$LoginButton', x=>26, y=>10);
(the 26,10 x,y coords are from an examination of the headers using livehttpheaders) I've tried including the ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState parameter as well as some other tomfoolery.

I'm hoping the presence of js on the page isn't going to preclude me from using Mech, just that I have to figure out what's going on, and, if necessary, cut out the middleman.


I like computer programming because it's like Legos for the mind.

Replies are listed 'Best First'.
Re: Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize
by Corion (Pope) on Sep 30, 2011 at 22:26 UTC

    Have you looked at what your browser sends as data?

      Hey you know what? I did go ahead and look more closely at the headers after your post and I had been using a non-URL-decoded value for a parameter called ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState. The encoded value was %7B%22logEntries%22%3A%5B%5D%2C%22value%22%3A%22Subscriber%22%2C%22text%22%3A%22Subscriber%22%2C%22enabled%22%3Atrue%7D and I used to decode it so now the following works:
      $mech->set_fields( 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState => '{"logEntries":[],"value":"Subscriber","text":"Subscriber" +,"enabled":true}' ); my $response = $mech->click_button( name => 'ctl00$ContentPlaceHolder1$ctl00$L +oginButton', x=>26, y=>10 );

      I like computer programming because it's like Legos for the mind.
      Yes, if you read my post again, I'm sure you'll see I used "live http headers," which is a Firefox extension that allows me to examine what POST data my browser sends.

      I like computer programming because it's like Legos for the mind.

        You will need to replicate all data you see in the HTTP Live Headers, at least as long as submitting the form fails otherwise. You should also include ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState in your request, and also the __EVENTTARGET and other fields that the form has. The Live HTTP Headers should show you all values submitted, and you should also send all those values unless you know better.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://928943]
Approved by ww
[stevieb]: ahem... s/no idea how to use it/didn't read enough as to what it's used for/
[stevieb]: interestingly enough, someone else got my Devel::Examine:: Subs distribution for their PRC, and I applaud the change. This dist is extremely complicated and mostly obfu, but the person doing it understood PPI enough to change...
[stevieb]: ...something I had overlooked in the extreme depths of the core functionality. After merging, then a couple of extra tweaks, I still have 100% test coverage. Yay for people who write tests!

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (5)
As of 2017-01-24 01:18 GMT
Find Nodes?
    Voting Booth?
    Do you watch meteor showers?

    Results (199 votes). Check out past polls.