Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize

by OfficeLinebacker (Chaplain)
on Sep 30, 2011 at 22:17 UTC ( #928943=perlquestion: print w/replies, xml ) Need Help??
OfficeLinebacker has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, esteemed monks!

I'm running into a wall with trying to have my Mech program log into the following site: .

I think it's because it's doing something tricky with the User Type field. It's not a typical select, it's something weird. The response content I get back is what I would get if I did not specify a User Type, ie the red '* required' text is visible next to the User Type box.

Here is my code:
#!/usr/bin/perl -- use strict; use warnings; use LWP::UserAgent; use WWW::Mechanize; use Readonly; use HTML::TreeBuilder; #use HTML::Element qw(Table); use HTML::TableExtract;# qw(tree); use HTML::Encoding 'encoding_from_http_message'; use Encode; use File::Slurp; use Date::Manip; Readonly::Scalar my $url => ''; Readonly::Scalar my $un => ''; Readonly::Scalar my $pw =>'IheartPERL'; my $mech = WWW::Mechanize->new( agent => 'Mozilla/5.0 (Windows NT 6.1; + WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2' ); $mech->get($url); #The user name input is ctl00$ContentPlaceHolder1$ctl00$txtUserName #PW field is ctl00$ContentPlaceHolder1$ctl00$txtPassword #"User Type field (actually a select but think I can treat as a regula +r input) my $response = $mech->submit_form( form_number => 1, fields => { 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, } ); if ($response->is_success) { print $response->decoded_content; # or whatever $mech->save_content('resp.html'); } else { die $response->status_line; }
I have also tried
$mech->set_fields( 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, ); my $response = $mech->click_button( name => 'ctl00$ContentPlaceHolder1 +$ctl00$LoginButton', x=>26, y=>10);
(the 26,10 x,y coords are from an examination of the headers using livehttpheaders) I've tried including the ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState parameter as well as some other tomfoolery.

I'm hoping the presence of js on the page isn't going to preclude me from using Mech, just that I have to figure out what's going on, and, if necessary, cut out the middleman.


I like computer programming because it's like Legos for the mind.

Replies are listed 'Best First'.
Re: Help with logging in to a tricky ASP web form with JavaScript using WWW::Mechanize
by Corion (Pope) on Sep 30, 2011 at 22:26 UTC

    Have you looked at what your browser sends as data?

      Hey you know what? I did go ahead and look more closely at the headers after your post and I had been using a non-URL-decoded value for a parameter called ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState. The encoded value was %7B%22logEntries%22%3A%5B%5D%2C%22value%22%3A%22Subscriber%22%2C%22text%22%3A%22Subscriber%22%2C%22enabled%22%3Atrue%7D and I used to decode it so now the following works:
      $mech->set_fields( 'ctl00$ContentPlaceHolder1$ctl00$rcbusertype_Input' => "Su +bscriber", 'ctl00$ContentPlaceHolder1$ctl00$txtUserName' => $un, 'ctl00$ContentPlaceHolder1$ctl00$txtPassword' => $pw, ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState => '{"logEntries":[],"value":"Subscriber","text":"Subscriber" +,"enabled":true}' ); my $response = $mech->click_button( name => 'ctl00$ContentPlaceHolder1$ctl00$L +oginButton', x=>26, y=>10 );

      I like computer programming because it's like Legos for the mind.
      Yes, if you read my post again, I'm sure you'll see I used "live http headers," which is a Firefox extension that allows me to examine what POST data my browser sends.

      I like computer programming because it's like Legos for the mind.

        You will need to replicate all data you see in the HTTP Live Headers, at least as long as submitting the form fails otherwise. You should also include ctl00_ContentPlaceHolder1_ctl00_rcbusertype_ClientState in your request, and also the __EVENTTARGET and other fields that the form has. The Live HTTP Headers should show you all values submitted, and you should also send all those values unless you know better.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://928943]
Approved by ww
[stevieb]: 16.96 hours of runtime of a Raspberry Pi on a 12v, 7.2Ah battery :) Now to watch now long it takes to recharge the battery completely with my 18v solar cells in the sun :)
[stevieb]: my high-range estimate was 18 hours

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (7)
As of 2017-03-26 15:18 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (315 votes). Check out past polls.