Monks,
I return from my quest with success!! It took me a couple of days and many ponderings on Monks' responses to my previous postings, but I now have code that will enter into my selected user/pass protected https webpage and cruise around internally to do almost whatever I want or need. I thought I would post and share as there were no definitive answers during my quest on how to get this done.
Comments for improvement are welcome.
#!/usr/bin/perl
use strict;
use Crypt::SSLeay;
use LWP::UserAgent;
use WWW::Mechanize;
use HTTP::Cookies;
use HTTP::Request;
##The first two lines of code are used because
##I was sending my results to a web browser.
use CGI::Carp qw/ fatalsToBrowser /;
print "Content-type: text/html\n\n";
my $user = 'MyUser';
my $pass = 'MyPass';
my $base_address = 'https://wwws.stocksRus.com';
my $output = '';
my $dv_data = '';
my $webpage = '';
my $url = 'https://wwws.stocksRus.com/cgi-bin/LogIn';
my $agent = WWW::Mechanize->new( autocheck => 1);
# Set up cookie jar
$agent->cookie_jar(HTTP::Cookies->new);
##Go through login to get appropriate cookies
##so that I can then move onto
##the following pages with my cookie jar properly loaded.
$agent->get($url);
die $agent->response->status_line unless $agent->success;
#In order to login properly it was necessary to send
#a hidden input called DV_DATA along with user/pass that
#appeared to be time based and was assigned on entry to
#the main login page. I extracted this value and
#assigned it as one of the inputs for the form.
#This is a critical component of the login.
$output = $agent->content;
for ($output =~ /name=\"DV_DATA\" type=\"hidden\" VALUE=\"(.*?)\">/smi
+){
$dv_data = $1;
}
$agent->form_name( 'LoginFormName' );
$agent->set_fields(
USERID => $user,
PASSWORD => $pass,
DV_DATA => $dv_data
);
$agent -> submit();
###END Login
##I now have the appropriate cookies and may continue.
#On the main page there is some javascript that codes for
#the pull down menus. I could not access these using
#WWW::Mechanize and one of them had the link I needed.
#To solve, I logged into the webpage in a normal browser
#and found the url of the page that I wanted and plugged
#it in below.
##Accessing this page directly does not work as there
##is some Authorization procedure that requires going
##through login first.
$agent->get('https://wwws.stocksRus.com/cgi-bin/Quotes');
die $agent->response->status_line unless $agent->success;
#This page has a form that holds a symbol that I can
#submit on to get data on that symbol.
$agent -> field ('ticker','AMGN');
$agent -> submit();
#At first the webpage wouldn't come up in my web browser
#(which is where the output from this run was sent),
#because I didn't have the base URL. All the addresses
#were similar to "/cgi-bin/menu/equity".
#I substitute the base URL in below.
$webpage = $agent -> content;
$webpage =~ s/\"\//\"$base_address\//smig;
$webpage =~ s/\'\//\"$base_address\//smig;
#Prints out my webpage of interest.
print $webpage;
exit;
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|