Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

WWW::Mechanize redirection problem

by eisdrache (Novice)
on Nov 28, 2006 at 05:07 UTC ( #586414=perlquestion: print w/replies, xml ) Need Help??

eisdrache has asked for the wisdom of the Perl Monks concerning the following question:

I have website running javaserver pages that requires authentication via a login form on the front page.

If I access the page via a web browser I can submit my info and enter the site without a problem. However, when using WWW::Mechanize with the same exact login credentials I am denied entrance.

Here is the basic site setup:
login.jsp -> search.jsp -> mainIndex.jsp

When a user logs in the request is sent from login.jsp to search.jsp where the information is examined. If successful the request is redirected to the main page and the user is logged in, otherwise the request is redirected back to login.jsp with an error message asking to login to access the main page.

Here is a section of the code I am running:
my $mech = WWW::Mechanize->new(); $mech->cookie_jar(HTTP::Cookies->new()); $mech->get($target); $mech->form_name('LoginForm'); $mech->field('loginId' => $login_id); $mech->field('password' => $password); $mech->click();
For whatever reason the search.jsp page does not like the request coming from WWW::Mechanize and I cannot figure out the problem, hence the reason for the post.

I realize this info is vague but I at least needed to create a starting point. Any help is greatly appreciated!

Replies are listed 'Best First'.
Re: WWW::Mechanize redirection problem
by perrin (Chancellor) on Nov 28, 2006 at 06:27 UTC
    The best I can give you with a vague question is a vague answer: try it in Firefox with the LiveHTTPHeaders extension and watch the traffic between the browser and the server. Adjust your mech script so that it does the same thing Firefox is doing. It's impossible for the JavaScript to do anything other than affect the HTTP exchange between browser and server, so everything that matters is contained in that exchange.
      Thanks for the vague answer, it actually helped. I was previously unaware of the LiveHTTPHeaders extension, so after installing and using that I believe I found the problem. Apparently, the web request and the mechanize request were posting to two different pages.

      I checked the source code of the login page and the form has an action value of seach.jsp, however the submit button actually calls a javascript function which sends the request to login.jsp.

      Thanks again for your help!
Re: WWW::Mechanize redirection problem
by sanPerl (Friar) on Nov 28, 2006 at 09:04 UTC
    You can try Win32::IE::Mechanize module. It will open IE's session. If your javascript is checking browser then you can fool it.
    Also check if, before accessing new page, 'sleep' for few seconds could do the job. Sometimes the script works too fast and you do not pull-out page till that time.
Re: WWW::Mechanize redirection problem
by former33t (Scribe) on Nov 28, 2006 at 12:47 UTC
    I would think that sleeping for a second or two on the redirect might help. Also check for values that your browser may be POST'ing when the redirect is thrown. These are the two things that have screwed me up in the past with WWW::Mechanize.
      I had problems with redirected posts as well. See LWP::UserAgent Docs. LWP::UserAgent does not follow redirects for posts by default, you have to tell it you want to do that.

      Since WWW::Mechanize is a subclass of LWP::UserAgent, you should be able to:
      push @{ $mech->requests_redirectable }, 'POST';

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://586414]
Approved by McDarren
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (3)
As of 2022-06-28 00:06 GMT
Find Nodes?
    Voting Booth?
    My most frequent journeys are powered by:

    Results (89 votes). Check out past polls.