Seeking assistance with WWW::Mechanize Form Submission

[tspang]

Hello everyone - long time reader (as google turns up perlmonks quite frequently) first time poster, as I'm frustratingly stuck. The Mechanize perl doc even recommends asking for assistance here, so I'm giving it a shot.

The problem: I'm attempting to use WWW::Mechanize to log into an apc network management card to scrape the temperature data contained in one of the pages. This should be an easy task... I would think.

I started by creating a super simple form on my own to verify I could submit a form with WWW::Mechanize. I quickly got this functional.

Now, with basic functional code, I used mech-dump to get the form information that I would need to submit (and verified this by recording logging into the interface with HTTP Watch). HTTP Watch also shows that after submitting the correct username, password, and language, you recieve a 303 response directing you to a new page that contains your session ID as part of the URI - so I MUST use the data returned by the 303 to get any further into the interface.

mech-dump returned this: login_username= (text) login_password= (password) prefLanguage=00000000 (option) *00000000/English submit=Log On (submit) <NONAME>=<UNDEF> (reset)

Given this information, I modified my (functional) test script to use the data provided by mech-dump and HTTP watch:

#!/usr/bin/perl
use WWW::Mechanize;
use Crypt::SSLeay;

my $mech = WWW::Mechanize->new( autocheck => 0);
$url="https://192.168.0.2";
$mech->get( $url );

print $mech->status;

$mech->form_name('HashForm1');
$mech->field("login_username",'username');
$mech->field("login_password",'password');
$mech->field("prefLanguage",00000000);
$mech->submit_form();

print "form submitted, reponse was ",$mech->status," printing header\n
+";

    my $response = $mech->response();
    for my $key ( $response->header_field_names() ) {
        print $key, " : ", $response->header( $key ), "\n";
    }
exit;

The problem at this point is, I always recieve a 500 back from the page. To make matters worse, I've tried to modify this script to submit to several other interfaces (IBM AMM, HP ILOM, etc) and none of them accept the data that I'm submitting, so it appears to be a problem with my perl.

Any help would be GREATLY appreciated, even if its just links to some extra documentation that I can read to find my own way (I've already tried the perdoc and the cookbook here: http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize/Cookbook.pod)

Thanks!

[Gangabass]

First I recommend you to start by checking if your form works in the browser with JavaScript disabled: remove all cookies for this site, disable JavaScript and try to submit your form.

If the problem is not in JavaScript you need to try to add button => ... to your code. Many sites expects x and y coordinates of the submit button you click:

$mech->submit_form(
    with_fields => {
        "login_username" =>'username',
        "login_password" => 'password',
        "prefLanguage" => '00000000',
    },
    button => "your_submit_button_name",
);

to your code. Many sites expects x and y coordinates of the submit button you click:

[tspang]

I removed all existing cookies, disabled javascript and cookies in firefox prior to logging in, and was still able to successfully log in, so it doesn't look like javascript or cookie issue (even though the site does set a cookie per HTTP Watch).

I changed the structure of my form submission to this:

    $mech->submit_form(
        form_name => 'HashForm1',
        fields => {
            login_password => 'username',
            login_username => 'password',
            prefLanguage => '00000000',

        },
        button => "submit",
    );

Still to no success. Is the above code functionally different than what I had been using earlier?

$mech->form_name('HashForm1');
$mech->field("login_username",'username');
$mech->field("login_password",'password');
$mech->field("prefLanguage",00000000);
$mech->submit_form();

Thanks for the help so far, everyone. I'm looking into HTTP:Recorder right now, and I'm feeling like I'm moving at least a little toward a solution at this point.

[Gangabass]

Are you sure that your submit button is named "submit"? Yes, the code is different (you're now sending button info too).

I recommend you to use LiveHttpHeaders for Firefox and make exactly same request with $mech->post()

Thanks.
Roman

[Anonymous Monk]

500 after submit? What is $mech->dump of the 500 error?

is JS involved?

try HTTP::Recorder, Web Testing with HTTP::Recorder )

[tspang]

I'll take a look at HTTP::Recorder.

There doesn't appear to be a dump method - just dump-forms, dump-images, etc. If I set autocheck to 1 and let it exit it gives me a quite generic error:

Server closed connection without sending any data back

As for javascript, the only javascript on the page sets focus to the first text field. I've included it here for completeness:

script type="text/javascript"
function placeFocus() {
if (document.forms.length > 0) {
var field = document.forms[0];
for (i = 0; i < field.length; i++) {
if ((field.elements[i].type == "text") || (field.elements[i].type == "
+textarea")) {
document.forms[0].elements[i].focus();
break;
}
}
}
}
/script

[Anonymous Monk]

It is common for an image on the page to set a cookie, so HTTP::Recorder or LiveHttpHeaders or WireShark will show that if you're looking :)

[Anonymous Monk]

:) $mech->res->dump