Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

WWW::Mechanize and session id

by seaver (Pilgrim)
on Sep 28, 2004 at 15:46 UTC ( [id://394639]=perlquestion: print w/replies, xml ) Need Help??

seaver has asked for the wisdom of the Perl Monks concerning the following question:

Dear all,

Im using WWW::Mechanize to scrape a online library of citations. It seems to be working well, and easily. However, this online library needs a session id, persistent throughout its search, which is present in a hidden input field.

Here's my code so far:

#!/usr/bin/perl -w require strict; use WWW::Mechanize; my $mech = WWW::Mechanize->new( autocheck => 1 ); $mech->get("http://isiknowledge.com/?DestApp=WOS&Func=Frame"); $mech->follow_link(tag => 'frame', n => 2); $mech->form_number(1); $mech->click('General Search', [2,2]); $mech->form_number(1); $mech->field("topic","emergence AND complexity"); $mech->click(); $mech->form_number(1); $mech->field('mark_from',1); $mech->field('mark_to',500); $mech->field('mark_selection','range'); my $res=$mech->click('Submit'); print $res->content;
Its pretty straight forward, and for the first couple of pages, it goes well, but on the last 'Submit', the response page claims I need to start a new session!! I've encountered this before, and I actually did try extracting the session id from previous pages, and inserting it into the hidden field in the last page, but WWW::Mechanize wont let me, as it says that field is 'read-only' I assume because it is hidden.

The funny thing is, I stopped using HTTP::Request::Form, because there was a bug concerning multiple checkboxes of the same name, however doing this same thing with HRF, I could get through to the next page, so WWW::Mechanize seems to be dropping the session id, what's going on?

Is it possible that there's a redirect that it's following automatically WITHOUT the session id?

Thanks
Seaver

Replies are listed 'Best First'.
Re: WWW::Mechanize and session id
by johnnywang (Priest) on Sep 28, 2004 at 20:17 UTC
    From the WWW::Mechanize FAQ:
    If you really do want to change a hidden value, make the changes in a +scope that has warnings turned off: { local $^W = 0; $agent->field( name => $value ); }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://394639]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (5)
As of 2024-03-28 09:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found