Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: Lexis-Nexis Academic spider

by mpeters (Chaplain)
on Apr 26, 2005 at 16:27 UTC ( [id://451659]=note: print w/replies, xml ) Need Help??


in reply to Lexis-Nexis Academic spider

Regardless of whether or not the page uses Javascript it still has to communicate with the server via HTTP. All you need to do is find out if the Javascript is affecting what is sent to the server or not. You might also have to step outside of WWW::Mechanize and use HTTP::Request your self. It's really not that bad, and you can even use the two together.
#build out your request my $request = HTTP::Request::Common::POST->new(....); $mech->request($request);
And then you can continue using Mech.

If you are having a hard time figuring out what HTTP the browser is sending to the server, check out something like the LiveHTTPHeaders plugin for Mozilla, or ethereal.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://451659]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (2)
As of 2024-04-25 06:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found