Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: My first LWP script

by Qiang (Friar)
on Jun 15, 2004 at 19:35 UTC ( [id://367015]=note: print w/replies, xml ) Need Help??


in reply to My first LWP script

LWP is perfect use for web automation. usually for web login, you need to take care the referer/browser user agent/cookie as the web application may check if this is a valid request ( shallow but does work sometimes ). use ethereal to sniff the packet if you can visit from brower but can't with LWP.

for more info on how to do that check out %perldoc lwpcookbook and this article on perl.com by Sean M. Burke. I found it's very useful for people who are new to LWP. if you want to get detail on LWP and all the html processing, check his book "Perl & LWP".

Replies are listed 'Best First'.
Better to sniff with lynx
by mako132 (Initiate) on Jun 15, 2004 at 20:46 UTC
    If you can log in with the lynx text browser, then you can turn on its trace mode and get a trace file (~/Lynx.trace) that shows exactly what's going on.
    lynx -trace http://foo.bar/
    The trace file is a bit cluttered with debug info concerning how Lynx parses and renders HTML, but it is still pretty easy to follow.
      forgot to mention.. alternative, if you use firefox from mozilla, there is the 'Live Http Header' plugin. it's simpler and easier to use.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://367015]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (4)
As of 2024-04-19 14:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found