Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re: LWP posting

by SnackySmorez (Initiate)
on May 14, 2013 at 19:28 UTC ( #1033537=note: print w/ replies, xml ) Need Help??

in reply to LWP posting

Hi PerlSufi, try encoding the url. I seem to remember having to encode urls for some sites before submitting it. Also as the other poster replied, try Mechanize. If you have a choice use Mechanize.

Comment on Re: LWP posting
Replies are listed 'Best First'.
Re^2: LWP posting
by PerlSufi (Friar) on May 14, 2013 at 19:59 UTC
    changing the crawler to use WWW::Mechanize for my sites gave me this error:
    Error GETing +ASHBOARD: Can't connect to (certificate verify failed) at ac l ine 44.

    I think I may need to use LWP instead. Maybe with HTML::Form?
    And actually, I should mention that I use mechanize to login. Just haven't had much success navigating with it because of js stuff
        Thanks for the insight, although.. I'm not sure what that means I need to do to mechanize for it to work.. I'll read on
Re^2: LWP posting
by PerlSufi (Friar) on May 14, 2013 at 21:27 UTC
    Hi, Thanks for your insight. Actually, the header does say it is form encoded.. but how do I do that?

      You can try forming your own url string and using url::encode. Also worth checking out is uri::escape for %-encode and %-decode unsafe characters

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1033537]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (13)
As of 2015-09-02 19:03 GMT
Find Nodes?
    Voting Booth?

    My preferred temperature scale is:

    Results (87 votes), past polls