Beefy Boxes and Bandwidth Generously Provided by pair Networks Frank
Think about Loose Coupling
 
PerlMonks  

Re: Investigating connection reset with LWP to an https site

by Gangabass (Priest)
on Mar 05, 2012 at 23:46 UTC ( #957995=note: print w/ replies, xml ) Need Help??


in reply to Investigating connection reset with LWP to an https site

I recommend you login via browser with Javascript disabled and check all params sended by your browser to the target site (via Firefox's HTTPFox or Opera's Dragonfly).

Next you just need to send all these params from your script (Referer, cursor click x and y etc.)

Thanks.
Roman


Comment on Re: Investigating connection reset with LWP to an https site
Re^2: Investigating connection reset with LWP to an https site
by sedusedan (Monk) on Mar 06, 2012 at 02:02 UTC
    To restate my problem: I'm getting a connection reset error instead of the site changing HTML/layout/design/JavaScript (it doesn't, BTW). The module can't even retrieve a single page, like the front page or the login page, from the site. So I'm still guessing it has to do with the network layer like LWP or Crypt::SSLeay.

      RESET can happen for lots of different reasons. They could even be suddenly breaking the connection after they see your user agent string.

      I'd get more information by capturing packets with wireshark to compare the different scenarios.

      - tye        

        Thanks for the advice. Since HTTPS is encrypted and the LWP's source is at hand, I just let LWP print out (uncomment one line, really) the HTTP header it's sending, and the culprit turns out to be the Transfer-Encoding (TE) header. I'll be bugging Gisle Aas again to give LWP the option of not sending TE header.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://957995]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (12)
As of 2014-04-23 18:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (553 votes), past polls