http://www.perlmonks.org?node_id=1062391


in reply to Re^2: LWP 400 Bad Request on POST
in thread LWP 400 Bad Request on POST

Open a local port. Send the same request (from your browser) a) to that local port directly (temporarily disabling the proxy); b) and via the proxy. Compare the requests received.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^4: LWP 400 Bad Request on POST
by dsb (Chaplain) on Nov 13, 2013 at 19:06 UTC
    It's a corporate proxy so I cannot disable it.

    however I have made a little progress insofar as understanding the issue. not so much for how to fix it.

    i used ig's suggestion on wireshark to watch the packet transfers. First thing I noticed is that the browser is doing a CONNECT request, not a POST. When I switched to a CONNECT method in the script I immediately got a 200 return code. Still no content though.

    after re-checking the packet capture I saw that after the initial CONNECT and 200, the browser and server do a short handshake dance followed by several application data packets coming from the server to the browser. The LWP request doesn't capture any of that since (I guess) they aren't direct responses to the original request.

    Just in case you were curious...


    dsb
    This @ISA my( $cool ) %SIG
      First thing I noticed is that the browser is doing a CONNECT request, not a POST.... Just in case you were curious...

      You've moved way beyond my knowledge of this stuff now. Sorry.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
      First thing I noticed is that the browser is doing a CONNECT request, not a POST.

      A browser issuing a CONNECT request is typical for access to a HTTPS URL (instead of a HTTP URL). CONNECT bypasses most of the proxy (especially the filter logic), so it is usually restricted. Typically you are only allowed to connect to the HTTPS standard port 443, and no other port.

      When I switched to a CONNECT method in the script I immediately got a 200 return code. Still no content though.

      You should not have to change your code from POST to CONNECT, in fact, this should not work. LWP::UserAgent should issue a CONNECT request to the proxy, make a SSL connection to the HTTPS server, and issue your POST request through the SSL encrypted connection. When you issue a CONNECT manually, the proxy just answers 200 OK and switches to forwarding all data between your script and the destination server.

      If you can access HTTPS URLs from a browser through the proxy, LWP::UserAgent should be able to do so, too. Try something simple first, like a GET request for https://www.google.com/. If that does not work, talk with the proxy admin, make him/her look into the proxy logfiles. Talk with the network admins, try to get a way out around the proxy (directly through the firewall).

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)