Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

I'm trying to use LWP to GET a page using the https protocol, and for the life of me, no matter what I try, I get a "400 Bad request" response from the server (or at least I assume it's from the server). In the source below, I use LWP to first GET https://mail.google.com, which fails with a rc 400, then I GET http://mail.google.com, which gets OK. I can read both addresses using my browser. I am behind a firewall, but I am able to get through it OK for the http protocol, and I think I'm gettign through with https as well, based on the rc 400.

This topic has been brought up in the past, and I've gathered snippets from all the threads I could find, to no avail. Any suggestions? How can I go to the next level of debugging (obviously, I can use -d)? Is there some way to look at the request that was sent to see if I can make out how my request is bad? I'm at my wit's end on this one.


Source

#!/usr/bin/perl -w use strict; use warnings; use Data::Dumper; use LWP::UserAgent; use Crypt::SSLeay; use Net::SSL (); # From Crypt-SSLeay $Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; # Force use of Net::SSL (? +) my $proxyinfo='http://bday:adsfa3@144.92.221.19'; $ENV{HTTPS_VERSION} = '2'; $ENV{HTTP_PROXY} = $proxyinfo; $ENV{HTTPS_PROXY} = $proxyinfo; $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0; $ENV{PERL_LWP_ENV_PROXY} = $proxyinfo; my $ua; my $url = 'https://mail.google.com'; print "--------------------------- Test #1: $url --------------------- +------\n"; $ua = LWP::UserAgent->new(show_progress=>1); $ua->cookie_jar( {} ); $ua->protocols_allowed( [ 'http','https'] ); $ua->env_proxy; # $ua->proxy(['https'], $proxyinfo); my $req = HTTP::Request->new('GET',$url); print Dumper($req); my $res = $ua->request($req); print $res->status_line; print $res->as_string; #$url = 'https://www.helsinki.fi/'; $url = 'http://mail.google.com/'; print "--------------------------- Test #2: $url --------------------- +------\n"; #my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 1 }); #$ua->protocols_allowed( [ 'http', 'https'] ); #bd added - may be unne +cessary $res = $ua->get( $url ); $res->is_success or die "Failed to GET '$url': ", $res->status_line; print $res->as_string; print "--------------------------- SSL in \%INC ---------------------- +-----\n"; print "$_\n" for grep { $_ =~ /SSL/ } keys %INC;

Results

Name "Net::HTTPS::SSL_SOCKET_CLASS" used only once: possible typo at . +/lwpssltest.pl line 8. --------------------------- Test #1: https://mail.google.com --------- +------------------ $VAR1 = bless( { '_content' => '', '_uri' => bless( do{\(my $o = 'https://mail.google.co +m')}, 'URI::https' ), '_headers' => bless( {}, 'HTTP::Headers' ), '_method' => 'GET' }, 'HTTP::Request' ); ** GET https://mail.google.com ==> 400 Bad Request (1s) 400 Bad RequestHTTP/1.1 400 Bad Request Cache-Control: no-cache Connection: close Pragma: no-cache Content-Length: 691 Content-Type: text/html; charset=utf-8 Client-Date: Mon, 24 Sep 2012 14:31:14 GMT Client-Peer: 144.72.231.19:80 Client-Response-Num: 1 Proxy-Connection: close Title: Request Error <HTML><HEAD> <TITLE>Request Error</TITLE> </HEAD> <BODY> <FONT face="Helvetica"> <big><strong></strong></big><BR> </FONT> <blockquote> <TABLE border=0 cellPadding=1 width="80%"> <TR><TD> <FONT face="Helvetica"> <big>Request Error (invalid_request)</big> <BR> <BR> </FONT> </TD></TR> <TR><TD> <FONT face="Helvetica"> Your request could not be processed. Request could not be handled </FONT> </TD></TR> <TR><TD> <FONT face="Helvetica"> This could be caused by a misconfiguration, or possibly a malformed re +quest. </FONT> </TD></TR> <TR><TD> <FONT face="Helvetica" SIZE=2> <BR> For assistance, contact your network support team. </FONT> </TD></TR> </TABLE> </blockquote> </FONT> </BODY></HTML> --------------------------- Test #2: http://mail.google.com/ --------- +------------------ ** GET http://mail.google.com/ ==> 200 OK HTTP/1.1 200 OK Cache-Control: private, max-age=604800 Connection: close Date: Mon, 24 Sep 2012 14:31:14 GMT Server: GSE Content-Length: 232 Content-Type: text/html; charset=ISO-8859-1 Expires: Mon, 24 Sep 2012 14:31:14 GMT Client-Date: Mon, 24 Sep 2012 14:31:14 GMT Client-Peer: 144.72.231.19:80 Client-Response-Num: 1 Refresh: 0;URL=http://mail.google.com/mail/ X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-XSS-Protection: 1; mode=block <html><head><meta http-equiv="Refresh" content="0;URL=http://mail.goog +le.com/mail/" /></head><body><script type="text/javascript" language= +"javascript"><!-- location.replace("http://mail.google.com/mail/") --></script></body></html> --------------------------- SSL in %INC --------------------------- Crypt/SSLeay.pm Crypt/SSLeay/X509.pm Net/SSL.pm

Versions

SunOS 5.10 Perl v5.10.1. LWP 6.04 Crypt::SSLeay 0.57 Net::SSL 2.84 LWP::UserAgent 6.04 LWP::Protocol::https 6.03
{}think; #Think outside of the brackets

In reply to LWP GET of https URL: 400 Bad Request by {}think

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others examining the Monastery: (9)
    As of 2014-11-28 19:39 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      My preferred Perl binaries come from:














      Results (200 votes), past polls