Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

I'm trying to use LWP to GET a page using the https protocol, and for the life of me, no matter what I try, I get a "400 Bad request" response from the server (or at least I assume it's from the server). In the source below, I use LWP to first GET https://mail.google.com, which fails with a rc 400, then I GET http://mail.google.com, which gets OK. I can read both addresses using my browser. I am behind a firewall, but I am able to get through it OK for the http protocol, and I think I'm gettign through with https as well, based on the rc 400.

This topic has been brought up in the past, and I've gathered snippets from all the threads I could find, to no avail. Any suggestions? How can I go to the next level of debugging (obviously, I can use -d)? Is there some way to look at the request that was sent to see if I can make out how my request is bad? I'm at my wit's end on this one.


Source

#!/usr/bin/perl -w use strict; use warnings; use Data::Dumper; use LWP::UserAgent; use Crypt::SSLeay; use Net::SSL (); # From Crypt-SSLeay $Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; # Force use of Net::SSL (? +) my $proxyinfo='http://bday:adsfa3@144.92.221.19'; $ENV{HTTPS_VERSION} = '2'; $ENV{HTTP_PROXY} = $proxyinfo; $ENV{HTTPS_PROXY} = $proxyinfo; $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0; $ENV{PERL_LWP_ENV_PROXY} = $proxyinfo; my $ua; my $url = 'https://mail.google.com'; print "--------------------------- Test #1: $url --------------------- +------\n"; $ua = LWP::UserAgent->new(show_progress=>1); $ua->cookie_jar( {} ); $ua->protocols_allowed( [ 'http','https'] ); $ua->env_proxy; # $ua->proxy(['https'], $proxyinfo); my $req = HTTP::Request->new('GET',$url); print Dumper($req); my $res = $ua->request($req); print $res->status_line; print $res->as_string; #$url = 'https://www.helsinki.fi/'; $url = 'http://mail.google.com/'; print "--------------------------- Test #2: $url --------------------- +------\n"; #my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 1 }); #$ua->protocols_allowed( [ 'http', 'https'] ); #bd added - may be unne +cessary $res = $ua->get( $url ); $res->is_success or die "Failed to GET '$url': ", $res->status_line; print $res->as_string; print "--------------------------- SSL in \%INC ---------------------- +-----\n"; print "$_\n" for grep { $_ =~ /SSL/ } keys %INC;

Results

Name "Net::HTTPS::SSL_SOCKET_CLASS" used only once: possible typo at . +/lwpssltest.pl line 8. --------------------------- Test #1: https://mail.google.com --------- +------------------ $VAR1 = bless( { '_content' => '', '_uri' => bless( do{\(my $o = 'https://mail.google.co +m')}, 'URI::https' ), '_headers' => bless( {}, 'HTTP::Headers' ), '_method' => 'GET' }, 'HTTP::Request' ); ** GET https://mail.google.com ==> 400 Bad Request (1s) 400 Bad RequestHTTP/1.1 400 Bad Request Cache-Control: no-cache Connection: close Pragma: no-cache Content-Length: 691 Content-Type: text/html; charset=utf-8 Client-Date: Mon, 24 Sep 2012 14:31:14 GMT Client-Peer: 144.72.231.19:80 Client-Response-Num: 1 Proxy-Connection: close Title: Request Error <HTML><HEAD> <TITLE>Request Error</TITLE> </HEAD> <BODY> <FONT face="Helvetica"> <big><strong></strong></big><BR> </FONT> <blockquote> <TABLE border=0 cellPadding=1 width="80%"> <TR><TD> <FONT face="Helvetica"> <big>Request Error (invalid_request)</big> <BR> <BR> </FONT> </TD></TR> <TR><TD> <FONT face="Helvetica"> Your request could not be processed. Request could not be handled </FONT> </TD></TR> <TR><TD> <FONT face="Helvetica"> This could be caused by a misconfiguration, or possibly a malformed re +quest. </FONT> </TD></TR> <TR><TD> <FONT face="Helvetica" SIZE=2> <BR> For assistance, contact your network support team. </FONT> </TD></TR> </TABLE> </blockquote> </FONT> </BODY></HTML> --------------------------- Test #2: http://mail.google.com/ --------- +------------------ ** GET http://mail.google.com/ ==> 200 OK HTTP/1.1 200 OK Cache-Control: private, max-age=604800 Connection: close Date: Mon, 24 Sep 2012 14:31:14 GMT Server: GSE Content-Length: 232 Content-Type: text/html; charset=ISO-8859-1 Expires: Mon, 24 Sep 2012 14:31:14 GMT Client-Date: Mon, 24 Sep 2012 14:31:14 GMT Client-Peer: 144.72.231.19:80 Client-Response-Num: 1 Refresh: 0;URL=http://mail.google.com/mail/ X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-XSS-Protection: 1; mode=block <html><head><meta http-equiv="Refresh" content="0;URL=http://mail.goog +le.com/mail/" /></head><body><script type="text/javascript" language= +"javascript"><!-- location.replace("http://mail.google.com/mail/") --></script></body></html> --------------------------- SSL in %INC --------------------------- Crypt/SSLeay.pm Crypt/SSLeay/X509.pm Net/SSL.pm

Versions

SunOS 5.10 Perl v5.10.1. LWP 6.04 Crypt::SSLeay 0.57 Net::SSL 2.84 LWP::UserAgent 6.04 LWP::Protocol::https 6.03
{}think; #Think outside of the brackets

In reply to LWP GET of https URL: 400 Bad Request by {}think

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others meditating upon the Monastery: (10)
    As of 2015-07-06 09:38 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









      Results (71 votes), past polls