Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Grab a page using openSSL

by CiceroLove (Monk)
on Jan 25, 2001 at 04:44 UTC ( #54169=perlquestion: print w/replies, xml ) Need Help??

CiceroLove has asked for the wisdom of the Perl Monks concerning the following question:

I am just wanting to connect to a site via https, download the page, parse through it, and reupload it securely. As it stands, My problem is three-fold: 1) Everyone and their monk have told me they do it all the time, 2) I cannot for the life of me see where I am any different from any other code I have seen on the net, and 3) I get a repsonse from snippet one that I don't understand. I am using (or have used) openSSL 0.9.3-0.9.6 as well as all current version of SSLeay.pm and LWP. Your help in bringing a young acolyte to inner harmony and joy will be greatly rewarded in the afterlife. Both snippets begin with:
use LWP; use LWP::UserAgent; use LWP::Protocol::https;
and then:
my $connect = new LWP::UserAgent; my $request = new HTTP::Request 'POST', 'https://secure.directnic.com/ +myaccount/myaccount.php'; my $response = $connect->request($request); print $response;
This prints HTTP::Response=HASH(nxnnnennn) where n is some number. The second snippet changes
print $response;
to:
print $response->content; print $response->as_string;
and gives me 500 (Internal Server Error) cannot connect to secure.directnic.com:443. I know that I can conect to it when I telnet to that port. Any suggestions from anyone?

Replies are listed 'Best First'.
Re (tilly) 1: Grab a page using openSSL
by tilly (Archbishop) on Jan 25, 2001 at 07:14 UTC
    I am going to go out on a limb and guess that your problem is that you do not have your proxy server set correctly for the https protocol and you are being blocked by a firewall. On Unix try this from the shell first:
    export https_proxy=http://your.proxy.machine:8080/
    where the data you put there is appropriate for whatever your local proxies are. And then try the following (switch quotes on Windows):
    perl -e 'use LWP::Simple; getprint("https://www.safeweb.com/");'
    Hopefully that will work. (If not then you have a very short script that should work to play with.)
Re: Grab a page using openSSL
by bsb (Priest) on Jan 25, 2001 at 08:46 UTC
    LWP::Debug might give you some more details of what is going on.

    use LWP::Debug qw(+);

    The LWP GET command line util can help:
    GET -eSU url

    I can't see any probs in your code, you need to be sure that it's not somewhere else (the proxy suggested earlier)

    Brad

      Proxy settings seem correct as I get a 404 when I change them and a 500 when they are reverted. However, the LWP::Debug was a goldmine. When run I get,
      LWP::UserAgent::new() LWP::UserAgent::request: () LWP::UserAgent::simple_request: POST url LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: ()
      When I try and change the last line to something like
      LWP::Protocol::https::request 'POST', 'url';
      in my script it tells me that it can't locate object method "new" via package "LWP::Protocol::https::Request". Does this mean that the https request method is not implemented? Thanks so much for your continued help and guidance.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://54169]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (11)
As of 2021-06-22 21:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)












    Results (108 votes). Check out past polls.

    Notices?