Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Quest for the Elusive Non-Blocking SSL Client

by bill.bbennett (Novice)
on Dec 17, 2010 at 16:27 UTC ( #877656=perlmeditation: print w/ replies, xml ) Need Help??

My Quest for the Elusive Non-blocking SSL Client

I have been trying to construct an SSL client which works in non-blocking mode on a variety of platforms (i.e. it needs to run under UNIX (or Linux) and Windows). It uses a custom select dispatcher, which I am at this point unwilling to change (it would affect too much existing code).

I have looked at the following modules for implementing this client:

  • Net::SSLeay - I have achieved partial success. I was able to get a non-blocking SSL client working. It required a lot of time-consuming experimentation. The problem is that I can't get Net::SSLeay to install on the PC. Further, my impression is that Net::SSLeay seems to be fading, being replaced by "better" modules, like Net::SSL, and IO::Socket::SSL. In the section "How I got Net::SSLeay to work" below, I explain what I wound up with at the end of the day for Net::SSLeay. It may be of interest to those on a similar quest.
  • IO::Socket::SSL - This looks like it would be the best way to go, if I could get it to work, it would simply be a matter of replacing the IO::Socket::INET objects with IO::Socket::SSL objects. There are two flies in this ointment:
    1. I can't get it to install under either UNIX or Windows.
    2. Its implementors claim that:
      Non-blocking and timeouts (which are based on non-blocking) are not supported on Win32, because the underlying IO::Socket::INET does not support non-blocking on this platform.
      ( I'm not sure what they're getting at, I use non-blocking IO::Socket::INET objects all the time in Win32, and they seem to work (although they exhibit behavior which is slightly different from UNIX). )
  • Net::SSL - This installs on both UNIX (i.e. snow leopard) and Windows, but I haven't been able to get even a blocking client working yet. Could anyone offer me some clues (the documentation is rather sparse). In particular:
    • What is the "configure" method all about? Do I need to invoke it, and, if so, how?
    • What do I need to do with the "ssl_context" method?
    • What is the proper invocation of the "connect" method?

How I got Net::SSLeay to work

Being primarily a UNIX guy, I usually implement first on UNIX (in this case snow leopard), and then port to Windows. After much weeping and gnashing of teeth, I was able to get such a client working on my Mac using Net::SSLeay. The weeping and gnashing of teeth occured primarily not because Net::SSLeay is bad, or difficult, but because it isn't really well documented. In the process of implementation, here's what I arrived at:

  • At the begining there are a number of magical incantations you have to make to make Net::SSLeay work. And they "have to be executed once". Here's what I found on the web (I can't vouch too much for it, but it seems to work):
    use Net::SSLeay qw( die_now die_if_ssl_error ); Net::SSLeay::load_error_strings(); eval 'no warnings "redefine"; sub Net::SSLeay::load_error_strings () {} '; die $@ if $@; Net::SSLeay::SSLeay_add_ssl_algorithms(); eval 'no warnings "redefine"; sub Net::SSLeay::SSLeay_add_ssl_algorithms () {} '; die $@ if $@; Net::SSLeay::ENGINE_load_builtin_engines(); eval 'no warnings "redefine"; sub Net::SSLeay::ENGINE_load_builtin_engines () {} '; die $@ if $@; Net::SSLeay::ENGINE_register_all_complete(); eval 'no warnings "redefine"; sub Net::SSLeay::ENGINE_register_all_complete () {} '; die $@ if $@; Net::SSLeay::randomize(); eval 'no warnings "redefine"; sub Net::SSLeay::randomize (;$$) {} '; die $@ if $@;
  • You have to understand that you are dealing with an IO::Socket object (which you can make non-bocking and do a select upon) and a Net::SSLeay object, to which you connect the IO::Socket (the SSL "connect" is not to be confused with the IO::Socket "connect", they are different). Further, you have to understand (and be willing to live with) the fact that sometimes you will wind up doing a "quasi-busy" wait, i.e. you will continue to select true and retry io operations while Net::SSLeay does its connecting/packing/unpacking operations underneath you. This does not appear to be too much of a practical problem in my experience.
  • To initiate a connection, you first connect an IO::Socket in the normal way:
    my $socket = IO::Socket::INET->new( PeerAddr => $host, PeerPort => 'https(443)', AutoFlush => 1, Blocking => 0, Proto => 'tcp' ) or die "can't connect to https server";
  • Then you create an "ssl" context and object (I suppose the "context" could be created once at the beginning and reused of all of your ssl objects, but what I've got seems to work, and I'm a little reluctant to make gratuitous changes.):
    my $ctx = Net::SSLeay::CTX_new() or die("Failed to create SSL_CTX $!"); Net::SSLeay::CTX_set_options($ctx, &Net::SSLeay::OP_ALL) and ssl_check_die("ssl ctx set options"); my $ssl = Net::SSLeay::new($ctx) or die_now("Failed to create SSL $!");
  • By the way, this little fragment uses the following little functions, which I stole off the web (along with the magical incantations above):
    sub ssl_get_error { my $errors = ""; my $errnos = []; while(my $errno = Net::SSLeay::ERR_get_error()) { push @$errnos, $errno; $errors .= Net::SSLeay::ERR_error_string($errno) . "\n"; } return $errors, $errnos if wantarray; return $errors; } sub ssl_check_die { my ($message) = @_; my ($errors, $errnos) = ssl_get_error(); die "${message}: ${errors}" if @$errnos; return; }
  • Once you have a socket and an ssl object, you need to connect the two. Since the socket is non-blocking, this will fail with $! eq "resource temporarily unavailable" until the key exchange is complete. So, you need to select on the socket for both read and write, and whenever it selects true for either, execute the following code:
    Net::SSLeay::set_fd($ssl, fileno($socket)); my $res = Net::SSLeay::connect($ssl) and Net::SSLeay::die_if_ssl_error("ssl connect"); if($res < 0){ # Here the connect failed because the exchange is not complete; # continue to select on this socket and retry every time the # socket selects true until it succeeds. return; } else { # Here the connect is complete. This is where to but the code # that changes the "select" handler for the socket for the next # phase (writing the data). }
  • Once you have connected the ssl object to the socket, then you need to select on the socket for write. Whenever it selects true, write your header and content to it sequentially. You need to check for EAGAIN (which in this case manifests itself as "Resource temporarily unavailable") and check the number of bytes you actually wrote, etc. Here's a fragment of code (which is actually working at this time, as part of the "write" select handler):
    my $written = $ssl->write(substr($cur_buff, $cur_count)); if($written <= 0){ unless($! eq "Resource temporarily unavailable"){ ## An error has occurred - recovery code goes here } return; } $cur_count += $written; if($cur_count == length($cur_buff)){ $cur_count = 0; $cur_buff = undef; ### We have emptied the current buffer ### Here is where code specific to select dispatcher ### must determine whether we have written all the data... }
  • When all of the data has been written to the socket, you need to "shutdown" the write side of the socket so that it will be unpacked on the other side:
    my $res = CORE::shutdown $socket, 1;
  • Then you have to arrange to read the response. This means you select on the socket for read, and when it selects true, do the following:
    my $rb = $ssl->read(16384); ssl_check_die("SSL read"); if(undefined($rb) or length($rb) <= 0){ unless($! eq "Resource temporarily unavailable"){ ## Here we are done - do whatever is necessary to ## shutdown select dispatcher and close sockets, release ## ssl and context } if($rb){ ## you have read some data in $rb -- do something with it } }
  • Cleaning up after this involves arranging (in the context of your select dispatcher) to have the following code executed at the end:
    Net::SSLeay::free($ssl); Net::SSLeay::CTX_free($ctx); $socket->close();

References

The code which I "stole" off the web, came for the following source:

http://devpit.org/wiki/OpenSSL_with_nonblocking_sockets_(in_Perl)

Since I was writing a client and his sample code was for a server, I didn't use much of his code, but the magical incantations were useful. I didn't implement his "drain the socket on read strategy", because the server always closes when its finished writing and "read" always selects true on a closed socket, so I don't have to worry about orphan data in the SSL buffer.

Comment on Quest for the Elusive Non-Blocking SSL Client
Select or Download Code
Re: Quest for the Elusive Non-Blocking SSL Client
by syphilis (Canon) on Dec 18, 2010 at 22:23 UTC
    IO::Socket::SSL builds and installs fine for me on Windows. What was the problem you came up against ?

    Cheers,
    Rob

      Rob,

      Thanks for the reply. Here's what I'm seeing:

      • The install of IO::Socket::SSL fails because it can't find Net::SSLeay.
      • Of course, if I could install Net::SSLeay, I wouldn't need IO::Socket::SSL. I already have code which works with Net::SSLeay (at least under snow leopard).
      • Net::SSLeay fails because it can't find openssl. It asks me to set the OPENSSL_PREFIX environment variable.
      • It looks like there is a directory under C: which contains OpenSSL (after I download and install the latest openssl, which for some reason, doesn't set the OPENSSL_PREFIX environment variable). So I set the environment variable.
      • However, this version of openssl is newer than the openssl with which Net::SSLeay was built (which is not surprising, because I just downloaded it).
      • It still tries to install, but ultimately, it fails...

      Does this mean that I have to abandon the package managers and go rebuild Net::SSLeay from source?

      Thanks,

      Bill B

      Hm. I have IO::Socket::SSL running on Linux box. I didn't even install it, it came bundled with perl (gentoo linux)

      But as a side note, the openssl library really sucks for not providing direct support for non-blocking reading/writing. I guess thinking only in threads plays its role in design. Please, don't take it as complain. Just pointing out, that things are not simple even outside of Perl :)

Re: Quest for the Elusive Non-Blocking SSL Client
by ambrus (Abbot) on Dec 19, 2010 at 14:45 UTC

    Did you try AnyEvent::Handle? It comes with SSL support, backed by Net::SSLeay, which, funnily, uses the openssl library (not the ssleay library).

      Thanks for the suggestion, but the problem isn't doing event handling, its getting Net::SSLeay to install under windows. If I could get Net::SSLeay to install, then I could try my working code (which works very well on snow leopard) under Windows.

      Also, when I say "Windows", I guess I mean "ActiveState Perl on Windows", and when I say "Install", I mean "Install using cpan or ActiveState Package Manager".

      I'm really not much of a Windows person.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://877656]
Approved by Arunbear
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (13)
As of 2014-07-22 12:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (111 votes), past polls