Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: Does IO::Select work? (Problem partially resolved)

by BrowserUk (Pope)
on Oct 21, 2012 at 10:56 UTC ( #1000203=note: print w/ replies, xml ) Need Help??


in reply to Does IO::Select work? Anywhere?

The problems (long term; I previously put it down to my not using the module correctly), I've been encountering with IO::Select come down to the fact that the module relies upon fileno to do its thing.

Where this falls down is if the handle gets closed before the program gets around to remove()ing it from the IO::Select object. For example, if you code:

... $client->close; $sel->remove( $client ); ...

You won't get any errors or warnings, but your server simply will not work correctly.

Because the socket was closed before it was removed, fileno( $client ) will return undef and the attempt to remove the handle from the select object fails. SILENTLY!

After that, pretty much nothing works properly.

I realise that the above code can be seen to be in error, and switching the order of the two statements makes it work. But that doesn't cover all the bases, because it is perfectly possible for a file handle to get closed without the program doing it explicitly.

This turns out to be a known problem that was reported in 2010, previously reported in 2005 and (apparently) probably existed since circa. 1998..

And, despite that the RT suggests a fix has been applied in May 2010, inspecting the CPAN source shows that it will still fail -- silently -- today.

Even a simple warning that you've made an attempt to remove a closed handle would help; but it isn't hard to fall back on a linear search of the array of file handles and relate their position in the array back to a fileno and hence bit vector bit.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong


Comment on Re: Does IO::Select work? (Problem partially resolved)
Download Code
Re^2: Does IO::Select work? (Problem partially resolved)
by Anonymous Monk on Oct 21, 2012 at 11:02 UTC

      Hm. So, the CPAN source is *NOT* the place to go for the latest version. That sucks!

      And, according to IO::Select: allow removal of IO::Handle objects without fileno the fix was implemented in January 2011, and I am using 5.16:

      C:\test>asyncServer2.pl Perl -v: 5.016001 IO::Socket: 1.34 IO::Select: 1.21 ... C:\test>head -20 \perl64-16\lib\IO\Select.pm # IO::Select.pm # # Copyright (c) 1997-8 Graham Barr <gbarr@pobox.com>. All rights reser +ved. # This program is free software; you can redistribute it and/or # modify it under the same terms as Perl itself. package IO::Select; use strict; use warnings::register; use vars qw($VERSION @ISA); require Exporter; $VERSION = "1.21"; @ISA = qw(Exporter); # This is only so we can do version checking sub VEC_BITS () {0} sub FD_COUNT () {1} sub FIRST_FD () {2}

      which I'm pretty sure came out after that and still seeing the problem?

      Perhaps because the 'fix', removes the handle from the array, but fails to remove it from the bitmask:

      } else { # remove if ( ! defined $fn ) { # remove if fileno undef'd defined($_) && $_ == $f and do { $vec->[FD_COUNT]--; $_ = und +ef; } for @{$vec}[FIRST_FD .. $#$vec]; next; ####################################################### +#### ???? } my $i = $fn + FIRST_FD; next unless defined $vec->[$i]; $vec->[FD_COUNT]--; vec($bits, $fn, 1) = 0; $vec->[$i] = undef; }

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      RIP Neil Armstrong

Re^2: Does IO::Select work? (Problem partially resolved)
by andal (Friar) on Oct 22, 2012 at 07:32 UTC
    But that doesn't cover all the bases, because it is perfectly possible for a file handle to get closed without the program doing it explicitly.

    Are you sure about this? Well, I've never programmed under Windows, but under Linux I haven't seen yet any single case when the file descriptor would be closed by the kernel. The rule is simple, if the user opens a file descriptor, the user has to close it. There could be libraries that close it for you, but those libraries are running in the application space.

    So, if you make sure, that you remove file descriptor from "select" before you close it, then things work fine.

    As to example, I've never used IO::Select module, but many years ago I've created more or less simple module that uses "select" directly. This module still works for many of my application, even though I consider it to be very poor. You can get it at http://vandal.sdf-eu.org/NetHandling.pm if you want.

    I guess the main reason why one can't find good examples of using "select" is just the fact, that it is not that simple. To make things really working, one has to take into account many details. As result the code becomes hard to understand. Actually, if one tries to implement the same stuff using threads, then at the end the code would become equally complex and hard to understand. So I don't believe one would be able to provide simple but yet robust example of using select or threads for networking.

      Are you sure about this?

      Yes. Pretty sure.

      I've never programmed under Windows,

      Don't write this off as a "windows problem".

      but under Linux I haven't seen yet any single case when the file descriptor would be closed by the kernel.

      You may not have, but others have:

      Platform: osname=linux, osvers=2.6.24-27-server, archname=i486-linux-gnu-thread +-multi uname='linux vernadsky 2.6.24-27-server #1 smp fri mar 12 01:45:06 ut +c 2010 i686 gnulinux '
      As to example, I've never used IO::Select module, but many years ago I've created more or less simple module that uses "select" directly. This module still works for many of my application, even though I consider it to be very poor. You can get it at http://vandal.sdf-eu.org/NetHandling.pm if you want.

      Thanks for teh offer, but the way I eventually stopped writing off my long term lack of success with IO::Select as "user error" was by re-implementing (most) of the API without reference to the module source:

      I guess the main reason why one can't find good examples of using "select" is just the fact, that it is not that simple.

      Maybe, but wrapping over some of the low-level nitty-gritty in an IO:Select-type module certainly makes it easier.

      But if there has been a latent bug in that module for the last 12 or more years that prevented people's attempts from working, it would have had a significant effect upon the availability of good examples.

      And given that there are plenty of examples of many other things that are at least as complex, I find that a pretty persuasive argument for that inhibitory effect.

      I've never really needed to use select for serious work -- I find threading far simpler, more intuitive and more powerful -- on windows at least -- so my previous attempts have been half-hearted -- usually with a view to demonstrating the virtues of threading over the select model -- and so I never had any real incentive to work past the problems.

      Now I have understood the problem that has dogged my attempts for years, I feel the need to write a (pair of) full-featured servers. In part so I know I have done so. In part to allow me to make the real-world comparisons and measurements that will test my gut feel hypothesis about the relative merits of the two approaches.

      Watch this space. (But don't hold your breath :)


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      RIP Neil Armstrong

        You may not have, but others have

        Just to clarify things a little. The page you have referenced, does not provide any specific example where the file descriptor would be closed by something other than the program itself. And I dare to say, that Linux kernel never closes handles opened by application unless application requests it. I don't believe perl would do it either. If you know about such place in perl, please point it out, so that I could avoid this swampy area in the future.

        I don't know how the things are done on Windows. But really, if Windows kernel closes your handles behind your back, then it is another reason for me not to use it :) It's a joke of course. But practical outcome is then, that IO::Select is not usable on Windows, but it shouldn't introduce any problems on Linux, as long as the user does the right thing. IMHO.

      If BrowserUK said it, yes, he is sure.   And sure to be right.   Seriously.
        If BrowserUK said it, yes, he is sure. And sure to be right. Seriously.

        Well, if BrowserUK is so sure, and you know that he is right, then all I'd like to see is the description of situations where the kernel would close your file descriptor without you requesting it. Once you show it to me, I'll be happy to say, that he is right. Until you've done it, I prefer to stick to my own experience. I believe this is fair enough :)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1000203]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (11)
As of 2014-12-17 22:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (36 votes), past polls