I thought I did. :-) IO::Select can be used to set a timeout on reading a socket filehandle. Why aren't you using it? I thought it was in one of the links I listed above.
FWIW, the method which I found to be workable, comes from the way event-loop systems, like Glib or AnyEvent does it. In those systems, when you you get the meta data from the socket connection that an 'IN' condition has occured, but yet there is no data there to be read, then your connection is caput. See Gtk2 server and client GUI's with root messaging for an example.
I believe that the "IO :: Poll," is better than "IO :: Selekt" because no limit on the number of sockets. I do not use "IO :: Seleсt" because I use IO :: Poll.
But this does not solve the problem.
Happy new year!