First, you setup your server:
in reply to How do I do a non-blocking accept?
Then make it non-blocking, for perls that don't obey the Blocking param above:
my $server = IO::Socket::INET->new( %params, Blocking => 0 );
Then you add that server to your select vector watching for read events. When a client is waiting to be accept()ed, the select will return your server as ready to read. Just compare the sockets using == to determine if the socket is your server, and if it is, issue accept(). Note that by the time you get to it the client may have changed his mind, and so your accept could return EWOULDBLOCK or EAGAIN, so you have to check for that.