I recently discovered
Net::Sockets::NonBlock which is a nice solution for the server side, taking care of all the IO and select calls for you. The example in the documentation is a proxy server, which was exactly what I was looking for, but if you get rid of some of the code you have a ready to use listen server, you just need to add in some code to parse/handle data coming from clients.
Threads aren't required for what you want to do, although they could be used, depending on what exactly you want to do. Net::Sockets::NonBlock isn't thread compatible though, so you'll need to find another solution if you want to use a threaded solution.