That made for some interesting reading and you covered a lot of ground. Have you considered converting this into a tutorial? I can see where I would of benefited of such an article when I ran into problem with the LWP::* branch of modules a few weeks back.
Use head to determine if a site is up
The only thing I would add here is that you do not always get a valid HEAD response back from some web servers. This really screwed me for days when trying to write a Link Checker a few weeks back and I was directed to crzyinsomniac's LWP head replacement which works great and much more reliable than solely relying on the HEAD response.
(wil) Re: Getting more out of LWP::Simple