The problem is that you do NOT have any right to know that I installed your module from the CPAN, and it IS an invasion of my privacy for you to grab my IP address. When I intentionally go to your web site with my browser, it is a given that I am choosing to have you record who I am. Here, you are doing it without my permission or knowledge. You are capturing data about me against my will, which is a de facto breach of my privacy.
There's no two ways about it.
Your idea is not bad, but is done incorrectly. The whole thing would be solved if you asked permission before fetching it. I know that is a burden, but tough. You do not have the right to collect my data without my persmission, explicit or otherwise.