Find or get an email that says that they want you to remove the CPAN modules. Something like this,
You: 'My boss' had previously approved my releasing modules on CPAN before without issues.
Them: We know and we changed our minds because we don't like it.
Why do this? It is written proof that they acknowledge allowing you to release the code. You are giving yourself legal standing to say you were not a cowboy who was releasing code willy-nilly without some sort of approval. Keeps you from getting in big trouble.
