My general position on this is that the CPAN distribution is not the development tree; the CPAN distribution is a product of the development tree.
My repo may contain a bunch of extra files (author tests, Devel::Cover output, benchmarking scripts, etc) which are not included in the distro. And similarly the distribution will contain a bunch of auto-generated files (like META.yml, Changes, LICENSE, etc) that I have no desire to check into my repo.
However if I go under a bus tomorrow and somebody else needs to pick up the distro, they don't actually need any of that stuff. They could just download the tarball, extract the files, make a few changes, bump the version numbers, and then use tar to create a new tarball, and upload it.
OK, that might seem like a pain for them not to be able to use the author tests, etc I have in my repo, but actually if you think about it, it makes it easier for them to transition the distribution to their own build tools. They can start working on the distribution the way they like to work on software, not the way I like to work on software. (Especially considering I'm somewhat of an oddball in my development style!)