Or, in looking, they didn't evaluate well enough to determine that the goals and/or implementation of another project really were
in line with what they were doing. It may not have worked "because of Y", but "Y" may have been perfectly in line with both their own goals and the the goals of "X module".
For example, I had developed a module that seemed like a nice generalist solution to coding web applications. I'd even done some research first to see if I could use something else, but hadn't seen anything that seemed quite in the vein I was considering; at the time, I can see now, I wasn't terribly clear on the scope of my own project. However, when I started looking around for a namespace after developing my implementation, I ran across CGI::Application again, and, in thoroughly reviewing it, discovered that it did what I was doing, and better. I could have saved a lot of effort if I'd been more clear in what I needed to do and more flexible in seeing how another module could be adapted to suit my needs.
I think that's the issue I see most frequently with duplicate projects on CPAN -- new authors haven't completely reviewed the scope of their own projects, much less that of existing projects, including the TODO lists and goals. More often than not, it appears that they could have simply contributed to existing modules, thus improving the codebase out there and preventing needless duplication of effort.