I probably don't know what I'm talking about, but I'm wondering if there is an issue with Math::Pari (and/or required libraries/modules/etc.) and having them in a 64-bit version.
For a comparison, I took a quick look at what Strawberry Perl includes in their distributions. For 32-bit 184.108.40.206 (release notes), they do include the Math::Pari module. However, for 64-bit 220.127.116.11 (release notes), they do not include that module. In the "Known Issues" section of the release notes, they have a link to a bug report (https://rt.cpan.org/Public/Bug/Display.html?id=52146). That bug report might give you some clues to help you with your situation.