|Keep It Simple, Stupid|
Take a look at Bill Pratt's book on image processing ISBN 0 471 01888-0 on page 232. It describes Two dimensional unitary transforms. (Such as the Fourier transform.) In particular on page 233 he describes the fact that such transforms are separable by dimension (equations 10.1-5 and 10.1-6). This process says take the one dimensional transform along columns and leave the data in place. Then run over the rows with one dimensional transforms. You should be able to extend this into further dimensions in order. You are left with the N-dimensional transform in place. So handling N-dimensions as single dimension arrays is not so punishing as you have to work that way anyway.
Here's another ref - Digital Image Processing by Gonzalez and Wintz -ISBN 0-201-02597-3 page 50 where they describe the separability of using one dimensional FT's as applied to each dimension in succesion.
So you may have to the reinvent part of the wheel, but it is a reasonable extension from the Math::FFT module that exists.
It sounds like a great module to put back into CPAN when you have it nailed.
Its interesting that MATLAB has a foundation in Perl. It installs Perl in order to run.
Best of Luck,