Although tybalt89 has already said several of these things, just to summarize: -M is the "Script start time minus file modification time, in days." The "Script start time" is kept in $^T, which is in seconds since the epoch. The file's mtime is also reported in seconds. So internally, Perl is doing a division and subtraction - it's too late in the evening here for me to go digging around in the Perl sources, but maybe there's some inaccuracy happening there.
My opinion: just don't use -M in the first place. Use stat or File::stat to get the mtime and do the comparisons to time and/or $^T yourself, then you've got everything in seconds and can do the calculations on your terms. Since you're already using -M, I assume you don't need to-the-nanosecond precision, so calculating with 1 day = 86400 seconds is probably good enough.
