I would rather not have to install this module
Just do it. You're not really using Perl if you refuse to take advantage of modules. Dependencies are a good
thing. You're likely afraid of change and overestimating the effort of installing, but with the right knowledge and tools it's easy. I don't want to go off-topic in this thread, so open a new question if you want further advice.
Is it not possible to do it with the current way I have the command?
No, there's only one Perl module distro that accurately calculates date and time, and that's the one I named. POSIX and Time::Piece offer only crude calculation and place the burden on the implementing programmer to get the details right, which is wrong because it always results in inaccuracy.
would I have to convert the "amount of time into the past" into all seconds … then subtract them
Sounds like a plan that can't go wrong… except you neglect to account for leap seconds. The magic number 86400 you used earlier is likewise flawed due to daylight saving time changing the length of a day.