Yep, this is normal. In their documentation for PerlEx, it states the following... see 2nd paragraph in particular:
in reply to PerlEx and .PM recompilation
"Before a Perl script is executed, the precompiler compares the last modification time of the script file against the last time it was compiled and cached by PerlEx. If the modification time is more recent, the PerlEx precompiler throws away the cached state, recompiles, and caches the compiled state of the script. The cached state of the script is then executed.
This ensures that PerlEx recognizes any changes to the script that you may make while PerlEx is running. However, any files or modules loaded by the script via require or use will not be reloaded by PerlEx in this way. If you change any modules used by your scripts, restart the web server or reload the PerlEx interpreters."