I guess that I'm the "devil's advocate" to the "devil's advocate"?
re: File::Find - I think that we could cooperate and possibly increase internal performance (I'm game for that), but the interface is "spot on" - it works!.
My suggested modifications to the OP's code represents a massive simplification of program logic.
There is only one file system operation that happens per $File:Find::name. Maybe File::Find does some more "under the covers"? I'm not sure what you are proposing... But basically, I see no problem with code that makes a single decision based upon a single input.
I'm game to increase the performance of File::Find - are you willing to help me do it?
I think that will be be a pretty hard undertaking.
I'm not sure that it is even possible.
But if it is, let's go for it!