The first thing I would look at is App::Staticperl, which mostly builds a Perl without dynamic module loading. Adding a selection of modules to that build should be fairly easy.

Another approach would be to just build the miniperl from source and then use that. This gives you a minimal Perl without any loadable modules and wiull give you a lot of headaches once you want to add more modules that need a C compiler.

I would at least talk to the people maintaining other Perl distributions to learn about their experiences with building and maintaining Perl binary distributions. For example the authors of Perl::Dist::Strawberry likely have experiences with how to add modules from CPAN into their build.