This is something like needing the runtime module for Visual Basic programs. If you make your .exe program fully self-contained, it will run on computers that don't have Perl installed, and won't require addional .dll's. The problem is that the self-contained .exe must have all this functionality in it, which makes it large.
It is possible to select options in perlapp that will create much smaller .exe binaries. You can do this, if you know your .exe is going to be run on machines that have Perl installed, and that have perl*.dll. When using perlapp, click on the "Size" tab and check the boxes for "Make dependent executable", and "Exclude perl58.dll from executable".
If you're running several .exe programs, it may pay to have Perl and the .dll installed, and creating smaller .exe files that can use them. If it's only one or two programs, or if you don't have the option of installing Perl, then you are pretty well stuck with a large .exe and the memory use.