I believe you haven't set your PERL5LIB variable correctly. Assuming your home directory is /home/me, you'll need to set your PERL5LIB environment variable to /home/me/perl_mods/lib/site_perl/5.8.4. You may also have to include the architecture-specific directory, even though the perlrun man page says it searches them automatically. If so, set the variable to /home/me/perl_mods/lib/site_perl/5.8.4:/home/me/perl_mods/lib/site_perl/5.8.4/sun4-solaris-64int.
Try that and see if it works.
After you set it, you can run perl -V and see if it's picked up the change. Look in the @INC: section to see if it shows your new directory.