Can you provide some code? It sounds like you've generated a reference somewhere that doesn't want to go away. Perl's garbage collector is based on reference counting, i.e. it reclaims a variable's memory once no references to it remain. If you've managed some convoluted circular reference magic (or DBI has done some of it for you, which happens if you don't clean up your handles properly), you could have a variable at the end of your program that resists removal. It's not a critical failure, like you say, but an annoying one.
Short of seeing some code, I'd suggest you go back and finish all your handles after you're done with them; that has helped me with similar problems in the past.