++Juerd great detective work!
I raised this exact issue on the perl-xml list earlier this week after I received email from an XML::Simple user who was having problems with a memory leak. I did manage to reduce it to a simple test case that involved only XML::Parser, but I hadn't worked out how to proceed with identifying the cause. (I foolishly assumed Devel::Symdump would only work with package globals and was unlikely therefore to help in my quest).
vek is correct that matts offered to take XML::Parser under his wing earlier this year. However, he's been very busy recently so if someone else wanted to volunteer, then I'd suggest taking the discussion to the perl-xml mailing list.