Yes, in general, 5.x.z will be compatible with 5.x.y. Changes that break this compatibility ("binary compatibility") should not get backported.
Of course, this only holds true for identical configuration options, either in the Win32/Makefile or via Configure. If you configure a Perl differently (with/without threads, different float sizes, ...), you won't get compatibility between Perl versions, or within the same Perl version even.
Now that I think/write about this, I don't think there is a real test between Perl versions at release time to ensure that compatibility test. I think the smoke testers will find out such a breaking change quickly, but a (manual) test step should be good there ...