It's not "unpredictable" in the sense of being random but it is undefined in the sense that it is not specified in the documentation and could potentially change in a future release. Quoting perlsyn:
in reply to Re^4: self-feeding infinite loop
in thread self-feeding infinite loop
If any part of LIST is an array, foreach will get very confused if you add or remove elements within the loop body, for example with splice. So don't do that.
I should add for the record that I once asked the same question myself.