Are you unfamiliar with how an LALR(1) grammar is parsed? (No, I don't know if Perl strictly constitutes an LALR(1) grammar, but it does use yacc / bison and so at least much of the parsing actually follows the LALR(1) rules.)
Your conclusion would only be correct if both PHP and Ruby behaved the same for the equivalent of Perl's:
which I couldn't even get PHP to compile (no Ruby handy).
If you think Perl is bending some rules, then please point out the rule-bending code that it gives to yacc / bison. The fact is that declared precedence only comes into play when a "shift/reduce conflict" is found in generating the parser state machine.
What I see is simply:
Perl could make the grammar definition more complicated by not using yacc / bison declared precedence to implement the precedence of just the one ternary operator since it is the only case where there is a "middle argument" where declared precedence doesn't matter. But that would involve specifying yet another primitive for "expression" and having to pick the right one to use in each of the dozens of places where the current 'term' is used.
I find it very Perlish that ?: precedence is enforced by declared precedence, not proscriptive grammar definition.
(Updated: to add the relevant bits from perly.y)