Those remarks in perlre are not specific to character classes, and one regularly thinks these character classes are more special.
Explicit mentioning of $ being special in character classes is found in perlretut#Using-character-classes:
…The special characters for a character class are -]\^$ (and the pattern delimiter, whatever it is). ] is special because it denotes the end of a character class. $ is special because it denotes a scalar variable.…
So indead not only punctation variables are being expanded:
E:\Temp>perl -Mstrict -we "my $foo=7;die qq(matched '$&'\n) if '123456