This is an excellent question, and I don't know the answer. The behaviour you describe is surprising, at least to me. The development version of perl exhibits the same behaviour. I wonder whether this is a known limitation, because there is a warning in perlfunc
Though if you try to re-open C<STDOUT> or C<STDERR> as an "in memory"
file, you have to close it first:
open STDOUT, '>', \$variable or die "Can't open STDOUT: $!";
Of course this isn't talking about encodings, but I think it might be the same problem.
Unless a PerlIO guru pops up here, I'd suggest asking p5p.
PS. If you just want a minimal demonstration script, you can delete most of your code:
# close STDOUT; # with this it works
open STDOUT, ">:encoding(cp37)", "foo.out" or die "Cannot redirect STD
print "yadda yadda\n";