Re: The Unicode Bug with Transliteration or Substitution

[Anonymous Monk]

It could be :) so you have sample data to play with?

Have you tried utf8::upgrade($string)? Maybe you can try Unicode::Semantics

[choroba]

You can use the Japanese Wikipedia Perl page . Perl 5.8.3 at work outputs different files for

tr/ / /s; tr/\t/ /s;

and

s/ +/ /g; s/\t+/ /g;

I tested with diff -w against the original, i.e. ignoring whitespace.

utf8::upgrade didn't change anything, before or after the substitution/transliteration.