Re^5: Unicode substitution regex conundrum

in reply to Re^4: Unicode substitution regex conundrum
in thread Unicode substitution regex conundrum

Hi, I may be too late, seeing your message has been here since nearly two months, but it could still be of use to you or someone else.

I may be wrong, but it seems to me like the problem does not reside with the whitespaces, but with the definition of word in Perl : \w+ does not match chinese characters.

On my system (with unicode locale and chinese readable in the console) :

$ perl -le 'print "ok" if ("&#25105;&#36208;" =~ m/\w+/)'
$ perl -le 'print "ok" if ("hi" =~ m/\w+/)'
ok
[download]

(Chinese chars were jumbled, I didn't put the codes in the one-liner)

Furthermore, I played a bit with your code, and when I replaced

$terms =~ s/($word)\p{IsSpace}*($word)/$1 AND $2/g for 1..2;
[download]

with

$terms =~ s/(\p{IsSpace}/ AND /g;
[download]

it did the job I expected of it.

The quickest workaround I see at the moment would be to declare $word using CJK character ranges instead of \w.

Hope I could be of help.

Lu.

In Section Seekers of Perl Wisdom