Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re: Regex Matching Unicode and Regex Classes

by choroba (Abbot)
on Nov 02, 2011 at 14:05 UTC ( #935404=note: print w/ replies, xml ) Need Help??

in reply to Regex Matching Unicode and Regex Classes

Two days ago, I was having the same issues. My testing script is a bit more elaborate, but confirms your problem: \w does not work under use locale, whatever the locale is. If you need locale, use posix classes or unicode character properties (provided your locale is unicode based).
Discussion on the details can be found in perlunicode and perllocale, but the relevant paragraphs are quite different in each version of Perl.

#!/usr/bin/perl use warnings; use strict; my @p_locale = qw/C cs_CZ.UTF8 en_US.UTF8/; my @locale = ('no locale', 'use locale'); my @posix = (q(), 'use POSIX qw/locale_h/; setlocale LC_ALL,"C"', 'use POSIX qw/locale_h/; setlocale LC_ALL,"cs_CZ.UTF8"', ); for my $p_locale (@p_locale) { for my $locale (@locale) { for my $posix (@posix) { print "$p_locale $locale $posix\n"; open my $OUT, '>', 'l.perl' or die "$!"; print {$OUT} << " OUT"; $locale; $posix; my \@chars = qw/283 269 345 225 32 98 99 48 49 50 51 52 32 353 253 382 237/; my \$string = join q[], map chr, \@chars; binmode STDOUT, ':utf8'; my \@regex = (qr/(\\w+)/, qr/([[:alnum:]]+)/, qr/(\\p{W +ord}+)/); for my \$regex (\@regex) { print "\t\$1" while \$string =~ /\$regex/g; print "\n"; } print sort qw/ci ch/; print "\n"; OUT close $OUT; $ENV{LC_ALL} = $p_locale; system 'perl', 'l.perl'; } } } unlink 'l.perl';

Comment on Re: Regex Matching Unicode and Regex Classes
Select or Download Code
Re^2: Regex Matching Unicode and Regex Classes
by McA (Deacon) on Nov 02, 2011 at 14:58 UTC
    Hi choroba,
    thank you for your answer. I just checked it on my platform.

    By the way: A program generating code, mutating and running it, is called a virus. ;-)

    Best regards

Re^2: Regex Matching Unicode and Regex Classes
by choroba (Abbot) on Nov 04, 2011 at 10:34 UTC
    Tested on several versions of Perl: In 5.10.1 and 5.12.1, \w does not work. In 5.14.1, \w and [[:alnum:]] do not work. Should I report a bug?

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://935404]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (22)
As of 2014-07-23 10:29 GMT
Find Nodes?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:

    Results (140 votes), past polls