Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Regex Matching Unicode and Regex Classes

by choroba (Abbot)
on Nov 02, 2011 at 14:05 UTC ( #935404=note: print w/ replies, xml ) Need Help??


in reply to Regex Matching Unicode and Regex Classes

Two days ago, I was having the same issues. My testing script is a bit more elaborate, but confirms your problem: \w does not work under use locale, whatever the locale is. If you need locale, use posix classes or unicode character properties (provided your locale is unicode based).
Discussion on the details can be found in perlunicode and perllocale, but the relevant paragraphs are quite different in each version of Perl.

#!/usr/bin/perl use warnings; use strict; my @p_locale = qw/C cs_CZ.UTF8 en_US.UTF8/; my @locale = ('no locale', 'use locale'); my @posix = (q(), 'use POSIX qw/locale_h/; setlocale LC_ALL,"C"', 'use POSIX qw/locale_h/; setlocale LC_ALL,"cs_CZ.UTF8"', ); for my $p_locale (@p_locale) { for my $locale (@locale) { for my $posix (@posix) { print "$p_locale $locale $posix\n"; open my $OUT, '>', 'l.perl' or die "$!"; print {$OUT} << " OUT"; $locale; $posix; my \@chars = qw/283 269 345 225 32 98 99 48 49 50 51 52 32 353 253 382 237/; my \$string = join q[], map chr, \@chars; binmode STDOUT, ':utf8'; my \@regex = (qr/(\\w+)/, qr/([[:alnum:]]+)/, qr/(\\p{W +ord}+)/); for my \$regex (\@regex) { print "\t\$1" while \$string =~ /\$regex/g; print "\n"; } print sort qw/ci ch/; print "\n"; OUT close $OUT; $ENV{LC_ALL} = $p_locale; system 'perl', 'l.perl'; } } } unlink 'l.perl';


Comment on Re: Regex Matching Unicode and Regex Classes
Select or Download Code
Re^2: Regex Matching Unicode and Regex Classes
by McA (Priest) on Nov 02, 2011 at 14:58 UTC
    Hi choroba,
    thank you for your answer. I just checked it on my platform.

    By the way: A program generating code, mutating and running it, is called a virus. ;-)

    Best regards
    Andreas

Re^2: Regex Matching Unicode and Regex Classes
by choroba (Abbot) on Nov 04, 2011 at 10:34 UTC
    Tested on several versions of Perl: In 5.10.1 and 5.12.1, \w does not work. In 5.14.1, \w and [[:alnum:]] do not work. Should I report a bug?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://935404]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (9)
As of 2014-12-22 11:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (116 votes), past polls