Beefy Boxes and Bandwidth Generously Provided by pair Networks RobOMonk
No such thing as a small change
 
PerlMonks  

Regexp with unicode problem

by demis (Novice)
on Feb 08, 2010 at 12:48 UTC ( #822000=perlquestion: print w/ replies, xml ) Need Help??
demis has asked for the wisdom of the Perl Monks concerning the following question:

I am reading data from an excel file(ParseExcel0.32) in a local language.What I need to do is say:

if ($field=~/local_alphabet_string_that_I_copy_from_the_excel_cell_int +o_the_perl_script/){$field=...;}

any idea how to do it? It does not seem to work if I just copy the string in the regexp. I could convert to csv, but I wonder if there is a way to get the regexp to work as it stands(or with some modification)

Comment on Regexp with unicode problem
Download Code
Re: Regexp with unicode problem
by Gangabass (Priest) on Feb 08, 2010 at 12:56 UTC

    Try to convert all your text strings into UTF-8

      how do I do this? Not with use utf? This is for having utf in the code e.g.
      my $string='ΔΙΑΜΕΡΙΣΜ&#91 +3;'; if($field =~/$string/){ ....}
        Perhaps this (untested)
        use Unicode::String; Unicode::String->stringify_as( 'utf8' ); my $string = "This is latin text."; my $string_utf8 = Unicode::String::latin1( $string ); print $string_utf8;
        Never had to use it myself, but had this snippet from a long time ago.
        /\ Sierpinski
Re: Regexp with unicode problem
by ikegami (Pope) on Feb 08, 2010 at 15:04 UTC
    Could you adjust your script to look like
    use Data::Dumper; $Data::Dumper::Useqq = 1; my $pat = "local_alphabet_string_that_I_copy_from_the_excel_cell_into_ +the_perl_script"; print(Dumper($pat)); print(Dumper($field)); if ($field=~/$pat/){$field=...;}
    and give us the output. It'll tell us which variable is wrong and how it's wrong.
Re: Regexp with unicode problem
by Polyglot (Monk) on Feb 09, 2010 at 13:08 UTC
    As others have explained to me here previously, if you use unicode characters in your code, then you need to specify that in your uses. For example, if you are using utf8 characters within your code, you should specify "use utf8;". However, you do not need to make this declaration for characters that your code will act upon, only for those utf8 characters that are in the code itself.

    For characters outside your code, you may wish to make sure that they are recognized as utf8 by perl. The following examples present some ways of doing this--though you should not need to use them all at once. These are all lines which I have used at one time or another to deal with utf8.

    #FOR WORKING WITH UTF8, AS NECESSARY WITH CJK use Encode; #PARTICULARLY NEED THE ENCODE/DECODE FUNCTIONS use Encode qw(encode decode); #TO MAKE THE DEFAULT STANDARD ENCODING BE UTF8 use open qw( :std :encoding(UTF-8) ); #LIKE print CGI::header(); FOR UTF8 OUTPUT TO HTML print "Content-type: text/html; charset=utf-8\n\n"; #SET DEFAULT I/O TO UTF8 binmode STDOUT, ":utf8"; #OPEN/READ A UTF8 FILE open (DAT, '<:encoding(utf8)', $Data) or die "Can't open file! $!\n"; $source = <DAT>; close DAT;

    Blessings,

    ~Polyglot~

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://822000]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (8)
As of 2014-04-18 13:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (468 votes), past polls