print unicode characters from hex format

[sistermaryguacamole]

Greetings,

I have what I thought would be a simple issue. I'm reading a YAML file full of user names, email, phone, etc. Many are French-Canadian, and have accented characters. In the file I'm reading, it looks like:

jean-fran\xe7ois chr\xe9tien

I know that \xe7 is "į", and \xe9 is "é", etc., but it prints to the terminal as just \xe7, \xe9.

I've looked up all sorts of stuff: use utf8; use Encode; binmode(STDOUT, ":utf8"), blah blah blah.

I just want to print the stupid messed up characters the way they're supposed to look; for the love of God, please, help me.

(The next step, of course, is to forbid our French-Canadian employees to use ridiculous non-english characters when creating user accounts - but one thing at a time).

Regards & God Bless,

Sister Mary Guacamole

[choroba]

You can use substitution to replace the codes by acutal characters, and set the IO layer of the output to accept the encoding:

#!/usr/bin/perl
use warnings;
use strict;
use feature qw{ say };

my $string = 'jean-fran\xe7ois chr\xe9tien';
$string =~ s/\\x(..)/chr hex $1/ge;
binmode STDOUT, 'encoding(UTF-8)';
say $string;

Also note that telling anyone their name is "ridiculous" might sound inpolite.

Update: Simplified using binmode.

[kennethk]

How are you importing the YAML? The issue here is not that the data is being written incorrectly (you have the literals properly represented in the in-memory string) but that whatever is doing your data import is not doing the necessary character unescaping. You could fix this as choroba recommends with a regex post-fix, but this should really be handled in your serialization/deserialization layer.

[RonW]

Expanding on kennethk's post, the YAML::XS will process the character escapes for you.

[Anonymous Monk]

In case you are talking about Windows, there are some obstacles:

I recently found that Data::Dumper on a Windows Console migth mess it up, maybe you find
$Data::Dumper::Useperl = 1;
helpful in that case or avoid Data::Dumper. I havenīt dug deeper than that, as it was just the debugging output that was not as expected.

To change the consoleīs codepage on Windows to utf-8:
chcp 65001
and the window should use a utf8-capable font like Lucida console.

On Linux you could check if a utf-8 locale is set, if you use a certain terminal like Putty, check for that being set to e.g. UTF8. In case of doubt, do the output and convert to hex to see what is going on and what changes throughout changes.

If those characters are formatted right in memory, saving them to a file requires that files output mode set to UTF-8.

e.g. for File::Slurp
write_file ($filename, {binmode => ':utf8'}, $string);