Re: The “real length" of UTF8 strings

length returns the number of characters. To get the length in bytes, you have to convert the string into a given encoding:

my $s="\x{5fcd}\x{65e0}\x{53ef}\x{5fcd}";
use Encode;
print length encode("utf8", $s), "\n"; # 12
[download]

Since Unicode strings are stored in utf8 internally, you can use a number of hacks to avoid the explicit re-encoding:

print do {use bytes; length($s)}, "\n"; # 12 (see perldoc -f length)

# or

utf8::encode($s); # resets the utf8 flag
print length($s), "\n"; # 12
[download]

Comment on Re: The “real length" of UTF8 strings Select or Download Code

Replies are listed 'Best First'.
Re^2: The “real length" of UTF8 strings by moritz (Cardinal) on Sep 23, 2008 at 20:35 UTC
In general the correlation between byte length in UTF-8 and visual character width is only a weak one. For example many european non-ASCII-characters are printed with visual width of only one character, but encoded as two bytes. The Euro sign `€` is even encoded as three bytes, and still printed with a width of only one.	[reply] [d/l]
Re^3: The “real length" of UTF8 strings by betterworld (Curate) on Sep 23, 2008 at 21:54 UTC
You're right. I misunderstood the question and did not realize that we are looking for the "visual length". Well, I think it depends on the font then, doesn't it?	[reply]
Re^4: The “real length" of UTF8 strings by Anonymous Monk on Sep 24, 2008 at 04:21 UTC
Yes betterworld, I should have choosen a better title like "the visual length of UTF8 strings" instead of the "real length" which leads to confusion.	[reply]
Re^4: The “real length" of UTF8 strings by moritz (Cardinal) on Sep 24, 2008 at 07:59 UTC
Well, I think it depends on the font then, doesn't it? Maybe, but there are double-width characters that even fixed-width fonts display with the width of two normal characters (like the ones in the OP).	[reply]
Re^3: The “real length" of UTF8 strings by Anonymous Monk on Sep 24, 2008 at 04:18 UTC
You're right, the correlation between visual length and the actual number of characters is weak, and maybe only font dependent... But when I print these strings with both chinese and ASCII characters using the mysql command (SELECT * FROM...), it prints an array on stdout and is absolutely not confused with the visual and character lengths. That's why I think the solution must exist ^_^*	[reply]


go ahead... be a heretic
	PerlMonks