Re: How does the built-in function length work?

Replies are listed 'Best First'.
Re^2: How does the built-in function length work? by ikegami (Patriarch) on Dec 02, 2011 at 19:30 UTC
I usually always agree 100% with your posts, but not today. Perl operators that deal with text (regex and uc and the like) expect Unicode code points. (Formerly: Expected Unicode code points or ASCII depending on UTF8 flag.) Perl operators that deal with file names (open, stat, etc) expect the file names to be bytes. Perl never assumes or expects iso-8859-1.	[reply]
Re^3: How does the built-in function length work? by moritz (Cardinal) on Dec 02, 2011 at 21:02 UTC
(Formerly: Expected Unicode code points or ASCII depending on UTF8 flag.) I guess that what you call "Unicode code point" is what I call "ISO-8859-1". ISO-8859-1 is simply the encoding that maps the byte values from 0 to 255 to the Unicode codepoints from 0 to 255, in that order. Perl never assumes or expects iso-8859-1. `$ echo -e "\xE4"\|perl -wE 'say <> ~~ /\w/' 1 $ # this a perl 5.14.1` [download] Since no decoding step happened here, and `<>` is a binary operation, and the regex match a text operation, perl has to assume a character encoding. And that happens to be ISO-8859-1. Or what do you think it is, if not ISO-8859-1? Perl 6 - second systems done right	[reply] [d/l] [select]
Re^4: How does the built-in function length work? by JavaFan (Canon) on Dec 02, 2011 at 21:27 UTC
Or what do you think it is, if not ISO-8859-1? EBCDIC? Binary? ISO-8859-15?	[reply]
Re^4: How does the built-in function length work? by ikegami (Patriarch) on Dec 02, 2011 at 21:48 UTC
perl has to assume a character encoding. Not at all. If it must assume an encoding, and that encoding is iso-8859-1 for `"\x{E4}" =~ /\w/` [download] then what encoding is assumed for the following? `"\x{2660}" =~ /\w/` [download] It never deals with any encoding. It always deals with string elements (characters). And those string elements (characters) are ~~assumed~~required to be Unicode code points. Character E4 is taken as Unicode code point E4, not some byte produced by iso-8859-1. Character 2660 is taken as Unicode code point 2660, not some byte produced by iso-8859-1. It's entirely up to you to create a string with the right elements, which may or may not involve character encodings. Or what do you think it is, if not ISO-8859-1? A Unicode code point, regardless of the state of the UTF8 flag. Character E4 (UTF8=0) is taken as Unicode code point E4, not some byte produced by iso-8859-1. Character E4 (UTF8=1) is taken as Unicode code point E4, not some byte produced by iso-8859-1. In short, you're over complicating things. It's NOT: Each character is expected to be an iso-8859-1 byte if UTF8=0 or a Unicode code point if UTF8=1. It's simply: Each character is expected to be a Unicode code point.	[reply] [d/l] [select]
Re^5: How does the built-in function length work? by moritz (Cardinal) on Dec 03, 2011 at 06:22 UTC
Re^6: How does the built-in function length work? by ikegami (Patriarch) on Dec 03, 2011 at 07:53 UTC


Keep It Simple, Stupid
	PerlMonks