Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Jargon relating to Perl strings

by wrog (Monk)
on Jan 17, 2012 at 21:34 UTC ( #948422=note: print w/ replies, xml ) Need Help??


in reply to Jargon relating to Perl strings

Under Basics

The sequence of string elements in a string. This is not affected by the string's UTF8 flag.
This is not what you want to say (it immediately confused me because my first thought was, "This is wrong because if you have a string with non-ASCII characters in it and change its UTF8 flag, that will change the sequence of elements.")

What I think you meant to say

The sequence of string elements in a string, irrespective of any particular choice of memory representation being used for that string
and only bring up the UTF8 flag later.

Also, IMHO there needs to be distinct terminology for

  1. a grouping of (usually but not always) 8 consecutive bits of physical storage
  2. the abstract array element in the case where all elements are expected to be in the range 0-255, regardless of the actual storage format
the problem being (as noted by others) that most people associate "byte" with (1). Using it for (2) is unlikely to reduce the confusion out there and not having distinct terminology makes it difficult to talk about storage formats.

One could possibly commandeer "octet" for (2) but realize that "octet" originated in the RFC world where a word was needed to refer to physical storage in the specific case where bytes explicitly are known to be 8 bits. On the other hand, most of the stuff in the RFC world is indeed trying to abstract away from specific hardware, so one could justify its usage in a more abstract sense that way. And "octet" does, at least, immediately imply 0-255, unlike "byte"

There's also the small matter that it really doesn't make a whole lot of sense to use UTF-8-flag-on format to store something that is composed of octets, even if it is indeed possible to do. Which is why people conflate octet strings with the UTF-8-flag-off format and its 1-1 correspondence between octets and bytes
(...and thus why it is indeed important to point out that UTF-8-flag-on octet strings are possible, albeit silly...)

Update: yeah I edited this... sorry.


Comment on Re: Jargon relating to Perl strings
Re^2: Jargon relating to Perl strings
by ikegami (Pope) on Jan 18, 2012 at 02:22 UTC

    This is not what you want to say

    Indeed. I discovered this reading JavaFan's post.

    What I think you meant to say

    Perfect. I shall update.

    a grouping of (usually but not always) 8 consecutive bits of physical storage

    UTF8=0 storage format.

    the problem being (as noted by others) that most people associate "byte" with (1)

    If so, then reading 5 bytes produces a variable number of bytes*. That doesn't jive.

    If I read 5 bytes from a file, what I get if 5 bytes as far as I'm concerned. I'm open to a better word that "byte" for this, but I haven't come across one.

    One could possibly commandeer "octet"

    An octet is simply an 8-bit byte. But since that's what byte means in all relevant circumstances anyway*, "octet" is no better than "byte".

    * — Perl doesn't currently support systems with byte sizes other than 8.

    There's also the small matter that it really doesn't make a whole lot of sense to use UTF-8-flag-on format to store something that is composed of octets, even if it is indeed possible to do.

    No, but it can happen. Say you have:

    # as C9 in source. print $bin_fh "AX100X";

    And say one day you convert your source files to UTF-8.

    # as C3 E9 in source. use utf8; print $bin_fh "AX100X";

    The code is still fine, yet the string in the latter has UTF8=1.

      Perfect. I shall update.
      except you updated the wrong thing. It's the sentence "This is not affected by the string's UTF8 flag," under Basics->"String Value" that's tripping me (and apparently also javafan) up and that needs to either go away or be changed
      a grouping of (usually but not always) 8 consecutive bits of physical storage
      UTF8=0 storage format.
      No, any storage format. In order to talk about storage formats at all you need a word for the raw underlying bytes whatever they are and however they're to be interpreted, and redefining "byte" to mean something else makes this really difficult.

      You need a different word, and you're probably right that "octet" isn't a great choice either, so I had another thought: How about one of the following to refer to string elements that are constrained to lie in the 0-255 range?

      • "octetchar"
      • "octet-character"
      • "bytecharacter"
      • "byte-character"
      • "bytechar"
      as opposed to "general character" or "Unicode character" when the full Unicode (or UV) range is possible. This way you're emphasizing that they're still characters in the sense that everybody agrees on (i.e., they're elements of a string and we're abstracting away from how they're represented). If you then say that a single octetchar can actually be multiple bytes in the UTF8=1 storage format, the meaning is clear.

        except you updated the wrong thing.

        I had *missed* one of the three spots where I used the problem phrase. Fixed the one I have missed.

        In order to talk about storage formats at all you need a word for the raw underlying bytes whatever they are and however they're to be interpreted,

        It's not really something that comes up, but "contents of the PV".

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://948422]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (10)
As of 2014-07-25 22:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (175 votes), past polls