http://www.perlmonks.org?node_id=198647


in reply to Re: Re: Re: Re: Handling Mac, Unix, Win/DOS newlines at readtime...
in thread Handling Mac, Unix, Win/DOS newlines at readtime...

You're both right and wrong here. Let me first quote a bit of the HTML 4 specification:

4.3 The text/html content type

HTML documents are sent over the Internet as a sequence of bytes accompanied by encoding information (described in the section on character encodings). The structure of the transmission, termed a message entity, is defined by RFC2045 and RFC2616. A message entity with a content type of "text/html" represents an HTML document.

Here, the HTML specification explicitly indicates a reliance upon RFC2045 (and by association, RFC2046).

An HTML document is stored as a native plain-text document either on a user's PC or a web server. Newlines at this point are native to the OS.

Do not make the assumption that HTTP servers are filesystem-based! It just so happens that a filesystem-oriented document root, with URI contents represented as files, is the simplest and most prevalent way HTTP servers are implemented, but do not blindly assume that HTTP was written with the intent that a URI should map to a file on a filesystem. That's just not accurate.

But because most HTTP servers do this, they have to do a certain number of things to ensure that a requested resource is being delivered in a fashion consistent with HTTP and MIME. This usually involves examining a file extension for a MIME type, and delivering the contents of the file in a fashion consistent with that MIME type. If an HTML document is being stored on a filesystem with native newlines, an HTTP server that relies on filesystem-oriented content should take steps to ensure that "special cases" like newlines are addressed as well. Conversion should be performed by the web server as a consequence of the web server's filesystem-oriented implementation of an HTTP service.

What is the alternative? Turn HTML files into what are effectively binary files due to their quirkly (with respects to the native text format) line endings? If not, what else is supposed to be converting newlines here?

Think of this from the user agent's point of view. It's expecting content with the MIME type of text/html. MIME explicitly states that text/html must have line endings in CRLF fashion. How does it get that way? If the server isn't responsible for it, what is?

The HTTP servers have assumed this responsibility as a consequence of choosing a filesystem-oriented mechanism for storing content. They have to live with content stored with native line endings and should thus be responsible for getting that converted into something appropriate when delivering text/* content over the Internet.