Printing the Unprintable

[Zaxo]

Sometimes the Mark I Eyeball is still the best tool. It is useful to be able to see problems with unprintable characters. The various line endings, old databases with now-uncommon delimiters, escape sequences of all kinds, strings like that all are peppered with unprintable ASCII characters. When problems arise, we need to see what's there. The usual solution is an eye-crossing hex dump with man ascii open in another terminal.

The old DEC VT-x terminals had a lovely solution. A terminal mode was available which printed ASCII control characters as single-character representations of their ASCII names. For instance tab, "\t", has the official name 'HT', for "Horizontal Tab". The DEC control font represents that as a miniature ^H_T.

Moved by japhy's recent discussion, I was browsing the unicode character list. There amongst the math and the exotic languages, I found that DEC's control character set has a home at code point 2400, "Control Pictures". A light came on.

Here's a demo. I'll use HTML::Entities to make the result browser friendly.

#!/usr/bin/perl
use strict;
use warnings;

use HTML::Entities;

$_ = "\0foo\tbar\tbaz\r\n\a";

tr/\x00-\x1f\x7f/\x{2400}-\x{241f}\x{2421}/;
print encode_entities $_;

__END__
␀foo␉bar␉baz␍␊␇

That's rendered by your browser as:
␀foo␉bar␉baz␍␊␇
(assuming your unicode fonts have these characters). Of course this is much more interesting in a unicode-enabled terminal, but that is a whole different problem.

Added: graff, I don't recall seeing extended control characters on VT-100, either. The unicode list has those control characters themselves, with names like "START OF GUARDED AREA", "END OF GUARDED AREA", "STRING TERMINATOR", and the frightening "OPERATING SYSTEM COMMAND". I have seen no hint of Control Pictures for them.

More Added: There is room for extended control characters. The range of Control Pictures is 2400-243F, but only a few characters are assigned in the upper half of that range. They are 2420 ␠ (SPACE), 2421 ␡ (DELETE), 2422 ␢ (BLANK), 2423 ␣ (OPEN BOX), 2424 ␤ (NEWLINE), 2425 ␥ (DELETE FORM TWO), 2426 ␦ (SUBSTITUTE FORM TWO). The NEWLINE character would serve nicely for NEXT LINE. The rest don't seem to correspond. I suppose the unicode group would consider designs for the missing symbols. Who uses the extended control characters? Are they part of some standard, ISO-8859 maybe?

[hv]

Many years ago I wrote assembler for a 6800-based mini with very simple display screens - 8 rows of 32 characters, each character being encoded by 6 bits for the value (no lower case), along with one bit for "underline" and one bit for "flashing".

The displays were directly memory mapped in 256-byte blocks starting at 0xc000, and the O/S also split memory up into 256-byte buffers. So if you wanted to see what was actually going on, you just poked 0xc1 into a buffer-id to get a real-time display on VDU 1 of what the program on VDU 0 was doing.

Tell people these days that you used to work with a multi-tasking O/S on an 8-bit platform capable of driving 48 VDUs and 56 printers with 48K of RAM and they laugh at you. :(

Hugo

[graff]

Ohmygod!! That is too cool! (I can say that because the unicode font I'm using at the moment happens to have those characters...)

But now I want unicode "control pictures" for values between 0x80 and 0x9F, if any such set exists -- I don't recall whether the VT-100 ever came to terms with those. Hmmm, if necessary, I guess one could just use the "low-table" pictures with a different color, or reverse-video, or bold, or something like that.

[japhy]

That is very cool, Zaxo. You've gotten a well-deserved ++ from me.