Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Printing the Unprintable

by Zaxo (Archbishop)
on May 20, 2004 at 04:28 UTC ( #354858=perlmeditation: print w/ replies, xml ) Need Help??

Sometimes the Mark I Eyeball is still the best tool. It is useful to be able to see problems with unprintable characters. The various line endings, old databases with now-uncommon delimiters, escape sequences of all kinds, strings like that all are peppered with unprintable ASCII characters. When problems arise, we need to see what's there. The usual solution is an eye-crossing hex dump with man ascii open in another terminal.

The old DEC VT-x terminals had a lovely solution. A terminal mode was available which printed ASCII control characters as single-character representations of their ASCII names. For instance tab, "\t", has the official name 'HT', for "Horizontal Tab". The DEC control font represents that as a miniature HT.

Moved by japhy's recent discussion, I was browsing the unicode character list. There amongst the math and the exotic languages, I found that DEC's control character set has a home at code point 2400, "Control Pictures". A light came on.

Here's a demo. I'll use HTML::Entities to make the result browser friendly.

#!/usr/bin/perl use strict; use warnings; use HTML::Entities; $_ = "\0foo\tbar\tbaz\r\n\a"; tr/\x00-\x1f\x7f/\x{2400}-\x{241f}\x{2421}/; print encode_entities $_; __END__ ␀foo␉bar␉baz␍␊␇
That's rendered by your browser as:
(assuming your unicode fonts have these characters). Of course this is much more interesting in a unicode-enabled terminal, but that is a whole different problem.

Added: graff, I don't recall seeing extended control characters on VT-100, either. The unicode list has those control characters themselves, with names like "START OF GUARDED AREA", "END OF GUARDED AREA", "STRING TERMINATOR", and the frightening "OPERATING SYSTEM COMMAND". I have seen no hint of Control Pictures for them.

More Added: There is room for extended control characters. The range of Control Pictures is 2400-243F, but only a few characters are assigned in the upper half of that range. They are 2420 ␠ (SPACE), 2421 ␡ (DELETE), 2422 ␢ (BLANK), 2423 ␣ (OPEN BOX), 2424 ␤ (NEWLINE), 2425 ␥ (DELETE FORM TWO), 2426 ␦ (SUBSTITUTE FORM TWO). The NEWLINE character would serve nicely for NEXT LINE. The rest don't seem to correspond. I suppose the unicode group would consider designs for the missing symbols. Who uses the extended control characters? Are they part of some standard, ISO-8859 maybe?

After Compline,

Comment on Printing the Unprintable
Download Code
Re: Printing the Unprintable
by graff (Chancellor) on May 20, 2004 at 05:45 UTC
    Ohmygod!! That is too cool! (I can say that because the unicode font I'm using at the moment happens to have those characters...)

    But now I want unicode "control pictures" for values between 0x80 and 0x9F, if any such set exists -- I don't recall whether the VT-100 ever came to terms with those. Hmmm, if necessary, I guess one could just use the "low-table" pictures with a different color, or reverse-video, or bold, or something like that.

Re: Printing the Unprintable
by hv (Parson) on May 20, 2004 at 10:14 UTC

    Many years ago I wrote assembler for a 6800-based mini with very simple display screens - 8 rows of 32 characters, each character being encoded by 6 bits for the value (no lower case), along with one bit for "underline" and one bit for "flashing".

    The displays were directly memory mapped in 256-byte blocks starting at 0xc000, and the O/S also split memory up into 256-byte buffers. So if you wanted to see what was actually going on, you just poked 0xc1 into a buffer-id to get a real-time display on VDU 1 of what the program on VDU 0 was doing.

    Tell people these days that you used to work with a multi-tasking O/S on an 8-bit platform capable of driving 48 VDUs and 56 printers with 48K of RAM and they laugh at you. :(


Re: Printing the Unprintable
by japhy (Canon) on May 20, 2004 at 14:15 UTC
    That is very cool, Zaxo. You've gotten a well-deserved ++ from me.
    Jeff[japhy]Pinyan: Perl, regex, and perl hacker, who'd like a job (NYC-area)
    s++=END;++y(;-P)}y js++=;shajsj<++y(p-q)}?print:??;

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://354858]
Approved by broquaint
Front-paged by gmax
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (10)
As of 2014-10-31 00:11 GMT
Find Nodes?
    Voting Booth?

    For retirement, I am banking on:

    Results (211 votes), past polls