A Mystery of Unix History

I wrote recently about buying a Digital (DEC) vt420 and hooking it up to Linux. Among my observations on the vt420, which apparently were among the most popular to use with Unix systems, are these:

  • DEC keyboards had no Esc key
  • DEC keyboards had no key that could be used as Alt or Meta

The two most popular historic editors on Unix, vi and emacs, both make heavy use of these features (Emacs using Esc when Alt or Meta is unavailable). Some of the later entries in the DEC terminal line, especially the vt510, supported key remapping or alternative keyboards, which can address the Esc issue, but not entirely.

According to the EmacsOnTerminal page and other research, at least the vt100 through the vt420 lacked Esc by default. Ctrl-3 and Ctrl-[ could send the character. However, this is downright terrible for both vi and Emacs (as this is the only way to trigger meta commands in Emacs).

What’s more, it seems almost none of these old serial terminal support hardware flow control, and flow control is an absolute necessity on many. That implies XON/XOFF, which use Ctrl-S and Ctrl-Q — both of which are commonly used in Emacs.

Both vi and Emacs trace their roots back to the 1970s and were widely used in the serial terminal era, running on hardware dominated by DEC and its serial terminals.

So my question is: why would both of these editors be developed in such a way that they are downright inconvenient to use on the hardware on which they most frequently ran?

Update 2019-11-20: It appears that the vt100 did have the Esc key, but it was dropped with the vt220. At least the vt420 and later, and possibly as far back as the vt220, let you map one of a few other keys to be Esc. This still leaves the Ctrl-S mystery in Emacs though.

44 thoughts on “A Mystery of Unix History

  1. I wrote my PhD thesis many years ago using emacs on a DEC VT220 terminal. I don’t recall any problems in using emacs in the terminal and most notably don’t recall ever missing the ESC key. Indeed, I am pretty sure the terminals I had access to all had an ESC key.

  2. The original code for vi was written by Bill Joy in 1976. Joy used a Lear Siegler ADM-3A terminal. On this terminal, the Escape key was at the location now occupied by the Tab key on the widely used IBM PC keyboard (on the left side of the alphabetic part of the keyboard, one row above the middle row).

    https://en.wikipedia.org/wiki/Vi

  3. An hypothesis I’ve always had about why the Escape key disappeared in VT220s and after is that none of the DEC-supplied programs in VMS used a single Escape key. They made heavy use of function keys. EDT, the main DEC-supplied editor, didn’t use a single Escape key. (Ironically, TECO, the editor that many people used on RSX-11 and RT-11(?), used a single Escape key so I wonder what happened inside DEC that made them get rid of the Escape key.)

    (I’ve internalized using Cntl-[ so long ago that I don’t use the Escape key even on keyboards that have it.).

    1. > An hypothesis I’ve always had about why the Escape key
      > disappeared in VT220s and after is that none of the DEC-supplied
      > programs in VMS used a single Escape key. They made heavy use of
      > function keys. EDT, the main DEC-supplied editor, didn’t use a
      > single Escape key. (Ironically, TECO, the editor that many people
      > used on RSX-11 and RT-11(?), used a single Escape key so I wonder
      > what happened inside DEC that made them get rid of the Escape
      > key.)

      Your hypothesis is wrong, in some sense.

      Those Digital word processing-style editors were wedded to
      Digital terminal cursor arrow, numeric keypad, and function keys
      that sent escape sequences
      (multi-character sequences starting with ESC).
      _Those_ programs’ command sets couldn’t
      accommodate additional editing commands using ESC
      because it would be ambiguous.

      But that’s not why VT220 terminals shipped without escape keys.

      When DEC Terminals Engineering was developing the VT220,
      they implemented many complex commands and modes –
      virtually all invoked from the remote computer
      by sending to the terminal escape sequences starting
      with Escape, Open Square Bracket.

      When Terminals Engineering developed the (detached/corded) LK201
      keyboard for the VT220, they made the doctrinaire proclamation
      that the LK201 would not have a dedicated Escape key,
      because toothless rube customer end-users would
      confuse the VT220 by hitting Escape,
      followed by an invalid escape sequence.

      This despite the fact that the overwhelming majority of terminal use
      was over a full-duplex link which did not echo keyboard-entered
      Escape characters back to the terminal.

      And despite the existence of applications like TECO
      even EMACS that used escape commands.
      Not to mention the entire TOPS-20 command line interface’s
      command and file completion feature; preserved in Unix tcsh
      using horizontal tab instead of ESC as the completion character.

      I _think_ there was even a gap in the LK201 inner keyboard
      framework to hold an ESC keyswitch.
      If so, they planned for an ESC key,
      and then deleted it,
      Because Doctrine.

      Internal users screamed.
      Terminals Engineering ignored them,
      Because Doctrine.

      The product shipped.
      Customers screamed.
      Terminals Engineering told them they were wrong,
      Because Doctrine.

      Competitors ran ads bragging that their terminals had Escape keys.

      https://i.imgur.com/bpPotS8.png

      Follow-on terminals and keyboards magically acquired
      a supported ESC key.
      Funny how that happened, eh?

      1. Hah. Thank you for all that color on this! Clearly I wasn’t thinking like a DEC product manager :-)

        1. (In this specific historical instance, not thinking like the LK201
          leadership seems like a net positive, doesn’t it?).

          And thanks for your calm consideration of my rant.
          The historical record is certainly on the thin side
          at this late date, and you had no good way to know
          about the origin of the lossage.

  4. The VT100 did have an ESC key. I learned Emacs (starting with actual TECO Emacs on the PDP-10!) on VT52s and VT100s in the early 80s and can confirm this firsthand. (Note that the EmacsOnTerminal page doesn’t say that it didn’t, just that the VT220 forward didn’t.)

  5. The lack of an ESC key was why I stopped using DEC terminals when the VT220 came out. Looking for the presence of an ESC key was key in evaluating new terminals. This is why we ended up buying hundreds of Wyse WY-75s.

    1. Curious about the comment that “most terminals that people used didn’t need.” My vt420 corrupts the screen at Emacs load time unless I either have XON/XOFF enabled or the line speed is 4800bps or below. The vt510 might be able to tolerate 9600bps, maybe. The IBM 3151 is a bit better, at least in 3151 mode, but still has cases where it would need it. Note the vt420 supports 38400bps, the vt510 115200bps and the 3151 19200bps – all of which are far faster than they can process certain types of data. None of these support RTS/CTS flow control as we understand it now. The vt510 seems to maybe support something like it, but using the DTR/DSR lines instead. So based on this I simply extrapolate that XON/XOFF was mandatory in a lot of settings. Am I wrong about that?

  6. It’s only a mystery if you assume all Unix development was done using those DEC terminals with those particular keyboards. The DEC LK421 keyboard, designed for Unix, has an escape key. The ADM-3a on which vi was developed has an escape key. There were many, many text terminals in the early years of Unix.

  7. CTRL [ , I used it extensively in emacs back in the day. I never missed the ESC key. It was never an issue. I used DEC terminals and later WYSE terminals and even when I had an ESC key I was so used to CTRL [ that I never used the ESC key.

  8. Set your VT220 to VT52 or VT100 mode (commonly done in the 80s) and F11 send just ESC. The standard function key overlay strip that fits in the tray by the F keys shows this.

  9. Keep in mind that EMACS does not output scrolling text at the bottom of the screen. It is operations like that, which need to copy bunches of screen memory around, that trigger XOFF. Mostly, EMACS is positioning absolutely and repainting.

    For display optimizations, such as region scrolling when inserting or deleting lines, an effective alternative is to send a stream of padding characters (rubouts or nulls) to give the terminal time to catch up. And that is what PDP-10 EMACS on VT100’s did. Look at VT1UP3 in http://pdp-10.trailing-edge.com/mit_emacs_170_teco_1220/01/emacs/tectrm.mid.html. For a true VT100, send 10 pad characters. For a VT132, send 32ms worth.

    I should also mention that some terminals of that day, such as Datamedia, were hacked to have a META key. A key’s switch is wired (more or less) directly to what should be the parity bit that the terminal sends.

    1. This tickled something at the far back of my brain. I checked, and indeed the terminfo(5) manpage specifies various things around padding! Interesting. I don’t think I’ve ever seen this used in the wild on Linux, but I’m probably too new to it. I doubt it would work very well today, what with programs running over ssh/network connections that don’t have direct knowledge of the terminal’s baud rate, but I could totally see this being useful back in the day.

  10. The confusion about ctl-s/ctl-q is because there were two different notions of flow control at the time.

    One used handshake lines on the RS232 wires. That was what you needed to have to deal with a machine that could send faster then the terminal could accept.

    The other was user-level flow control. Ctrl-S and Ctrl-Q were special characters that a user could type to say “pause output” and “resume output” to the Unix terminal driver at the other end. Had nothing to do with the hardware-level flow control at all.

    Thing is, those characters were only interpreted in what we called cooked mode. The default. Editors like Emacs put themselves in raw mode, in which those characters were passed directly through, not interpreted by the TTY driver, and that’s available to be used as commands.

    I know about this because I personally wrote some of the TTY handling support in early Emacs, and in a couple of other early screen oriented programs like nethack as well. I never touched vi but I knew what it was doing.

    1. Interesting – so I was wondering about the apparent lack of hardware flow control in these terminals. In my vt420 (a copy of the manual is here https://vt100.net/docs/vt420-uu/contents.html but I also have the original printed book) there is a communications set up screen at https://vt100.net/docs/vt420-uu/chapter5.html#S5.9 . This is what I have found. The XOFF option sends an XOFF when the terminal’s small buffer reaches a certain level. Nothing at all about hardware flow control (and indeed in testing, it did nothing with the RTS/CTS lines).

      My vt510 — see https://vt100.net/docs/vt510-rm/chapter2.html#S2.9.7 — does appear to be able to do hardware flow control, but not with CTS/RTS — it uses DSR/DTR. Unfortunately Linux doesn’t support this, and I haven’t spent the time to build the relevant conversion cable, so I haven’t tested it.

      My IBM 3151 — one of the relatively few “ASCII terminals” that IBM made — has a manual at https://www.manualslib.com/manual/887102/Ibm-3151.html and on page 2-29 it describes “line control” and “pacing”. While the line control options mention the RTS and CTS lines, none of those senses do flow control (they appear to be more related to old half-duplex signaling, which I gather RTS/CTS was more originally for). The “Pacing” option is XON/XOFF.

      So that’s three terminals, and only the vt510 supports modern flow control (though on the “wrong” pins); the vt510 was only briefly made by DEC before they spun off their terminal business; it was pretty much past the heydey of serial terminals, I gather.

      So, I wonder – were the DEC and IBM terminals unusually crappy in this regard? Perhaps the budget competitors like Wyse had hardware flow control? I went and checked some Wyse manuals. It appears that the WY-50 and WY-60 can do XON/XOFF and DSR/DTR handshaking.

      I don’t know the popularity of the Wyse vs. DEC terminals. I had understood that DEC terminals were quite popular, but maybe others copied them and put better RS-232 support in their clones?

      Do you know anything about the transition from DTR/DSR to RTS/CTS flow control? It looks like FreeBSD may still support DTR/DSR flow control but I see no evidence that Linux ever has.

      1. You are reminding me of things I hadn’t thought about in decades.

        You’re right, there was variation in what lines were used for hardware flow control, and some terminals did issue XON/XOFF (^S/^Q) automatically. I knew that at one time but had forgotten it, as I never happened to use any of those. They would have played merry hell with Emacs.

        Sorry, I don’t know when the industry settled on RTS/CTS flow control either. It’s the only variant I remember seeing on the terminals I used, though I vaguely knew there were others.

        Most of my tube time was on Wyse 50, 60, and 75 terminals.

  11. Emacs was not originally developed on Unix. It was originally used on terminals at MIT in, I believe the 60’s, and they had a lot of extra shift keys of different kinds. Later it was a question of mapping all those shift combinations onto keyboards that really didn’t fit well.

    1. Not in the 1960s. Terminals in the sense we think of them today didn’t really develop until, at the most expensive interpretation, around 1969. The classic cursor addressable terminal as we now think of it actually dates to the ADM-3A in 1976.

      The era of weird keyboards at MIT actually peaked in a few years around 1980.

      They did indeed have a lot of wacky shift keys. Read the Wikipedia article on the “space cadet keyboard” for details.

      Most of these modifiers didn’t survive. Shift came from typewriters of course, and Ctrl from early serial terminals. Of the space cadet modifiers only Alt outlasted the experimental MIT and Lisp machine designs.

      Though years later Microsoft would introduce a flag key that could be pressed into service as an equivalent of the space cadet super or hyper shifts. That was a completely independent development.

  12. @jgoerzen @SDF The terminal? It came in a lot of vintage minicomputer stuff I bought a couple of years ago. The PDP-11/34A it’s _supposed_ to be hooked to is behind where I’m sitting when I took the photo. ;-)I was on SDF MANY years ago, but lapsed, and have only recently rejoined. I love the idea of a shared Unix system like I used to enjoy when shared systems were a Thing.

  13. @elb @SDF I do check the local timeline there about as often as the local timeline where I’m at. I wound up using a different Mastodon host due to the dated installation there at the time (since fixed) and some performance issues, but SDF is a fantastic organization (along side the Tildes)Now if I could just get them to process my #UUCP feed request… 🙂
    uucp

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.