I wrote recently about buying a Digital (DEC) vt420 and hooking it up to Linux. Among my observations on the vt420, which apparently were among the most popular to use with Unix systems, are these:
- DEC keyboards had no Esc key
- DEC keyboards had no key that could be used as Alt or Meta
The two most popular historic editors on Unix, vi and emacs, both make heavy use of these features (Emacs using Esc when Alt or Meta is unavailable). Some of the later entries in the DEC terminal line, especially the vt510, supported key remapping or alternative keyboards, which can address the Esc issue, but not entirely.
According to the EmacsOnTerminal page and other research, at least the vt100 through the vt420 lacked Esc by default. Ctrl-3 and Ctrl-[ could send the character. However, this is downright terrible for both vi and Emacs (as this is the only way to trigger meta commands in Emacs).
What’s more, it seems almost none of these old serial terminal support hardware flow control, and flow control is an absolute necessity on many. That implies XON/XOFF, which use Ctrl-S and Ctrl-Q — both of which are commonly used in Emacs.
Both vi and Emacs trace their roots back to the 1970s and were widely used in the serial terminal era, running on hardware dominated by DEC and its serial terminals.
So my question is: why would both of these editors be developed in such a way that they are downright inconvenient to use on the hardware on which they most frequently ran?
Update 2019-11-20: It appears that the vt100 did have the Esc key, but it was dropped with the vt220. At least the vt420 and later, and possibly as far back as the vt220, let you map one of a few other keys to be Esc. This still leaves the Ctrl-S mystery in Emacs though.
I wrote my PhD thesis many years ago using emacs on a DEC VT220 terminal. I don’t recall any problems in using emacs in the terminal and most notably don’t recall ever missing the ESC key. Indeed, I am pretty sure the terminals I had access to all had an ESC key.
The VT220 didn’t have an ESC key. https://external-preview.redd.it/SiLVPTrm_c8MfLplX48AxZySG5xhACu8p3i1reo1_dM.jpg?auto=webp&s=26d297073cdd338eaa1454f8fd6dbbda4b2a043d
It kinda looks like the entire bar above the function row is an escape key :)
Wow, an ADM3A! Do you still have it?
Authoritative source I should have tried first: zoom in on the photo in en.m.wikipedia.org/wiki/VT100 and see ESC, next to 1. So the funny stuff started later.
Some history around early visual editors [0], can be found at “A Quarter Century of UNIX” by Peter H. Salus. Specifically the day when ‘Editor for Mortals’ author visited Berkeley, etc..
https://en.wikipedia.org/wiki/Visual_editor
The original code for vi was written by Bill Joy in 1976. Joy used a Lear Siegler ADM-3A terminal. On this terminal, the Escape key was at the location now occupied by the Tab key on the widely used IBM PC keyboard (on the left side of the alphabetic part of the keyboard, one row above the middle row).
https://en.wikipedia.org/wiki/Vi
An hypothesis I’ve always had about why the Escape key disappeared in VT220s and after is that none of the DEC-supplied programs in VMS used a single Escape key. They made heavy use of function keys. EDT, the main DEC-supplied editor, didn’t use a single Escape key. (Ironically, TECO, the editor that many people used on RSX-11 and RT-11(?), used a single Escape key so I wonder what happened inside DEC that made them get rid of the Escape key.)
(I’ve internalized using Cntl-[ so long ago that I don’t use the Escape key even on keyboards that have it.).
> An hypothesis I’ve always had about why the Escape key
> disappeared in VT220s and after is that none of the DEC-supplied
> programs in VMS used a single Escape key. They made heavy use of
> function keys. EDT, the main DEC-supplied editor, didn’t use a
> single Escape key. (Ironically, TECO, the editor that many people
> used on RSX-11 and RT-11(?), used a single Escape key so I wonder
> what happened inside DEC that made them get rid of the Escape
> key.)
Your hypothesis is wrong, in some sense.
Those Digital word processing-style editors were wedded to
Digital terminal cursor arrow, numeric keypad, and function keys
that sent escape sequences
(multi-character sequences starting with ESC).
_Those_ programs’ command sets couldn’t
accommodate additional editing commands using ESC
because it would be ambiguous.
But that’s not why VT220 terminals shipped without escape keys.
When DEC Terminals Engineering was developing the VT220,
they implemented many complex commands and modes –
virtually all invoked from the remote computer
by sending to the terminal escape sequences starting
with Escape, Open Square Bracket.
When Terminals Engineering developed the (detached/corded) LK201
keyboard for the VT220, they made the doctrinaire proclamation
that the LK201 would not have a dedicated Escape key,
because toothless rube customer end-users would
confuse the VT220 by hitting Escape,
followed by an invalid escape sequence.
This despite the fact that the overwhelming majority of terminal use
was over a full-duplex link which did not echo keyboard-entered
Escape characters back to the terminal.
And despite the existence of applications like TECO
even EMACS that used escape commands.
Not to mention the entire TOPS-20 command line interface’s
command and file completion feature; preserved in Unix tcsh
using horizontal tab instead of ESC as the completion character.
I _think_ there was even a gap in the LK201 inner keyboard
framework to hold an ESC keyswitch.
If so, they planned for an ESC key,
and then deleted it,
Because Doctrine.
Internal users screamed.
Terminals Engineering ignored them,
Because Doctrine.
The product shipped.
Customers screamed.
Terminals Engineering told them they were wrong,
Because Doctrine.
Competitors ran ads bragging that their terminals had Escape keys.
https://i.imgur.com/bpPotS8.png
Follow-on terminals and keyboards magically acquired
a supported ESC key.
Funny how that happened, eh?
Hah. Thank you for all that color on this! Clearly I wasn’t thinking like a DEC product manager :-)
(In this specific historical instance, not thinking like the LK201
leadership seems like a net positive, doesn’t it?).
And thanks for your calm consideration of my rant.
The historical record is certainly on the thin side
at this late date, and you had no good way to know
about the origin of the lossage.
The VT100 did have an ESC key. I learned Emacs (starting with actual TECO Emacs on the PDP-10!) on VT52s and VT100s in the early 80s and can confirm this firsthand. (Note that the EmacsOnTerminal page doesn’t say that it didn’t, just that the VT220 forward didn’t.)
On my vt420, there is a <> key to the left of the Z that I can put them on. It’s all a bit weird. The vt510 can take a PS/2 keyboard and it also supports full remapping of everything.
The lack of an ESC key was why I stopped using DEC terminals when the VT220 came out. Looking for the presence of an ESC key was key in evaluating new terminals. This is why we ended up buying hundreds of Wyse WY-75s.
http://www.columbia.edu/kermit/vt320-keyboard.jpg
Looks like F11 is Escape.
PDP-10 EMACS (TECO) did have a mode where ^S/^Q were left on in TOPS-20. You then have to assign search and quote commands to something different.
https://github.com/PDP-10/its/blob/master/src/_teco_/teco.1213#L683-L685
But most terminals that people used didn’t need.
Curious about the comment that “most terminals that people used didn’t need.” My vt420 corrupts the screen at Emacs load time unless I either have XON/XOFF enabled or the line speed is 4800bps or below. The vt510 might be able to tolerate 9600bps, maybe. The IBM 3151 is a bit better, at least in 3151 mode, but still has cases where it would need it. Note the vt420 supports 38400bps, the vt510 115200bps and the 3151 19200bps – all of which are far faster than they can process certain types of data. None of these support RTS/CTS flow control as we understand it now. The vt510 seems to maybe support something like it, but using the DTR/DSR lines instead. So based on this I simply extrapolate that XON/XOFF was mandatory in a lot of settings. Am I wrong about that?
It’s only a mystery if you assume all Unix development was done using those DEC terminals with those particular keyboards. The DEC LK421 keyboard, designed for Unix, has an escape key. The ADM-3a on which vi was developed has an escape key. There were many, many text terminals in the early years of Unix.
Understood. I just thought that the DEC terminals were fairly dominant in the marketplace. Perhaps I am wrong about that.
420 lol….Animus
CTRL [ , I used it extensively in emacs back in the day. I never missed the ESC key. It was never an issue. I used DEC terminals and later WYSE terminals and even when I had an ESC key I was so used to CTRL [ that I never used the ESC key.
Set your VT220 to VT52 or VT100 mode (commonly done in the 80s) and F11 send just ESC. The standard function key overlay strip that fits in the tray by the F keys shows this.
Keep in mind that EMACS does not output scrolling text at the bottom of the screen. It is operations like that, which need to copy bunches of screen memory around, that trigger XOFF. Mostly, EMACS is positioning absolutely and repainting.
For display optimizations, such as region scrolling when inserting or deleting lines, an effective alternative is to send a stream of padding characters (rubouts or nulls) to give the terminal time to catch up. And that is what PDP-10 EMACS on VT100’s did. Look at VT1UP3 in http://pdp-10.trailing-edge.com/mit_emacs_170_teco_1220/01/emacs/tectrm.mid.html. For a true VT100, send 10 pad characters. For a VT132, send 32ms worth.
I should also mention that some terminals of that day, such as Datamedia, were hacked to have a META key. A key’s switch is wired (more or less) directly to what should be the parity bit that the terminal sends.
This tickled something at the far back of my brain. I checked, and indeed the terminfo(5) manpage specifies various things around padding! Interesting. I don’t think I’ve ever seen this used in the wild on Linux, but I’m probably too new to it. I doubt it would work very well today, what with programs running over ssh/network connections that don’t have direct knowledge of the terminal’s baud rate, but I could totally see this being useful back in the day.
I don’t recall whether (ITS/Twenex) EMACS itself always at best only used
absolute addressing and overwriting to do screen updates. But the CMU
TOPS-10 EMACS clone FINE (Fine Is Not Emacs) acquired dynamic programming
algorithms that would leverage line- and character- insert and delete operations
in smart terminals which supported the features. The algorithm was adopted
with ferocity at Stevens Institute for the computer center staff’s video editor fork
of DEC TECO when the insight was published by Gosling on behalf of Mike Kazar,
et. al.
https://dl.acm.org/doi/abs/10.1145/800209.806463
https://donhopkins.com/home/archive/emacs/p123-gosling.pdf
Known as LID and CID (pronounced “sid”), escape sequences let the host open
up a gap between existing lines or characters to insert new ones (or to delete
entire lines or characters, causing subsequent lines to scroll upward or leftwards
to replace what was deleted).
In the general case, these functions were not used by the editors merely during
insertion or editing. The screen update algorithm was an adaptation of the
dynamic programming solution to the String-to-String Correction Problem.
If the end-user performed a page or line scroll operation (in either direction)
where the resulting screen contained fragments of text identical to the previous
screenful, would compute the LID/CID operations that would require the
minimal number of new characters to be transmitted over a serial line to the
terminal to fill in the blanks. The algorithm was quite ingenious. If scrolling
through tabular text – such as short subroutines with a lot of boilerplate routine
headers – the screen would just shimmy up and down a little, and the new page
would be filled in very quickly. Likewise for scrolling through text using a
substantial overlap between the bottom of the old screenful and the top of the
new screenful.
The performance was uncanny. With rapid scrolling through templated text at
middling baud rates (1200-2400), you could almost get seasick at all the
gyrations from minimal screen updates.
The confusion about ctl-s/ctl-q is because there were two different notions of flow control at the time.
One used handshake lines on the RS232 wires. That was what you needed to have to deal with a machine that could send faster then the terminal could accept.
The other was user-level flow control. Ctrl-S and Ctrl-Q were special characters that a user could type to say “pause output” and “resume output” to the Unix terminal driver at the other end. Had nothing to do with the hardware-level flow control at all.
Thing is, those characters were only interpreted in what we called cooked mode. The default. Editors like Emacs put themselves in raw mode, in which those characters were passed directly through, not interpreted by the TTY driver, and that’s available to be used as commands.
I know about this because I personally wrote some of the TTY handling support in early Emacs, and in a couple of other early screen oriented programs like nethack as well. I never touched vi but I knew what it was doing.
Interesting – so I was wondering about the apparent lack of hardware flow control in these terminals. In my vt420 (a copy of the manual is here https://vt100.net/docs/vt420-uu/contents.html but I also have the original printed book) there is a communications set up screen at https://vt100.net/docs/vt420-uu/chapter5.html#S5.9 . This is what I have found. The XOFF option sends an XOFF when the terminal’s small buffer reaches a certain level. Nothing at all about hardware flow control (and indeed in testing, it did nothing with the RTS/CTS lines).
My vt510 — see https://vt100.net/docs/vt510-rm/chapter2.html#S2.9.7 — does appear to be able to do hardware flow control, but not with CTS/RTS — it uses DSR/DTR. Unfortunately Linux doesn’t support this, and I haven’t spent the time to build the relevant conversion cable, so I haven’t tested it.
My IBM 3151 — one of the relatively few “ASCII terminals” that IBM made — has a manual at https://www.manualslib.com/manual/887102/Ibm-3151.html and on page 2-29 it describes “line control” and “pacing”. While the line control options mention the RTS and CTS lines, none of those senses do flow control (they appear to be more related to old half-duplex signaling, which I gather RTS/CTS was more originally for). The “Pacing” option is XON/XOFF.
So that’s three terminals, and only the vt510 supports modern flow control (though on the “wrong” pins); the vt510 was only briefly made by DEC before they spun off their terminal business; it was pretty much past the heydey of serial terminals, I gather.
So, I wonder – were the DEC and IBM terminals unusually crappy in this regard? Perhaps the budget competitors like Wyse had hardware flow control? I went and checked some Wyse manuals. It appears that the WY-50 and WY-60 can do XON/XOFF and DSR/DTR handshaking.
I don’t know the popularity of the Wyse vs. DEC terminals. I had understood that DEC terminals were quite popular, but maybe others copied them and put better RS-232 support in their clones?
Do you know anything about the transition from DTR/DSR to RTS/CTS flow control? It looks like FreeBSD may still support DTR/DSR flow control but I see no evidence that Linux ever has.
You are reminding me of things I hadn’t thought about in decades.
You’re right, there was variation in what lines were used for hardware flow control, and some terminals did issue XON/XOFF (^S/^Q) automatically. I knew that at one time but had forgotten it, as I never happened to use any of those. They would have played merry hell with Emacs.
Sorry, I don’t know when the industry settled on RTS/CTS flow control either. It’s the only variant I remember seeing on the terminals I used, though I vaguely knew there were others.
Most of my tube time was on Wyse 50, 60, and 75 terminals.
Emacs was not originally developed on Unix. It was originally used on terminals at MIT in, I believe the 60’s, and they had a lot of extra shift keys of different kinds. Later it was a question of mapping all those shift combinations onto keyboards that really didn’t fit well.
Not in the 1960s. Terminals in the sense we think of them today didn’t really develop until, at the most expensive interpretation, around 1969. The classic cursor addressable terminal as we now think of it actually dates to the ADM-3A in 1976.
The era of weird keyboards at MIT actually peaked in a few years around 1980.
They did indeed have a lot of wacky shift keys. Read the Wikipedia article on the “space cadet keyboard” for details.
Most of these modifiers didn’t survive. Shift came from typewriters of course, and Ctrl from early serial terminals. Of the space cadet modifiers only Alt outlasted the experimental MIT and Lisp machine designs.
Though years later Microsoft would introduce a flag key that could be pressed into service as an equivalent of the space cadet super or hyper shifts. That was a completely independent development.
At least _some_ terminals with paper tape readers (Teletype ASR33?) had
the property that they would start the reader if a ^S was sent to them
_by the computer_ (dataset), and would stop the reader if a ^Q was sent.
http://bitsavers.org/pdf/dec/pdp10/TOPS10/DEC-10-OMCMA-A_D_Monitor_Calls_Ver_6.01_May74.pdf#page=228
Similarly, some would start/stop the paper tape punch via reception of
^R/^T. However, these features were _not_ triggered if the control
characters were merely entered from the keyboard – the host computer had
to send (or echo) the commands back to the TTY.
http://bitsavers.org/pdf/dec/pdp10/TOPS10/DEC-10-OMCMA-A_D_Monitor_Calls_Ver_6.01_May74.pdf#page=214
Terminals with more features than brains (e.g., VT5x, VT100, VT2xx)
which required the ability to throttle computer output to prevent data
overruns arrogated to themselves the “software” flow control protocol
of _sending_ ^S/^Q to the computer – as opposed to RTS/CTS. Computer
systems which supported the ^S/^Q protocol would naturally also afford
end-users the feature of output pause/resume by typing ^S/^Q from the
keyboard.
But consider the proposition that computer systems _first_ supported
^S/^Q pause/resume to support (semi-)smart terminals at high baud rates
– _not_ to provide pause/resume to end-users trying to read rapidly
scrolling output streams. I don’t have definitive historical proof of
this at my fingertips; however…
The TOPS-10 V5.07/V6.01 monitor (O/S) API provided for a terminal mode
where you could inform the system of the screen height (say, 24), the
system would keep count of the number of lines output between pauses for
user input, and if more than (24) lines were output, the system would
pause _as if_ a ^S had been typed (and would resume if and when a ^Q was
typed). But while the TRMOP. function code .TOSTP (01022) for this had
been defined, it was not yet implemented
http://bitsavers.org/pdf/dec/pdp10/TOPS10/DEC-10-OMCMA-A_D_Monitor_Calls_Ver_6.01_May74.pdf#page=225
Based on the marginal changebars in the V5.05 TOPS-10 Operating System
Commands manual, even the “manual” pause/resume features were very
recent additions ca. Dec-71/Jun-72
http://bitsavers.org/pdf/dec/pdp10/TOPS10/1972_PDP-10_Users_Handbook/08_commands.pdf#page=242
(You may be amused by the SET TTY FILL padding table on the prior page).
There’s no hint of the latter day SET TTY PAGE n command which had
appeared in TOPS-10 V7.04 by Oct-88:
http://bitsavers.org/pdf/dec/pdp10/TOPS10_softwareNotebooks/vol02/AA_0916F_Operating_System_Commands_Manual_Oct88.pdf#page=319
Also, the 1972 introductory section 1.3.1.1 “Special Characters” mentions
^C, ^O, ^U, but breathes not a hint of the utility of ^S/^Q.
http://bitsavers.org/pdf/dec/pdp10/TOPS10/1972_PDP-10_Users_Handbook/08_commands.pdf#page=18
This is in marked contrast to section 1.6 of Oct-88, where ^S/^Q are
touted as end-user commands, with no hint _at that place in the manual_
that contemporary video terminals may throw a flurry of ^S/^Q’s at the
system when overtaxed by complex cursor operations.
http://bitsavers.org/pdf/dec/pdp10/TOPS10_softwareNotebooks/vol02/AA_0916F_Operating_System_Commands_Manual_Oct88.pdf#page=14
A historical midpoint is available to us. The Mar-77 printing of the
(VT5x) DECscope Users’ Manual says that the _only_ reason a -52 class
terminal would send ^S to the host computer to prevent data overrun is
if it was equipped with the hardcopy option, and was sending characters
from the screen memory to the printer.
http://www.bitsavers.org/pdf/dec/terminal/vt52/EK-VT5X-OP-001_DECscope_Users_Manual_Mar77.pdf#page=22
Otherwise, the 5x’s were studly enough to keep up with input at the
highest baud rates.
_However_, there was a SCROLL key which would sip characters from the
(13 character long, LOL) “Silo” (buffer) and display as many as would
fit on a line (or a screen, with Shift-SCROLL). This only worked if the
computer put the terminal in a magical Hold-Screen Mode via “ESC-[”.
And the terminal was willing to send ^S/^Q to the computer to mediate
lines (or screenfuls) of output even without a printer-induced data
overrun.
All this evidence is merely suggestive – not dispositive. And this is a
very DEC-centric history – other vendors may have had a different
timeline. But back in the day DEC both wrote their own O/S’s and
eventually built their own terminals. So it’s plausible that the
software and hardware groups collaborated on how to solve terminal data
overruns. Actually, I can’t even say that the very first terminals to
adopt ^S/^Q flow control were third party hardware. Perhaps some system
vendors had to scramble to support the ^S/^Q protocol in order to be
compatible with someone else’s (semi-) smart terminal product.
/AHM/THX
P. S. See also a reply I intend to enter this morning to MMcM’s
21-Nov’19 post about EMACS smart terminal addressing.
Grrr; web site ate my nice paragraph spacing.
@jgoerzen this one is technically a VT105, it has (virtually unused) VT55 graphics modes.
@elb That is FANTASTIC! (Love the ssh to @SDF also; I’ve been a member for yeras) How did you get ahold of it?
@jgoerzen @SDF The terminal? It came in a lot of vintage minicomputer stuff I bought a couple of years ago. The PDP-11/34A it’s _supposed_ to be hooked to is behind where I’m sitting when I took the photo. ;-)I was on SDF MANY years ago, but lapsed, and have only recently rejoined. I love the idea of a shared Unix system like I used to enjoy when shared systems were a Thing.
@jgoerzen @SDF I try to rep SDF in my terminal posts since they’re hosting my Mastodon. 🙂
@elb @SDF I do check the local timeline there about as often as the local timeline where I’m at. I wound up using a different Mastodon host due to the dated installation there at the time (since fixed) and some performance issues, but SDF is a fantastic organization (along side the Tildes)Now if I could just get them to process my #UUCP feed request… 🙂
uucp
@jgoerzen @SDF Well, keep an eye on my feed for (I’m sure) more terminals logged into SDF; this is already number two. I hope to get a Tek 4012 up and running in the not-so-distant future, that’ll be a treat. (I have a working 4010, but since it’s ASCII 63, I can’t log in anywhere!)
@jgoerzen @derkern @szczezuja well I taught UNIX for Nixdorf with their own terminal hardware, AIX i accessed via terminal emulation on a PC iirc, and I used VT100(original) connected to a PDP-11 at the Luftwaffe programming center, where i was drafted to after graduating from university (instead of being drafted before uni). did you read Brian Kernighans Unix memories?
@PeterSommerlad @derkern @szczezuja I literally have a copy of #Kernighan‘s “UNIX: A History and a Memoir” sitting on my desk in my to-read stack, one which sadly seems to be growing faster than it is shrinking! Thanks for sharing; those are names I recognize (and interesting being drafted to do programming!)I was stuck when I visited the computing history museum at #BletchleyPark how different computing hardware was in US vs. Europe until the 90s. A lot of stuff there I’d never heard of.
bletchleypark
kernighan