Following up to yesterday’s article about how NNCP rehabilitates asynchronous communication with modern encryption and onion routing, here is the first of my posts showing how to put it into action.
Email is a natural fit for async; in fact, much of early email was carried by UUCP. It is useful for an airgapped machine to be able to send back messages; errors from cron, results of handling incoming data, disk space alerts, etc. (Of course, this would apply to a non-airgapped machine also).
The NNCP documentation already describes how to do this for Postfix. Here I will show how to do it for Exim.
A quick detour to UUCP land
When you encounter a system such as email that has instructions for doing something via UUCP, that should be an alert to you that “here is some very relevant information for doing this same thing via NNCP.” The syntax is different, but broadly, here’s a table of similar NNCP commands:
| Purpose | UUCP | NNCP |
|---|---|---|
| Connect to remote system | uucico -s, uupoll | nncp-call, nncp-caller |
| Receive connection (pipe, daemon, etc) | uucico (-l or similar) | nncp-daemon |
| Request remote execution, stdin piped in | uux | nncp-exec |
| Copy file to remote machine | uucp | nncp-file |
| Copy file from remote machine | uucp | nncp-freq |
| Process received requests | uuxqt | nncp-toss |
| Move outbound requests to dir (for USB stick, airgap, etc) | N/A | nncp-xfer |
| Create streaming package of outbound requests | N/A | nncp-bundle |
If you used UUCP back in the day, you surely remember bang paths. I will not be using those here. NNCP handles routing itself, rather than making the MTA be aware of the network topology, so this simplifies things considerably.
Sending from Exim to a smarthost
One common use for async email is from a satellite system: one that doesn’t receive mail, or have local mailboxes, but just needs to get email out to the Internet. This is a common situation even for conventionally-connected systems; in Exim speak, this is a “satellite system that routes mail via a smarthost.” That is, every outbound message goes to a specific target, which then is responsible for eventual delivery (over the Internet, LAN, whatever).
This is fairly simple in Exim.
We actually have two choices for how to do this: bsmtp or rmail mode. bsmtp (batch SMTP) is the more modern way, and is essentially a derivative of SMTP that explicitly can be queued asynchronously. Basically it’s a set of SMTP commands that can be saved in a file. The alternative is “rmail” (which is just an alias for sendmail these days), where the data is piped to rmail/sendmail with the recipients given on the command line. Both can work with Exim and NNCP, but because we’re doing shiny new things, we’ll use bsmtp.
These instructions are loosely based on the Using outgoing BSMTP with Exim HOWTO. Some of these may assume Debianness in the configuration, but should be easily enough extrapolated to other configs as well.
First, configure Exim to use satellite mode with minimal DNS lookups (assuming that you may not have working DNS anyhow).
Then, in the Exim primary router section for smarthost (router/200_exim4-config_primary in Debian split configurations), just change transport = remote_smtp_smarthost to transport = nncp.
Now, define the NNCP transport. If you are on Debian, you might name this transports/40_exim4-config_local_nncp:
nncp: debug_print = "T: nncp transport for $local_part@$domain" driver = pipe user = nncp batch_max = 100 use_bsmtp command = /usr/local/nncp/bin/nncp-exec -noprogress -quiet hostname_goes_here rsmtp .ifdef REMOTE_SMTP_HEADERS_REWRITE headers_rewrite = REMOTE_SMTP_HEADERS_REWRITE .endif .ifdef REMOTE_SMTP_RETURN_PATH return_path = REMOTE_SMTP_RETURN_PATH .endif
This is pretty straightforward. We pipe to nncp-exec, run it as the nncp user. nncp-exec sends it to a target node and runs whatever that node has called rsmtp (the command to receive bsmtp data). When the target node processes the request, it will run the configured command and pipe the data in to it.
More complicated: Routing to various NNCP nodes
Perhaps you would like to be able to send mail directly to various NNCP nodes. There are a lot of ways to do that.
Fundamentally, you will need a setup similar to the UUCP example in Exim’s manualroute manual, which lets you define how to reach various hosts via UUCP/NNCP. Perhaps you have a star topology (every NNCP node exchanges email with a central hub). In the NNCP world, you have two choices of how you do this. You could, at the Exim level, make the central hub the smarthost for all the side nodes, and let it redistribute mail. That would work, but requires decrypting messages at the hub to let Exim process. The other alternative is to configure NNCP to just send to the destinations via the central hub; that takes advantage of onion routing and doesn’t require any Exim processing at the central hub at all.
Receiving mail from NNCP
On the receiving side, first you need to configure NNCP to authorize the execution of a mail program. In the section of your receiving host where you set the permissions for the client, include something like this:
exec: {
rsmtp: ["/usr/sbin/sendmail", "-bS"]
}
The -bS option is what tells Exim to receive BSMTP on stdin.
Now, you need to tell Exim that nncp is a trusted user (able to set From headers arbitrarily). Assuming you are running NNCP as the nncp user, then add MAIN_TRUSTED_USERS = nncp to a file such as /etc/exim4/conf.d/main/01_exim4-config_local-nncp. That’s it!
Some hosts, of course, both send and receive mail via NNCP and will need configurations for both.
Very interesting, finally a modernized UUCP! Looking forward to NNCP appearing in Debian.
Yes, same here. I started packaging it, but it’s a Go package with a custom build system and I didn’t have enough time to sort through all that at the time. Still on my list though!
I’m playing with UUCP (over TCP/540 and TOR/TCP/540, other transports, especially modems and license free transceivers will follow) on Debian/Devuan/Raspian (BSDs will follow later).
Go is no go for me, so NNCP never was a topic for me.
Would it be compaible with UUCP?
@elb @ajroach42 8/ So if you have a VPS or a machine “in town” or whatever, you can do some pretty nice things; take the photos you copied into the “to upload” Syncthing folder and upload them, then delete them out of there. Or a laptop can run those commands directly “in town”
@elb @ajroach42 9/ Finally the two best ways to improve your 4G signal are: 1) height, and 2) antenna. I got one of these https://smile.amazon.com/gp/product/B01NBSLNJ6 with a Nighthawk M1 awhile back. Tremendous difference. A booster can only boost what it can receive. A good antenna, mounted high, hardwired into the access point will almost certainly be better. That antenna has “gain”, meaning it’s directional, so figure out where your best towers are and point it at those.
@elb @ajroach42 10/ Also point-to-point wireless may help; if there’s a good place you can get Internet and you have line-of-sight from your house, you may be able to work something out, even something surprisingly fast. For more challenging conditions, LoRA or XBee could work… but at 100Kbps or less. Not suitable for browsing but could work for email.
@elb @ajroach42 11/ Finally, don’t understimate the utility of sshing to a VPS somewhere and reading email in text. My qualifications to anser: have lived in Internet-challenged areas for 20 years, frequently travel into no-Internet areas, have experience with modern communication over extremely low bandwidth links (1 to 100Kbps) including LoRA/XBee radio, AX.25 packet radio, and satellite.
@elb @ajroach42 Also, @joeyh lives off-grid and may have lots of ideas to contribute here.
@jgoerzen @elb Thick canopy and mountains, so there’s only so much improving to do. We’ll stick an antenna up pretty high, and run that in to our little booster. It’ll get the job done (I’m posting now over that cellular connection. It works, when you’re in the right location.)
@jgoerzen @elb I did some experimenting with LoRA and tried to do some work with XBee for keeping nodes of a distributed BBS in sync over multiple KM, but ultimately we just didn’t have the mesh density, and I ended up building a solution that used a device I carried with me to rsync each location over wifi. Using syncthing and a cellphone (and NNCP) seems like a more viable longterm solution.
@ajroach42 @jgoerzen @elb I still say we should try the trebuchet of usb flash drives thing at some point
@djsundog @ajroach42 @jgoerzen @elb question: what would be the bandwidth of 90 kg of flash drives launched 300 m?How fast would that be travelling
@nev @djsundog @jgoerzen @elb https://what-if.xkcd.com/31/
FedEx Bandwidth
@ajroach42 @djsundog @jgoerzen @elb this doesn’t answer my question!!
@ajroach42 @djsundog @jgoerzen @elb I googled it myself. Top speed 70 m/s, apparently…someone do the math
@nev a USB drive on average weighs about 30 grams, according to the two sources I found that weighed USB drives. Microcenter cells 256 GB flash drives. You could fit 3000 of them in a 90 KG payload. So that’s 768 terabytes.A full sized trebuchet can launch a 90kg at roughly 70m/s or 156.586 MPH. So you’re sending 768 TB at 300 meters in 4.2 seconds for a transfer speed of 182.857143 terabytes TBps.
@ajroach42 @nev the bottleneck is going to be reading from the interface
@nev Or, in more conventional terms, 1462857144 Mbps, or 1462857144000 Kbps or 1.462857144e+15 bits per second.
@ajroach42 thank you for this incredible contribution to modern science.
@ajroach42 @nev this works as a value for peak throughput, but i think in order to give a full picture of the capabilities, this calculation should also consider loading time rather than just launching time
@ajroach42 @nev I thought 30g sounded like a /really/ heavy flashdrive, so I weighed the small jar of flashdrives on my desk. their weights: 10g, 9g, 4g, 8g, 16g. which would increase data throughput :) I can weigh some SD cards too I guess… nvm. microSD card doesn’t make it up to 1g on my scales. 0.5g would be best guess. regular sized SD card is about 2 or 2.5g.
@epoch @ajroach42 okay so by my calculation, assuming a micro SD card is 0.5g and holds 1 TB, a 90kg payload contains 180 PB…
@nev @epoch and it’ll travel 300 meters in 4.2 seconds. So 180PB/4.2 seconds is 342 Petabits per second.