On mail.quux, a node of NNCPNET (the NNCP-based peer-to-peer email network), I started noticing emails not being delivered. They were all in the queue, frozen, and Exim’s log had entries like:
unable to set gid=5001 or uid=5001 (euid=100): local delivery to [redacted] transport=nncp
Weird.
Stranger still, when I manually ran the queue with sendmail -qff -v, they all delivered fine.
Huh.
Well, I thought, it was a one-off weird thing. But then it happened again.
Upon investigating, I observed that this issue was happening only on messages submitted by SMTP. Which, on these systems, aren’t that many.
While trying different things, I tried submitting a message to myself using SMTP. Nothing to do with NNCP at all. But look at this:
jgoerzen@[redacted] R=userforward defer (-1): require_files: error for /home/jgoerzen/.forward: Permission denied
Strraaannnge….
All the information I could find about this, even a FAQ entry, said that the problem is that Exim isn’t setuid root. But it is:
-rwsr-xr-x 1 root root 1533496 Mar 29 2025 /usr/sbin/exim4
This problem started when I upgraded to Debian Trixie. So what changed there?
There are a lot of possibilities; this is running in Docker using my docker-debian-base system, which runs a regular Debian in Docker, including systemd.
I eventually tracked it down to Exim migrating from init.d to systemd in trixie, and putting a bunch of lockdowns in its service file. After a bunch of trial and error, I determined that I needed to override this set of lockdowns to make it work. These overrides did the trick:
ProtectClock=false PrivateDevices=false RestrictRealtime=false ProtectKernelModules=false ProtectKernelTunables=false ProtectKernelLogs=false ProtectHostname=false
I don’t know for sure if the issue is related to setuid. But if it is, there’s nothing that immediately jumps out at me about any of these that would indicate a problem with setuid.
I also don’t know if running in Docker makes any difference.
Anyhow, problem fixed, but mystery not solved!
 
						
A similar issue was brought up on the exim-users mailing list, with the red-flag log words “Read-only file system”. The solution suggestion there was systemd’s ProtectSystem and/or ProtectHome configuration.
Usually the Debian guys are better than this.
Debian didn’t have ProtectHome enabled. It did have ProtectSystem, but disabling that wasn’t necessary to fix these issues.
https://etbe.coker.com.au/2025/01/17/systemd-hardening-mail/
After writing the above blog post about similar issues I received the following comment by email:
Exim has the “queue_only = ” or “-odq” option to disable the “fork to background here and now” delivery. However it doesn’t ping a running Exim daemon that a message has been queued (unlike say Postfix), so the messages will only be sent during the next queue run depending on the daemon’s -q option, which may take a while unless you add some inotify watcher on top.
(Overall, that’s one of the reasons I prefer Postfix for send-only systems.)
When Arch was introducing systemd, one of my contributions was to add an exim-queue.path that would start a queue-runnner as soon as the queue directory became non-empty (which fit well together with a .timer to emulate the -q30m periodic flush).