Monthly Archives: June 2018

Making a difference

Politics

Every day, ask yourself this question: What one thing can I do today that will make this democracy stronger and honor and support its institutions? It doesn’t have to be a big thing. And it probably won’t shake the Earth. The aggregation of them will shake the Earth.

– Benjamin Wittes

I have written some over the past year or two about the dangers facing the country. I have become increasingly alarmed about the state of it. And that Benjamin Wittes quote, along with the terrible tragedy, spurred me to action. Among other things, I did two things I never have done before:

I registered to protest on June 30.

I volunteered to do phone banking with SwingLeft.

And I changed my voter registration from independent to Republican.

No, I have not gone insane. The reason for the latter is that here in Kansas, the Democrats rarely field candidates for most offices. The real action happens in the Republican primary. So if I can vote in that primary, I can have a voice in keeping the crazy out of office. It’s not much, but it’s something.

Today we witnessed, hopefully, the first victory in our battle against the abusive practices happening to children at the southern border. Donald Trump caved, and in so doing, implicitly admitted the lies he and his administration have been telling about the situation. This only happened because enough people thought like Wittes: “I am small, but I can do SOMETHING.” When I called the three Washington offices of my senators and representatives — far-right Republicans all — it was apparent that I was by no means the first to give them an earful about this, and that they were changing their tone because of what they heard. Mind you, they hadn’t taken any ACTION yet, but the calls mattered. The reporting mattered. The attention mattered.

I am going to keep doing what little bit I can. I hope everyone else will too. Let us shake the Earth.

Memories, Father’s Day, and an 89-year-old plane

Aviation, Family

“Oh! I have slipped the surly bonds of Earth
And danced the skies on laughter-silvered wings;
Sunward I’ve climbed, and joined the tumbling mirth
of sun-split clouds, — and done a hundred things”

– John Gillespie Magee, Jr.

I clicked on the radio transmitter in my plane.

O’Neill Traffic, Bonanza xx departing to the south. And Trimotor, thanks for flight . We really enjoyed it.

And we had. Off to my left, a 1929 Ford Trimotor airliner was heading off into the distance, looking as if it were just hanging in the air, glinting in the morning sun, 1000 feet above the ground. Earlier that morning, my boys and I had been passengers in that very plane. But now we had taken off right after them, as they were taking another load of passengers up for a flight and we were flying back home. To my right was my 8-year-old, and my 11-year-old was in back, both watching out the windows. The radio clicked on, and the three of us heard the other pilot’s response:

Oh thank you. We’re glad you came!

A few seconds later, they were gone out of sight.

The experience of flying in an 89-year-old airliner is quite something. As with the time we rode on the Durango & Silverton railroad, it felt like stepping back into a time machine — into the early heyday of aviation.

Jacob and Oliver had been excited about this day a long time. We had tried to get a ride when it was on tour in Oklahoma, much closer, but one of them got sick on the drive that day and it didn’t work out. So Saturday morning, we took the 1.5-hour-flight up to northern Nebraska. We’d heard they’d have a pancake breakfast fundraiser, and the boys were even more excited. They asked to set the alarm early, so we’d have no risk of missing out on airport pancakes.

Jacob took this photo of the sunrise at the airport while I was doing my preflight checks:

IMG_1574

Here’s one of the beautiful views we got as we flew north to meet the Trimotor.

IMG_20180616_070810_v1

It was quite something to share a ramp with that historic machine. Here’s a photo of our plane not far from the Trimotor.

IMG_20180616_082051

After we got there, we checked in for the flight, had a great pancake and sausage breakfast, and then into the Trimotor. The engines fired up with a most satisfying low rumble, and soon we were aloft — cruising along at 1000 feet, in that (by modern standards) noisy, slow, and beautiful machine. We explored the Nebraska countryside from the air before returning 20 minutes later. I asked the boys what they thought.

“AWESOME!” was the reply. And I agreed.

IMG_20180616_090828

Jacob and Oliver have long enjoyed pretending to be flight attendants when we fly somewhere. They want me to make airline-sounding announcements, so I’ll say something like, “This is your captain speaking. In a few moments, we’ll begin our descent into O’Neill. Flight attendants, prepare the cabin for arrival.” Then Jacob will say, “Please return your tray tables that you don’t have to their full upright and locked position, make sure your seat belt is tightly fastened, and your luggage is stowed. This is your last chance to visit the lavatory that we don’t have. We’ll be on the ground shortly.”

Awhile back, I loaded up some zip-lock bags with peanuts and found some particularly small bottles of pop. Since then, it’s become tradition on our longer flights for them to hand out bags of peanuts and small quantities of pop as we cruise along — “just like the airlines.” A little while back, I finally put a small fridge in the hangar so they get to choose a cold beverage right before we leave. (We don’t typically have such things around, so it’s a special treat.)

Last week, as I was thinking about Father’s Day, I told them how I remembered visiting my dad at work, and how he’d let me get a bottle of Squirt from the pop machine there (now somewhat rare). So when we were at the airport on Saturday, it brought me a smile to hear, “DAD! This pop machine has Squirt! Can we get a can? It’s only 75 cents!” “Sure – after our Trimotor flight.” “Great! Oh, thank you dad!”

I realized then I was passing a small but special memory on to another generation. I’ve written before of my childhood memories of my dad, and wondering what my children will remember of me. Martha isn’t old enough yet to remember her cackles of delight as we play peek-a-boo or the books we read at bedtime. Maybe Jacob and Oliver will remember our flights, or playing with mud, or researching dusty maps in a library, playing with radios, or any of the other things we do. Maybe all three of them will remember the cans of Squirt I’m about to stock that hangar fridge with.

But if they remember that I love them and enjoy doing things with them, they will have remembered the most important thing. And that is another special thing I got from my parents, and can pass on to another generation.

Syncing with a memory: a unique use of tar –listed-incremental

Uncategorized

I have a Nextcloud instance that various things automatically upload photos to. These automatic folders sync to a directory on my desktop. I wanted to pull things out of that directory without deleting them, and only once. (My wife might move them out of the directory on her computer, and I might arrange them into targets on my end.)

In other words, I wanted to copy a file from a source to a destination, but remember what had been copied before so it only ever copies once.

rsync doesn’t quite do this. But it turns out that tar’s listed-incremental feature can do exactly that. Ordinarily, it would delete files that were deleted on the source. But if we make the tar file with the incremental option, but extract it without, it doesn’t try to delete anything at extract time.

Here’s my synconce script:

#!/bin/bash

set -e

if [ -z "$3" ]; then
    echo "Syntax: $0 snapshotfile sourcedir destdir"
    exit 5
fi

SNAPFILE="$(realpath "$1")"
SRCDIR="$2"
DESTDIR="$(realpath "$3")"

cd "$SRCDIR"
if [ -e "$SNAPFILE" ]; then
    cp "$SNAPFILE" "${SNAPFILE}.new"
fi
tar "--listed-incremental=${SNAPFILE}.new" -cpf - . | \
    tar -xf - -C "$DESTDIR"
mv "${SNAPFILE}.new" "${SNAPFILE}"

Just have the snapshotfile be outside both the sourcedir and destdir and you’re good to go!

Running Digikam inside Docker

Uncategorized

After my recent complaint about AppImage, I thought I’d describe how I solved my problem. I needed a small patch to Digikam, which was already in Debian’s 5.9.0 package, and the thought of rebuilding the AppImage was… unpleasant.

I thought – why not just run it inside Buster in Docker? There are various sources on the Internet for X11 apps in Docker. It took a little twiddling to make it work, but I did.

My Dockerfile was pretty simple:

FROM debian:buster
MAINTAINER John Goerzen 

RUN apt-get update && \
    apt-get -yu dist-upgrade && \
    apt-get --install-recommends -y install firefox-esr digikam digikam-doc \
         ffmpegthumbs imagemagick minidlna hugin enblend enfuse minidlna pulseaudio \
         strace xterm less breeze && \
    apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN adduser --disabled-password --uid 1000 --gecos "John Goerzen" jgoerzen && \
    rm -r /home/jgoerzen/.[a-z]*
RUN rm /etc/machine-id
CMD /usr/bin/docker

RUN mkdir -p /nfs/personalmedia /run/user/1000 && chown -R jgoerzen:jgoerzen /nfs /run/user/1000

I basically create the container and my account in it.

Then this script starts up Digikam:

#!/bin/bash

set -e

# This will be unnecessary with docker 18.04 theoretically....  --privileged see
# https://stackoverflow.com/questions/48995826/which-capabilities-are-needed-for-statx-to-stop-giving-eperm
# and https://bugs.launchpad.net/ubuntu/+source/docker.io/+bug/1755250

docker run -ti \
       -v /tmp/.X11-unix:/tmp/.X11-unix -v "/run/user/1000/pulse:/run/user/1000/pulse" -v /etc/machine-id:/etc/machine-id \
       -v /etc/localtime:/etc/localtime \
       -v /dev/shm:/dev/shm -v /var/lib/dbus:/var/lib/dbus -v /var/run/dbus:/var/run/dbus -v /run/user/1000/bus:/run/user/1000/bus  \
       -v "$HOME:$HOME" -v "/nfs/personalmedia/Pictures:/nfs/personalmedia/Pictures" \
     -e DISPLAY="$DISPLAY" \
     -e XDG_RUNTIME_DIR="$XDG_RUNTIME_DIR" \
     -e DBUS_SESSION_BUS_ADDRESS="$DBUS_SESSION_BUS_ADDRESS" \
     -e LANG="$LANG" \
     --user "$USER" \
     --hostname=digikam \
     --name=digikam \
     --privileged \
     --rm \
     jgoerzen/digikam "$@"  /usr/bin/digikam

The goal here was not total security isolation; if it had been, then all the dbus mounting and $HOME mounting was a poor idea. But as an alternative to AppImage — well, it worked perfectly. I could even get security updates if I wanted.

Please stop making the library situation worse with attempts to fix it

Linux

I recently had a simple-sounding desire. I would like to run the latest stable version of Digikam. My desktop, however, runs Debian stable, which has 5.3.0, not 5.9.0.

This is not such a simple proposition.


$ ldd /usr/bin/digikam | wc -l
396

And many of those were required at versions that weren’t in stable.

I had long thought that AppImage was a rather bad idea, but I decided to give it a shot. I realized it was worse than I had thought.

The problems with AppImage

About a year ago, I wrote about the problems Docker security. I go into much more detail there, but the summary for AppImage is quite similar. How can I trust all the components in the (for instance) Digikam AppImage image are being kept secure? Are they using the latest libssl and libpng, to avoid security issues? How will I get notified of a security update? (There seems to be no mechanism for this right now.) An AppImage user that wants to be secure has to manually answer every one of those questions for every application. Ugh.

Nevertheless, the call of better facial detection beckoned, and I downloaded the Digikam AppImage and gave it a whirl. The darn thing actually fired up. But when it would play videos, there was no sound. Hmmmm.

I found errors like this:

Cannot access file ././/share/alsa/alsa.conf

Nasty. I spent quite some time trying to make ALSA work, before a bunch of experimentation showed that if I ran alsoft-conf on the host, and selected only the PulseAudio backend, then it would work. I reported this bug to Digikam.

Then I thought it was working — until I tried to upload some photos. It turns out that SSL support in Qt in the AppImage was broken, since it was trying to dlopen an incompatible version of libssl or libcrypto on the host. More details are in the bug I reported about this also.

These are just two examples. In the rather extensive Googling I did about these problems, I came across issue after issue people had with running Digikam in an AppImage. These issues are not limited to the ALSA and SSL issues I describe here. And they are not occurring due to some lack of skill on the part of Digikam developers.

Rather, they’re occurring because AppImage packaging for a complex package like this is hard. It’s hard because it’s based on a fiction — the fiction that it’s possible to make an AppImage container for a complex desktop application act exactly the same, when the host environment is not exactly the same. Does the host run PulseAudio or ALSA? Where are its libraries stored? How do you talk to dbus?

And it’s not for lack of trying. The scripts to build the Digikam appimage support runs to over 1000 lines of code in the AppImage directory, plus another 1300 lines of code (at least) in CMake files that handle much of the work, and another 3000 lines or so of patches to 3rd-party packages. That’s over 5000 lines of code! By contrast, the Debian packaging for the same version of Digikam, including Debian patches but excluding the changelog and copyright files, amounts to 517 lines. Of course, it is reusing OS packages for the dependencies that were already built, but this amounts to a lot simpler build.

Frankly I don’t believe that AppImage really lives up to its hype. Requiring reinventing a build system and making some dangerous concessions on security for something that doesn’t really work in the end — not good in my book.

The library problem

But of course, AppImage exists for a reason. That reason is that it’s a real pain to deal with so many levels of dependencies in software. Even if we were to compile from source like the old days, and even if it was even compatible with the versions of the dependencies in my OS, that’s still a lot of work. And if I have to build dependencies from source, then I’ve given up automated updates that way too.

There’s a lot of good that ELF has brought us, but I can’t help but think that it wasn’t really designed for a world in which a program links 396 libraries (plus dlopens a few more). Further, this world isn’t the corporate Unix world of the 80s; Open Source developers aren’t big on maintaining backwards compatibility (heck, both the KDE and Qt libraries under digikam have both been entirely rewritten in incompatible ways more than once!) The farther you get from libc, the less people seem to care about backwards compatibility. And really, who can blame volunteers? You want to work on new stuff, not supporting binaries from 5 years ago, right?

I don’t really know what the solution is here. Build-from-source approaches like FreeBSD and Gentoo have plenty of drawbacks too. Is there some grand solution I’m missing? Some effort to improve this situation without throwing out all the security benefits that individually-packaged libraries give us in distros like Debian?