I recently had a simple-sounding desire. I would like to run the latest stable version of Digikam. My desktop, however, runs Debian stable, which has 5.3.0, not 5.9.0.
This is not such a simple proposition.
$ ldd /usr/bin/digikam | wc -l
396
And many of those were required at versions that weren’t in stable.
I had long thought that AppImage was a rather bad idea, but I decided to give it a shot. I realized it was worse than I had thought.
The problems with AppImage
About a year ago, I wrote about the problems Docker security. I go into much more detail there, but the summary for AppImage is quite similar. How can I trust all the components in the (for instance) Digikam AppImage image are being kept secure? Are they using the latest libssl and libpng, to avoid security issues? How will I get notified of a security update? (There seems to be no mechanism for this right now.) An AppImage user that wants to be secure has to manually answer every one of those questions for every application. Ugh.
Nevertheless, the call of better facial detection beckoned, and I downloaded the Digikam AppImage and gave it a whirl. The darn thing actually fired up. But when it would play videos, there was no sound. Hmmmm.
I found errors like this:
Cannot access file ././/share/alsa/alsa.conf
Nasty. I spent quite some time trying to make ALSA work, before a bunch of experimentation showed that if I ran alsoft-conf on the host, and selected only the PulseAudio backend, then it would work. I reported this bug to Digikam.
Then I thought it was working — until I tried to upload some photos. It turns out that SSL support in Qt in the AppImage was broken, since it was trying to dlopen an incompatible version of libssl or libcrypto on the host. More details are in the bug I reported about this also.
These are just two examples. In the rather extensive Googling I did about these problems, I came across issue after issue people had with running Digikam in an AppImage. These issues are not limited to the ALSA and SSL issues I describe here. And they are not occurring due to some lack of skill on the part of Digikam developers.
Rather, they’re occurring because AppImage packaging for a complex package like this is hard. It’s hard because it’s based on a fiction — the fiction that it’s possible to make an AppImage container for a complex desktop application act exactly the same, when the host environment is not exactly the same. Does the host run PulseAudio or ALSA? Where are its libraries stored? How do you talk to dbus?
And it’s not for lack of trying. The scripts to build the Digikam appimage support runs to over 1000 lines of code in the AppImage directory, plus another 1300 lines of code (at least) in CMake files that handle much of the work, and another 3000 lines or so of patches to 3rd-party packages. That’s over 5000 lines of code! By contrast, the Debian packaging for the same version of Digikam, including Debian patches but excluding the changelog and copyright files, amounts to 517 lines. Of course, it is reusing OS packages for the dependencies that were already built, but this amounts to a lot simpler build.
Frankly I don’t believe that AppImage really lives up to its hype. Requiring reinventing a build system and making some dangerous concessions on security for something that doesn’t really work in the end — not good in my book.
The library problem
But of course, AppImage exists for a reason. That reason is that it’s a real pain to deal with so many levels of dependencies in software. Even if we were to compile from source like the old days, and even if it was even compatible with the versions of the dependencies in my OS, that’s still a lot of work. And if I have to build dependencies from source, then I’ve given up automated updates that way too.
There’s a lot of good that ELF has brought us, but I can’t help but think that it wasn’t really designed for a world in which a program links 396 libraries (plus dlopens a few more). Further, this world isn’t the corporate Unix world of the 80s; Open Source developers aren’t big on maintaining backwards compatibility (heck, both the KDE and Qt libraries under digikam have both been entirely rewritten in incompatible ways more than once!) The farther you get from libc, the less people seem to care about backwards compatibility. And really, who can blame volunteers? You want to work on new stuff, not supporting binaries from 5 years ago, right?
I don’t really know what the solution is here. Build-from-source approaches like FreeBSD and Gentoo have plenty of drawbacks too. Is there some grand solution I’m missing? Some effort to improve this situation without throwing out all the security benefits that individually-packaged libraries give us in distros like Debian?
I get
ldd /usr/bin/digikam | wc -l
265
I’m also running Debian stable/stretch/9.4. And the Digikam version is 4:5.3.0-1. Why the disparity?
That was from 5.9.0 on buster.
Oh, ok. That wasn’t totally clear. So the situation gets worse in buster.
What do you think about Snap and Flatpak? There, applications don’t use any libraries from the host, so at least the problems that causes on AppImage shouldn’t happen. Of course they still need to talk to services running on the host like X, D-Bus, or PulseAudio. And base snaps/runtimes should make the security situation a bit better.
I haven’t looked into them in detail, but I thought AppImage was supposed to do pretty much the same. snap seems to be something of a hybrid, though; I haven’t ever quite understood why it’s a whole new packaging tool. https://askubuntu.com/questions/866511/what-are-the-differences-between-snaps-appimage-flatpak-and-others has a somewhat interesting read
I didn’t even mention GLX extensions because my use case didn’t need them. The 3D needs in any of those environments are going to be a mess, because some of those .so files are tied to which driver is in use.
Well, as for your situation with your obsolete Debian distro, either you change it on something with newest digikam version, or just use some kind of virtualization. Doesn’t simple with AppImage? Ok, try others. Better to use something like LXC, and do something with sound from guest to host environment.
As for libraries as a whole, modern solution will be to use modern languages like Go&Rust and others, who gives you almost statical binaries. And either rebuild them by yourself, or get updates binaries from developer. One binary without dependencies to rule them all, baby!
That just makes the situation worse, too. Imagine if you have OpenSSL statically linked into every binary. Instead of replacing one library, it’s now dozens — probably hundreds — of binaries to recompile or relink.
You should have a look at GNU Guix, and the way Guix and Nix do packaging. It has some of the advantages of dynamic linking, but as you’re normally linking against libraries in an immutable store, you get a load of guarantees about stability as well.
You should try Nix..oh Chris Baines beat me to it again.
There is a fundamental problem involved we will not fix so easily. If we want to share code on a system, and we sometimes want so that we can replace a single lib to apply (security) fixes, we inherently introduce breakage if we change the ABIs.
Note: NIX also cannot fully solve the problem, if the ABI is incompatible, NIX needs to increase the number of the installed lib’s versions.
So imho there is no way around having reliable low-level libraries with stable ABIs. For higher-level application-specific stuff (like most of what digikam depends on) there are plenty of solutions: old-fashioned LD_LIBRARY_PATH/RPATH/.. (like it is done for libreoffice, also in Debian, see /usr/lib/libreoffice/program for libreoffice’s libs) or one of the containerization solutions already mentioned here.
An important question is: what should be part of these trustworthy low-level components.
And to solve your concrete problem: Maybe we should package KDE differently so that KDE applications can be in backports.
If I understand the suggestion correctly – permit multiple ABI versions of the needed qt/kde components akin to what is done with OpenSSL – that’s a darn good one. I like it. Maybe instead of, for instance, kdelibs-bin providing files in /usr/lib/kde4, kdelibs-bin4.14 would provide files in /usr/lib/kde4.14? That would permit multiple versions to be installed side-by-side.
Typo:
“wrote about the problems Docker security” -> “the problems *with* Docker”