A (Partial) Defense of Debian

I was sad to read on his blog that Michael Stapelberg is winding down his Debian involvement. In his post, he outlined some critiques of Debian. In his post, I want to acknowledge that he is on point with some of them, but also push back on others. Some of this is also a response to some of the comments on Hacker News.

I’d first like to discuss some of the assumptions I believe his post rests on: namely that “we’ve always done it this way” isn’t a good reason to keep doing something. I completely agree. However, I would also say that “this thing is newer, so it’s better and we should use it” is also poor reasoning. Newer is not always better. Sometimes it is, sometimes it’s not, but deeper thought is generally required.

Also, when thinking about why things are a certain way or why people prefer certain approaches, we must often ask “why does that make sense to them?” So let’s dive in.

Debian’s Perspective: Stability

Stability, of course, can mean software that tends not to crash. That’s important, but there’s another aspect of it that is also important: software continuing to act the same over time. For instance, if you wrote a C program in 1985, will that program still compile and run today? Granted, that’s a bit of an extreme example, but the point is: to what extent can you count on software you need continuing to operate without forced change?

People that have been sysadmins for a long period of time will instantly recognize the value of this kind of stability. Change is expensive and difficult, and often causes outages and incidents as bugs are discovered when software is adapted to a new environment. Being able to keep up-to-date with security patches while also expecting little or no breaking changes is a huge win. Maintaining backwards compatibility for old software is also important.

Even from a developer’s perspective, lack of this kind of stability is why I have handed over maintainership of most of my Haskell software to others. Some of my Haskell projects were basically “done”, and every so often I’d get bug reports that it no longer compiles due to some change in the base library. Occasionally I’d have patches with those bug reports, but they would inevitably break compatibility with older versions (even though the language has plenty good support for something akin to a better version of #ifdefs to easily deal with this.) The culture of stability was not there.

This is not to say that this kind of stability is always good or always bad. In the Haskell case, there is value to be had in fixing designs that are realized to be poor and removing cruft. Some would say that strcpy() should be removed from libc for this reason. People that want the latest versions of gimp or whatever are probably not going to be running Debian stable. People that want to install a machine and not really be burdened by it for a couple of years are.

Debian has, for pretty much its entire life, had a large proportion of veteran sysadmins and programmers as part of the organization. Many of us have learned the value of this kind of stability from the school of hard knocks – over and over again. We recognize the value of something that just works, that is so stable that things like unattended-upgrades are safe and reliable. With many other distros, something like this isn’t even possible; when your answer to a security bug is to “just upgrade to the latest version”, just trusting a cron job to do it isn’t going to work because of the higher risk.

Recognizing Personal Preference

Writing about Debian’s bug-tracking tool, Michael says “It is great to have a paper-trail and artifacts of the process in the form of a bug report, but the primary interface should be more convenient (e.g. a web form).” This is representative of a personal preference. A web form might be more convenient for Michael — I have no reason to doubt this — but is it more convenient for everyone? I’d say no.

In his linked post, Michael also writes: “Recently, I was wondering why I was pushing off accepting contributions in Debian for longer than in other projects. It occurred to me that the effort to accept a contribution in Debian is way higher than in other FOSS projects. My remaining FOSS projects are on GitHub, where I can just click the “Merge” button after deciding a contribution looks good. In Debian, merging is actually a lot of work: I need to clone the repository, configure it, merge the patch, update the changelog, build and upload. “

I think that’s fair for someone that wants a web-based workflow. Allow me to present the opposite: for me, I tend to push off contributions that only come through Github, and the reason is that, for me, they’re less convenient. It’s also harder for me to contribute to Github projects than Debian ones. Let’s look at this – say I want to send in a small patch for something. If it’s Github, it’s going to look like this:

  1. Go to the website for the thing, click fork
  2. Now clone that fork or add it to my .git/config, hack, and commit
  3. Push the commit, go back to the website, and submit a PR
  4. Github’s email integration is so poor that I basically have to go back to the website for most parts of the conversation. I can do little from the comfort of mu4e.
  5. Remember to clean up my fork after the patch is accepted or rejected.

Compare that to how I’d contribute with Debian:

  1. Hack (and commit if I feel like it)
  2. Type “reportbug foo”, attach my patch
  3. Followup conversation happens directly in email where it’s convenient to reply

How about as the developer? Github constantly forces me to their website. I can’t very well work on bug reports, etc. without a strong Internet connection. And it’s designed to push people into using their tools and their interface, which is inferior in a lot of ways to a local interface – but then the process to pull down someone else’s set of patches involves a lot of typing and clicking, much more that would be involved from a simple git format-patch. In short, I don’t have my shortcut keys, my environment, etc. for reviewing things – the roadblocks are there to make me use theirs.

If I get a contribution from someone in debbugs, it’s so much easier. It’s usually just git apply or patch -p1 and boom, I can see exactly what’s changed and review it. A review comment is just a reply to an email. I don’t have to ever fire up a web browser. So much more convenient.

I don’t write this to say Michael is wrong about what’s more convenient for him. I write it to say he’s wrong about what’s more convenient for me (or others). It may well be the case that debbugs is so inconvenient that it pushes him to leave while github is so inconvenient for others that it pushes them to avoid it.

I will note before leaving this conversation that there are some command-line tools available for Github and a web interface to debbugs, but it is still clear that debbugs is a lot easier to work with from within my own mail reader and tooling, and Github is a lot easier to work with from within a web browser.

The case for reportbug

I remember the days before we had reportbug. Over and over and over again, I would get bug reports from users that wouldn’t have the basic information needed to investigate. reportbug gathers information from the system: package versions, configurations, versions of dependencies, etc. A simple web form can’t do this because it doesn’t have a local agent. From a developer’s perspective, trying to educate users on how to do this over and over as an unending, frustrating, and counter-productive task. Even if it’s clearly documented, the battle will be fought over and over. From a user’s perspective, having your bug report ignored or told you’re doing it wrong is frustrating too.

So I think reportbug is much nicer than having some github-esque web-based submission form. Could it be better? Sure. I think a mode to submit the reportbug report via HTTPS instead of email would make sense, since a lot of machines no longer have local email configured.

Where Debian Should Improve

I agree that there are areas where Debian should improve.

Michael rightly identifies the “strong maintainer” concept as a source of trouble. I agree. Though we’ve been making slow progress over time with things like low-threshold NMU and maintainer teams, the core assumption that a maintainer has a lot of power over particular packages is one that needs to be thrown out.

Michael, and commentators on HN, both identify things that boil down to documentation problems. I have heard so many times that it’s terribly hard to package something up for Debian. That’s really not the case for most things; dh_make and similar tools will do the right thing for many packages, and all you have to do is add some package descriptions and such. I wrote a “concise guide” to packaging for my workplace that ran to only about 2 pages. But it is true that the documentation on debian.org doesn’t clearly offer this simple path, so people are put off and not aware of it. Then there were the comments about how hard it is to become a Debian developer, and how easy it is to submit a patch to NixOS or some such. The fact is, these are different things; one does not need to be a Debian Developer to contribute to Debian. A DD is effectively the same as a patch approver elsewhere; these are the people that can ultimately approve software for insertion into the OS, and you DO want an element of trust there. Debian could do more to offer concise guides for drive-by contributions and the building of packages that follow standard language community patterns, both of which can be done without much knowledge of packaging tools and inner workings of the project.

Finally, I have distanced myself from conversations in Debian for some time, due to lack of time to participate in what I would call excessive bikeshedding. This is hardly unique to Debian, but I am glad to see the project putting more effort into expecting good behavior from conversations of late.


14 thoughts on “A (Partial) Defense of Debian

  1. anarcat says:

    A lot of the problems Michael identified boil down to Debian being an old project. It’s fair to identify those things, but it’s unfair when you compare them with workflows like GitHub or Golang, that have been there for barely a decade. We seen to have entered a culture where it’s “hip” to just trash talk about whatever came before and how all the new stuff is so much better. We just start from scratch every time and reinvent the wheel, thinking we’re so much better than everyone else that came before. What I find really hilarious is that there are many, MANY different such tracks going sometimes in opposite directions (e.g. Rust vs Javascript)…

    Anyways, I mostly wanted to say that I also see a problem with the Debian documentation. It’s hard work, documentation, and it’s not valued enough in Debian. Old tools here as well: I’m pretty sure no one really enjoys writing Docbook and WML anymore.. I wrote my own documentation as well because I didn’t want to fight with all that legacy as well, but unfortunately it means it lives outside Debian.org. But it does mean I don’t have to get into futile flamewars about git-buildpackage vs gitpkg or dgit or whatever: the tools I pick are the best for me and I can just provide a complete track, something which is dearly missing in Debian as well (which Michael also correctly identified).

    Anyways, all good suggestions, I wish there was a good way to do all of that…

    1. John Goerzen says:

      You make good points. Indeed, I once wrote a book using DocBook and will be happy to never do so again.

      Good point on tooling. There’s a lot of wheel reinvention that happens out there, for often marginal gain. There are some things we could improve, and others that frankly are old but I don’t really see a problem with.

      1. Martin says:

        OT, but anyway: I wrote user manuals (200..250 pages) using DocBook and so far, I’ll choose DocBook again. With Emacs nxml-mode, it’s much cleaner than editing LaTeX, and one has neither to type the long element names nor remember them. po4a allows translations. dblatex produces very nice output. It is very easy to filter documents e.g. by release, by architecture, etc. to produce special versions etc. I know, I should learn to love the ReST, markdown, asciidoc and all of their dialects and strange syntactic somersaults now…

  2. Thorsten says:

    Just my two cents about the Debian BTS:

    I have now been using Debian for more than 10 years almost exclusively on both personal servers and desktops. And I have not once reported a bug to the BTS or participated in the discussion of a bug. One big reason for that (but not the only one) is that I do not want my email address published on the web, to the maintainers of the package, etc. And creating and using a different “fake” email account just for reporting bugs to Debian is not worth it IMHO.

    Note that I do not claim that this is a common problem or that there’s even a single other person on the planet who has this problem.

    1. John Goerzen says:

      That is a valid point and something that should be addressed. I guess my email address has been out on the net since the 90s due to participation in mailing lists, Usenet, etc., that it doesn’t much matter to me anymore.

    2. Adam says:

      How is this higher friction than having to create a dummy account on a website to contribute (assuming you don’t want to your real website account associated, equivalent to not wanting to use your email address)?

    3. John says:

      I want to report bugs.

      Without an email, I can bookmark the page, and return to respond. Something similar to how anonymous Quora answers work.

      There’s also a chance that I am mistaken in reporting it. This among a few other reasons has led to me not reporting any bugs.

  3. Josh says:

    For projects on github, there are tools, so that you can just write “git hub clone https://github.com/foo/bar“, edit, commit,`git hub pull new”, and never leave the command line.

    Also, for your Debian contribution example, you started with “hack”, but where’s the step where you obtain the repository? You mentioned cloning in the github case, but what about the Debian case?

    1. John Goerzen says:

      You are correct; I’ve used “hub” and it is something, though it isn’t quite the same. Both scenarios assume I already had source from somewhere before I decided to patch it; this is usually the case for me (“hmm, why is this thing doing what it’s doing? Let me read the source.” … “Ah, I need to make a patch.”) It could be apt-get source, or some other thing (git checkout, etc.) There is no need in that case to make a fork and such.

  4. Martin says:

    Random idea: Let’s reverse “low treshold NMU logic, i.e.make low the default and allow package maintainers to list them on https://wiki.debian.org/HighThresholdNmu (TBD).

  5. James says:

    Many good points, especially on tooling. I’ve always been a Debian fan so this was an interesting read for me after all these years.

    Take care and stay safe
    James

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.