Making a difference

Every day, ask yourself this question: What one thing can I do today that will make this democracy stronger and honor and support its institutions? It doesn’t have to be a big thing. And it probably won’t shake the Earth. The aggregation of them will shake the Earth.

– Benjamin Wittes

I have written some over the past year or two about the dangers facing the country. I have become increasingly alarmed about the state of it. And that Benjamin Wittes quote, along with the terrible tragedy, spurred me to action. Among other things, I did two things I never have done before:

I registered to protest on June 30.

I volunteered to do phone banking with SwingLeft.

And I changed my voter registration from independent to Republican.

No, I have not gone insane. The reason for the latter is that here in Kansas, the Democrats rarely field candidates for most offices. The real action happens in the Republican primary. So if I can vote in that primary, I can have a voice in keeping the crazy out of office. It’s not much, but it’s something.

Today we witnessed, hopefully, the first victory in our battle against the abusive practices happening to children at the southern border. Donald Trump caved, and in so doing, implicitly admitted the lies he and his administration have been telling about the situation. This only happened because enough people thought like Wittes: “I am small, but I can do SOMETHING.” When I called the three Washington offices of my senators and representatives — far-right Republicans all — it was apparent that I was by no means the first to give them an earful about this, and that they were changing their tone because of what they heard. Mind you, they hadn’t taken any ACTION yet, but the calls mattered. The reporting mattered. The attention mattered.

I am going to keep doing what little bit I can. I hope everyone else will too. Let us shake the Earth.

Memories, Father’s Day, and an 89-year-old plane

“Oh! I have slipped the surly bonds of Earth
And danced the skies on laughter-silvered wings;
Sunward I’ve climbed, and joined the tumbling mirth
of sun-split clouds, — and done a hundred things”

– John Gillespie Magee, Jr.

I clicked on the radio transmitter in my plane.

O’Neill Traffic, Bonanza xx departing to the south. And Trimotor, thanks for flight #1. We really enjoyed it.

And we had. Off to my left, a 1929 Ford Trimotor airliner was heading off into the distance, looking as if it were just hanging in the air, glinting in the morning sun, 1000 feet above the ground. Earlier that morning, my boys and I had been passengers in that very plane. But now we had taken off right after them, as they were taking another load of passengers up for a flight and we were flying back home. To my right was my 8-year-old, and my 11-year-old was in back, both watching out the windows. The radio clicked on, and the three of us heard the other pilot’s response:

Oh thank you. We’re glad you came!

A few seconds later, they were gone out of sight.

The experience of flying in an 89-year-old airliner is quite something. As with the time we rode on the Durango & Silverton railroad, it felt like stepping back into a time machine — into the early heyday of aviation.

Jacob and Oliver had been excited about this day a long time. We had tried to get a ride when it was on tour in Oklahoma, much closer, but one of them got sick on the drive that day and it didn’t work out. So Saturday morning, we took the 1.5-hour-flight up to northern Nebraska. We’d heard they’d have a pancake breakfast fundraiser, and the boys were even more excited. They asked to set the alarm early, so we’d have no risk of missing out on airport pancakes.

Jacob took this photo of the sunrise at the airport while I was doing my preflight checks:

IMG_1574

Here’s one of the beautiful views we got as we flew north to meet the Trimotor.

IMG_20180616_070810_v1

It was quite something to share a ramp with that historic machine. Here’s a photo of our plane not far from the Trimotor.

IMG_20180616_082051

After we got there, we checked in for the flight, had a great pancake and sausage breakfast, and then into the Trimotor. The engines fired up with a most satisfying low rumble, and soon we were aloft — cruising along at 1000 feet, in that (by modern standards) noisy, slow, and beautiful machine. We explored the Nebraska countryside from the air before returning 20 minutes later. I asked the boys what they thought.

“AWESOME!” was the reply. And I agreed.

IMG_20180616_090828

Jacob and Oliver have long enjoyed pretending to be flight attendants when we fly somewhere. They want me to make airline-sounding announcements, so I’ll say something like, “This is your captain speaking. In a few moments, we’ll begin our descent into O’Neill. Flight attendants, prepare the cabin for arrival.” Then Jacob will say, “Please return your tray tables that you don’t have to their full upright and locked position, make sure your seat belt is tightly fastened, and your luggage is stowed. This is your last chance to visit the lavatory that we don’t have. We’ll be on the ground shortly.”

Awhile back, I loaded up some zip-lock bags with peanuts and found some particularly small bottles of pop. Since then, it’s become tradition on our longer flights for them to hand out bags of peanuts and small quantities of pop as we cruise along — “just like the airlines.” A little while back, I finally put a small fridge in the hangar so they get to choose a cold beverage right before we leave. (We don’t typically have such things around, so it’s a special treat.)

Last week, as I was thinking about Father’s Day, I told them how I remembered visiting my dad at work, and how he’d let me get a bottle of Squirt from the pop machine there (now somewhat rare). So when we were at the airport on Saturday, it brought me a smile to hear, “DAD! This pop machine has Squirt! Can we get a can? It’s only 75 cents!” “Sure – after our Trimotor flight.” “Great! Oh, thank you dad!”

I realized then I was passing a small but special memory on to another generation. I’ve written before of my childhood memories of my dad, and wondering what my children will remember of me. Martha isn’t old enough yet to remember her cackles of delight as we play peek-a-boo or the books we read at bedtime. Maybe Jacob and Oliver will remember our flights, or playing with mud, or researching dusty maps in a library, playing with radios, or any of the other things we do. Maybe all three of them will remember the cans of Squirt I’m about to stock that hangar fridge with.

But if they remember that I love them and enjoy doing things with them, they will have remembered the most important thing. And that is another special thing I got from my parents, and can pass on to another generation.

Syncing with a memory: a unique use of tar –listed-incremental

I have a Nextcloud instance that various things automatically upload photos to. These automatic folders sync to a directory on my desktop. I wanted to pull things out of that directory without deleting them, and only once. (My wife might move them out of the directory on her computer, and I might arrange them into targets on my end.)

In other words, I wanted to copy a file from a source to a destination, but remember what had been copied before so it only ever copies once.

rsync doesn’t quite do this. But it turns out that tar’s listed-incremental feature can do exactly that. Ordinarily, it would delete files that were deleted on the source. But if we make the tar file with the incremental option, but extract it without, it doesn’t try to delete anything at extract time.

Here’s my synconce script:

#!/bin/bash

set -e

if [ -z "$3" ]; then
    echo "Syntax: $0 snapshotfile sourcedir destdir"
    exit 5
fi

SNAPFILE="$(realpath "$1")"
SRCDIR="$2"
DESTDIR="$(realpath "$3")"

cd "$SRCDIR"
if [ -e "$SNAPFILE" ]; then
    cp "$SNAPFILE" "${SNAPFILE}.new"
fi
tar "--listed-incremental=${SNAPFILE}.new" -cpf - . | \
    tar -xf - -C "$DESTDIR"
mv "${SNAPFILE}.new" "${SNAPFILE}"

Just have the snapshotfile be outside both the sourcedir and destdir and you’re good to go!

Running Digikam inside Docker

After my recent complaint about AppImage, I thought I’d describe how I solved my problem. I needed a small patch to Digikam, which was already in Debian’s 5.9.0 package, and the thought of rebuilding the AppImage was… unpleasant.

I thought – why not just run it inside Buster in Docker? There are various sources on the Internet for X11 apps in Docker. It took a little twiddling to make it work, but I did.

My Dockerfile was pretty simple:

FROM debian:buster
MAINTAINER John Goerzen 

RUN apt-get update && \
    apt-get -yu dist-upgrade && \
    apt-get --install-recommends -y install firefox-esr digikam digikam-doc \
         ffmpegthumbs imagemagick minidlna hugin enblend enfuse minidlna pulseaudio \
         strace xterm less breeze && \
    apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN adduser --disabled-password --uid 1000 --gecos "John Goerzen" jgoerzen && \
    rm -r /home/jgoerzen/.[a-z]*
RUN rm /etc/machine-id
CMD /usr/bin/docker

RUN mkdir -p /nfs/personalmedia /run/user/1000 && chown -R jgoerzen:jgoerzen /nfs /run/user/1000

I basically create the container and my account in it.

Then this script starts up Digikam:

#!/bin/bash

set -e

# This will be unnecessary with docker 18.04 theoretically....  --privileged see
# https://stackoverflow.com/questions/48995826/which-capabilities-are-needed-for-statx-to-stop-giving-eperm
# and https://bugs.launchpad.net/ubuntu/+source/docker.io/+bug/1755250

docker run -ti \
       -v /tmp/.X11-unix:/tmp/.X11-unix -v "/run/user/1000/pulse:/run/user/1000/pulse" -v /etc/machine-id:/etc/machine-id \
       -v /etc/localtime:/etc/localtime \
       -v /dev/shm:/dev/shm -v /var/lib/dbus:/var/lib/dbus -v /var/run/dbus:/var/run/dbus -v /run/user/1000/bus:/run/user/1000/bus  \
       -v "$HOME:$HOME" -v "/nfs/personalmedia/Pictures:/nfs/personalmedia/Pictures" \
     -e DISPLAY="$DISPLAY" \
     -e XDG_RUNTIME_DIR="$XDG_RUNTIME_DIR" \
     -e DBUS_SESSION_BUS_ADDRESS="$DBUS_SESSION_BUS_ADDRESS" \
     -e LANG="$LANG" \
     --user "$USER" \
     --hostname=digikam \
     --name=digikam \
     --privileged \
     --rm \
     jgoerzen/digikam "$@"  /usr/bin/digikam

The goal here was not total security isolation; if it had been, then all the dbus mounting and $HOME mounting was a poor idea. But as an alternative to AppImage — well, it worked perfectly. I could even get security updates if I wanted.

Please stop making the library situation worse with attempts to fix it

I recently had a simple-sounding desire. I would like to run the latest stable version of Digikam. My desktop, however, runs Debian stable, which has 5.3.0, not 5.9.0.

This is not such a simple proposition.


$ ldd /usr/bin/digikam | wc -l
396

And many of those were required at versions that weren’t in stable.

I had long thought that AppImage was a rather bad idea, but I decided to give it a shot. I realized it was worse than I had thought.

The problems with AppImage

About a year ago, I wrote about the problems Docker security. I go into much more detail there, but the summary for AppImage is quite similar. How can I trust all the components in the (for instance) Digikam AppImage image are being kept secure? Are they using the latest libssl and libpng, to avoid security issues? How will I get notified of a security update? (There seems to be no mechanism for this right now.) An AppImage user that wants to be secure has to manually answer every one of those questions for every application. Ugh.

Nevertheless, the call of better facial detection beckoned, and I downloaded the Digikam AppImage and gave it a whirl. The darn thing actually fired up. But when it would play videos, there was no sound. Hmmmm.

I found errors like this:

Cannot access file ././/share/alsa/alsa.conf

Nasty. I spent quite some time trying to make ALSA work, before a bunch of experimentation showed that if I ran alsoft-conf on the host, and selected only the PulseAudio backend, then it would work. I reported this bug to Digikam.

Then I thought it was working — until I tried to upload some photos. It turns out that SSL support in Qt in the AppImage was broken, since it was trying to dlopen an incompatible version of libssl or libcrypto on the host. More details are in the bug I reported about this also.

These are just two examples. In the rather extensive Googling I did about these problems, I came across issue after issue people had with running Digikam in an AppImage. These issues are not limited to the ALSA and SSL issues I describe here. And they are not occurring due to some lack of skill on the part of Digikam developers.

Rather, they’re occurring because AppImage packaging for a complex package like this is hard. It’s hard because it’s based on a fiction — the fiction that it’s possible to make an AppImage container for a complex desktop application act exactly the same, when the host environment is not exactly the same. Does the host run PulseAudio or ALSA? Where are its libraries stored? How do you talk to dbus?

And it’s not for lack of trying. The scripts to build the Digikam appimage support runs to over 1000 lines of code in the AppImage directory, plus another 1300 lines of code (at least) in CMake files that handle much of the work, and another 3000 lines or so of patches to 3rd-party packages. That’s over 5000 lines of code! By contrast, the Debian packaging for the same version of Digikam, including Debian patches but excluding the changelog and copyright files, amounts to 517 lines. Of course, it is reusing OS packages for the dependencies that were already built, but this amounts to a lot simpler build.

Frankly I don’t believe that AppImage really lives up to its hype. Requiring reinventing a build system and making some dangerous concessions on security for something that doesn’t really work in the end — not good in my book.

The library problem

But of course, AppImage exists for a reason. That reason is that it’s a real pain to deal with so many levels of dependencies in software. Even if we were to compile from source like the old days, and even if it was even compatible with the versions of the dependencies in my OS, that’s still a lot of work. And if I have to build dependencies from source, then I’ve given up automated updates that way too.

There’s a lot of good that ELF has brought us, but I can’t help but think that it wasn’t really designed for a world in which a program links 396 libraries (plus dlopens a few more). Further, this world isn’t the corporate Unix world of the 80s; Open Source developers aren’t big on maintaining backwards compatibility (heck, both the KDE and Qt libraries under digikam have both been entirely rewritten in incompatible ways more than once!) The farther you get from libc, the less people seem to care about backwards compatibility. And really, who can blame volunteers? You want to work on new stuff, not supporting binaries from 5 years ago, right?

I don’t really know what the solution is here. Build-from-source approaches like FreeBSD and Gentoo have plenty of drawbacks too. Is there some grand solution I’m missing? Some effort to improve this situation without throwing out all the security benefits that individually-packaged libraries give us in distros like Debian?

Emacs #5: Documents and Presentations with org-mode

The Emacs series

This is fifth in a series on Emacs and org-mode.

This blog post was generated from an org-mode source and is available as: a blog page, slides (PDF format), and a PDF document.

1 About org-mode exporting

1.1 Background

org-mode isn't just an agenda-making program. It can also export to lots of formats: LaTeX, PDF, Beamer, iCalendar (agendas), HTML, Markdown, ODT, plain text, man pages, and more complicated formats such as a set of web pages.

This isn't just some afterthought either; it's a core part of the system and integrates very well.

One file can be source code, automatically-generated output, task list, documentation, and presentation, all at once.

Some use org-mode as their preferred markup format, even for things like LaTeX documents. The org-mode manual has an extensive section on exporting.

1.2 Getting started

From any org-mode document, just hit C-c C-e. From there will come up a menu, letting you choose various export formats and options. These are generally single-key options so it's easy to set and execute. For instance, to export a document to a PDF, use C-c C-e l p or for HTML export, C-c C-e h h.

There are lots of settings available for all of these export options; see the manual. It is, in fact, quite possible to use LaTeX-format equations in both LaTeX and HTML modes, to insert arbitrary preambles and settings for different modes, etc.

1.3 Add-on packages

ELPA containts many addition exporters for org-mode as well. Check there for details.

2 Beamer slides with org-mode

2.1 About Beamer

Beamer is a LaTeX environment for making presentations. Its features include:

  • Automated generating of structural elements in the presentation (see, for example, the Marburg theme). This provides a visual reference for the audience of where they are in the presentation.
  • Strong help for structuring the presentation
  • Themes
  • Full LaTeX available

2.2 Benefits of Beamer in org-mode

org-mode has a lot of benefits for working with Beamer. Among them:

  • org-mode's very easy and strong support for visualizing and changing the structure makes it very quick to reorganize your material.
  • Combined with org-babel, live source code (with syntax highlighting) and results can be embedded.
  • The syntax is often easier to work with.

I have completely replaced my usage of LibreOffice/Powerpoint/GoogleDocs with org-mode and beamer. It is, in fact, rather frustrating when I have to use one of those tools, as they are nowhere near as strong as org-mode for visualizing a presentation structure.

2.3 Headline Levels

org-mode's Beamer export will convert sections of your document (defined by headings) into slides. The question, of course, is: which sections? This is governed by the H export setting (org-export-headline-levels).

There are many ways to go, which suit people. I like to have my presentation like this:

#+OPTIONS: H:2
#+BEAMER_HEADER: \AtBeginSection{\frame{\sectionpage}}

This gives a standalone section slide for each major topic, to highlight major transitions, and then takes the level 2 (two asterisks) headings to set the slide. Many Beamer themes expect a third level of indirection, so you would set H:3 for them.

2.4 Themes and settings

You can configure many Beamer and LaTeX settings in your document by inserting lines at the top of your org file. This document, for instance, defines:

#+TITLE:  Documents and presentations with org-mode
#+AUTHOR: John Goerzen
#+BEAMER_HEADER: \institute{The Changelog}
#+PROPERTY: comments yes
#+PROPERTY: header-args :exports both :eval never-export
#+OPTIONS: H:2
#+BEAMER_THEME: CambridgeUS
#+BEAMER_COLOR_THEME: default

2.5 Advanced settings

I like to change some colors, bullet formatting, and the like. I round out my document with:

# We can't just +BEAMER_INNER_THEME: default because that picks the theme default.
# Override per https://tex.stackexchange.com/questions/11168/change-bullet-style-formatting-in-beamer
#+BEAMER_INNER_THEME: default
#+LaTeX_CLASS_OPTIONS: [aspectratio=169]
#+BEAMER_HEADER: \definecolor{links}{HTML}{0000A0}
#+BEAMER_HEADER: \hypersetup{colorlinks=,linkcolor=,urlcolor=links}
#+BEAMER_HEADER: \setbeamertemplate{itemize items}[default]
#+BEAMER_HEADER: \setbeamertemplate{enumerate items}[default]
#+BEAMER_HEADER: \setbeamertemplate{items}[default]
#+BEAMER_HEADER: \setbeamercolor*{local structure}{fg=darkred}
#+BEAMER_HEADER: \setbeamercolor{section in toc}{fg=darkred}
#+BEAMER_HEADER: \setlength{\parskip}{\smallskipamount}

Here, aspectratio=169 sets a 16:9 aspect ratio, and the remaining are standard LaTeX/Beamer configuration bits.

2.6 Shrink (to fit)

Sometimes you've got some really large code examples and you might prefer to just shrink the slide to fit.

Just type C-c C-x p, set the BEAMER_opt property to shrink=15.

(Or a larger value of shrink). The previous slide uses this here.

2.7 Result

Here's the end result:

screenshot1

3 Interactive Slides

3.1 Interactive Emacs Slideshows

With the org-tree-slide package, you can display your slideshow from right within Emacs. Just run M-x org-tree-slide-mode. Then, use C-> and C-< to move between slides.

You might find C-c C-x C-v (which is org-toggle-inline-images) helpful to cause the system to display embedded images.

3.2 HTML Slideshows

There are a lot of ways to export org-mode presentations to HTML, with various levels of JavaScript integration. See the non-beamer presentations section of the org-mode wiki for details.

4 Miscellaneous

4.1 Additional resources to accompany this post

4.2 Up next in my Emacs series…

mu4e for email!

Emacs #4: Automated emails to org-mode and org-mode syncing

This is fourth in a series on Emacs and org-mode.

Hopefully by now you’ve started to see how powerful and useful org-mode is. If you’re like me, you’re thinking:

“I’d really like to have this in sync across all my devices.”

and, perhaps:

“Can I forward emails into org-mode?”

This being Emacs, the answers, of course, are “Yes.”

Syncing

Since org-mode just uses text files, syncing is pretty easily accomplished using any number of tools. I use git with git-remote-gcrypt. Due to some limitations of git-remote-gcrypt, each machine tends to push to its own branch, and to master on command. Each machine merges from all the other branches and pushes the result to master after a merge. A cron job causes pushes to the machine’s branch to happen, and a bit of elisp coordinates it all — making sure to save buffers before a sync, refresh them from disk after, etc.

The code for this post is somewhat more extended, so I will be linking to it on github rather than posting inline.

I have a directory $HOME/org where all my org-stuff lives. In ~/org lives a Makefile that handles the syncing. It defines these targets:

  • push: adds, commits, and pushes to a branch named after the machine’s hostname
  • fetch: does a simple git fetch
  • sync: adds, commits, pulls remote changes, merges, and (assuming the merge was successful) pushes to the branch named after the machine’s hostname plus master

Now, in my user’s crontab, I have this:

*/15   *   *  *   *      make -C $HOME/org push fetch 2>&1 | logger --tag 'orgsync'

The accompanying elisp code defines a shortcut (C-c s) to cause a sync to occur. Thanks to the cronjob, as long as files were saved — even if I didn’t explicitly sync on the other boxen — they’ll be pulled in.

I have found this setup to work really well.

Emailing to org-mode

Before going down this path, one should ask the question: do you really need it? I use org-mode with mu4e, and the integration is excellent; any org task can link to an email by message-id, and this is ideal — it lets a person do things like make a reminder to reply to a message in a week.

However, org is not just about reminders. It’s also a knowledge base, authoring system, etc. And, not all of my mail clients use mu4e. (Note: things like MobileOrg exist for mobile devices). I don’t actually use this as much as I thought I would, but it has its uses and I thought I’d document it here too.

Now I didn’t want to just be able to accept plain text email. I wanted to be able to handle attachments, HTML mail, etc. This quickly starts to sound problematic — but with tools like ripmime and pandoc, it’s not too bad.

The first step is to set up some way to get mail into a specific folder. A plus-extension, special user, whatever. I then use a fetchmail configuration to pull it down and run my insorgmail script.

This script is where all the interesting bits happen. It starts with ripmime to process the message. HTML bits are converted from HTML to org format using pandoc. an org hierarchy is made to represent the structure of the email as best as possible. emails can get pretty complicated, with HTML and the rest, but I have found this does an acceptable job with my use cases.

Up next…

My last post on org-mode will talk about using it to write documents and prepare slides — a use for which I found myself surprisingly pleased with it, but which needed a bit of tweaking.

A Grand Adventure: Sailing the Springtime Sky

And forget not that the earth delights to feel your bare feet and the winds long to play with your hair.

– Khalil Gibran

To be a pilot of a small plane is to be a scientist, a mathematician, and a poet. We read charts, analyze weather reports and forecasts, calculating what the headwinds will do to our fuel situation.

But in the end, the wise ones let the earth speak to us through these charts, and go where it invites us — where the skies are smooth and blue.

And so it was last week that we did not go to California as planned, but instead to the mountains near Santa Fe, a canyon near Amarillo, and a remote museum in far southwest Kansas — all the while hearing the delighted exclamations of “wow!” from our children.

IMG_20180317_091955

As we sailed along up there in our flying machine, down below we saw the rugged, craggy mesas of New Mexico, here and there punctuated by a lake, a little town, or an isolated airport — each a friendly sight in its own way. Our boys read some books, and sometimes pressed their noses to the windows, while little Martha mostly slept and sometimes played or ate — she enjoys flying more than driving.

IMG_4629_bw

Mountains have a way of reminding us all that the earth is larger than we are. We drive around them, fly around them, and even on a pleasant day they make the air bumpy. But once down on the ground, Oliver got out of the plane, and looked at the mountains all around us. He couldn’t stop saying “Wow! Dad, wow! Amazing! Look at that!” Jacob was more excited that an American Airlines plane was taxiing by right where we had been a minute before.

IMG_4667_v1

The boys helped us plan our trip. They’re the ones that chose for us to head southwest, and the #1 thing on their Santa Fe agenda was riding the New Mexico Rail Runner Express. So, despite a strong and cold wind, ride it we did, all the way to Albuquerque for pizza, then back to Santa Fe. When they weren’t busy listening to the “meep meep” sound the doors make when they’re about to close, they were excitedly reading the timetable or taking in the world as it whizzed past their window.

IMG_4680_bw

Martha, too, took in the train — though she still enjoyed her chew toys. Those things out the window don’t fit into a mouth so easily.

IMG_4743

Up in the mountains, the Puye Cliff Dwellings brought home the history of the place. The stories of the peaceful people that lived there, told by their descendants, members of the Santa Clara pueblo. Our guide Elijah picked up a shard of pottery, many of which remain on the mesa. He explained why there were no intact pottery examples up there. When his ancestors were done with a pot, then would throw it on the ground, shattering it — to give it back with thanks to the earth from where it came. One gets the sense that these ancient peoples knew quite a few things that our modern societies have not yet learned.

IMG_4639

After a full day, a cool evening in our hotel was welcome. Our room had a wood-burning fireplace, burning the pinyon pine that gives Santa Fe such a distinct sweet smell in the winter. Jacob would gaze into the fireplace for quite awhile.

P1070297_v1

I have never seen a photo do justice to Palo Duro Canyon. As you drive along the desolate, high Texas prairie, complete with tumbleweeds, all of a sudden you go around a corner and the earth opens up. It’s the “painted canyon” for a reason, and even though we’d been there before, as we rounded that bend, I heard exclamations of “Dad, this is AMAZING” from the back seat once again.

The vastness of the place cannot be captured on a screen. How can one capture 60 miles of color, ridges and gorges stretching out into the horizon, in a few inches?

MVI_4844

The boys were excited, bubbly, and bouncy as we hiked along some trails on the canyon floor. They’d make up games to play, most of which involved teasing me in some way.

IMG_4841

Oliver particularly loved to tease me.

IMG_4812

Jacob insisted I take a picture with him and Martha.

IMG_4942

But even these excited, bubbly kids would stop to reflect sometimes. Sometimes Jacob would say, “Dad, I have GOT to take a picture of this!” And sometimes they would just stare, maybe even with a mouth agape. Children know beauty, too.

MVI_5075

The three siblings delight in each other, too. Oliver would play a version of peek-a-boo, saying “I’m alive! Horse pill!” (he’d say silly things, and whatever made Martha laugh he’d keep saying.)

map

All told, we traveled over a thousand miles by air, spending some 7 or 8 hours in the plane. Had we attempted to drive it, it would have been more than 30 hours. There’s something amazing about seeing so much of the world in such a short amount of time.

IMG_20180228_180503_bw

Perhaps this is why many pilots secretly give their plane a little pat after a long journey. After all the smiles, laughter, the memories, the feeling of being part of sky — if you pay attention, it truly is more poetry than physics.

And that is why, though it is always nice to return to our home, in my mind’s eye, the hangar door is always open.

Emacs #3: More on org-mode

This is third in a series on Emacs and org-mode.

Todo tracking and keywords

When using org-mode to track your TODOs, it can have multiple states. You can press C-c C-t for a quick shift between states. I have set this:

(setq org-todo-keywords '(
  (sequence "TODO(t!)" "NEXT(n!)" "STARTED(a!)" "WAIT(w@/!)" "OTHERS(o!)" "|" "DONE(d)" "CANCELLED(c)")
))

Here, I set up 5 states that are for a task that is not yet done: TODO, NEXT, STARTED, WAIT, and OTHERS. Each has a single-character shortcut (t, n, a, etc). The states after the pipe symbol are ones that are considered “done”. I have two: DONE (for things that I have done) and CANCELED (for things that I haven’t done, but for whatever reason, won’t).

The exclamation mark means to log the time when an item was changed to a state. I don’t add this to the done states because those are already logged anyhow. The @ sign means to prompt for a reason; so when switching to WAIT, org-mode will ask me why and add this to the note.

Here’s an example of an entry that has had some state changes:

** DONE This is a test
   CLOSED: [2018-03-02 Fri 03:05]
  
   - State "DONE"       from "WAIT"       [2018-03-02 Fri 03:05]
   - State "WAIT"       from "TODO"       [2018-03-02 Fri 03:05] \\
     waiting for pigs to fly
   - State "TODO"       from "NEXT"       [2018-03-02 Fri 03:05]
   - State "NEXT"       from "TODO"       [2018-03-02 Fri 03:05]

Here, the most recent items are on top.

Agenda mode, schedules, and deadlines

When you’re in a todo item, C-c C-s or C-c C-d can set a schedule or a deadline for it, respectively. These show up in agenda mode. The difference is in intent and presentation. A schedule is something that you expect to work on at around a time, while a deadline is something that is due at a specific time. By default, the agenda view will start warning you about deadline items in advance.

And while we’re at it, the agenda view will show you the items that you have coming up, offers a nice way to search for items based on plain text or tags, and handles bulk manipulation of items even across multiple files. I covered setting the files for agenda mode in part 2 of this series.

Tags

Of course org-mode has tags. You can quickly set them with C-c C-q.

You can set shortcuts for tags you might like to use often. Perhaps something like this:

  (setq org-tag-persistent-alist 
        '(("@phone" . ?p) 
          ("@computer" . ?c) 
          ("@websurfing" . ?w)
          ("@errands" . ?e)
          ("@outdoors" . ?o)
          ("MIT" . ?m)
          ("BIGROCK" . ?b)
          ("CONTACTS" . ?C)
          ("INBOX" . ?i)
          ))

You can also add tags to this list on a per-file basis, and also set tags for something on a per-file basis. I use that for my inbox.org and email.org files to set an INBOX tag. I can then review all items tagged INBOX from the agenda view each day, and the simple act of refiling them into other files will cause them to lost the INBOX tag.

Refiling

“Refiling” is moving things around, either within a file or elsewhere. It has completion using your headlines. C-c C-w does this. I like these settings:

(setq org-outline-path-complete-in-steps nil)         ; Refile in a single go
(setq org-refile-use-outline-path 'file)

Archiving

After awhile, you’ll get your files all cluttered with things that are done. org-mode has an archive feature to move things out of your main .org files and into some other files for future reference. If you have your org files in git or something, you may wish to delete these other files since you’d have things in history anyhow, but I find them handy for grepping and searching.

I periodically want to go through and archive everything in my files. Based on a stackoverflow discussion, I have this code:

(defun org-archive-done-tasks ()
  (interactive)
  (org-map-entries
   (lambda ()
     (org-archive-subtree)
     (setq org-map-continue-from (outline-previous-heading)))
   "/DONE" 'file)
  (org-map-entries
   (lambda ()
     (org-archive-subtree)
     (setq org-map-continue-from (outline-previous-heading)))
   "/CANCELLED" 'file)
)

This is based on a particular answer — see the comments there for some additional hints. Now you can run M-x org-archive-done-tasks and everything in the current file marked DONE or CANCELED will be pulled out into a different file.

Up next

I’ll wrap up org-mode with a discussion of automatically receiving emails into org, and syncing org between machines.

Resources to accompany this article

Emacs #2: Introducing org-mode

In my first post in my series on Emacs, I described returning to Emacs after over a decade of vim, and org-mode being the reason why.

I really am astounded at the usefulness, and simplicity, of org-mode. It is really a killer app.

So what exactly is org-mode?

I wrote yesterday:

It’s an information organization platform. Its website says “Your life in plain text: Org mode is for keeping notes, maintaining TODO lists, planning projects, and authoring documents with a fast and effective plain-text system.”

That’s true, but doesn’t quite capture it. org-mode is a toolkit for you to organize things. It has reasonable out-of-the-box defaults, but it’s designed throughout for you to customize.

To highlight a few things:

  • Maintaining TODO lists: items can be scattered across org-mode files, contain attachments, have tags, deadlines, schedules. There is a convenient “agenda” view to show you what needs to be done. Items can repeat.
  • Authoring documents: org-mode has special features for generating HTML, LaTeX, slides (with LaTeX beamer), and all sorts of other formats. It also supports direct evaluation of code in-buffer and literate programming in virtually any Emacs-supported language. If you want to bend your mind on this stuff, read this article on literate devops. The entire Worg website
    is made with org-mode.
  • Keeping notes: yep, it can do that too. With full-text search, cross-referencing by file (as a wiki), by UUID, and even into other systems (into mu4e by Message-ID, into ERC logs, etc, etc.)

Getting started

I highly recommend watching Carsten Dominik’s excellent Google Talk on org-mode. It is an excellent introduction.

org-mode is included with Emacs, but you’ll often want a more recent version. Debian users can apt-get install org-mode, or it comes with the Emacs packaging system; M-x package-install RET org-mode RET may do it for you.

Now, you’ll probably want to start with the org-mode compact guide’s introduction section, noting in particular to set the keybindings mentioned in the activation section.

A good tutorial…

I’ve linked to a number of excellent tutorials and introductory items; this post is not going to serve as a tutorial. There are two good videos linked at the end of this post, in particular.

Some of my configuration

I’ll document some of my configuration here, and go into a bit of what it does. This isn’t necessarily because you’ll want to copy all of this verbatim — but just to give you a bit of an idea of some of what can be configured, an idea of what to look up in the manual, and maybe a reference for “now how do I do that?”

First, I set up Emacs to work in UTF-8 by default.

(prefer-coding-system 'utf-8)
(set-language-environment "UTF-8")

org-mode can follow URLs. By default, it opens in Firefox, but I use Chromium.

(setq browse-url-browser-function 'browse-url-chromium)

I set the basic key bindings as documented in the Guide, plus configure the M-RET behavior.

(global-set-key "\C-cl" 'org-store-link)
(global-set-key "\C-ca" 'org-agenda)
(global-set-key "\C-cc" 'org-capture)
(global-set-key "\C-cb" 'org-iswitchb)

(setq org-M-RET-may-split-line nil)

Configuration: Capturing

I can press C-c c from anywhere in Emacs. It will capture something for me, and include a link back to whatever I was working on.

You can define capture templates to set how this will work. I am going to keep two journal files for general notes about meetings, phone calls, etc. One for personal, one for work items. If I press C-c c j, then it will capture a personal item. The %a in all of these includes the link to where I was (or a link I had stored with C-c l).

(setq org-default-notes-file "~/org/tasks.org")
(setq org-capture-templates
      '(
        ("t" "Todo" entry (file+headline "inbox.org" "Tasks")
         "* TODO %?\n  %i\n  %u\n  %a")
        ("n" "Note/Data" entry (file+headline "inbox.org" "Notes/Data")
         "* %?   \n  %i\n  %u\n  %a")
        ("j" "Journal" entry (file+datetree "~/org/journal.org")
         "* %?\nEntered on %U\n %i\n %a")
        ("J" "Work-Journal" entry (file+datetree "~/org/wjournal.org")
         "* %?\nEntered on %U\n %i\n %a")
        ))
(setq org-irc-link-to-logs t)

I like to link by UUIDs, which lets me move things between files without breaking locations. This helps generate UUIDs when I ask Org to store a link target for future insertion.


(require 'org-id)
(setq org-id-link-to-org-use-id 'create-if-interactive)

Configuration: agenda views

I like my week to start on a Sunday, and for org to note the time when I mark something as done.


(setq org-log-done 'time)
(setq org-agenda-start-on-weekday 0)

Configuration: files and refiling

Here I tell it what files to use in the agenda, and to add a few more to the plain text search. I like to keep a general inbox (from which I can move, or “refile”, content), and then separate tasks, journal, and knowledge base for personal and work items.

  (setq org-agenda-files (list "~/org/inbox.org"
                               "~/org/email.org"
                               "~/org/tasks.org"
                               "~/org/wtasks.org"
                               "~/org/journal.org"
                               "~/org/wjournal.org"
                               "~/org/kb.org"
                               "~/org/wkb.org"
  ))
  (setq org-agenda-text-search-extra-files
        (list "~/org/someday.org"
              "~/org/config.org"
  ))

  (setq org-refile-targets '((nil :maxlevel . 2)
                             (org-agenda-files :maxlevel . 2)
                             ("~/org/someday.org" :maxlevel . 2)
                             ("~/org/templates.org" :maxlevel . 2)
                             )
        )
(setq org-outline-path-complete-in-steps nil)         ; Refile in a single go
(setq org-refile-use-outline-path 'file)

Configuration: Appearance

I like a pretty screen. After you’ve gotten used to org a bit, you might try this.

(require 'org-bullets)
(add-hook 'org-mode-hook
          (lambda ()
            (org-bullets-mode t)))
(setq org-ellipsis "⤵")

Coming up next…

This hopefully showed a few things that org-mode can do. Coming up next, I’ll cover how to customize TODO keywords and tags, archiving old tasks, forwarding emails to org-mode, and using git to synchronize between machines.

You can also see a list of all articles in this series.

Resources to accompany this article