Monthly Archives: April 2016

Count me as a systemd convert

Back in 2014, I wrote about some negative first impressions of systemd. I also had a plea to debian-project to end all the flaming, pointing out that “jessie will still boot”, noting that my preference was for sysvinit but things are what they are and it wasn’t that big of a deal.

Although I still have serious misgivings about the systemd upstream’s attitude, I’ve got to say I find the system rather refreshing and useful in practice.

Here’s an example. I was debugging the boot on a server recently. It mounts a bunch of NFS filesystems and runs a third-party daemon that is started from an old-style /etc/init.d script.

We had a situation where the NFS filesystems the daemon required didn’t mount on boot. The daemon then was started, and unfortunately it basically does a mkdir -p on startup. So it started running and processing requests with negative results.

So there were two questions: why did the NFS filesystems fail to start, and how could we make sure the daemon wouldn’t start without them mounted? For the first, journalctl -xb was immensely helpful. It logged the status of each individual mount, and it turned out that it looked like a modprobe or kernel race condition when a bunch of NFS mounts were kicked off in parallel and all tried to load the nfsv4 module at the same time. That was easy enough to work around by adding nfsv4 to /etc/modules. Now for the other question: refusing to start the daemon if the filesystems weren’t there.

With systemd, this was actually trivial. I created /etc/systemd/system/mydaemon.service.requires (I’ll call the service “mydaemon” here), and in it I created a symlink to /lib/systemd/system/remote-fs.target. Then systemctl daemon-reload, and boom, done. systemctl list-dependencies mydaemon will even show the the dependency tree, color-coded status of each item on it, and will actually show every single filesystem that remote-fs requires and the status of it in one command. Super handy.

In a non-systemd environment, I’d probably be modifying the init script and doing a bunch of manual scripting to check the filesystems. Here, one symlink and one command did it, and I get tools to inspect the status of the mydaemon prerequisites for free.

I’ve got to say, as someone that has occasionally had to troubleshoot boot ordering and update-rc.d symlink hell, troubleshooting this stuff in systemd is considerably easier and the toolset is more powerful. Yes, it has its set of poorly-documented complexity, but then so did sysvinit.

I never thought the “world is falling” folks were right, but by now I can be counted among those that feels like systemd has matured to the point where it truly is superior to sysvinit. Yes, in 2014 it had some bugs, but by here in 2016 it looks pretty darn good and I feel like Debian’s decision has been validated through my actual experience with it.

A Year of Flight

“Dad-o, I’m so glad you’re a pilot!”

My 9-year-old son Jacob has been saying that, always with a big hug and his fond nickname for me (“dad-o”). It has now been a year since the first time I sat in the pilot’s seat of a plane, taking my first step towards exploring the world from the sky. And now, one year after I first sat in the pilot’s seat of an airborne plane, it’s prompted me to think back to my own memories.

vlcsnap-2015-07-24-20h35m16s96_1

Flying over the airport at Moundridge, KS

Memories

Back when I was a child, maybe about the age my children are now, I’d be outside in the evening and see this orange plane flying overhead. Our neighbor Don had a small ultralight plane and a grass landing strip next to his house. I remember longing to be up in the sky with Don, exploring the world from up there. At that age, I didn’t know all the details of why that wouldn’t work — I just knew I wanted to ride in it.

It wasn’t until I was about 11 that I flew for the first time. I still remember that TWA flight with my grandma, taking off early in the morning and flying just a little ways above the puffy clouds lit up all yellow and orange by the sunrise. Even 25 years later, that memory still holds as one of the most beautiful scenes I have ever seen.

Exploring

I have always been an explorer.

When I go past something interesting, I love to go see what it looks like inside. I enjoy driving around Kansas with Laura, finding hidden waterfalls, old county courthouses, ghost towns, beautiful old churches, even small-town restaurants. I explore things around me, too — once taking apart a lawnmower engine as a child, nowadays building HF antennas in my treetops or writing code for Linux. If there is little to learn about something, it becomes less interesting to me.

I see this starting to build in my children, too. Since before they could walk, if we were waiting for something in a large building, we’d “go exploring.”

IMG_7231

A patch of rain over Hillsboro, KS

The New World

A pilot once told me, “Nobody can become a pilot without it changing the way they see the world — and then, changing their life.”

I doubted that. But it was true. One of the most poetic sights I know is flying a couple thousand feet above an interstate highway at night, following it to my destination. All those red and white lights, those metal capsules of thousands of lives and thousands of stories, stretching out as far as the eye can see in either direction.

IMG_7099

Kansas sunset from the plane

When you’re in a plane, that small town nowhere near a freeway that always seemed so far away suddenly is only a 15-minute flight away, not even enough time to climb up to a high cruise altitude. Two minutes after takeoff, any number of cities that are an hour’s drive away are visible simultaneously, their unique features already recognizable: a grain elevator, oil refinery, college campus, lake, whatever.

And all the houses you fly over — each with people in them. Some pretty similar to you, some apparently not. But pretty soon you realize that we all are humans, and we aren’t all that different. You can’t tell a liberal from a conservative from the sky, nor a person’s race or religion, nor even see the border between states. Towns and cities are often nameless from the sky, unless you’re really low; only your navigation will tell you where you are.

I’ve had the privilege to fly to small out-of-the-way airports, the kind that have a car that pilots can use for free to go into town and get lunch, and leave the key out for them. There I’ve met many friendly people. I’ve also landed my little Cessna at a big commercial airport where I probably used only 1/10th of the runway, on a grass runway that was barely maintained at all. I’ve flown to towns I’d driven to or through many times, discovering the friendly folks at the small airport out of town. I’ve flown to parts of Kansas I’ve never been to before, discovered charming old downtowns and rolling hills, little bursts of rain and beautiful sunsets that seem to turn into a sea.

Smith Center, KS airport terminal

Parked at the Smith Center, KS airport terminal, about to meet some wonderful people

For a guy that loves exploring the nooks and crannies of the world that everyone else drives by on their way to a major destination, being a pilot has meant many soul-filling moments.

Hard Work

I knew becoming a pilot would be a lot of hard work, and thankfully I remembered stories like that when I finally concluded it would be worth it. I found that I had an aptitude for a lot of things that many find difficult about being a pilot: my experience with amateur radio made me a natural at talking to ATC, my fascination with maps and navigation meant I already knew how to read aviation sectional maps before I even started my training and knew how to process that information in the cockpit, my years as a system administrator and programmer trained me with a careful and methodical decision-making process. And, much to the surprise of my flight instructor, I couldn’t wait to begin the part of training about navigating using VORs (VHF radio beacons). I guess he, like many student pilots, had struggled with that, but I was fascinated by this pre-GPS technology (which I still routinely use in my flight planning, as a backup in case the GPS constellation or a GPS receiver fails). So that left the reflexes of flight, the “art” of it, as the parts I had to work on the hardest.

The exam with the FAA is not like getting your driver’s license. It’s a multi-stage and difficult process. So when the FAA Designated Pilot Examiner said “congratulations, pilot!” and later told my flight instructor that “you did a really good job with this one,” I felt a true sense of accomplishment.

IMG_20151021_193137

Some of my prep materials

Worth It

Passengers in a small plane can usually hear all the radio conversations going on. My family has heard me talking to air traffic control, to small and big planes. My 6-year-old son Oliver was playing yesterday, and I saw him pick up a plane and say this:

“Two-four-niner-golf requesting to land on runway one-seven…. Two-four-niner-golf back-taxi on one-seven… Two-four-niner-golf ready to takeoff on runway one-seven!”

That was a surprisingly accurate representation of some communication a pilot might have (right down to the made-up tailnumber with the spelling alphabet!)

20160408_203110

It just got more involved from there!

Jacob and Oliver love model train shows. I couldn’t take them to one near us, but there was one in Joplin, MO. So the day before Easter, while Laura was working on her Easter sermon, two excited boys and I (frankly also excited) climbed into a plane and flew to Joplin.

We had a great time at the train show, discovered a restaurant specializing in various kinds of hot dogs (of course they both wanted to eat there), played in a park, explored the city, and they enjoyed the free cookies at the general aviation terminal building while I traded tips on fun places to fly with other pilots.

When it comes right down to it, the smiles of the people I fly with are the most beautiful thing in the air.

IMG_20151205_183440

Jacob after his first father-son flight with me