Married!

One week before the wedding, to Laura: “Mono won’t just clear up right away.”

One week before the wedding, to me: “That’s going to need stitches.”

Yes, not long before the wedding, Laura had come down with mononucleosis and I had cut into my finger with a very sharp knife while cutting bread requiring a trip to the emergency room to get stitches. Two days before we got married, instead of moving furniture, I was getting stitches out of my finger.

It wasn’t the kind of week we had planned.

But it was the happiest, most amazing occasion I could have ever imagined.

As I wrote last month, I am richly blessed indeed.

Our wedding was three days after Christmas. The church was still decorated for Christmas, with the tree in on corner, glittering stars suspended in mid-air on cables from the walls, wreaths and candles in the windows, and it was a joy-filled day.

Before the ceremony, we took pictures — the only part of the day Jacob and Oliver weren’t thrilled with. Nevertheless, we got some fun ones.

Laura and I seem to know quite a few pastors between us – and not just because Laura is a pastor. My brother officiated with the wedding vows, his wife with scripture and a prayer of blessing, and the church’s pastor gave the message.

Laura and I wrote in our wedding program, “Music has long been a thread running through both our lives. We have enjoyed singing together, playing piano and pennywhistle duets, attending concerts, and even exploring old hymnals. Music is also one of the best ways to have a conversation – even a conversation with God.” We wrote a page in the program about each of the hymns that were a part of the wedding, and why we picked them. The combined church choirs of my home church and Laura’s church sang John Rutter’s beautiful arrangement of For the Beauty of the Earth (click here to listen to a different choir). Hearing “For the beauty of each hour”, “For the joy of human love”, and “Lord of all, to thee we raise this our joyful hymn of praise” was perfect for the day.

It was with such great happiness that we walked out of the sanctuary, a married couple, to the sound of the congregation singing Joy to the World!

Jacob and Oliver were so very excited on our wedding day. They happily explored the church while waiting for things to happen. We had them help us light our unity candle, and they were pleased with that. Jacob loved his suit, which made him look just like me. And they were, of course, delighted with the cake and in the middle of it all.

For our honeymoon, we managed to get two weeks of vacation, and spent about half of it at home. We had looked at various options for retreats in the country, but eventually concluded that our house is a retreat in the country, so might as well enjoy it at home.

We also went to the Palo Duro Canyon area near Amarillo in the Texas panhandle, staying in a small B&B in Canyon, TX. Palo Duro is the second-largest canyon in North America, and quite colorful year-round. What a beautiful place to go for our honeymoon! By the time we got there, Laura was getting past mono, and we went for hikes in the canyon on two different days — hiking a total of 10 miles, including a hike up the side of the canyon.

After we got back home, on the last weekday of our honeymoon, we went back to the Flint Hills of Kansas, to some of the same places we had spent our third date. We climbed the windy staircase at the Chase County Courthouse, the oldest courthouse still in use in Kansas.

And peered out its famous oval window.

We found the last remnant of the old ghost town of Elk, ate at the same restaurant we had that day. It brought back wonderful memories, and it was a good day in itself. Because even though it was a gold, drizzly, overcast day in January, this time, we were married.

And this year, Thanksgiving is all year.


Richly Blessed

“It’s wedding week! Wedding week! Wedding week! Wedding week! Oh, also Christmas. Oh dad, it’s wedding week! I can’t believe it! It’s finally here! Wedding week!” – Jacob, age 7, Sunday

“Oh dad, this is the best Christmas EVER!” – Jacob, Wednesday

“Dad, is the wedding TODAY?” – Oliver, age 4, every morning this week

This has certainly been a Christmas like no other. I have never known something to upstage Christmas for Jacob, but apparently a wedding can!

Laura and I got to celebrate our first Christmas together this year — together, of course, with the boys. We enjoyed a wonderful day in the middle of a busy week, filled with play, family togetherness, warmth, and happiness. At one point, while I was helping the boys with their new model train components, Laura was enjoying playing Christmas tunes on the piano. Every time she’d reach the end, Jacob paused, and said, “That was awesome!”, beating me to it.

That’s a few days before Christmas — Jacob and Oliver demanding snow ice cream, and of course who am I to refuse?

Cousins opening presents

After his school Christmas program, Jacob has enjoyed singing. Here he is after the Christmas Eve program, where he excitedly ran up into the choir loft, picked up a hymnal, and pretended to sing.

And, of course, opening of presents at home.

Sometimes I think about how I didn’t know life could get this good. Soon Laura and I will be married, and it will be even better. Truly we have been richly blessed.

Delicious Holiday Recipes

I’ve come up with some new favorites this season. The boys and Laura were around for all three, and I am happy to report there were many kitchen smiles over these!

From-Scratch Hot Chocolate

There’s something about hot chocolate made from scratch, with chocolate melted into milk, instead of a powder stirred in. It takes quite a bit more time, and probably has more calories, but it is quite delicious.

The key to a delicious result where milk is concerned is to take things slow and keep stirring. You don’t want the chocolate to scorch at the bottom of the pan. Heating up the milk before the chocolate should help things mix in more easily as well.

  • Begin with 3 cups milk and 1 cup heavy whipping cream. Heat slowly over moderate to low heat, stirring periodically. Once you see bubbles start to form around the edges, it is plenty hot (or even a bit more hot than it needs to be).
  • Add one cup of semisweet chocolate chips, 1 teaspoon sugar, and 1/2 tsp vanilla extract.
  • Stir constantly until all the chocolate is melted and well mixed. There will still be some small bits of chocolate within, but if it is all done slowly like this, the chocolate should be pretty well melted.

The basis for this recipe was here, and it called for 2 cups milk and 2 cups half-and-half. I trust my heavy whipping cream was fine! <grin> There are also some other variations on that site.

This nearly made my little cast iron kettle overflow, so next time I made a 3/4 recipe.

Hot Spiced Cider

We put up a Christmas tree yesterday, so I thought hot spiced cider would be perfect for the occasion. I went searching for recipes, and many of them called for cloves (which have to be sifted out later or put in a spice bag). I wasn’t going to have time to delay two boys from setting up a Christmas tree long enough for that, so I found this basic recipe to work well. However, I, as usual, made some modifications ;-)

  • Warm 4 cups apple cider (not juice, as the recipe suggests) in a pot.
  • Add 1/2 tsp cinnamon
  • Add 1/4 tsp nutmeg or allspice (I used allspice because I was mysteriously out of nutmeg, but will probably use nutmeg next time)
  • Add 1 tbsp brown sugar
  • Stir constantly until sufficiently dissolved. Pour immediately before drinking, as the contents will tend to separate.

Mmmmm…. yum….

Turkey or Chicken Noodle Soup

The annual “what to do with all that leftover turkey” quest strikes again. I like chicken noodle soup, so why not a turkey noodle soup done the same way?

Here’s what I used, roughly, in my large 6-quart cast iron cooking pot (aka “Dutch oven”):

  • 9 cups chicken/turkey broth. Your own if you have it, or the canned variety works. Or make your own with boullion if you have it.
  • 2 chopped yellow onions. (I added half a chopped red onion as well because I had it sitting around. Nobody complained, but 2.5 onions was a little much.)
  • 4 tsp fresh basil or 1 tsp dried basil
  • 4 tsp fresh oregano or 1 tsp dried oregano
  • 1/2 tsp pepper
  • 1/2 tsp salt
  • 1 tsp beef bouillon
  • 2 bay leaves
  • 20 oz frozen mixed vegetables (I’d probably add more than that next year; this wasn’t quite enough)
  • Plenty of wide egg noodles. The recipe I used called for 1 cup, which was laughably inadequate. I just dumped until it looked right, and then the package was almost empty so I dumped the rest in too.
  • 4 cups cooked turkey or chicken, cubed (a kitchen scissors makes quick work of that)
  • Two 14.5-oz cans diced tomatoes (do not drain)

Start with the broth, onion, basil, oregano, pepper, and bay leaf. Heat up the mixture and add the vegetables. Bring it to boiling, then add the uncooked noodles. Return to boiling, then reduce heat, cover, and simmer for 8 minutes. Add the turkey or chicken and diced tomatoes, and simmer until hot enough to serve.

The nice thing about soups is that they freeze well and make great winter leftovers. This recipe makes quite a lot of soup; you may wish to halve it.

This recipe was adapted from one in a Better Homes & Gardens cookbook.

Results with btrfs and zfs

The recent news that openSUSE considers btrfs safe for users prompted me to consider using it. And indeed I did. I was already familiar with zfs, so considered this a good opportunity to experiment with btrfs.

btrfs makes an intriguing filesystem for all sorts of workloads. The benefits of btrfs and zfs are well-documented elsewhere. There are a number of features btrfs has that zfs lacks. For instance:

  • The ability to shrink a device that’s a member of a filesystem/pool
  • The ability to remove a device from a filesystem/pool entirely, assuming enough free space exists elsewhere for its data to be moved over.
  • Asynchronous deduplication that imposes neither a synchronous performance hit nor a heavy RAM burden
  • Copy-on-write copies down to the individual file level with cp --reflink
  • Live conversion of data between different profiles (single, dup, RAID0, RAID1, etc)
  • Live conversion between on-the-fly compression methods, including none at all
  • Numerous SSD optimizations, including alignment and both synchronous and asynchronous TRIM options
  • Proper integration with the VM subsystem
  • Proper support across the many Linux architectures, including 32-bit ones (zfs is currently only flagged stable on amd64)
  • Does not require excessive amounts of RAM

The feature set of ZFS that btrfs lacks is well-documented elsewhere, but there are a few odd btrfs missteps:

  • There is no way to see how much space subvolume/filesystem is using without turning on quotas. Even then, it is cumbersome and not reported with df like it should be.
  • When a maxmium size for a subvolume is set via a quota, it is not reported via df; applications have no idea when they are about to hit the maximum size of a filesystem.

btrfs would be fine if it worked reliably. I should say at the outset that I have never lost any data due to it, but it has caused enough kernel panics that I’ve lost count. I several times had a file that produced a panic when I tried to delete it, several times when it took more than 12 hours to unmount a btrfs filesystem, behaviors where hardlink-heavy workloads take days longer to complete than on zfs or ext4, and that’s just the ones I wrote about. I tried to use btrfs balance to change the metadata allocation on the filesystem, and never did get it to complete; it seemed to go into an endless I/O pattern after the first 1GB of metadata and never got past that. I didn’t bother trying the live migration of data from one disk to another on this filesystem.

I wanted btrfs to work. I really, really did. But I just can’t see it working. I tried it on my laptop, but had to turn of CoW on my virtual machine’s disk because of the rm bug. I tried it on my backup devices, but it was unusable there due to being so slow. (Also, the hardlink behavior is broken by default and requires btrfstune -r. Yipe.)

At this point, I don’t think it is really all that worth bothering with. I think the SuSE decision is misguided and ill-informed. btrfs will be an awesome filesystem. I am quite sure it will, and will in time probably displace zfs as the most advanced filesystem out there. But that time is not yet here.

In the meantime, I’m going to build a Debian Live Rescue CD with zfsonlinux on it. Because I don’t ever set up a system I can’t repair.

Why are we still backing up to hardlink farms?

A person can find all sorts of implementations of backups using hardlink trees to save space for incrementals. Some of them are fairly rudimentary, using rsync --link-dest. Others, like BackupPC, are more sophisticated, doing file-level dedup to a storage pool indexed by a hash.

While these are fairly space-efficient, they are really inefficient in other ways, because they create tons of directory entries. It would not be surprising to find millions of directory entries consumed very quickly. And while any given backup set can be deleted without impact on the others, the act of doing so can be very time-intensive, since often a full directory tree is populated with every day’s backup.

Much better is possible on modern filesystems. ZFS has been around for quite awhile now, and is stable on Solaris, FreeBSD and derivatives, and Linux. btrfs is also being used for real workloads and is considered stable on Linux.

Both have cheap copy-on-write snapshot operations that would work well with a simple rsync --inplace to achieve the same effect ad hardlink farms, but without all the performance penalties. When creating and destroying snapshots is a virtually instantaneous operation, and the snapshots work at a file block level instead of an entire file level, and preserve changing permissions and such as well (which rsync --link-dest can have issues with), why are we not using it more?

BackupPC has a very nice scheduler, a helpful web interface, and a backend that doesn’t have a mode to take advantage of these more modern filesystems. The only tool I see like this is dirvish, which someone made patches for btrfs snapshots three years ago that never, as far as I can tell, got integrated.

A lot of folks are rolling a homegrown solution involving rsync and snapshots. Some are using zfs send / btrfs send, but those mechanisms require the same kind of FS on the machine being backed up as on the destination, and do not permit excluding files from the backup set.

Is this an area that needs work, or am I overlooking something?

Incidentally, hats off to liw’s obnam. It doesn’t exactly do this, but sort of implements its own filesystem with CoW semantics.

Two Kittens

Almost every time he got off the bus for the past month and a half, Jacob started his afternoon in the same way. Before toys, before his trains and his toy bus, before anything indoors, he went for our cats. Here he is, cradling his favorite, Tigger:

Laura and I both grew up around cats. We had been talking about kittens, and shortly after we got engaged, one of my relatives offered us some free kittens. We went to his place one evening and selected two of them – one calico and one tiger-colored. Since what is now my place will soon be our place, they came to live with me. Our cats were one of the first things we did to prepare for our lives together.

Oliver wanted to name them some rather impractical sentence-long names (“The Cat Who Always Likes To Run”), so Laura and I suggested some names from one of their favorite books: Tigger and Roo. They both liked the names, but Oliver thought they should be called “Tigger the Digger” and “Roo the Runner”. Never mind that they were just 6 weeks old at the time, and not really old enough to either dig or run. Here’s Oliver with Roo, the day after the kittens arrived here.

I have always had outside cats, both because I’m allergic to cats so I need them to be outside, and because they sometimes literally quiver with joy of being outdoors. Tigger and Roo often chased insects, wrestled with each other, ran up (and slowly came back down) trees, and just loved the outside. Sometimes, I have taken my laptop and wireless headset and work from the back porch. The kittens climb up my jeans, inspect the laptop, and once Roo even fell asleep on my lap at one of those times.

Jacob has been particularly attached to Tigger, calling him “my very best friend.” When Jacob picks him up after school, Tigger often purrs while cradled in Jacob’s arms, and Jacob comments that “Tigger loves me. Oh dad, he knows I am his friend!”

The kittens have been growing, and becoming more and more comfortable with their home in the country. Whenever I go outside, it isn’t long before there are two energetic kittens near my feet, running back and forth, sometimes being very difficult to avoid stepping on. I call and I see little heads looking at me, from up in a tree, or peeking out from the grain elevator door, or from under the grill. They stare for just a second, and then start running, sometimes comically crashing into something in their haste.

Yesterday when I went to give them food, I called and no cats came. I was concerned, and walked around the yard, but at some point either they come or they don’t.

Yesterday afternoon, just after the bus dropped off Jacob, I discovered Tigger on the ground, motionless. Once Jacob was in the house, I went to investigate, and found Tigger was dead. As I was moving his body, I saw Roo was dead, too. Both apparently from some sort of sudden physical injury — a bit mysterious, because neither of them were at a place where they had ever gone before. While all this was happening, I had to also think about how I was going to tell the boys about this.

I tried to minimize what he could see, Jacob had caught an unavoidable glimpse of Tigger as we were walking back from the bus, but didn’t know exactly what had happened. He waited in the house, and when I came back, asked me if Tigger was dead. I said he was. Jacob started crying, saying, “Oh Dad, I am so sad”, and reached up for a hug. I picked him up and held him, then sat down on the couch and let him curl up on my lap. I could quite honestly let him know he wasn’t alone, telling him I am sad, too.

Oliver arrived not long after, and he too was sad, though not as much as Jacob. Both boys pretty soon wanted to see them. I decided this was important for them for closure, and to understand, so while they waited in the house, I went back out to arrange the kittens to hide their faces, the part that looks most unnatural after they die. The boys and I walked out to where I put them, then I carried both of them the last few feet. We stood a little ways back — close enough to see who was there, far enough to not get too much detail — and they were both sniffling. I tried to put voice to the occasion, saying, “Goodbye, Tigger and Roo. We love you.” Oliver asked if they could hear us. I said “No, but I told them what I felt like anyway.” Jacob, through tears, said, “Dad, maybe they are in heaven now.”

We went back inside. Jacob said, “Oh dad, I am so sad. This is the saddest day of my life. My heart is breaking.” Hearing a 7-year-old say that isn’t exactly easy for a dad. Pretty soon he was thinking of sort of comfort activities to do, saying “I think I would feel better if we did…” So they decided to watch a favorite TV program. Jacob asked if Laura knew yet, and when I said no, he got his take-charge voice and said, “Dad, you will start the TV show for us. While we are watching, you will send Laura an email to tell her about Tigger and Roo. OK?” What could I say, it wasn’t a bad idea.

Pretty soon both boys were talking and laughing. It was Big Truck Night last night, at a town about half an hour away. It’s an annual event we were already planning to attend, where all sorts of Big Trucks – firetrucks, school bus, combine, bucket truck, cement truck, etc – show up and are open for kids to climb in and explore. It’s always a highlight for them. They played and sang happily as we drove, excitedly opened and closed the big door on the school bus and yelled “All Aboard!” from the top of the combine. We ate dinner, and drove back home. When we got home, Jacob mentioned the cats again, in a sort of matter-of-fact way, and also wanted to make sure he knew Laura had got the message.

A person never wakes up expecting to have to dump a bowl of un-eaten cat food, or to give an impromptu cat funeral for little boys. As it was happening, I wished they hadn’t been around right then. But in retrospect, I am glad they were. They had been part of life for those kittens, and it is only right that they could be included in being part of death. They got visual closure this way, and will never wonder if the cats are coming back someday. They had a chance to say goodbye.

Here is how I remember the kittens.


Earthrise

Today I link you to a video narrated by the legendary Carl Sagan – The Frontier is Everywhere.

Partial quotes:

We were hunters and foragers. The frontier was everywhere. We were bounded only by the earth, and the ocean, and the sky. The open road still softly calls. Our little terraqueous globe as the madhouse of those hundred thousand millions of worlds. We, who cannot even put our own planetary home in order, riven with rivalries and hatreds; are we to venture out into space?

By the time we are ready to settle even the nearest other planetary systems, we will have changed… For all our failings, despite our limitations and fallibilities, we humans are capable of greatness… How far will our nomadic species have wandered, by the end of the next century, and the next millennium?

Our remote descendants, safely arrayed on many worlds through the solar system, and beyond, will be unified, by their common heritage, by their regard for their home planet, and by the knowledge that, whatever other life may be, the only humans in all the universe, come from Earth.

They will gaze up and strain to find the blue dot in their skies. They will marvel at how vulnerable the repository of all our potential once was, how perilous our infancy, how humble our beginnings, how many rivers we had to cross, before we found our way.

Can you imagine seeing Earthrise?

Earthrise, 1968 December 24

Incredible.

Engaged!

Today I have the delightful chance to write about a deep and wonderful joy.

Yes, Laura and I are engaged!

There are no words adequate for this kind of occasion, but there is a picture that gets close:

I never imagined a person could find a friend so wonderful, a person that enjoys so much in common, someone that understands me and that I understand so well. And yet, here I am, engaged to that friend. “Amazing” only begins to describe the feeling.

One of the first things Laura and I talked about was a hymn I was tinkering with, typesetting with GNU LilyPond. That hymn ends like this:

“Since Love is Lord of heav’n and earth, how can I keep from singing?”

It is a wonderful thought, and very true. Even literally true; I often find myself singing, humming, or playing the penny whistle during my day.

One of my good friends once told me, “I am completely sure that your happiest days lie ahead of you.” And he was right. I have already experienced them. This is a wonderful time, and every day brings plentiful reasons to be thankful. And as Laura and I prepare for a life together, I know that it is true not only that my friend was right, but that he is right — my happiest days are yet to come, and our happiest days are yet to come, too.

I have been blessed in many ways, and feel like the luckiest man alive.

To be loved, and to love, a person so wonderful is truly a remarkable gift.

A 4-year-old’s sudden interest in German monorail

Today, Oliver and Jacob heard about monorails. I showed them a wikipedia article about monorails, and it had a picture of a monorail in Germany.

Oliver, age 4: I have been on that monorail!

Me: Sorry, Oliver, but that monorail is in Germany. You have never been to Germany.

Oliver: But I HAVE been to Germany!

Me: No, you have never been to Germany.

Oliver: Dad, I HAVE been to Germany. I love Germany!

Jacob, taking an interest: Dad, can we go to Germany sometime?

Me: Yes, we probably could sometime.

Jacob: Great! Then we could ride a Deutsche Bahn train!

At this point, we had a brief discussion about the fact that we can’t take a train from the United States to Germany, but we can fly there and then take trains.

Oliver: Dad, we should go the the airport and tell them to take us to Germany right now!

Development Freedom in New Platforms

I started writing code when I was somewhere around 1st grade, hacking some rudimentary BASIC on a TRS-80. Since then, I’ve used more programming languages than I can count; everything from C to Prolog, Python to Haskell, and probably a dozen others. I enjoy diversity in programming languages and tools.

I’ve had an interest in both web and mobile development for some time, but have done little with either. Reflecting on that lately, I realize that both platforms severely restrict my freedom to use the tools I choose. Even on an old x86 with 640K of RAM, I had choices; BASIC, Pascal, and C would work under DOS. (Plus assembler, but I had yet to learn that at the time.) On Android, my choices are Java or… Java. (OK, or something that compiles to Java, but those selections are still limited.) Plus it seems like I’d have to use Eclipse, which seems to have taken the kitchen sync crown from Emacs long ago, and not in a good way. On iOS, the choices are Objective C. And on the web, it’s JavaScript.

The problem with all this is that there’s little room for language innovation on those platforms. If predictions about the decline of development on the PC pan out, how will the next advance in programming languages gain traction if the hot spots for development are locked down to three main languages? Will we find ourselves only willing to consider languages that can compile to Java/Dalvik bytecode? And won’t that put limits on the possible space for innovation to occur within?

Yes, I know about things like Mono and the Android Scripting Environment, which offer some potential for hope, but I think it’s still safe to say that these platforms are pretty closed to development in “unofficial” languages. I don’t know of major apps in the Android market written in Python, for instance.

I know of efforts such as Ubuntu’s, but even they simply lock down development to a different toolkit. There is no choice there of GTK vs. Qt, or such things.

Sadly, I don’t think it has to be this way. My phone has way more power than my computer did just a few years ago, and that computer was capable of not just executing, but even compiling, code written in all sorts of different languages.

But I don’t know if there’s much we can do to change things. Even with native code on Android, plenty still has to go through Dalvik (at least that’s my understanding). If you wanted to write a Python app with a full graphical touch interface on Android, I don’t think this would be all that easy of a task.