Monthly Archives: April 2010

Some More Gopher Data

Yesterday, I invited you to download a piece of Internet history. Today I have unearthed a bit more data to add to it.

There is a new torrent with 1.5GB more data (3GB uncompressed). This includes the FTP site from boombox.micro.umn.edu/pub/gopher, which was a large collection of Gopher client and server software. Also included was ftp.unicom.com/pub/gn-tools, the old GN server, and wiretap.area.com. This last site was mirrorable via FTP, and included the UMN Gopherd .cap and .Links files, so it can be served up via PyGopherd. The full contents are available in my post to the Gopher mailing list.

Incidentally, thank you to everyone that has helped seed, download, store, host, and otherwise preserve the 2007 archive. It is much appreciated. I have contacted several institutions recommended to me in comments or emails to try to arrange a stable long-term home for the data.

Enjoy.

Update: This torrent is now available elsewhere; see this post for the most current location.

Download A Piece of Internet History

Back in the early 1990s, before there was a World Wide Web, there was the Internet Gopher. It was a distributed information system in the same sense as the web, but didn’t use hypertext and was text-based. Gopher was popular back then, as it made it easy to hop from one server to the next in a way that FTP didn’t.

Gopher has hung on over the years, and is still clinging to life in a way. Back in 2007, I was disturbed at the number of old famous Gopher servers that had disappeared off the Internet without a trace. Some of these used to be known by most users of the Internet in the early 90s. To my knowledge, no archive of this data existed. Nobody like archive.org had ever attempted to save Gopherspace.

So I decided I would. I wrote Gopherbot, a spidering archiver for Gopherspace. I ran it in June 2007, and saved off all the documents and sites it could find. That saved 40GB of data, or about 780,000 documents. Since that time, more servers have died. To my knowledge, this is the only comprehensive archive there is of what Gopherspace was like. (Another person is working on a new 2010 archive run, which I’m guessing will find some new documents but turn up fewer overall than 2007 did.)

When this was done, I compressed the archive with tar and bzip2 and split it out to 4 DVDs and mailed copies to a few people in the Gopher community.

Recently, we’ve noted that hard disk failures have hobbled a few actually maintained Gopher sites, so I read this archive back in and posted it on BitTorrent. If you’d like to own a piece of Internet history, download the torrent file and go to town (and please stick around to seed if you can). This is 15GB compressed, and also includes a rare video interview with two of the founders of Gopher.

There are some plans to potentially host this archive publicly in the manner of archive.org; we’ll have to wait and see if anything comes of it.

Finally, I have tried to find a place willing to be a permanent host of this data, and to date have struck out. If anybody knows of such a place, please get in touch. I regret that so many Gopher sites disappeared before 2007, but life is what it is, and this is the best snapshot of the old Gopherspace that I’m aware of and would like to make sure that this piece of history is preserved.

Update: The torrents are now permaseeded at ibiblio.org. See the 2007 archive and the 2006 mirror collection.

Update: The ibiblio mirror is now down, but you can find them on archive.org. See the 2007 archive and the 2006 mirror collection.

Moral obligations of Free Software authors?

I’ve got a bit of a problem.

I enjoy writing software. I often write software to solve some sort of problem that I’ve had. Usually virtually any code I write winds up in my git repositories, on the theory that it might be useful to someone else. Some of the code that I think might really be useful to people gets even better treatment. OfflineIMAP, for instance, has a very comprehensive manpage, heavily commented example config file, wiki, mailing list, public bug tracker, etc. Most of these I did the majority of the work to create, but OfflineIMAP does occasionally receive code and documentation contributions from others.

Now here’s my dilemma. For my purposes, OfflineIMAP is, well, finished. It does everything I ever wanted it to do, and does it better than I ever expected it would. There are some people that would like it to do other things; for instance, optimize performance for IMAP folders with 100,000 messages in them, do UTF-8 folder name translation, and retry a sync if a connection is lost rather than crash (OfflineIMAP was designed to crash gracefully and be automated, so this has never bothered me.)

None of these are features that I care about, and I don’t have much time to devote to OfflineIMAP these days. It is not an interesting problem to me anymore as, well, I’ve solved it already.

Yet I’ll be honest and say I feel guilty about the bug reports that are stacking up in the OfflineIMAP bug tracking system. OfflineIMAP is used by people that have an expectation for improvement. My efforts to hand over maintainership of OfflineIMAP have failed (the people have gone AWOL shortly after agreeing to maintain it).

This problem is even more acute for hpodder, my command-line podcatcher. hpodder works great and is simple. But I no longer listen to podcasts. At all. (I blame my Kindle for that.) Therefore I no longer even use hpodder. Again, I feel guilty for not working on it; for instance, when language changes broke its UTF-8 support, I haven’t gone in to fix it. Neither has anybody else, for that matter.

This leads me to a dilemma. If I do nothing with my code but toss it on git.complete.org, few people will benefit from it. Most people need documentation, packages for their distribution, etc. git repos don’t tend to show up highly in search engines. So, although technically I’ve shared things with the world by putting them there, practically speaking I haven’t done people many favors.

On the other hand, if I go the whole “responsible maintainer” route, writing documentation, wiki, mailing list, Debian packages, etc., then I have the problem of, well, actual users who want actual support. I feel bad if I’m not in a position to give it to them. Many people seem to have the expectation that software is never “finished” and will continue to be improved. (Ah, if only my name were Knuth I might stand a chance to evade that one. But only a chance, given all the TeX spinoffs.) This expectation, in turn, reduces my enthusiasm for publishing my code online as Free Software. Because now I can’t just toss it up there and say “help yourselves”. Now I get angry emails about all the bug reports piling up. On the other hand, I also get occasional small contributions via my PayPal “tip jar”, which are awesome and motivating.

Despite my grumbling, I do continue to maintain OfflineIMAP, primarily as a patch reviewer these days. I take an aggressive stance on quality, and when I get patches that add features without documentation, I usually write documentation for them before committing them. If I have evidence that a patch hurts quality, I yank it (as I had to do with IDLE support, which was a great feature, but the patch caused all sorts of stability problems due to its requirement of imaplib2.py). For that reason, I suspect, forks haven’t taken off.

So what’s a person that writes niche software to do? OfflineIMAP isn’t at the level of popularity of something like Gnome, or even debhelper, and never will be; its userbase consists of people that think that IMAP support in $MUA isn’t good enough. hpodder is the somewhat small domain of console-loving podcast listeners.

What are your suggestions? Should I abolish the bug trackers and just go for simple? Is there something more I could be doing to make the community feel more empowered? Is simple posting on git.complete.org not as bad as I thought?

Jacob has a new computer — and a favorite shell

Earlier today, I wrote about building a computer with Jacob, our 3.5-year-old, and setting him up with a Linux shell.

We did that this evening, and wow — he loves it. While the Debian Installer was running, he kept begging to type, so I taught him how to hit Alt-F2 and fired up cat for him. That was a lot of fun. But even more fun was had once the system was set up. I installed bsdgames and taught him how to use worm. worm is a simple snake-like game where you use the arrow keys to “eat” the numbers. That was a big hit, as Jacob likes numbers right now. He watched me play it a time or two, then tried it himself. Of course he crashed into the wall pretty quickly, which exits the game.

I taught him how to type “worm” at the computer, then press Enter to start it again. Suffice it to say he now knows how to spell worm very well. Yes, that’s right: Jacob’s first ever Unix command was…. worm.

He’d play the game, and cackle if he managed to eat a number. If he crashed into a wall, he’d laugh much harder and run over to the other side of the room.

Much as worm was a hit, the Linux shell was even more fun. He sometimes has a problem with the keyboard repeat, and one time typed “worrrrrrrrrrrrrrrrrrm”. I tried to pronounce that for him, which he thought was hilarious. He was about to backspace to fix it, when I asked, “Jacob, what will happen if you press Enter without fixing it?” He looked at me with this look of wonder and excitement, as if to say, “Hey, I never thought of that. Let’s see!” And a second later, he pressed Enter.

The result, of course, was:

-bash: worrrrrrrrrrrrrrrrrrm: command not found

“Dad, what did it do?”

I read the text back, and told him it means that the computer doesn’t know what worrrrrrrrrrrrrrrrrrm means. Much laughter. At that point, it became a game. He’d bang at random letters, and finally press Enter. I’d read what it said. Pretty soon he was recognizing the word “bash”, and I heard one time, “Dad, it said BASH again!!!” Sometimes if he’d get semicolons at the right place, he’d get two or three “bashes”. That was always an exciting surprise. He had more fun at the command line than he did with worm, and I think at least half of it was because the shell was called bash.

He took somewhat of an interest in the hardware part earlier in the evening, though not quite as much. He was interested in opening up other computers to take parts out of them, but bored quickly. The fact that Terah was cooking supper probably had something to do with that. He really enjoyed the motherboard (and learned that word), and especially the CPU fan. He loved to spin it with his finger. He thought it interesting that there would be a fan inside his computer.

When it came time to assign a hostname, I told Jacob he could name his computer. Initially he was confused. Terah suggested he could name it “kitty”, but he didn’t go for it. After a minute’s thought, he said, “I will name it ‘Grandma Marla.'” Confusion from us — did he really understand what he was saying? “You want to name your computer ‘Grandma Marla?'” “Yep. That will be silly!” “Sure you don’t want to name it Thomas?” “That would be silly! No. I will name my computer ‘Grandma Marla.”” OK then. My DNS now has an entry for grandma-marla. I had wondered what he would come up with. You never know with a 3-year-old!

It was a lot of fun to see that sense of wonder and experimentation at work. I remember it from the TRS-80 and DOS machine, when I would just try random things to see what they would do. It is lots of fun to watch it in Jacob too, and hear the laughter as he discovers something amusing.

We let Jacob stay up 2 hours past his bedtime to enjoy all the excitement. Tomorrow the computer moves to his room. Should be loads of excitement then too.

Introducing the Command Line at 3 years

Jacob is very interested in how things work. He’s 3.5 years old, and into everything. He loves to look at propane tanks, as the pressure meter, and open the lids on top to see the vent underneath. Last night, I showed him our electric meter and the spinning disc inside it.

And, more importantly, last night I introduced him to the Linux command line interface, which I called the “black screen.” Now, Jacob can’t read yet, though he does know his letters. He had a lot of fun sort of exploring the system.

I ran “cat”, which will simply let him bash on the keyboard, and whenever he presses Enter, will echo what he typed back at him. I taught him how to hold Shift and press a number key to get a fun symbol. His favorite is the “hat” above the 6.

Then I ran tr a-z A-Z for him, and he got to watch the computer convert every lowercase letter into an uppercase letter.

Despite the fact that Jacob enjoys watching Youtube videos of trains and even a bit of Railroad Tycoon 3 with me, this was some pure exploration that he loves. Sometimes he’d say, “Dad, what will this key do?” Sometimes I didn’t know; some media keys did nothing, and some other keys caused weird things to appear. My keyboard has back and forward buttons designed to use with a web browser. He almost squealed with delight when he pressed the forward button and noticed it printed lots of ^@^@^@ characters on the screen when he held it down. “DAD! It makes LOTS of little hats! And what is that other thing?” (The at-sign).

I’ve decided it’s time to build a computer for Jacob. I have an old Sempron motherboard lying around, and an old 9″ black-and-white VGA CRT that’s pretty much indestructible, plus an old case or two. So it will cost nothing. This evening, Jacob will help me find the parts, and then he can help me assemble them all. (This should be interesting.)

Then I’ll install Debian while he sleeps, and by tomorrow he should be able to run cat all by himself. I think that, within a few days, he can probably remember how to log himself in and fire up a program or two without help.

I’m looking for suggestions for text-mode games appropriate to a 3-year-old. So far, I’ve found worm from bsdgames that looks good. It doesn’t require him to have quick reflexes or to read anything, and I think he’ll pick up using the arrow keys to move it just fine. I think that tetris is probably still a bit much, but maybe after he’s had enough of worm he would enjoy trying it.

I was asked on Twitter why I’ll be using the command line for him. There are a few reasons. One is that it will actually be usable on the 9″ screen, but another one is that it will expose the computer at a different level than a GUI would. He will inevitably learn about GUIs, but learning about a CLI isn’t inevitable. He won’t have to master coordination with a mouse right away, and there’s pretty much no way he can screw it up. (No, I won’t be giving him root yet!) Finally, it’s new and different to him, so he’s interested in it right now.

My first computer was a TRS-80 Color Computer (CoCo) II. Its primary interface, a BASIC interpreter, I guess counts as a command-line interface. I remember learning how to use that, and later DOS on a PC. Some of the games and software back then had no documentation and crashed often. Part of the fun, the challenge, and sometimes the frustration, was figuring out just what a program was supposed to do and how to use it. It will be fun to see what Jacob figures out.

Face to Face With Destruction

IMG_2641

Seeing these church bells in Lübeck wasn’t easy for me. They melted and fell to the ground after an air raid in 1942, and have been left exactly as they fell.

I wasn’t alive in 1942, and even my parents hadn’t yet been born. My grandfather was relocated by the American government because he refused to fight in the military for religious reasons.

And yet, I still felt a sense of embarrassment, being an American standing there in Germany next to those bells. My country* did things like this. Why? Even if there’s a war on, and whatever the other side may be doing, do we really have to bomb all their churches?

Then, we went to the Dom (cathedral). It has a small gallery of photos of the destruction in 1942, and how it took 40 years to restore (though some original works are simply irreplaceable.)

Burning Lübeck Cathedral after an air raid in 1942

I had just been walking through the beautiful Dom, and then saw pictures like that. It was very sad. I imagined the people in 1942, seeing the smoking remains of so many buildings that had been important to the town for centuries. Places where, no doubt, some of them worshiped, and their parents and grandparents had too. I imagined my sadness if a country bombed one of the churches that my ancestors helped start or build. I felt ashamed of what the democracy in which I live (and its allies) did to the Dom, to Marienkirche, and to the other churches and civilians there.

Then, a few days ago, Wikileaks posted a video from an American helicopter in which our soldiers begged “Come on, let us shoot!” at people, most or all of which were completely unarmed. They killed a bunch of people. Then a van came to try to get the wounded to the hospital. Then the soldiers in helicopter killed those people, and wounded some children. Even if they thought their lives were in danger, I’d expect to hear some quiet sadness at what they thought was a regrettable need to use force. Instead, you’d hear them cheering whenever they killed someone, as if they had just won a round on some video game. They killed civilians, and considered it a fun experience and a mission accomplished. There was no sadness for knocking a man to the ground, leaving him writhing. Just jokes. There was no more care for human life here than there was 60 years ago.

It saddens me that every year we celebrate our military on independence day — celebrating its ability to kill, not its ability to feed. I wish that we could rather celebrate the times Americans have saved lives, whether in Berlin or in Port-au-Prince, or helped to topple dictatorships. Military power shouldn’t be our first choice so often.

Lübeck wasn’t attacked for any strategic purpose; the British did it to destroy the “morale of the enemy civil population”, as the Americans did in Dresden or the Germans in London. In other words, those bells fell to the ground for no reason other than that they were German. People lost their lives for the same reason — in Germany and in London. The British had no reason to believe that those civilians meant them any harm, that they were anything but caught between two strong powers. And the same held true in Iraq.

When we went to Berlin, we walked through the replica of Checkpoint Charlie, with the big sign announcing that “you are leaving the American sector.” There is little positive to be said about the Cold War, but there is something the British and American military did of which I am not the least ashamed: the Berlin Airlift. Although I didn’t see it, I feel a sense of the USA having been a force for good when I read about the Berlin Airlift Monument at Tempelhof, which has the names of the 70 British and American men that lost their lives in the airlift (due to accidents), above the inscription “They gave their lives for the freedom of Berlin.” That’s the spirit I wish we’d see more often in the world today: people giving their lives not just for their own security, but for strangers in need on a different continent.

We Americans have been, compared to many countries in Europe, relatively sheltered from the ravages of war in our own midst. I wish that more of us would have the opportunity to gaze at fallen church bells, to wonder through a magnificent old church and then to see photos of it on fire, or to see a monument to our soldiers in a far away land. Perhaps then we could better learn from history, and see firsthand that some of our most positive contributions to the world have come in the form of coal and candy bars instead of incendiary devices.

A small post script

Many historic buildings or locations we saw in Germany had plaques on their exterior with a brief history, often in German and in English. I was interested to note that some (though not all) of these plaques had language such as “destroyed by fire in 1942”, with no mention that the cause of the fire was a bombing. I was even more interested to note that same sort of description at several sites in Prague Castle, mentioning that the original building was lost in the fire of 1142. It was only on the third or fourth building in the Castle that we learned that the fire of 1142 was caused by a military siege of Prague Castle.

* I know it was the British at Lübeck, but the Americans did similar things elsewhere.

I also acknowledge that, at a macro level at least, the Berlin Airlift wasn’t purely an altruistic act, as there were geopolitical factors in play. But I am speaking of the actions of the airmen here, who took on considerable risk to help others.

Begin questioning my sanity… NOW

It’s been a long and wet winter here. We live down a dirt/sand/gravel road, and when it rains, it’s difficult to get a car down the road due to mud. And impossible to get a bicycle down it. As a result, I’ve only been able to ride my bicycle to work once since November, and that was in January.

Last Thursday, I intended to ride in to work, but discovered my front tire wouldn’t hold air. I had heard about a wonderful local bike repair shop, so I dropped off the wheel there. The owner replaced the tube and checked it out, and charged me, yes, $4 — including labor and the tube. Nice.

So today was my first ride of the season. I rode a total of 21.8 miles (35 km) today, which was probably unwise enough for being as out of shape as I am. It’s 9.6 miles each way to work, plus I did some errands over lunch.

But add to that the winds — 30 MPH (48 kph) with gusts to 43 MPH (69 kph). This morning, they were weaker and also mostly at my back. This afternoon, though, they were mostly a vicious crosswind. If you’ve bicycled much, you’ll know that’s less annoying than a headwind, but is still quite annoying and takes a lot more energy to battle than you might think.

So, I am now rather sore. And the question is: will I be silly enough to do this again tomorrow?

The answer to that probably depends on how late I stay up watching the Butler/Duke game tonight, as I have to get up an hour earlier on days that I bicycle to work.

I found that the bicycle rack at work — which, somewhat to my annoyance, was moved indoors last year — has been rather disused. It is in a rather dusty and dirty part of our manufacturing shop, and there were large metal bins completely blocking the path to it, which I had to move before I could park my bike.

Then, of course, it was the usual comments — which I take with a smile — about somebody that shows up to work wearing cycling shorts & shirt.

It should be noted that I change into professional clothes at work. But my commute is too long to wear them on the way in and expect to be presentable, non-smelly, and pain-free.

In any case, evidence that this may not have been the best day to start my commute: it hurts to sit at the moment.