All posts by John Goerzen

Technology

Technology

Two days ago, I was out by the road digging a hole for our mailbox.

Along comes a guy on a tractor. The tractor was built in maybe the 1960s. It had no cab, and was loud, as tractors are. Attached to the back was a mower. It had long since lost any part of paint that was on it, and appeared to be made of rust.

The farmer driving this tractor was wearing a bluetooth earpiece for his cellphone.

(I don’t think there would be any way to use a phone while driving that tractor due to the noise, but hey.)

Now, the rest of the story…

This being rural Kansas, the farmer saw me and stopped to chat, turning off the tractor first, of course. He introduced himself as “that crazy guy from south of town”, and of course had a pretty good idea of who I was. After all, he’s lived around here for decades and knew my grandpa.

I told him why I was digging the hole for the mailbox. There’s going to be a second house down our mile, so now the post office will actually deliver to a mailbox at our driveway, rather than one almost a mile away. I thought he looked confused, so I pointed down the road and said that’s our mailbox on the corner.

“Oh, I know. I probably shouldn’t say this, but I knew when you moved in because I checked your mailbox one day and it didn’t have 40-year-old ads piling up anymore.”

We chatted a few more minutes, about the neighbor’s goats, how annoying it is that he can’t get DSL, why he wishes he could still use DOS instead of Windows. Then he went off towards home.

He had stopped his tractor in the middle of the road. But it wasn’t a problem. Nobody drove by that whole time anyway.

Desktop Linux: Gnome

Desktop Linux

I had been intending to write an entire series of posts about our corporate switch to Linux on the Desktop. To date, I wrote only one introducing the project and our reasons for switching from Windows. That was back in April.

Today I’d like to start talking about it all some more.

We have standardized on Gnome for our desktops. Given the Windows background of our user base, it was pretty much a given that we would have to use either Gnome or KDE. Something like fvwm or a non-integrated environment just wouldn’t be a good option.

We evaluated both Gnome and KDE. The very “clean” appearance of Gnome was a nice thing for us. KDE seemed to be to “chatty”, talked about entering in audiocd:/ when it shouldn’t have needed to, and generally violated the KISS and principle of least surprise too often. That said, I continue to run KDE for my personal desktop because Gnome just doesn’t have the flexibility that KDE does. It is too bad that Gnome has gone on this remove functionality kick, and KDE hasn’t gotten the KISS religion yet.

Anyway, Gnome worked well for the most part. We have set some defaults in gconf for things like panel icons. We also set a few mandatory defaults. I fixed a couple of bugs in the vfs system related to nfs4 support, which manifested themselves as icons for files newly saved to the desktop never showing up.

We wanted to present a customized menu to people based on what their job function is. That is, we are using a single system image, so all apps will be installed on all machines. But we didn’t want people to have to see a ton of software that they don’t use. That was easily enough accomplished for custom apps by creating desktop files with mode 0640 and setting the group to the set of people that should see the program on their menu. We removed a few stock programs (such as the terminal) from the menu as well, using dpkg-statoverride. That was also quite easily done. However, I will say that the entire Gnome XDG menu thing is woefully under-documented.

We use Firefox for the standard web browser. It is integrated well enough with Gnome and we have no problems there, aside from sites that are IE-only. We solve that with a Windows terminal server, which I’ll discuss later.

Our network printing was already based on Cups. The individual machines are set up as Cups clients only, which works fine. We did find, however, that gnome-cups-manager automatically installs a tray monitor for cups. This monitor puts little printer icons on the tray when printers are in use. Unfortunately, it figures out which printers are in use by polling the server, and it is turned on by default out of the box, with no good way to disable it short of dpkg-statoverriding it to 0000. You can imagine that hundreds of users times dozens of printers times numerous polls per minute created quite the load on the server. This was a really braindead design and the people that wrote it should have known better. It is also quite useless to have icons coming on for all the printers on the network, which on some networks could be thousands, and not even on the same continent as the user.

Printing is generally a bit iffy in Gnome. They seem to be transitioning between about 3 different printing toolkits, all of which have different print dialog boxes with different supported features and different ways of selecting printers. One chief annoyance is that the print box in evince (the document/PDF viewer) does not let people access printer-specific features such as hole punching and stapling. So we installed gtklp and xpdf for people. The people that print heavy PDFs are huge fans of gtklp these days; it’s a nicer solution than we had in Windows. Nobody really likes evince. We also have had some trouble with evince generating PostScript output that some printers can’t grok. It sounds like all this should be much better in newer versions of Gnome, which if true, would be welcome news.

The Gnome screenshot tool makes it easy to save off a screenshot to a file, or to drag it into an email, but it is difficult to print it (you have to save it first). That was a common complaint around here, so I wrote a little wrapper around xwd and gtklp for printing screenshots. People really like that because gtklp gives them lots of options about orientation and size of the image if they want it, or a simple “Print” button to click if they don’t care. We set a gconf default to bind this to Ctrl-PrintScr and it works well. KDE’s screenshot tool is much more capable, and if we were using KDE, we wouldn’t have had any problem with screenshots.

The bottom line on Gnome is that we, and are users, are happy with it after we’ve made these customizations. But we have had to do more customization that we should have. I still think that Gnome has been better for our users than KDE, but I do wonder how long we’ll be able to survive with our “no KDE libraries” policy, as people want ksnapshot, kolour, etc.

Time: Failing Our Geniuses

Uncategorized

An interesting article on Time today: Failing Our Geniuses about how the most talented students are being sidelined by current education policy. Some choice bits:

Since well before the Bush Administration began using the impossibly sunny term “no child left behind,” those who write education policy in the U.S. have worried most about kids at the bottom, stragglers of impoverished means or IQs. But surprisingly, gifted students drop out at the same rates as nongifted kids–about 5% of both populations leave school early. Later in life, according to the scholarly Handbook of Gifted Education, up to one-fifth of dropouts test in the gifted range.

It can’t make sense to spend 10 times as much to try to bring low-achieving students to mere proficiency as we do to nurture those with the greatest potential.

We take for granted that those with IQs at least three standard deviations below the mean (those who score 55 or lower on IQ tests) require “special” education. But students with IQs that are at least three standard deviations above the mean (145 or higher) often have just as much trouble interacting with average kids and learning at an average pace. Shouldn’t we do something special for them as well?

In a no-child-left-behind conception of public education, lifting everyone up to a minimum level is more important than allowing students to excel to their limit. It has become more important for schools to identify deficiencies than to cultivate gifts. Odd though it seems for a law written and enacted during a Republican Administration, the social impulse behind No Child Left Behind is radically egalitarian. It has forced schools to deeply subsidize the education of the least gifted, and gifted programs have suffered. The year after the President signed the law in 2002, Illinois cut $16 million from gifted education; Michigan cut funding from $5 million to $500,000. Federal spending declined from $11.3 million in 2002 to $7.6 million this year.

Linux Hardware Support Better Than Windows

Debian, Desktop Linux

Something I often hear from people that talk about Linux on the desktop is this: people want to be able to go to the store, buy hardware, and be confident that it will Just Work.

I would like to point out that things are rarely this simple on Windows. And, in fact, things are often simpler on Linux these days.

Here’s the example that prompted this post.

I have a computer that’s about 4 years old. It’s my main desktop machine at home. It was still fast enough for me, but has been developing all sorts of weird behaviors. Certain USB ports stopped working altogether a few months ago. Then it started hanging during POST whenever I’d try to reboot — but would still boot OK about 80% of the time after a power cycle. Then it started randomly losing contact with my USB mouse until a reboot. And the last straw was when the display started randomly going out. I’ve told everyone that my machine has cancer and is slowly dying.

The case is a pretty nice full tower — solid and sturdy. I have an 160GB IDE drive in it. So I figured I will upgrade the motherboard, CPU, RAM, and add a 500GB SATA drive since they’re so cheap these days and I’m running out of space. I’d also have to buy a new video card since my old one was AGP and the new motherboard only has PCI Express for video. So about $700 later from Newegg (I got a Core 2 Duo E6750), the parts arrived.

I spent some time installing it all. The motherboard had only one IDE channel, and I didn’t have any IDE cable long enough to connect both the IDE hard disk and the optical drive, so I popped in an old Maxtor/Promise PCI Ultra133 controller I had sitting around to use with the DVD burner.

Now, to recap, the hardware that the OS would see as new/different is: CPU, RAM, IDE controller, SATA controller, Promise IDE controller, integrated NIC, sound, video.

Then the magic smoke test.

I turned on the machine. Grub appeared. Linux started booting.

Even though I had switched from the default Debian “supports everything” kernel to a K7 kernel, it still booted.

And every single piece of hardware was supported immediately. There was no “add new hardware” wizard that popped up, no “I’ve found new hardware” boxes. It just worked, silently, with no need to tell me anything or have me click on anything.

Only one piece required configuration: the NIC, thanks to some udev design flaws (it got renamed from eth0 to eth1 by udev). That took 20 seconds. Debian saw the IDE HDD, the SATA drive, the Promise controller, the DVD burner, the video card, the sound, and it all worked automatically. And Debian is not even a distro that occurs to a lot of people when they think of great hardware support.

Now let’s turn to Windows.

The Windows Nightmare

I have a legal copy of Windows XP Home that was preinstalled on the machine when I got it. I resized its partition down to about 20GB so that I could use 140GB for Linux. I use it rarely, primarily for gaming, and I’ve bought about 3 games in the last 4 years. I usually disconnect the network when I boot to Windows, though I do keep it current with updates.

I did some research on what Windows was going to do when I replace the hardware. The general consensus from people on the ‘net is that you can’t just replace a motherboard and expect everything to be happy. There were generally three different approaches suggested: 1) don’t even try, just reinstall; 2) do a rescue install after you move over; and 3) use sysprep. The rescue install has to be done by booting from an XP install CD, then picking a rescue install option somewhere. It will overwrite your installed Windows with the version from the CD. That means that I’d have to re-apply SP2, though bits of it that didn’t get overwritten would still be on the hard disk, and who knows what would happen to the registry.

Option #3 was to download sysprep (must have the Genuine Disadvantage ActiveX to get the free download from MS). Sysprep is designed to be used just prior to taking an image with ghost for replication. It removes the hardware-specific config (but not the drivers), as well as the product key, from the machine, but otherwise leaves it untouched. On the next boot, you get the “Welcome to XP” wizard.

One other strike against is that Compaq “helpfully” didn’t ship any install CDs with the machine. Under Windows, they did have a “create rescue CD” tool, which burned 7 CDs for me. But they are full Compaq-specific CDs, not one of them an XP CD, *AND* they check on boot to see if you’re using the same Compaq motherboard, and exit if not. Highly useless.

So I went with sysprep. Before my new hardware even arrived, I downloaded the Windows drivers for all of it. I burned them to a CD, and installed as many as I could on the system in advance. About half of them refused to install since the new hardware wasn’t there yet. I then took a raw image of the partition with dd, just in case. Finally, right before I swapped the hardware, I ran sysprep and let it shut down the machine.

So after the new hardware was installed came the adventure.

Windows booted to the “welcome to XP” thingy. The video, keyboard, mouse, and IDE HDD worked. That’s about it.

I went through the “welcome to XP wizard”. But the network didn’t work yet, so I couldn’t activate it. So I popped my handy driver CD in the drive. But what’s this? Windows doesn’t recognize the DVD drive because it doesn’t have drivers for this Promise controller that came out in, what, 2001? Sigh. Downloaded the drivers with the imac, copy them to a CF card, plug the USB CF reader into Windows.

While I was doing that, about 6 “found new hardware” dialogs got queued up. Not one of them could actually find a driver for my hardware, but that didn’t prevent Windows from making me click through them all.

So, install Promise driver from CF card, reboot. Click through new hardware dialogs again. Install network driver, reboot, click through dialogs. Install sound driver. Install Intel “chipset” driver, click through dialogs. Reboot. Install SATA driver. Reboot.

So the hardware appears to all be working by this point, though I have a Creative volume control (from the old hardware) and a Realtek one in the tray. Minor annoyance to deal with later.

Now I have to re-activate XP. I dutifully key in the magic string from the sticker on my case. Surprise surprise, the Internet-based activation fails because my hardware is different. So I have to call the 800 number. I have to read in 7 blocks of 6 digits, one block at a time. Then I answer some questions: have I activated Windows before, have I changed hardware, was the old hardware defective (yes, yes, and yes). Then I get 7 blocks of 6 digits read to me. Finally Windows is activated. PHEW! Why they couldn’t ask those questions with the online tool is beyond me.

Anyhow. Linux took me 20 seconds to get working. Windows, about 2 hours, plus another 2 hours for prep and research.

I did zero prep for Linux. I made one config change (GUI users could have just configured their machine to use eth1).

Other cool Linux HW features

Say you buy a new printer and want to get it set up. On Windows, you insert the CD, let it install 200MB of print drivers plus ads plus crap plus add something to your taskbar plus who knows what else. Probably reboot. Then the printer might actually print.

On Debian, you plug in the printer to the USB port. You type printconf. 5 seconds later, your printer works.

I have been unpleasantly surprised lately by just how difficult hardware support in Windows really is, especially since everyone keeps saying how good it is. It’s not good. Debian’s is better, in my opinion.

There’s a redneck joke here somewhere

Humor

Back when I was in high school, I heard a Jeff Foxworthy CD once. He had this “you might be a redneck if” routine. We resemble one of his comments: “If the directions to your house include ‘turn off the paved road’, you might be a redneck.”

What if the road — the county road, not our driveway, mind you — has weeds growing down the middle? And you can tell that the county maintainer has been by recently because there are fewer, and shorter, weeds than before?

img_3383.jpg

Disasters happen

Software

It’s been one of those days.

It started with me forgetting my laptop bag at home. Then I had to work with an attorney on some things. He’s a nice guy, and does a good job, but you know, it’s not what I’d really like to spend my day doing.

Then in the middle of the afternoon, I hung up the phone and checked my email. About 50 messages from Samba telling me that processes with various PIDs had crashed unexpectedly. Uh-oh. I think we still have some 80ish people using that.

About a minute later, a coworker says, “John, I’ve been bad.” “Is that why I just got 50 emails from Samba?” “No. Well, yes. Well, maybe. I don’t know.”

It turned out he was working on a restore from tape that, out of necessity, grabbed more data than he needed. He meant to type rm -r ./var but typed rm -r /var instead. Oops. He hit Ctrl-C halfway through, so /var was still there enough to send email but not enough for Samba (or, apparently, NFS) to work.

As he dashed off to pull yesterday’s tapes from offsite storage, I prepared the restore and made a plan. We hadn’t installed any software since yesterday, so I restored var to a temporary location, took the server down into single-user mode, overwrote the /var that still existed, and rebooted the Xen instance in question. Everything back to normal. Except, that is, for the potentially dozens of users that will require assistance running SCANPST.EXE because their Outlook PST, being the fragile heap of garbage that it is, will have somehow been corrupted by this little incident.

So, what did we learn from this?

  • Deleting /var was probably the least annoying outage I’ve had to deal with yet. Certainly less nerve-wracking than the time I was working on a live, powered-up server and my wedding ring shorted out something on a circuit board. I didn’t know if that thing would just reboot or if we’d be down for hours waiting for parts…
  • It was really nice knowing what was going on, rather than trying to find that bit out
  • One coworker commented, “if he had to delete part of an operating system, at least it wasn’t Windows. We wouldn’t have recovered in 15 minutes if it was.” True.
  • Bacula is great.
  • Backups are great, even if you don’t use Bacula to make them.
  • I dislike programs that take server load from 0.3 to 9.5 just telling you that there’s something wrong with the server.

I suppose this means I’m a geek

Uncategorized

I work in an open-plan office. Normally I like to listen to some of my iPod’s music, or NPR or something, at some point during the day. It helps me tune out distractions when I’m coding or concentrating on something. My iPod, and my nice Etymotic headphones, get transported to and from work each day in my laptop bag. Today I forgot the laptop bag at home.

What to do? I could just work without headphones. I’d be fine, but you know, I’ve got standards here. My job involves working with computers, so I ought to be able to come up with a workaround, right?

So lesse… what do I have? One binaural (mono sound, but speakers for each hear) telephone headset. One Polycom SIP phone, connected to our corporate Asterisk system. One workstation with sound capabilities. One installation of Asterisk on this workstation for testing purposes. And, a pre-existing path from the corporate system to the workstation system for testing Asterisk. (Very handy that, and used a lot when we were doing active Asterisk work.)

So in less then five minutes I had music going via my telephone headset. Lo-fi, and not noise-dampening like the Etymotics, but I enjoyed it for the simple fact that it was being played *over the phone* at no cost to anyone. My desk phone supports multiple “lines”, so I still could place and receive calls just fine.

Should anyone care to look, they’d find a 5-hour call from me to myself deep in the Asterisk logs. My own workstation logs will show that I put myself on hold for 5 hours (since I used Asterik’s music-on-hold feature to play my own selections).

IP telephony is fun. So is Asterisk.

Haskell Fun

Programming

Bryan O’Sullivan noted over at the Real World Haskell blog that Haskell made quite the impact at OSCon. And I can attest to Simon Peyton-Jones having trouble leaving the building because of all the people that wanted to talk to him about Haskell. It was interesting to think about “why now” for Haskell’s popularity. Bryan’s post has links to the video of Simon’s talks, which are great. (Sample quote: “Oh look, a whiteboard has appeared as if by magic! What joy!”)

I followed Bryan’s link to the Haskell/OSCon-related blog posts at Technorati. Here’s an interesting one by chromatic, who gave some Perl talks at OSCon. Favorite quote:

I sat next to Nat Torkington at the tutorial. He kept rubbing his temples. At one point I leaned over and said, “The interesting thing about Haskell is that its functions only take one argument.” He turned green.

In all seriousness, well-factored Haskell code resembles well-factored Smalltalk code: if you have functions (or methods) longer than a handful of lines, you’re probably doing too much. Lower level languages such as C rarely give the opportunity for composition and abstraction that you can get out of functional languages. The presence of pure functions–functions which never change global state and which return the same output for the same input–is also immensely important.

It’s actually the combination of the two features which give these languages such power. When Haskell forces you to mark impure functions explicitly, it gives you tools to isolate behavior which can change global state in the smallest possible scopes, and prevents you from composing impure and pure code together accidentally. When Haskell lets you compose functions into larger functions, not only does it help you write code more concisely, but it provides well-defined units of behavior which work along well-defined and isolated boundaries.

So what is this business about Haskell functions taking only one argument? Let’s look at a quick example. Say I wanted to write a function to multiply two numbers. I’d write:

mul a b = a * b

I could give the type of this function like this:

mul :: Int -> Int -> Int

You could read that as “mul takes two Ints and returns an Int”. And you can think of it this way. But you could also write the type this way:

mul :: Int -> (Int -> Int)

It means the exact same thing and is valid to Haskell. To read it, you’d say “mul takes an Int and returns a function that takes an Int and returns an Int.” And truly this is what Haskell functions that take multiple parameters are doing. Now, I can say:

fifteen = mul 3 5
mulByThree = mul 3
fifteen' = mulByThree 5
fifteen'' = (mul 3) 5

mulByThree has type Int -> Int; it’s a function that we got by simply not applying “mul” to all its arguments. fifteen” illustrates what is going on internally when you write “mul 3 5”. It turns out that being able to call a function with multiple arguments is just some syntactic sugar to Haskell.

This is a tremendously useful feature. For instance:

filter (>= 10) [1..20]

I’ve applied >= to only one argument here. That’s fine; it returns a function that’s exactly what filter wants.

I’ve been watching the video of Simon’s Taste of Haskell talk. I could *hear* when the audience grasped the utility of this because there was a collective “Oooooo!” from them.

By the way, Haskell has type inference, so I didn’t have to give types at all. Oh, and you can also refer to the function itself by not giving any arguments.

mul = (*)

This is one small reason I like to say “Haskell manipulates functions with the same ease that Perl manipulates strings.” Mind-bending, isn’t it?

OSCon Friday

Software

Fudge Update

Yesterday I blogged about the guy handing out fudge. He saw my post and explained why he was doing it. Today he was around handing out fudge, and I thanked him for his comment. He gave me two pieces of fudge for blogging about it.

Pineapple

At most of the breaks, they have this truly wonderful real (non-canned) pineapple. I think I have eaten more pineapple this week than any other week, ever. Very insidious, O’Reilly.

Conference materials

Slides of talks are available. Apparently keynotes are being posted on Youtube and Google Video, though they haven’t provided a specific link AFAICT.

Philip Rosedale, Founder/CEO of Linden Lab

Second Life and the XPrize are two examples of escapism: you can escape to virtual earth, or escape the planet entirely.

In order for Second Life to grow, it has to become profoundly open. It has to be a standardized protocol. They are working on code to let people run their own servers. They are trying to make the server trusted and make it able to be open sourced as well. They see openness as the key way for it to grow.

If you are writing a web app that depends on the network effect — such as Facebook — you should open source everything right from the beginning. Not because it’s best for humanity, but because you will win.

Jimmy Wales

He’s trying to do open source search. Nice idea, could have been summarized in 45 seconds, I think.

Simon Wardli

Was going to talk about Zimki, which was going to open sourced, but they decided not to.

Funny and interesting talk, I sorta forgot to take notes while listening…

Nat Torkington

OSCon program chair.

Funny talk about Linux, an adolescent at 16 years old. Thinking about different languages and how they’re doing. How the Linux community is organized: are we fighting battles, and do we need to?

James Larsson

This guy takes old hardware and makes insane devices out of it. 15,000 volts running over a coat hanger used as a “game controller”, for instance.

In all, awesome keynotes today.

OSCon Thursday Part 2: Linux on laptops

Linux

Matthew Garrett

A lot of background on the state of laptop support in Linux. It worked reasonably well in the 90s, but with the migration to ACPI, has become much more complicated and less reliable in general, especially with suspend/resume and video.

Emporer Linux and System 76 produce Linux supported laptops.

I wanted some more in-depth technical information and found Matthew later on at the Intel booth. Here’s what I learned:

s2ram is little more than a wrapper around standard ACPI sleep that has options to do video mode save/restore

I asked him about all the many, many different userland laptop management tools. He recommends simple acpi-utils with the ondemand governor. laptop-mode-tools tries to do way too much, and there is little point to using a userland governor anymore.

I’ve been having a problem with my MacBook Pro (Core Duo) hanging on suspend about 10% of the time. I explained the symptoms and asked him how to go about debugging it. He suggested disabling console suspend and enabling PM debug in the kernel. I will give that a try and see what I get.