Two new bashisms

I learned about two bash features I hadn’t known about today.

From a colleague, GLOBIGNORE. A colon-separated glob of files to ignore when expanding globs. Helpful behavior when set to “*~” and used with grep.

From the Git FAQ, in a section explaining that it breaks the Git build process, CDPATH. A colon-separated search path to use when you type cd. Possibly useful to refer to subdirs of ~ or other common areas. Seems like it’s prone to break a ton of scripts if exported though.

Death sure is cost-effective, isn’t it?

I just read Death Be Not Proud (But It Is Cost-Effective) by Chez Pazienza. In his story, Chez talks about his stay in the hospital to have a marble-sized brain tumor removed. Across the room during his stay in neuro ICU, he saw a person far worse off than himself: staples all around his head, barely able to stay conscious, unable to speak. After a few days of this, Chez asked the nurse what had happened to the other person.

It was the same thing.

The difference? Chez had good insurance, and the other person didn’t. So Chez got the modern surgery with the latest technology, and the other guy got the Neolithic version. The other patient’s family came to visit, clearly heartbroken at his condition, not knowing whether he’d ever be the same. And knowing that even if he’d survive, he’d have years of physical therapy ahead of him.

Then there was the story of the girl whose insurance company denied a liver transplant, calling it “experimental”, sending her to her death. He says:

Regardless of what Fox business-creature Neil Cavuto may have to say on the subject, healthcare and profit are two thoroughly antithetical concepts. Giving CEOs the authority to stand on the edge of the arena and issue a final thumbs-up or down while we lay incapacitated or dying is like charging a lion with protecting the Christians.

I entirely agree.

Registrar Dynadot Conspires to Help Take Down Wikileaks?

Yesterday, Wired ran a story on the Cayman Islands bank that got Wikileaks.org blocked. This story said, in part:

When the bank’s lawyers indicated they would be filing a suit, she asked them to tell her where so that Wikileaks could find an attorney in the appropriate jurisdiction to represent it. She says the lawyers refused to tell her. Two and a half weeks later, the bank filed a restraining order against Dynadot and Wikileaks in San Francisco. Wikileaks received notice only a few hours before the case went to a judge who accepted the agreement between Dynadot and the bank.

(emphasis mine)

Now, Dynadot and this Cayman Islands bank apparently had an agreement to block wikileaks.org already. Not only did Dynadot effectively take wikileaks.org down, but also they “lock(ed) the wikileaks.org domain name to prevent transfer of the domain name to a different domain registrar.” The U.S. Disctrict Court for Northern California issued this injunction without ever giving Wikileaks a chance to respond. The bank only filed the request against Dynadot. Apparently Wikileaks received notification that this was going to happen only 6 hours before the hearing (an incredibly short time in legal terms), nowhere near enough time to prepare a case.

Now, the reason I post this is because I have looked at Dynadot as a registrar before. They have good prices and a whois privacy service that makes sense: where you remain the owner of record, making it easier if you need to transfer the domain or prove your ownership of it.

But before signing up, I read their AUP carefully. Among many alarming things, I noticed this paragraph:

You further agree that Dynadot, in its sole discretion and without liability to You for any resulting loss or damages, may take immediate corrective action, including, but not limited to, removal of all or a portion of Your domain services and/or deletion, suspension, cancellation, termination, or other interruption of domain services or Your customer account with Dynadot, at any time during the term of this Agreement, in the event of notice of any possible violation of this Agreement by You or Your end users, or if such service or account is used in association with morally objectionable activities, or for any reason whatsoever. In such cases, any and all fees paid to Dynadot will be non-refundable and ineligible for account credit.

So, I thought I would write to them about it. Here is an excerpt from their response:

We always conduct an investigation before taking action against a domain. We will give you a chance to respond to the complaints.

From Wired’s story, it doesn’t look like that really happened. The US Government has already issued advisories about Cayman Islands banks, and it is unclear (to me at least) what law Wikileaks broke, or how Dynadot could find their actions of exposing fraud “morally objectionable”. What’s more, collaborating with the bank to get a takedown order written, but not talking to Wikileaks, seems to go against their statements to me (assuming again that the Wired story is accurate).


Here is my entire mail. You may also find archive.org’s copy of the AUP from last July to be helpful. (I wrote the email in November, and they’ve added some sections since then, so the section numbers don’t necessarily match up)

Subject: Re: SITE: Questions about your AUP
From: Dynadot Info <info@dynadot.com>
Date: Sat, 3 Nov 2007 13:42 -0800
To: jgoerzen@complete.org

Hello,

Thank you for your email. Responses are below.

Best Regards,
Dynadot Staff

--------------------------------------------------
DYNADOT... $8.99 domain names... $1/mo. web hosting
http://www.dynadot.com



Hi,

I currently have several domains being hosted with Gandi.  I have long been looking for someone that can provide a level of privacy for my whois data in a sane way.  I think Dynadot is the first I've seen that looks like it can do that and still be affordable.

In preparation to transfer over a first test domain, I read your AUP and frankly am quite troubled by what I saw.  I hope that you can allay my fears.

In section 4 of part 2, the last paragraph states that Dynadot "for any reason whatsoever" may delete, cancel, or terminate my domain services or customer account.  It also lists "notice of possible violation" as a justification for that.  That makes me even more nervous -- any random person could send you an email claiming something nefarious is happing with my domain name, and I'm agreeing to just let you delete it because of that?


We always conduct an investigation before taking action against a domain. We will give you a chance to respond to the complaints. 

A very similar clause appears in section 7 of part 1 ("cancellation of services").  It goes on to cast a very broad net around objectionable material and says that Dynadot decides what's objectionable.  Now, if you go to my website at www.complete.org or blog at changelog.complete.org, you'll see I'm an upstanding netizen.  But the AUP says that things that "are designed to or effectively... embarrass" third parties could get my domain cancelled.  So if I post a review of Vista that says I think Microsoft did a poor job of engineering, would my account be yanked if one of their engineers complained?  What if I (legally!) linked to a Comedy Central sketch using real TV footage to mock George W. Bush or Hillary Clinton?


The cases you described above would never happen with us. Complaining about Microsoft or making fun of George Bush are protected free speech. The service agreement is designed to give us some flexibility in dealing with customers that break the law. 

I'm particularly concerned about this because apparently DynaDot felt it worthwhile to try to take down the website of someone that found a security hole: http://www.jhuskisson.com/friends/dynadot-fights-back-bans-nick-from-everything

While I wouldn't condone step-by-step cracking instructions in most cases, this is concerning to me.


He was posting a step by step guide to hacking our website, which is illegal. The hack did not work, but we noticed an upsurge in strange activity in our logs, so we requested the hacking guide be taken down. 

Finally, item (i) under the Domain Privacy Service section says that you could immediately reveal all my information upon the receipt of merely a *claim* (not even a court order), even if it's invalid.  But I thought that NOT doing this was what you were saying made your service better, over at http://www.dynadot.com/resource/article/qa.html?aid=0


Once again we need to build some flexibility into our service agreement to deal with people who use their domains for criminal activity. Otherwise we could be liable for the damages that they cause to others. So far, we have never dropped anyones privacy except in the few cases we were forced to by a FBI subpoena.

I know this is a long message, and I appreciate your time.  I really do want to use your service, but -- no offense intended -- I want to make sure I'm not dealing with scammers first, and from reading the AUP, I'm not so sure!


No offence taken. I will email our counsel to see if we can tighten up the agreement a bit. 

-- John Goerzen




A Cloud Filesystem

A Slashdot question today about putting to use all the unused disk space on corporate desktops got me to thinking. Now, before I start, comments there raised valid points about performance, reliability, etc.

But let’s say that we have a “cloud filesystem”. This filesystem would, at its core, have one configurable parameter: how many copies of each block of data must exist in the cloud. Now, we add servers with disk space to the cloud. As we add servers, the amount of available space on the cloud increases, subject to having enough space for replication according to our parameters.

Then, say we say we want a minimum of 3 copies of each block replicated. Each write to the filesystem will then cause a write to at least 3 different servers. Now, what if one server goes down? If the cloud filesystem is short on space, we may be down to only 2 copies of some blocks until that server comes back up. Otherwise, space permitting, it can rebuild that third copy on other servers.

Now, has this been done before? As far as I can tell, no. Wouldn’t it be sweet?

But there are some projects that are close. Most notably, GlusterFS. GlusterFS does all of the above, except the automated bits. You can have this 3 copy redundancy, but you have to manually tell it where each copy goes, manually reconfigure if a server goes offline, etc. Other options such as NBD, OpenAFS, GFS, DRBD, Lustre, GFS, etc. aren’t really well-suited for this scenario for various reasons.

So, what does everyone think? Can this work? Has it been done outside of Google?

xkcd author endorses Obama

The author of the xkcd comic has endorsed Obama for president. Among other things, he wrote:

Obama has shown a real commitment to open government. When putting together tech policy (to take an example close to home for xkcd) others might have gone to industry lobbyists. Obama went to Lawrence Lessig, founder of Creative Commons (under which xkcd is published) and longtime white knight in the struggle with a broken system over internet and copyright policy. Lessig was impressed by Obama’s commitment to open systems — for example, his support of machine-readable government information standards that allow citizens’ groups to monitor what our government is up to. Right now, the only group that can effectively police the government is the government itself, and as a result, it’s corrupt to the core. Through these excellent and long-overdue measures, Obama is working to fight this corruption.

Having Larence Lessig as an advisor, instead of some RIAA shill, speaks volumes about the candidate.

#1 on Google

I started this blog 5 years ago. In that time, I’ve written about Linux on the desktop, photography, international peace, version control systems, ice storms, Haskell, health care, rural life, Kansas, and infants.

Today I realized something: I am the hit on Google when you search for smelly ducting. This will take you to a story I wrote back in 2005 about an odd odor in our house. I believe it is my most popular story ever, with almost 100 comments — and comments continue to trickle in, a few a week.

What’s the right word to use for that? When the great worldwide democratizing force that is the Internet collectively decides that your most important blog post is about smelly ducting?

I’m glad folks know what matters. Wouldn’t want to worry too much about what’s going on in Iraq, Washington, or Kansas if your ducts smell, that’s for sure.

Obviously every single Presidential candidate has missed out on a key issue in voters’ minds. If only someone would come out with a freshly-scented ducting platform, this country could finally move forward!

Why Are We So Afraid of Socialized Medicine?

I’ve been thinking about this for awhile, so time to put down some thoughts.

First, what is socialized medicine? If we listen to the politicians that label health care as “socialized”, it seems to be “anything that is paid for by taxes and delivered free or cheaply to citizens.” Putting aside the question of whether that meets the academic definition of socialism for the moment, let’s look at things in the United States that are already socialized:

  • K-12 education
  • Police
  • Fire fighters
  • Public Libraries
  • Roads
  • Airports and air traffic control
  • Military defense and offense

That’s right. We trust the government with our children all day long for 13 years. For free!

Yet this is a country in which hospitals dump the homeless in the gutter for being unable to pay their bills. Even insured Americans find claims turned down for arbitrary reasons. People are afraid to change jobs for fear of losing health insurance.

Why is it bad to have the government pay for health care?

Here in the United States, our health care system is far from best in the world. It’s not even top 10. Or 20. Our system encourages minimizing health care, and doesn’t encourage preventative care.

I’d suggest that, in a democracy, it’s best to have the government pay for health care. That’s because, in a democracy, we are in control of the bureaucrats. If we wish to exercise common sense and pound into their heads that paying for preventative care makes good long-term sense, then we can do so at the ballot box.

So why the scare tactics about government being involved in health care?

Perhaps our real problem is that we have let government get out of our control? Perhaps we are too frightened of change to vote. Perhaps we’ve given up on a responsive government. Perhaps we think that the insurance companies and drug companies will never let us have a good health care system.

Yes, the lobbyists have a lot of power. But we have the power to remove it, and it’s high time we used it.

The audacity of Obama to have hope. To say that we can do better. When Hillary Clinton falls in line with the Republicans and accuses him of having “false hope”, effectively saying that we can’t do any better, then is she — or any Republicans — really a candidate of change? I think that all these accusations from conservatives and Hillary that Obama has “false hope” has finally convinced me that he’s the one to vote for. If everyone else claims that his ideas are too good, that his dreams too big, then I like him.

Oh, and you could substitute “college education” for “socialized medicine” everywhere in this article and get equally valid arguments.

LinuxCertified Laptop LC2100S

As you might know from reading my blog, at my workplace, we have largely standardized on Linux on the desktop and laptop.

We use systemimager to maintain a standard desktop image and a separate standard laptop image. These images differ because there are different assumptions. The desktop machines mount /home over NFS, authenticate to LDAP, etc. This doesn’t work on laptops. Moreover, desktops don’t use network-manager or wifi, but laptops do.

Our desktop image uses Debian’s hardware autodetection — plus a little hacking in /etc/init.d/gdm — to automatically adjust to a wide range of hardware. So far this has worked well.

Laptops are much more picky. Our standard laptop model had been the HP nc4400 — a small and light 12″ model that people here loved. HP discontinued that model. Their replacement was the 2510p. We ordered one in here for evaluation. Try as we might, we couldn’t get it to suspend and resume properly in Linux.

So I went out scouring the field of Linux laptops. Companies such as Emperor Linux buy retail laptops from people like Lenovo, test them for Linux, and sell them — at a premium. These were too expensive to justify at the quantities we need them.

Then I stumbled across Linux Certified. I’d never heard of them before. I called them up and asked a few questions. They don’t buy retail laptops, but instead have OEMs in Taiwan build laptops to their spec. They happen to use the same OEM that Fujitsu does, I believe. (No big company builds laptops in the USA these days). I asked them about wifi chipsets, video chipsets, whether they use stock kernels. I got clueful answers to all of these.

So we ordered one of their LC2100s models. They didn’t offer Debian preinstalled, but did offer Ubuntu, so I selected that. The laptop arrived a couple of days (!!) later, configured with the particular CPU, etc. that I selected.

I was surprised at the thrill I felt at taking a brand new laptop out of its box, turning it on, and watching Grub appear before my eyes. Ubuntu proceeded to boot. I then of course installed our regular Debian image on the thing to check it out.

It needed a kernel and xserver-xorg-video-intel from lenny, as well as the ipw3945 driver for wifi, but otherwise worked with the exact same software as our HP nc4400 image. (In fact, it wasn’t hard to support both laptops with that image, since both use a lot of Intel hardware.) The one trick was making hibernate call /etc/init.d/ipw3945d stop so that the ipw3945 module could be unloaded before suspend. (Why this particular chipset needs a daemon is beyond me, but oh well.)

The hardware is great. As far as I know, the ipw3945 was the only component that wasn’t directly and automatically supported by DFSG-free software in lenny main. The screen is sharp and high-contrast (it’s glossy, which I personally don’t like, but I bet our users will). The device itself feels sturdy. It’s small and dense. I haven’t opened it up, but it looks like all you need is a screwdriver to do so.

The only downside is that they don’t sell docking stations for it. Their standard answer on that is to buy a USB docking station. That’s a partial answer, but can’t handle power or video like a standard docking station will.

Also, the LC2100s is much cheaper than the HP laptop, even when configured when nicer specs in every way. That is no doubt partially due to the lack of the Windows tax.

I’m sending off an order for 4 more today, I believe.

DjVu: Almost Awesome

Earlier today, I started reading about the DjVu family of document formats. It really sounds slick: file sizes much smaller than PNG (and incredibly smaller than TIFF or PDF) for lossless data with the DjVuText format, file sizes much smaller than JPEG with equivalent quality for the DjVuPhoto format, and an advanced DjVuDocument format that separates the background photo from the foreground text and produces a quite nice output. There are wonderful plugins for browsers on all platforms, and server-side support already in Debian for sending pages incrementally as needed by clients.

I tried this out a bit and indeed it looks great on monochrome scans, and I made a quick try of DjVuPhoto as well. That part looks great.

So here’s the bad news.

Debian has no nice way to generate DjVuDocument files. There is a PS/PDF-to-DjVu converter that uses a djvu driver for Ghostscript. But Debian does not include that driver. Though, strangely, the program that depends on this driver is actually in Debian main. (Bug filed.) That program actually will make background-separated images, but only if they are separate objects in the input.

All Debian has is a program csepdjvu, which requires you to somehow manually separate the foreground and background images. Ugh.

So there is no way using software in Debian to produce DjVuDocument files with automatic separation, either from scans or from a digital source. It appears that there may not be Free Software to do this from scans either. This fact is not made clear at all in the DjVu documentation that is around.