Category Archives: Software

Haskell Time Travel

There is something very cool about a language in which the easiest, most direct way to explain how it solves a problem is to say, “When we pass the output of [this function] into the input for the oracle we are actually sending the data backwards in time. So when [the code] queries the oracle we get a result from the future.”

Sweet.

The story goes on to say, however, “Time travel is a very dangerous business. One false move and you can create a temporal paradox that will destroy the universe (which in this case means that the computation will diverge). When programming with values from the future, it is important never, never, to do anything with the values that might change the future. This is the temporal prime directive.”

Saving Power with CPU Frequency Scaling

Yesterday I wrote about the climate crisis. Today, let’s start doing something about it.

Electricity, especially in the United States and China, turns out to be a pretty dirty energy source. Most of our electricity is generated using coal, which despite promises of “clean coal” to come, burns dirty. Not only does it contribute to global warming, but it also has been shown to have an adverse impact on health.

So let’s start simple: reduce the amount of electricity our computers consume. Even for an individual person, this can add up to quite a bit of energy (and money) savings in a year. When you think about multiplying this over companies, server rooms, etc., it adds up fast. This works on desktops, servers, laptops, whatever.

The easiest way to save power is with CPU frequency scaling. This is a technology that lets you adjust how fast a running CPU runs, while it’s running. When CPUs run at slower speeds, they consume less power. Most CPUs are set to their maximum speed all the time, even when the system isn’t using them. Linux has support for keeping the CPU at maximum speed unless it is idle. By turning on this feature, we can save power at virtually no cost to performance. The Linux feature to handle CPU frequency scaling is called cpufreq.

Set up modules

Let’s start by checking to see whether cpufreq support is already enabled in your kernel. These commands will need to be run as root.

# cd /sys/devices/system/cpu/cpu0
# ls -l

If you see an entry called cpufreq, you are good and can skip to the governor selection below.

If not, you’ll need to load cpufreq support into your kernel. Let’s get a list of available drivers:

# ls /lib/modules/`uname -r`/kernel/arch/*/kernel/cpu/cpufreq

Now it’s guess time. It doesn’t really hurt if you guess wrong; you’ll just get a harmless error message. One hint, though: try acpi-cpufreq last; it’s the option of last resort.

On my system, I see:

acpi-cpufreq.ko     longrun.ko      powernow-k8.ko         speedstep-smi.ko
cpufreq-nforce2.ko  p4-clockmod.ko  speedstep-centrino.ko
gx-suspmod.ko       powernow-k6.ko  speedstep-ich.ko
longhaul.ko         powernow-k7.ko  speedstep-lib.ko

For each guess, you’ll run modprobe with the driver name. I have an Athlon64, which is a K8 machine, so I run:

#modprobe powernow-k8

Note that you leave off the “.ko” bit. If you don’t get any error message, it worked.

Once you find a working module, edit /etc/modules and add the module name there (again without the “.ko”) so it will be loaded for you on boot.

Governor Selection

Next, we need to load the driver that tells the kernel what governor to use. The governor is the thing that monitors the system and adjusts the speed accordingly.

I’m going to suggest the ondemand governor. This governor keeps the system’s speed at maximum unless it is pretty sure that the system is idle. So this will be the one that will let you save power with the least performance impact.

Let’s load the module now:

# modprobe cpufreq_ondemand

You should also edit /etc/modules and add a line that says simply cpufreq_ondemand to the end of the file so that the ondemand governor loads at next boot.

Turning It On

Now, back under /sys/devices/system/cpu/cpu0, you should see a cpufreq directory. cd into it.

To turn on the ondemand governor, run this:

# echo echo ondemand > scaling_governor

That’s it, your governor is enabled. You can see what it’s doing like this:

# cat cpuinfo_min_freq
800000
# cat cpuinfo_max_freq
2200000
# cat cpuinfo_cur_freq
800000

That shows that my CPU can go as low as 800MHz, as high as 2.2GHz, and that at the present moment, it’s running at 800MHz presently.

Now, check your scaling governor settings:

# cat scaling_min_freq
800000
# cat scaling_max_freq
800000

This is showing that the system is constraining the governor to only ever operate on an 800MHz to 800MHz range. That’s not what I want; I want it to scale over the entire range of the CPU. Since my cpuinfo_max_freq was 2200000, I want to write that out to scaling_max_freq as well:

echo 2200000 > scaling_max_freq

Making This The Default

The last step is to make this happen on each boot. Open up your /etc/sysfs.conf file. If you don’t have one, you will want to run a command such as apt-get install sysfsutils (or the appropriate one for your distribution).

Add a line like this:

devices/system/cpu/cpu0/cpufreq/scaling_governor = ondemand
devices/system/cpu/cpu0/cpufreq/scaling_max_freq = 2200000

Remember to replace the 2200000 with your own cpu_max_freq value.

IMPORTANT NOTE: If you have a dual-core CPU, or more than one CPU, you’ll need to add a line for each CPU. For instance:

devices/system/cpu/cpu1/cpufreq/scaling_governor = ondemand
devices/system/cpu/cpu1/cpufreq/scaling_max_freq = 2200000

You can see what all CPU devices you have with ls /sys/devices/system/cpu.

Now, save this file, and you’ll have CPU frequency scaling saving you money, and helping the environment, every time you boot. And with the ondemand governor, chances are you’ll never notice any performance loss.

This article showed you how to save power using CPU frequency scaling on Linux. I have no idea if it’s possible to do the same on Windows, Mac, or the various BSDs, but it would be great if someone would leave comments with links to resources for doing that if so.

Updated: added scaling_max_freq info

The Haskell Blog Tutorial

The first installment of Mark C. Chu-Carroll’s Haskell tutorial series went up last week.

It begins this way:

Before diving in and starting to explain Haskell, I thought it would be good to take a moment and answer the most important question before we start:

Why should you want to learn Haskell?

It’s always surprised me how many people don’t ask questions like that.

Farther down:

So what makes Haskell so wonderful? Or, to ask the question in a slightly better way: what is so great about the pure functional programming model as exemplified by Haskell?

The answer is simple: glue.

Languages like Haskell have absolutely amazing support for modular development.

An interesting and though-provoking article, even for someone that’s been using Haskell for more than 2 years now. (Yikes, I had no idea it was that long)

You can also see all his posts on Haskell, which include a couple more installments.

hpodder to be multithreaded… done right.

I’ll be hacking on my hpodder program this weekend. hpodder is a full-featured podcast aggregator that runs on the command line, and has many features over other command-line podcatchers like bashpodder, and even over GUI tools like iPodder.

I originally envisioned hpodder to be something that I’d cron up and run in the background. But I have tended to run it in the foreground more than in the background. Some others have too, and the requested hpodder feature is parallel downloads.

So I am working on that. I already have code working, in fact, that will parallelize both the hpodder update (downloading the feeds) and the hpodder download (downloading the actual episodes) commands.

Unlike ipodder, my code will make sure that no more than 1 thread will ever be downloading from a given server at a given time. ipodder had the terribly annoying habit of pointing all of its threads at a single server, thus pounding it while also providing little benefit for someone with a pipe fatter than the server’s.

Before all this multithreaded stuff could be written, I needed to write my own status bar code instead of just letting curl display its own status bar. (That wouldn’t work when there are 5 curls running at once)

I decided that I would write some generic status bar code, rather than something specific to hpodder. I took the apt-get status bar as an example, and whipped one up in Haskell and added it to my MissingH Haskell library.

But a status bar just begged for another feature: a generalized progress tracker. Something that could keep track of where a task (and its sub-tasks) are, calculate ETA, estimated time remaining, speed, etc. So I wrote that and made the status bar use it.

AND, a status bar begged for a generalized numeric formatter: something that could render 512 as 512, 2048 as 2K, 1048576 as 1M, etc. So I wrote that, and it’s general enough that it can render into both SI and “binary” units by default (and others that users may want).

Finally, I wrote a function to take a number of seconds and render it in something friendly like 23m5s like apt uses, and shoved that in MissingH as well.

So now hpodder will have a status bar, and any other Haskell program can use the same status bar code in minutes because it’s all generic. Or if someone just needs to render a number in megabytes, they can do that.

I really enjoy it when a program needs a solution that is generic enough to put in a larger library. I try to put as much of my Haskell code in MissingH as I can, so as to make it useful to others (and my other programs).

Desktop Linux: NFS or something else?

Recently, I asked for opinions on desktop Linux. Thanks very much to those that replied. I’ve set up an old laptop as an experiment. I’m using Debian, Gnome, and Systemimager. It’s been an interesting project (especially getting SystemImager and a splash screen program to do what I want).

I’d like for my desktop machines to mount /home over the network. I could use NFS, but of course that has all the well-known security risks. Is there a better network filesystem that is easy to use, fast, and more secure than NFS?

Desktop Linux Opinions?

I’m brainstorming about ways of setting up Linux desktops machines for people used to Window users on a LAN. It could be any size of LAN.

I’d like people to be able to sit down at any Linux machine on the LAN and log in — probably use a LDAP directory for that, and NFS-mounted home directories. I wouldn’t want to NFS-mount the entire thing for performance reasons.

So, some of the things I’m thinking about are:

  • Desktop environment: KDE or Gnome? Which would give Windows users all the tools they’d want? Which would they feel most at home with? I’m thinking it’s KDE, but Gnome has a more polished “feel” too it.
  • Image management. How could the desktops be updated? Just rsync everything except fstab over? Can we actually have a single system image? Is XOrg powerful enough to just recognize hardware at boot and Do The Right Thing? Can we build a unified initrd somehow?
  • Distribution. Debian, Ubuntu, Kubuntu? Do the Ubuntus bring anything to the table, if we take as a given that an experienced Debian admin is managing all this?
  • Laptops. What do we do about the home directories there? Some sort of automated rsync thingy?
  • Installation. FAI? Or some homegrown thing that just boots up, partitions, and runs rsync?

Managing Software

Recently I mentioned that I hate releasing software. It’s true, and I’ve decided that the first part of fixing it is to tackle the presentation of software to the world.

My current scheme of darcs.complete.org for repositories plus bare directories on my gopher (yes, that gopher) site leaves a lot to be desired. There is no bug tracker, there are few screenshots, there is no consistency. It is also not easy to empower others to work on them directly.

At the same time, I am the sole or primary contributor to most of them. These are not huge kernel-sized projects. These are smaller, bite-size projects. So I don’t want or need a lot of overhead. I’ve been thinking about my options.

  • I could just use sourceforge.net. I poked around there today, and all the advertising there is a real eyesore. Plus I figure that if anyone is getting paid for all my hard work, why should it be some random people that no longer write free software? On the other hand, it would be an easy way for my projects to gain visibility. Or I could use alioth, and give up both the advertisements and the larger visibility. But I don’t like giving up control over my site’s appearance, or behing beholden to others for backups, uptime, etc.
  • I could use trac. It’s nice, and is the only option that supports darcs, and has a very cool wiki integrated into everything (even parsing out keywords from changelogs). On the other hand, downloads are — at best — attachments to wiki pages. There is no download manager. And you have to set up a separate trac instance for each project. That is a non-starter for me. If I can’t see all my bug reports at one place, the bug tracker will be too annoying to use.
  • I could use gforge or savane. These are the sourceforge forks. Neither seems to be as resource-hungry as I expected, and debs are available for both. I could just install them locally and use them for my projects, though that seems like overkill. Plus, like SourceForge and Alioth, they have a crappy web-only bug tracker. I’d rather use something like RT that works by email. (Though RT is too resource-intensive to run on my server). However, web-only is better than nothing so I could hold my nose and use it.
  • I could write my own. But I’d rather not, if there’s already something workable out there.

Is anyone else thinking about this? What are your thoughts?

I use more than one computer

I use more than one computer, and quite a bit. I use three regularly, and two or three more on occasion.

But this seems to be a surprise to many programs.

I want to carry certain things with me from machine to machine, access them from anywhere, and have changes propogate across.

Things such as:

  • Bookmarks
  • newsrc files (to mark which Usenet articles are read)
  • mail (solved with my OfflineIMAP program)
  • A small set of files
  • Contacts
  • Calendar/scheduler (appointments)

Now, MacOS X seems to do some of this with their for-pay mac.com service. But I wonder why so few other apps do this out of the box?

The newsrc question is a particularly difficult one to crack, it seems. There are various schemes for synchronizing bookmarks, but none seem to work reliably.

Sigh.

I Hate Releasing Software

I’ve written a bunch of software. I like coding, I like debugging. I like getting e-mail from people that have used my software and are happy.

I don’t like actually having to make a release.

To do a good and proper release of a program, I’d be doing approximately these tasks:

  • Upload to Debian
  • Push to my darcs repo
  • Upload a tar.gz to my server
  • Update a webpage with the latest tar.gz
  • Announce the release to freshmeat
  • Announce the release to a mailing list
  • Update/post screenshots, if things have changed

So I have two wishes. First, I want a tool that maintains a website with software listings. Each program should have its own page, with a description, links to mailing lists, download links, links to the darcs repo, screenshots, etc. It should be simple but I’m too lazy to write it.

Secondly, there should be a tool that will do all of the above tasks (except the screenshots) for me. It should infer the name of the project and the version from the data in my working directory. It should be able to automate this while process without me having to lift a finger.

Sadly, no such thing seems to exist.

And, to date, I’ve been too lazy to write one. Does anyone know of such a thing?

Disk encryption support in Etch

Well, I got my new MacBook Pro 15″ in yesterday. I’ll write something about that shortly. The main OS for this machine is not Mac OS X, though, but Debian.

I decided that, being a laptop, I would like to run dm-crypt on here. Much to my delight, the etch installers support dm-crypt out of the box.

Not only that, but they supported this setup out of the box, too:

  • Two partitions for Debian — one for /boot, everything else on the second one
  • The second partition is completely encrypted
  • Inside the encrypted container is an LVM physical volume
  • Inside the LVM physical volume are logical volumes for /, /home, /usr, /var, and swap
  • XFS is used for each filesystem

Not only that, but it set up proper boot sequence for all of this out of the box, too.

So I turn on the unit, enter the password for the encrypted partition, and then the system continues booting.

Nice. Very nice.

Kudos to the debian-installer and initramfs teams.