Category Archives: Software

RedHat Gripes

Linux

Lately we are looking at groupware options, and have been looking at Scalix and Zimbra. We may need the features in the proprietary versions of these products, unfortunately.

So I downloaded an evaluation copy of Scalix.

They say they support RedHat and SuSE. Fine, I think, I’ll just alien the RPMs to debs and be happy.

Not so fast. They have a whole proprietary install system. They check for /etc/redhat_release or /etc/SuSE_release (or something like that) and do different things depending on what is there. Ugh. Why can’t these proprietary vendors just target LSB? The differences seem mostly related to init anyway.

So I touch /etc/SuSE_release into existence, run the installer again. It complains that DISPLAY is not set. UGH. I log in with ssh forwarding, to root (sigh), and run it again.

Now it complains that the SuSE_release file doesn’t contain a valid release. I google a bit, but the file format doesn’t seem to be documented anywhere. I extract it from an RPM somewhere, but no luck.

So, I figure at this point, let’s try an actual RPM distro. I’m running this in a Xen domain anyway, so it should be no big deal, right?

I think CentOS will be a good choice. It’s RHEL with the non-free stuff stripped out. And they support RHEL and don’t need any non-free stuff. I google, and find instructions for installing via rpmstrap for Xen uses.

Let me say, rpmstrap is not nearly the nice tool that cdebootstrap is. rpmstrap totally hosed the networking on the Xen host machine, requiring me to reboot to get it back to proper state. The resulting install wouldn’t boot, either — I later found out that, even though I listed explicit devices in /etc/fstab like usual, it requires labels on all my partitions to boot. Ugh. There are a host of other problems with the rpmstrap-installed chroot, and it’s broken beyond my ability to repair due to problems with the rpm database.

So then I downloaded the “Server” CD for CentOS, which is supposed to have just the stuff a person would need for a server, and leave off all the graphical tools, multimedia, etc. I fired up VMware and did an install. Then I booted Debian From Scratch in VMware and used tar and netcat to copy the installed image over to Xen.

I got it booting fairly easily. But now I start to remember why I had this instinctive gag reflex last time I used RHEL.

First off, the network configuration, by default, is tied to the MAC address of your ethernet card. So if you replace your Ethernet card, your network is broken by default.

Then, there’s the way the network is brought up. It uses arping as part of its procedure to bring up a NIC. If it sees a reply anywhere on the network with the IP you’re trying to assign, it leaves the NIC half-up — it’s been ifconfig’d up, but without an IP. So that’s right, if somebody happens to have a rogue device plugged in at the moment your server boots, your server will come up without a network configured. This is *Enterprise* Linux and it’s pulling this sort of thing. Terrible design.

Next, there’s the way the network is *configured*. There are commands such as system-config-network-tui, -gui, -cmd, -druid, etc. I go for -tui. to start with. It’s a dialog-like interface, and asks the basics like IP address, etc. It doesn’t have any way to configure more than one Ethernet card that I can tell. And some of the settings — like nameserver — apparently require you to press F12 to visit. But the program doesn’t recognize F12 as sent by an xterm, so it doesn’t work.

All the other options require X. So, I reluctantly ssh -X into it as root and run system-config-network-gui. It doesn’t work — complains it can’t find DISPLAY. Strange, I think; DISPLAY is set properly to localhost:whatever. It turns out that /etc/hosts is empty by default, so the thing can’t resolve localhost! Argh. I add a line to /etc/hosts and it fires up.

This tool works decently. I save, uncheck the tie to a MAC address box, and exit. I then think it might be good to fire it up again and see what it did. I try running it again, and get the same error about DISPLAY. The stupid tool blew away /etc/hosts and replaced it with an empty file! This is NOT what I would expect from an Enterprise Linux. You don’t blow away a config file the administrator touched without asking, EVER.

Next, I figure, let’s try installing the XFS tools so I can switch the root filesystem to xfs. I start with “yum update”, which doesn’t quite do what I expect. (It is more like apt-get update && apt-get -u dist-upgrade) So I hit Ctrl-C, but — surprise — IT DOESN’T WORK. I press it a few more times, and it seems to just make the downloader cycle through mirrors because of a “download error”. So I hit Ctrl-Z and kill %1. I have my prompt, but it’s STILL DOWNLOADING STUFF and spewing all over my console. Ugh.

I finally use ps and kill -9 and eventually get it killed off. Stupid thing.

I don’t understand why anybody would want to use RedHat Enterprise Linux in an enterprise. It seems more suited to a hobbyist system at home. From reading some forums, it seems there are quite a few people out there using Debian for enterprise systems for similar reasons.

So now, maybe I’ll have the chance to actually try Scalix.

(BTW, our intern got Zimbra installed on Debian just fine, so that’s a plus for it.)

Multipath is working

Linux

Yesterday, we got multipath working with our HP MSA1500cs SAN. We have a fully redundant setup with redundant controllers, fibre channel switches, and two FC controllers per host.

We had been having a lot of trouble getting things to work right with active/passive controllers. We could get failover to work in some cases, but getting everything to communicate correctly in the event of a failure was difficult, since every machine would have to flip over to the passive controller simultaneously.

With a firmware upgrade, the MSA 1500cs can support active/active controllers. With the dual-active setup, both controllers are active simultaneously and both are valid paths.

Despite HP support’s indications to the contrary, HP does have information on using built-in multipathd in Linux instead of their proprietary multipath solution. It’s document c00635587, part AA-RW8RA-TE.

We’ve configured multipathd.conf like this:

      path_grouping_policy  multibus
      path_checker              tur
      failback                  immediate
      no_path_retry             60
      path_selector             "round-robin 0"

Just put that in your default block and it should work.

Announcing hpodder

Software, ,

Today I’m finally announcing hpodder.

I’ve been trying different podcatchers in Linux, and have been generally unhappy. ipodder looked nice at first, but turned out to be horribly buggy.

bashpodder/podracer looked like a nice idea. However, it didn’t have enough flexibility for me, its XML parser has some well-known failures (it’s not a real XML parser, after all), etc.

So I wrote hpodder. hpodder is a command-line podcast downloader for Linux. It features:

  • Extensive manual (installed as manpage, or you can view the PDF versoin). Documents all command-line options, the config file, a quick start, plus some basic information about the internal database
  • Database of seen URLs (in Sqlite3) — for use both for downloads and when processing feeds
  • Graceful handling of Ctrl-C, shutdowns, network troubles, etc — including ability to resume downloads later, plus the ability to detect servers that don’t handle download resuming properly (libsyn)
  • Automatic setting of ID3 tags based on the episode title and podcast title from the podcast’s feed (as iTunes does) — dramatically helps with viewing of all sorts of podcasts on the iPod and your PC
  • Support for download rate limits, progress bars, etc. via Curl
  • Seems to be stable for me
  • Command-line tools to: add new podcasts, remove podcasts, update podcast feed URLs, scan podcast feeds, list known podcasts & status, list known episodes & status, alter episode status (mark for downloading or not), “catch up” podcasts, etc.
  • Automatic retry of downloads that failed due to transient errors

You can download a source tarball, or apt-get install hpodder if you run Debian sid.

hpodder is written in Haskell, and calls the curl and id3v2 binaries. It uses the Sqlite3 library and my HDBC database interface for Haskell.

But you’d never need to know or care about that unless you’re a programmer.

In future hpodder versions, I intend to improve the download status display, add last-seen date tracking, and add multithreaded downloading.

Why Is Expensive Software So Crappy?

Software

Today I was listening to Gary McGraw on Frontline Security talking about software security. One of his points was that a large part of security trouble is poor design.

That reminds me how I’ve been meaning to rant a bit about the really terrible security I’ve seen in proprietary software lately. Some of this is very expensive software that people pay lots of money for.

  • World-writable installations. In one case, the documentation for the software directed users to mark the entire program’s directory tree world-writable, including all files and directories within it. In a whole host of additional cases, consultants or support people tasked with installing the software make everything world-writable as a matter of routine. And some of these are programs specifically designed to be used on Unix shell hosts.
  • Overuse of telnet. There’s telnet use everywhere. One program actually telnets to a server to start their own server-side component and then send XML to it. So we have to have an account for the server-side component, put the password in plain text on the client side, and maintain *telnet* for the application. Have we never heard of, say, CGI, people???
  • Overuse of root. Again, I’ve seen this even in documentation — “run everything as root” or “if you have trouble, just run this as root.” I’ve seen installers actually check to see if they’re running as root, and fail if they’re not, even though they have no need for root privileges.

Sigh. Although I’ve seen some poor code out there in the Free Software community, I’ve never seen anything that even approaches this level of insanity.

Why is the most expensive software the least secure? And what can we do about it when the vendor doesn’t care?

HDBC 1.0.0

Software

This evening, I released HDBC 1.0.0.

HDBC is a database API for Haskell. I wrote it after being unsatisfied with HSQL. HDBC at a certain level feels similar to Perl’s DBI. But it is both simpler and more powerful, IMHO, thanks in large measure to Haskell language features.

The HDBC homepage is here.

How to solve “The following packages cannot be authenticated”

Debian, , ,

Users of Debian’s testing or unstable distributions may be noticing messages from apt saying things like:

WARNING: The following packages cannot be authenticated!
  foo bar baz
Install these packages without verification [y/N]?

I noticed today that google doesn’t turn up good hits for the fix. The fix is really simple:

apt-get install debian-archive-keyring
apt-get update

That’s it. You now have secure packages from Debian. Nice, eh?

An iPod under Linux

Hardware, Music, Software, ,

I finally purchased my first iPod: a black 60GB iPod video model. I had been holding off for years. The iPod sounded nifty, but I just didn’t quite go there.

The thing that finally won me over was the camera connector. It lets you plug your iPod directly in to a digital camera. The iPod can download photos from the camera to its internal disk without the need for a PC. Very slick.

So anyway, we got the iPod and the camera adapter at the Apple store in Cambridge — a quick subway ride from Usenix. They were out of stock on the FM tuner, so I ordered that online.

The next step was to get the iPod working with Linux. I currently have it working with both music and video. Here’s how I did it.

First Thougts on Xen

Software

At work, we’ve been using vserver for virtualization for some years now. Due to various reasons, we’re looking at Xen.

I’ve been trying to switch my workstation to use Xen. I’ve enountered a few issues so far. Probably these will go away as I learn the system.

Overall, my greatest gripe is the documentation. It is outdated and just plain wrong far too often. For instance, there’s a place where it says to run “make ARCH=xen xconfig”, but the Xen kernel patches don’t (any more, at least) provide a xen arch.

The next gripe is the very weird kernel build system. Xen doesn’t ship a diff against a kernel tree. They instead ship whole files to extract atop a particular kernel version. Annoying and unwieldy. There is a command to generate a diff, but you have to download the full kernel tree first.

A couple of other gripes: There is little documentation on memory management (can Xen adjust the RAM usage of running VMs?), on 64-bit systems (can you run a 32-bit kernel under a 64-bit hypervisor? how about a 64-bit kernel that supports 32-bit userspace?)

I’m also having trouble with my forcedeth card locking up under Xen.

However, I’ve heard of lots of people having good luck with it so I’m going to keep trying.

But one would think that basic docs could be actually worked on a bit more.

Announcing HSH, the Haskell Shell

Programming

Following the “release early, release often” motto, I am happy to announce version 0.1.0 of HSH, the Haskell shell.

You may obtain it with:

darcs get --tag 0.1.0 http://darcs.complete.org/hsh

Things are still very rough in many ways, but this version already lets you:

  • Run commands
  • Pipe things between commands
  • Pipe command input/output into and out of pure Haskell functions
  • Pure Haskell functions are as much a first-class citizen as is grep or cat

Here is an example session: (some lines wrapped for readability)

$ ghci -fglasgow-exts HSH

*HSH> run $ ("ls", ["."])
COPYING    HSH        HSH.hs    TODO    announcements  testsrc
COPYRIGHT  HSH.cabal  Makefile  _darcs  test.hs

*HSH> run $ ("ls", ["-l"]) -|- ("wc", ["-l"])
12

*HSH> :m +Text.Printf
*HSH Text.Printf> let countLines = (zipWith (\i line -> printf "%-5d %s" i line) 
       [(1::Int)..])::([String] -> [String])

*HSH Text.Printf> run $ ("ls", ["-l"]) -|- countLines -|- ("grep", ["hs$"])
6     -rw-r--r-- 1 jgoerzen jgoerzen  1285 Jun  6 09:43 HSH.hs
11    -rw-r--r-- 1 jgoerzen jgoerzen   565 Jun  6 09:43 test.hs

*HSH Text.Printf> :m +Data.List
*HSH Text.Printf Data.List> run $ ("ls", ["-l"]) -|- countLines -|- 
         filter (isSuffixOf "hs")
6     -rw-r--r-- 1 jgoerzen jgoerzen  1285 Jun  6 09:43 HSH.hs
11    -rw-r--r-- 1 jgoerzen jgoerzen   565 Jun  6 09:43 test.hs

*HSH Text.Printf Data.List> run $ ("ls", ["-l"]) -|- countLines -|- filter (isSuffixOf "hs") 
       -|- ("tr", ["a-z", "A-Z"])
6     -RW-R--R-- 1 JGOERZEN JGOERZEN  1285 JUN  6 09:43 HSH.HS
11    -RW-R--R-- 1 JGOERZEN JGOERZEN   565 JUN  6 09:43 TEST.HS

*HSH Text.Printf Data.List> let generator = \(_::String) -> unlines . map show $ [1..20]
*HSH Text.Printf Data.List> generator ""
"1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14\n15\n16\n17\n18\n19\n20\n"
*HSH Text.Printf Data.List> run $ generator -|- ("grep", ["1"])
1
10
11
12
13
14
15
16
17
18
19

Future versions will likely simplify syntax to make it easier to write scripts and introduce a sh to hsh converter. I also plan to add pure Haskell tools for some common shell-ish things that one could do in Haskell.

Bacula

Software, ,

Lately we’ve been looking at backup solutions at work.

And I’ve got to say that Bacula is looking downright awesome. It’s GPL’d and it has just about every feature a person could ask for.

I am a complete Bacula newbie. Today, after using Bacula for a total of about 30-60 minutes, I added the first client machine to my Linux test box. The client machine was running the Windows bacula client. It took about 10 minutes to install and configure the client and the server. And both backup and restore worked perfectly the first time. Nice. Setting up a *nix client is even easier.

I’ve been using Amanda for many years at home and at various workplaces. Looks like we’re going to be switching.

We’ve also ordered an HP MSL4048, a 48-tape LTO3 library with barcode support. Each tape has a native storage capacity of 400GB. Should be nice when it arrives. With that library and Bacula, we should be able to back up all our servers using a single backup system. And both our Windows and Unix people can manage the system, including running restores to any machine, from any authorized console machine.