All posts by John Goerzen

Announcing hpodder

Software, ,

Today I’m finally announcing hpodder.

I’ve been trying different podcatchers in Linux, and have been generally unhappy. ipodder looked nice at first, but turned out to be horribly buggy.

bashpodder/podracer looked like a nice idea. However, it didn’t have enough flexibility for me, its XML parser has some well-known failures (it’s not a real XML parser, after all), etc.

So I wrote hpodder. hpodder is a command-line podcast downloader for Linux. It features:

  • Extensive manual (installed as manpage, or you can view the PDF versoin). Documents all command-line options, the config file, a quick start, plus some basic information about the internal database
  • Database of seen URLs (in Sqlite3) — for use both for downloads and when processing feeds
  • Graceful handling of Ctrl-C, shutdowns, network troubles, etc — including ability to resume downloads later, plus the ability to detect servers that don’t handle download resuming properly (libsyn)
  • Automatic setting of ID3 tags based on the episode title and podcast title from the podcast’s feed (as iTunes does) — dramatically helps with viewing of all sorts of podcasts on the iPod and your PC
  • Support for download rate limits, progress bars, etc. via Curl
  • Seems to be stable for me
  • Command-line tools to: add new podcasts, remove podcasts, update podcast feed URLs, scan podcast feeds, list known podcasts & status, list known episodes & status, alter episode status (mark for downloading or not), “catch up” podcasts, etc.
  • Automatic retry of downloads that failed due to transient errors

You can download a source tarball, or apt-get install hpodder if you run Debian sid.

hpodder is written in Haskell, and calls the curl and id3v2 binaries. It uses the Sqlite3 library and my HDBC database interface for Haskell.

But you’d never need to know or care about that unless you’re a programmer.

In future hpodder versions, I intend to improve the download status display, add last-seen date tracking, and add multithreaded downloading.

Week 2 (ending Jul 8)

House Renovation

The big news this week: the kitchen is gone and the wash house was saved!

On Monday, I went out to take the week 1 pictures and clear out some junk from the wash house.

Workers started Wednesday. We thought we’d have to tear down the wash house, but they managed to save it! It’ll make a nice outbuilding somewhere.

They pulled down the kitchen/bathroom. So things are looking very different already.

Check out the week 2 photos. Here’s a sample:

Renovation: Week 1

House Renovation

I’m a week late in posting these, so pretend this was posted July 1.

We bought the place on June 27. The next day, somebody was already out removing the chimney from the house. Then over the weekend, dad and I went out there to remove the chimney from the wash house.

Our contractor happened by while we were out. He had heard that some concrete blocks would be available at an auction, so he bought them for us for $2 for the whole lot. Should save us some money.

Photos of the place after week 1 are now available.

Here’s one, featuring a vulture sitting on top of the barn:

Why Is Expensive Software So Crappy?

Software

Today I was listening to Gary McGraw on Frontline Security talking about software security. One of his points was that a large part of security trouble is poor design.

That reminds me how I’ve been meaning to rant a bit about the really terrible security I’ve seen in proprietary software lately. Some of this is very expensive software that people pay lots of money for.

  • World-writable installations. In one case, the documentation for the software directed users to mark the entire program’s directory tree world-writable, including all files and directories within it. In a whole host of additional cases, consultants or support people tasked with installing the software make everything world-writable as a matter of routine. And some of these are programs specifically designed to be used on Unix shell hosts.
  • Overuse of telnet. There’s telnet use everywhere. One program actually telnets to a server to start their own server-side component and then send XML to it. So we have to have an account for the server-side component, put the password in plain text on the client side, and maintain *telnet* for the application. Have we never heard of, say, CGI, people???
  • Overuse of root. Again, I’ve seen this even in documentation — “run everything as root” or “if you have trouble, just run this as root.” I’ve seen installers actually check to see if they’re running as root, and fail if they’re not, even though they have no need for root privileges.

Sigh. Although I’ve seen some poor code out there in the Free Software community, I’ve never seen anything that even approaches this level of insanity.

Why is the most expensive software the least secure? And what can we do about it when the vendor doesn’t care?

HDBC 1.0.0

Software

This evening, I released HDBC 1.0.0.

HDBC is a database API for Haskell. I wrote it after being unsatisfied with HSQL. HDBC at a certain level feels similar to Perl’s DBI. But it is both simpler and more powerful, IMHO, thanks in large measure to Haskell language features.

The HDBC homepage is here.

Mario Cuomo against Ronald Reagan

Politics, , ,

I finally have episode 3 of my podcast Sound of History online.

Episodes 1 and 2 both had two speeches each, and one of those speeches in each episode was from Ronald Reagan. So, for episode 3, I selected Mario Cuomo’s address to the 1984 Democratic National Convention. Quite an interesting listen.

It was quite interesting to hear how the rhetoric 20 years go, from both sides, is quite similar to today’s rhetoric.

Now I’m a little annoyed at HP

Hardware, ,

So, a little while ago, I wrote about why I like HP. This week, I’m starting to be annoyed at them.

My employer just bought nearly $100,000 worth of HP hardware. We get a new MSA1500cs Fibre Channel SAN (with redundant controllers, FC switches, disks, etc), a new blade enclosure system, three blades to start with (all of them, at minimum, dual dual-core Opterons with 4GB RAM, and some considerably more), a rack to put all this in, etc.

So we’re starting to set all this stuff up. I’ve got Debian installed on an NFS root for testing the blades and how they interact with the SAN.

The blades have an integrated dual-port QLogic QLA2312 Fibre Channel adapter. The Linux kernel has a built-in driver for this (qla2xxx), which detects it and, so far at least, works fine. We want to run kernel 2.6.17 because it’s the first version where XFS has decent semantics for write ordering to prevent corruption after a power failure. Plus we want at least a 2.6.16.x kernel because we want to run the latest Xen 3.0 on these blades. (Live migration of virtual servers from blade to blade — this will be great.)

But we learn that HP does not support the kernel qla2xxx driver. HP does not say WHY they don’t support it, just that their own driver is the only one that they support.

After plowing through several annoying scripts to get to their driver, I realize why it fails to install: it is OLD. At BEST, 2.6.14 is the most recent kernel it would even compile against (release date: October 2005), and I think the most recent version it supports is more like 2.6.8 (almost TWO YEARS OLD now). They reference a whole bunch of kernel symbols and macros that were removed somewhere between 2.6.8 and 2.6.17.

I sent a ticket to HP support. Their first request was to run their system information gathering tool and send them the results. Fine, that’s reasonable. I did so. Next they say, gee, you’re running Debian, and we don’t support that.

Argh…. If they tried to compile it against 2.6.17.1 on RedHat or SuSE, they’d get the exact same problem. I told them what symbols they were erroneously using, and a simple grep would have showed them that.

Besides, how many customers are going to be pleased with no upgrade path available for 2 years? I wouldn’t want our kernel version to be held hostage to HP’s slow driver development process.

Sigh.

Things are a happenin’

General,

So… Our big move (yes, we’re buying a farm) is moving along!

We’re tentatively set to close on it next Tuesday. The last two months have been busy. A lot of phone calls to arrange permits.

The director of planning and zoning retired recently, and they didn’t have a replacement ready. So all building permits were on hold until a replacement arrives. In the meantime, the county clerk’s office was trying to handle basic inquiries. I called it one day, and didn’t realize until later that I was speaking to the county clerk. I grew up in this county but obviously I’ve lived in larger ones too long if I’m surprised about this. Everyone was very friendly and as helpful as possible with an unfortunate situation.

I got a surprise phone call today. Apparently the telephone company is running new lines in rural areas of the county (upgrading to support DSL) and they wanted to know if that place was going to stay vacant. A “not at all!” later, and it looks like we’ll be able to get DSL by the time we move in. WOOHOO! We were bracing for dialup or wireless…

There are still some bits of paperwork that have to get done before Tuesday yet, so it’s not 100% guaranteed that we’ll close then. But we’re getting excited anyway!

New hosting provider: VPSLink

Reviews, Technology, , ,

Thanks to all of the helpful comments people left in response to my last post about hosting. I got some really helpful hints from them — including the one I eventually followed, to VPSLink.

I wound up going with VPSLink. Their prices are amazing and the performance is good, too.

I have to admit — I shot myself in the foot not once, not twice, but three times. The shorewall config I was using on my old VPS (which was hosted under UML) apparently doesn’t work well under OpenVZ (used at VPSLink). By “doesn’t work well”, I mean “blocks all traffic to or from the host on startup”.

So, I had rsynced over everything from my old host to the new, and rebooted the new. But it didn’t come back up. I was pretty sure this was why. I dropped off a ticket to the VPSLink folks asking them to please rm /etc/init.d/shorewall for me.

60 minutes later, they had done it. (And it looks like a bug in their ticket system prevented it from being flagged as “emergency” — they said they would have done it faster otherwise.)

I then tried to fix shorewall, and it looked like it was working, so I put the init script back and rebooted. Same problem! They fixed it again in about 60 minutes. (The ticket still had normal priority)

Finally, I deleted shorewall entirely, then rsynced my old host to the new one. Things looked good, so I rebooted…. and yes, guess what, that rsync brought back shorewall so it got hosed YET AGAIN. ARGH. This was now well into the overnight hours, but they still helped me out.

All in all, I am so far VERY impressed with VPSLink. I heartily recommend them. I’ll be sure to post updates as time goes by.

Regarding Memset, my current provider: Really great company. I heartily recommend them, too. My needs for RAM just wouldn’t be ecnomical under their current plans, but they are usually competitive. I also have experience with JohnCompanies. I used them way back when they only did FreeBSD VPSs, on through the Linux days, and we currently use their services at work. These are also great folks and I would recommend them to anyone. Like Mako’s suggestion of Rimuhost, both memset and JohnCompanies are “quality first” providers — not necessarily cheapest, but their systems work as advertised and are almost never down, and they support you with experienced Unix admins. BTW, Memset’s current plans use Xen and JohnCompanies uses Virtuozzo.