Monthly Archives: July 2012

Proof Humans Are Capable of Working Magic

What an astonishing thing a book is. It’s a flat object made from a tree with flexible parts on which are imprinted lots of funny dark squiggles. But one glance at it and you’re inside the mind of another person, maybe somebody dead for thousands of years. Across the millennia, an author is speaking clearly and silently inside your head, directly to you. Writing is perhaps the greatest of human inventions, binding together people who never knew each other, citizens of distant epochs. Books break the shackles of time.

A book is proof that humans are capable of working magic.

— Carl Sagan, The Persistence of Memory

How to get started programming?

I have been asked for advice from several people recently on how to get started programming, or how to further develop a nascent interest in coding or software engineering. The people asking the questions range in age from about 10 years old to older than me. These are people that, for various reasons, are not very easily able to take computer science courses right now.

One would think that, since I’ve been doing this for somewhere around a quarter century (oh I do feel old now), that I’d be ready to offer up some great advice. And offer some suggestions I have. But I’m not convinced they’re good ones.

I have two main tensions. The first is that I, like many in the communities I tend to hang out in such as Debian’s, have a personality that leads me to take a deep dive into details of anything that holds my interest. Whether it’s Linux, Haskell, or amateur radio, I want to do more than skim the surface if I’m having fun with it. Many people are not like that. They may have a lot of fun programming in Visual Basic, not really caring that other languages are out there. Or some people are not like this yet. I feel unqualified to provide good advice to people that are different from me in that way. To put it a different way: most people don’t want to wait 4 years to be useful, and want to start out right away and get better over time (and I was the same way too.)

The second is related. I learned programming at a time when, other than BASIC, interpreted languages were not really available to me. (Yes, they were available, but not to me.) I cut my teeth on BASIC, Pascal, and C. Although I rarely use C anymore, I can still drop into it at a moment’s notice and be perfectly comfortable. I feel it was a fundamentally valuable experience, and that it would be very hard to become a great programmer without ever having lived and breathed something like C, where memory and pointers must be managed manually. Having said that, it is probably possible to become a good coder without ever having touched C.

Here, then, is an edited version of some rambly advice I sent to someone recently, where learning OOP was particularly mentioned. I would welcome your comments and suggestions. I may point people that ask to this post in the future.

For simply learning how to write code, Dive Into Python has long been a decent resource, though it may assume more experience than some have. I haven’t read them myself, but I’ve also heard good things about the How to Think Like a Computer Scientist series from Green Tea Press. They’re all available as free PDF downloads, too!

Eric S. Raymond’s The Art of Unix Programming is another work I’ve heard good things about, despite having never read it myself. A quick glance at the table of contents makes me think that even if people don’t wind up working on Unix, the lessons and philosophy should be informative.

It seems that many Computer Science programs are using Java for the core of their instruction, or even almost exclusively. Whether that is good or bad, I’m not completely sure. It certainly gets people into OOP more deeply, but I’m a “right tool for the job” kind of person. Despite the hype, OO — like everything else — isn’t the right tool for every job.

It is fine for people to dive straight into OO and become good programmers/engineers. However, I think it would be difficult to become a great programmer/engineer without ever having a solid understanding of a more low-level language, such as C in particular. I did my CS work when it was mostly based in C, and am glad for it. If someone never has to manage memory or pointers, I suspect they will be at a disadvantage in the long run for not being able to understand or work with the system at a more fundamental level. If a person knows C, plus some concepts of OO and Functional Programming (FP), it should be easy to pick up just about any other language out there.

I used to think Python was a great first language, but during the 2.x series they added so much fluff and so many special cases that I’m less enthusiastic now, though I don’t know how much of that got cleaned up in 3.x. I am not too keen on Java as a first language, because too many things that should be simple aren’t. I have a fondness for Haskell, and its close relationship to mathematics could make it a great first language — or maybe a poor one, depending on your perspective.

One other thing – I think it’s important for good programmers to have experience with all three major models of programming (procedural, OO, functional.) Even if a person winds up working mostly in one universe, knowledge of and experience with the others is important and informative and, in my experience, leads to better algorithms and architecture all around.

  • Procedural languages: Obviously C, but also Unix shell
  • OO languages: Python, Java, plenty of other fine choices
  • Functional: Lisp, Scheme, Haskell (also the only lazy and pure language on this list)
  • Having said all that, more important than a choice of book or language is experience. I have heard people suggest that it takes 10,000 hours of practice to become a superstar at something, whatever that “something” is, and I wouldn’t doubt it. Seth Godin discusses that a bit, with some criticism of the idea too.

    So that leads to the most important piece of advice: dive in to whatever your interest is. Experiment, write code, put theory into practice in a way that holds interest and excitement. People that try to do things they don’t enjoy don’t seem to stick with them as long or execute as well, and thus will never become great.

A Linux-Based RFID Thing Finder

Sometimes I have things nicely organized. Power adapters for radios in one drawer, for cameras in the next. And sometimes… not so much. Sometimes I’m not sure if things are in the basement or the attic. It seems like technology should be able to help solve this problem, but as far as I can tell, no such solution exists yet.

So I’m planning to build one.

Here’s my general idea. Feedback, of course, is welcome.

Each item to be tracked can have an RFID tag of some sort attached to it. These tags can be read by an RFID reader at a distance of somewhere between 1ft and 1m in typical conditions. The reader could then act as a proximity alert to an object being searched for – a “you’re close” beep, for instance.

That helps, but is only part of the battle. It doesn’t help if you don’t even know which room to look in. So the second part of the plan is that the RFID reader is constantly talking to an object database. Besides the obvious association between RFID tag IDs and object descriptions, the database will also capture background reads of RFID tags. Logged with accurate timestamps, we can then conjecture that RFID reads that occurred within a few seconds of each other are probably physically nearby. If boxes have RFID tags on them, then I can probably get a reasonable idea, down to a box or two, of which box that elusive book is in. If I further put tags on certain immovable physical locations in the house, such as every few feet along shelving, then these will also be captured in the background and hopefully associated with objects nearby, giving a good physical idea of where things are.

What’s more, the simple act of looking for things using the RFID reader would help keep the proximity tables up-to-date, since it could of course log the RFID tags it sees on the way. The only discipline required to keep this info current is to periodically hover nearby storage areas when moving things around.

I like this concept a lot. That doesn’t mean it’s necessarily simple.

Implementation requirements

  • The cost of tagging an item must be less than $0.25 each. Ideally it would be $0.10 or less. We’re talking hundreds or thousands of items here, so even the $2.50 RFID tags for sale on hobbyist sites are way too expensive. RFID tags in industrial bulk quantities are needed, and cheaply.
  • Read range must be at least 1ft, and ideally 3ft, and ideally even around as many obstacles as possible. Shorter than that and it can’t even take in a whole box without pulling things out.
  • Cost kept as reasonable as possible.
  • Must be simple and unobtrusive, requiring little manual effort or discipline to maintain current data. This is the reason for RFID instead of barcodes; barcodes require much more specific action (scan the bin, scan the item) rather then just turn on the scanner and wave it around a bit.

Software side

I’m envisioning the software being split into two components. One would run on an embedded system with the RFID reader. Its job is simple transmitting of scanned RFIDs to the server, receiving instructions from the server, and generating a tone if it’s in search mode and the item being searched is nearby.

On the server lies a database. The database would contain descriptions for the objects that are tagged (so that the tag ID can be looked up). It would also contain the timestamped scan logs.

I envision a simple CGI-based frontend to it that is mobile-friendly, so a laptop, phone, tablet, etc. could be the user interface for the thing – saving the cost of a display and input device by reusing what most people already have.

This is the part I feel most qualified to work on already.

Hardware

The first question is what kind of RFID tags to use. Optimizing for cost per tag, the 800/900MHz UHF tags (EPC gen 2) seem ideal. I have found them in costs approaching $0.10 per tag when bought in rolls of 500 or 1000 Avery RFID labels. That’s reasonable.

The RFID reader is the more complicated part. UHF RFID readers are a lot more costly than their HF or LF counterparts. So far, the cheapest solution I have found started with a post about an Arduino UHF reader. It used a SolidDigi UHF reader board with UART interface for $177. That same board is also available with a USB interface at the same cost.

There is also the need for an antenna. There is a small 5dBi one rated 0-50cm for $8, or a $100 8dBi version rated 1-6m designed for wall mounting. They sell these in kit form that include power adapter, USB cables, etc. as well. The kit with the small antenna runs $192, including reader board.

Next is the compute platform. A DreamPlug might work for this at $160, though both the cost and the power consumption (5VDC 3A) are high. A Raspberry Pi seems perfect, though for whatever reason seem to be backordered by months everywhere and don’t include wifi. The Pi is $35, and the USB wifi is another $30, plus a cheap SD card, so we’re around $80 of computing. The running total of the project, then, is at least $272. Add on provisions for batteries, some sort of case, etc. and we’re probably past the $300 mark. That’s a lot cheaper than the $1000 for handheld RFID readers.

The final item is tags. Bulk tags can be found in the $100 to $200 range, making the total cost of the project $400 to $500. Higher than I’d like, but providing some valuable experience building something.

Risks

There are a few major risks to the project. First among them is read distance. If it’s not long enough, the usefulness of the project will be low. Fortunately, $100 gets a more high-gain antenna and I’d only be out the $8 if the cheaper one doesn’t pan out. But that’s something of a cold comfort, as it’s another, well, $100.

Having a bunch of RFID hardware can be used for all sorts of other interesting things, though, so it could perhaps be reused or repurposed.

Another risk is that RFID collisions wouldn’t be handled as intelligently as I’d like, meaning that the read range required to be useful would activate so many tags that collision algorithms break down. I don’t know enough about RFID collision algorithms to know if this is a serious councern.

Decisions on Listening to Music

A couple of days ago, I commented about my thoughts on finding a better way to listen to music, and asked for suggestions. I’ve checked out the avenues suggested, and here are my thoughts so far.

Streaming service: Spotify

Of the streaming service offerings, Spotify seems the most compelling. It has both Linux and Android clients. The Linux client is unofficial and a bit buggy, but serviceable. The service is fairly nice, and the serendipity of listening to “radio” of tracks like any track or artist I may pick is an incredible feature. It will mean a different mindset; rather than carefully choosing what I buy, I have a vast catalog at my fingertips. I’d have to give up some control, but might be better off for it.

Spotify can play local files, but sadly lacks support for FLAC that most of my collection is in. A little work with the shell, and I had a quick multi-core FLAC to MP3 conversion done. It can also identify local files and match them up with its catalog, but it does a somewhat poor job of it, despite the fact that my files are tagged by Musicbrainz. There are also a fair number of albums in my collection (from Magnatune and from local artists) that aren’t in Spotify’s catalog and probably never will be. I may wind up using git-annex or something like it to sync them between PCs.

Surprisingly, it doesn’t have any kind of web-based player available. It has a lot of features, but many of them are under-documented.

Subsonic / Supersonic

The idea of being able to stream music anywhere certainly has appeal, and the idea of streaming that music from my own machine — my own collection — is quite the appealing one. Subsonic seems to be the standard that people recommend for this. It has a web-based player that works well, as well as mobile clients for various platforms. The Android mobile client is particularly well-regarded and is probably the best one here, aside from Spotify. The playlists work well, as expected, and generally it has the appearance of a well-rounded system. There is a lot of flexibility in bitrates; maximum bitrates can be set per-user and per-client. It, of course, transcodes on the fly.

My chief gripe about Subsonic is that it has no true album/artist/genre browser. Its main browser is mostly a filesystem browser. I have used clients that show album/artist/genre panes or sortable lists, based on tags, for so long that my filesystem organization is rather different – centered around bitrate of rip, origin, etc. It is not very easy to navigate my collection in that way. The search feature can search artist and album, and this can partially make up for it. But there is simply no way to see a list of all artists in the collection.

The server is written in Java, and takes up 362MB at startup. That’s, I guess, decent for a Java server, but doesn’t make me pleased as I’d be running it on lower-end hardware.

Supersonic is a git-maintained actively synced subsonic fork which removes the license cost (the source is GPL3 anyhow so that wasn’t a big deal).

Ampache

Often mentioned in the same breath as Subsonic, this is yet another frequent suggestion for streaming your own collection. Its primary player is web-based, like Subsonic. It has full artist/genre browsers, and adds tag cloud and various other options as well – by default, it loaded the tag cloud with my genre tags, which was a nice touch. The search feature is a little less robust, as it simply searches anywhere and returns a list of all matching tracks, rather than listing matching albums and artists before going into the list of all tracks. But that is somewhat of a minor nit.

Ampache’s web-based player, liks Subsonic’s, uses Flash, but the implementation is really quite awkward. You can’t simply click a play button and have playback start. Rather, you must add things to the “playlist” (queue), a frame on the web interface. Then, you must click play in THAT frame, which launches a new window for the player and transfers its contents to it. You thus have two separate running playlists. And if you think you can just add to the playlist of the player window, think again; you can only replace it. It is simply not possible to add a track to the end of that list without losing your existing place. This makes the usefulness of the web-based player rather, well, low.

Playlist functionality in Ampache is rather odd. You can have playlists, but if you want to add a track to a saved playlist, first you have to clear you current playlist, then add the track to it, then save it to the existing playlist.

Frontends for Ampache exist in banshee and in amarok. Banshee’s is broken, and simply silently fails to do anything. Unless you look in the console, in which case you’ll see a .NET backtrace. Banshee has had these mysterious errors for me for years. The Amarok plugin is so basic that you can browse by artist but not by album, and is almost unusable. There is also a rhythmbox plugin in Debian, but it is apparently for an old version of Rhythmbox and no longer works.

The best frontend is a dedicated Ampache frontend called Viridian. It has a nice interface, provides features such as taskbar integration, and generally works pretty well. The only thing it’s bad at is playlists, and here it’s even worse than the web interface. It doesn’t support modifying the server-side playlists at all. And accessing one is an annoying “load playlist” operation, that replaces your current play queue (also called the “playlist”) with its contents by default.

Several Android clients exist for Ampache, though it seems only one has seen any updates since 2010.

mpd

This is an interesting program, as it is designed more to provide multiple points of control for a single player than it is to stream a single collection to multiple devices. Nevertheless, I tested its HTTP and icecast stream modules. (It can also stream with PulseAudio, but PulseAudio causes devious breakage on my workstation, so I refuse to bother.) This could be made to work reasonably well on my LAN using the gmpc client. There are several Android clients as well, though none of them integrate as well as Subsonic or Spotify. Most seem focused on controlling a remote player. The ones that do stream do so with some hiccups, and don’t integrate controls with the lock screen.

gmpc is a reasonable control program and, combined with mplayer, makes a decent player. It can browse the database every way I’d want to. It doesn’t let me right-click on tracks to add them to playlists, but I can use copy/paste to do so, or add them to the play queue and from there to a playlist. It’s annoying, but not completely unworkable.

But here’s the real concern. mpd has no provision whatsoever for any sort of encryption such as SSL. If I want to be listening to my music collection from on the road, I don’t really wish to open up a streaming music service for the entire Internet. Nor do I really want to mess with SSH tunnels and the like, which are cumbersome on Android (and can be in general). This is pretty much a fatal flaw to me.

Google Music

An interesting service, but ultimately one I don’t see the point for in my situation – where I already have my music on a PC that is always on. It would probably accommodate my entire collection free, but how many weeks would it take me to get it there? And how would it stay in sync? It simply doesn’t offer me any compelling advantage over streaming direct from my PC, so I don’t see much need to bother with it.

Others

I briefly considered Streeme (poor mobile support), Audiogalaxy (no Linux support), Jinzora (appears dead), and Sockso (no mobile support). I also briefly considered other streaming services such as Pandora, Slacker, rdio, etc., but based on reviews didn’t think that they were worth bothering with.

Conclusions

I’m left mainly considering Spotify and Subsonic. For right now, I’m going to try Spotify and see where that leads me, but may wind up getting back to Subsonic if I can’t find tracks I want with Spotify.

How to listen to music?

This seems like a simple question: how do you listen to music?

But in a way, I think I’ve fallen behind the times.

I’ve enjoyed listening to digital music for years, since well before MP3 even existed (MIDI files could be comfortably downloaded over dialup; some of you may even remember MOD files).

Anyhow, a few years ago, I digitized my entire CD collection, ripping it to FLAC and encoding to high-bitrate MP3 for storage on my 160GB iPod. Since, I’ve purchased some music from DRM-free MP3 retailers, and even FLAC files from Magnatune, but at this point I basically say that I have enough music. I have only rarely bought anything new in the last couple of years. I have used Musicbrainz Picard to enforce consistent tagging across almost my entire collection.

I am usually in front of a Linux machine. I run mt-daapd on my fileserver, so I can stream music from it to any other machine in the house. However, this has a lot of drawbacks. It is generally impossible to create or modify persistent playlists with this protocol. I can’t modify metadata. I can’t assign ratings. Though perhaps I don’t need to.

There are a lot of online services I could think of: Pandora, Spotify, Rdio, Slacker. Plus, of course, the upload your music to the cloud services such as Google and Amazon. Have people tried these? I’m hesitant to use a service where I pay a monthly fee, unless I wind up owning the music in a DRM-free fashion. And I’m a little nervous about the privacy implications of uploading a vast amount of music to the cloud.

There are some items in my collection that I am certain are not available on any of those others, but for the most part I’m willing to take what is out there if it meets my tastes. My tastes, by the way, typically run to classical and opera but also can go towards middle eastern or certain new age music (though I’m very picky about that, since I don’t care for most new age stuff I hear.) But these are huge categories; sometimes I’d like to hear some load and noisy classical (say, Beethoven’s Ninth conclusion, some Wagner, Verdi arias, etc.) And other times, something more contemplative (a Mozert concerto) or even quiet (a nice sonata) meet my mood. I have playlists for these on my iPod Classic – a bit of a problem, since I don’t use that device much anymore except in the car, and it takes a LOT of time to categorize things into these playlists. (An opera or a symphony could have parts in various styles and various levels of energy, for instance.) I keep a small manually-managed subset on my Android phone, which works fine. A recommendation engine for things like this could be useful, if it works well.

My requirements, basically: I want something that will play the same music on my Linux machines. I want to be able to have playlists kept in sync across my machines. And I don’t want a monthly fee. Ideally, the software will work on Android as well, and provide similar features there. Anything that requires Windows is a non-starter. I’ve considered NFS mounts, but that’s a bit annoying when the laptop disconnects from the network or when it’s not at home.

What are your thoughts? Ideas?