Category Archives: MythTV

MoinMoin as a Personal Wiki, Zen To Done, And A Bit of Ikiwiki

MythTV, , , ,

Since I last evaluated and complained about wikis last year, I’ve been using moinmoin for two sites: a public one and a personal one.

The personal site has notes on various projects, and my task lists. I’ve been starting out with the Zen To Done (ebook, PDF, paper) idea. It sounds great, by the way; a nice improvement on the better-known GTD.

My To Do Page

Anyhow, in MoinMoin, I have a ToDos page. At the top are links to pages with different tasks: personal, work, yard, etc. Below that, are the three “big rocks” (as ZTD calls them) for the day: three main goals for the day. I edit that section every day.

The Calendar

And below that, I use MoinMoin’s excellent MonthCalendar macro. I have three calendars in a row: this month, next month, and last month. Each day on the calendar is a link to a wiki page; for instance, ToDos/Calendar/2009-10-01. The day has a red background if the wiki page exists, and white otherwise. So when I need to do something on or by a specific day, I click on the link, click my TaskTemplate, and create a simple wiki page. When I complete all the tasks for that day, I delete that day’s wiki page (and can note what I did as the log message if I like). Very slick.

The Task Lists

My task pages are similar. They look like this:


= Personal =

<<NewPage(TaskTemplate,Create new task,@SELF)>>

<<Navigation(children,1)>>
<<BR>>

So, my personal task page has a heading, then it has an input form with a text box and a button that says “Create new task.” Type something in and that becomes the name for a wiki page, and takes you do the editor to describe it. Below the button is a list of all the sub-pages under the Personal page, which represent the tasks. When a task is done, I delete the page and off the list it goes. I can move items from one list to another by renaming the page. It works very, very nicely.

Collecting

Part of both ZTD and GTD is that it must be very easy to get your thoughts down. The idea is that if you have to think, “I’ve got to remember this,” then you’ll be stressed and worried about the things you might be forgetting. I have a “Collecting” page, like the Personal or Work pages, that new items appear on when I’m not editing my wiki. They get there by email.

MoinMoin has a nice email system. I’ve set up a secret email address. Mail sent there goes directly into MoinMoin. It does some checks on it, then looks at a combination of the From and Subject lines to decide what to do with it. If I name an existing page, it will append my message the the end. If it’s a new page, it’ll create it. I have it set up so that it takes the subject line as a page name to create/append to under ToDos/Collecting/$subject (by putting that as the “name” on the To line).

So, on my computers, I have a “newtodo” script that invokes mail(1), asks for a subject, and optionally lets me supply a body. Quick and painless.

Also, I’ve added the address to my mobile phone’s address book. That way I don’t have to carry around pen and paper. Need to get down some thought ? No problem. Hit send email, pull the last address sent to, give it a subject and maybe a body. Very slick.

Wiki Software

As a way of updating my posts from last year: I’ve been very happy with MoinMoin overall. It has some oddities, and the biggest one that concerns me is its attachment support. It doesn’t let you specify a maximum upload size, and doesn’t very well let you restrict attachment work to only certain people. But the biggest problem is that it doesn’t track history on attachments. If a vandal deletes the attachment on a page, it’s GONE. They expect to have that fixed in 2.0, coming out in approximately November, 2010.

I also looked at Ikiwiki carefully over the past few days. Several things impressed me. First, everything can be in git. This makes for a very nice offline mode, better than Moin’s offline sync. The comment module is nicer than anything in Moin, and the tagging system is as well. Ikiwiki truly could make a nice blog, and Moin just couldn’t. It also puts backlinks at the bottom of each page automatically, a nice feature. And it’s by Joey Hess, who writes very solid software.

There are also some drawbacks. Chief on that list is that ikiwiki has no built-in history of a page feature. Click History and it expects to take you to gitweb or ViewVC or some such tool. That means that reverting a page requires either git access or cut and pasting. That’s fine for me, but throwing newbies to gitweb suddenly might not be the most easy. Since ikiwiki is a (very smart) wiki compiler, its permission system is a lot less powerful than Moin’s, and notably can’t control read access to pages at all. If you need to do that, you’d have to do it at the webserver level. It does have a calendar, but not one that works like Moin’s does, though I could probably write one easily enough based on what’s there.

A few other minor nits: the email receiving feature is not as versatile as Moin’s, you can’t subscribe to get email notifications on certain pages (RSS feeds only, which would have to be manually tweaked later), and you can’t easily modify the links at the top of each page or create personal bookmarks.

Ikiwiki looks like an excellent tool, but just not quite the right fit for my needs right at the moment. I’ve also started to look at DokuWiki a bit. I was initially scared off by all the plugins I’d have to use, but it does look like a nice software.

I also re-visited MediaWiki, and once again concluded that it is way too complicated for its own good. There are something like a dozen calendar plugins for it, some of which even are thought to work. The one that looked like the one I’d use had a 7-step (2-page) installation process that involved manually running SQL commands and cutting and pasting some obscure HTML code with macros in it. No thanks.

How To Record High-Definition MythTV Files to DVD or Blu-Ray

MythTV,

I’ve long had a problem. Back on January 20, I took the day off work to watch the inauguration of Barack Obama. I saved the HD video recordings MythTV made of the day (off the ATSC broadcast), intending to eventually save them somehow. I hadn’t quite figured out how until recently, so there they sat: 9 hours of video taking up about 60GB of space on my disk.

MythTV includes a program called mytharchive that will helpfully transcode your files and burn a DVD from them. But it helpfully will transcode the beautiful 1920x1080i down to DVD resolution of — on a good day — 720×480. I couldn’t have that.

Background

My playback devices might be a PC, the MythTV, or the PlayStation 3. Of these, I figured the PS3 was going to be hardest to accommodate.

ATSC (HD) broadcasts in the United States are an MPEG Transport Stream (TS). Things are a bit complicated, because there may be errors in the TS due to reception problems, the resolution and aspect ratio may change multiple times (for instance, down to SD for certain commercials). And, I learned that some ATSC broadcasts are actually 1920×1088 because the vertical resolution has to be a multiple of a certain number, and those bottom 8 pixels shouldn’t be displayed.

Adding to the complexity, one file was 7 hours of video and about 50GB itself. I was going to have to do quite some splitting to get it onto 4.7GB DVD+Rs. I also didn’t want to re-encode any video, both for quality and for time reasons.

Attempts

So, I set out to try and figure out how to do this. My first approach was the sledgehammer one: split(1). split takes a large file and splits it on byte or line boundaries. It has no knowledge of MPEG files, so it may split them in the middle of frames. I figured that, worst case, I could always use cat to reassemble the file later.

Surprisingly, both mplayer and xine could play back any of these files, but the PS3 would only play back the first part. I remembered this as an option if all else failed.

Next, I tried avidemux. Quite the capable program — and I thought I could use it to cut my file into DVD-sized bits. But I couldn’t get it to let me copy the valid MPEG TS into another MPEG TS — it kept complaining of incompatible formats, but wouldn’t tell me in what way they were incompatible. I could get it to transcode to MPEG4, and produce a result that worked on the PS3, but that wasn’t really what I was after.

Then, I tried mpgsplit. Didn’t recognize the MPEG TS as a valid file, and even when I used a different tool to convert to MPEG PS, acted all suspicious as if it bought the MPEG from a shady character on a grungy street corner.

dvbcut

I eventually wound up using dvbcut to split up the ATSC (DVB) recordings. It understood the files natively and did exactly what I wanted. Well, *almost* exactly what I wanted. It has no command-line interface and didn’t have a way to split by filesize, but I calculated that about 35 minutes of the NBC broadcast and 56 minutes of the PBS broadcast could fit on a single DVD+R.

It worked very, very nicely. The resulting files tested out well on both the PS3 and the Linux box.

So after that, I wrote up an index.txt file to add to each disc and a little shell scripting later and I had a directory for each disc. I started burning them with growisofs. Discs 1 and 2 burned well, but then I got an error like this:


File disc06/VIDEO/0930-inaug-ksnw-06.mpg is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.

Eeeeepp. So apparently the ISO 9660 filesystem can’t represent files bigger than 4GB. My files on disc 1 had represented multiple different programs, and stayed under that limit; and disc 2’s file was surprisingly just a few KB short. But the rest of them weren’t. I didn’t want to have to go back and re-split the data to be under 4GB. I also didn’t want to waste 700MB per disc, or to have to make someone change video files every 15 minutes.

So I decided to investigate UDF, the filesystem behind Blu-Ray discs. mkisofs couldn’t make a pure UDF, only a hybrid 9660/UDF disc that risked compatibility issues with big files. There is a mkudffs, but it doesn’t take a directory of its own. So I wrote a script to do it. Note that this script may fail with dotfiles or files with spaces in them:

#!/bin/bash

set -e

if [ ! -d "$1" -o -e "$2" -o -z "$2" ]; then
   echo "Syntax: $0 srcdir destimage [volid]"
   echo "destimage must not exist"
   exit 5
fi

if [ "`id -u`" != "0" ]; then
   echo "This program must run as root."
fi

EXTRAARGS=""
if [ ! -z "$3" ]; then
   EXTRAARGS="--vid=$3"
fi

# Get capacities at http://en.wikipedia.org/wiki/DVD+R as of 9/27/2009

SECSIZE=2048

# I'm going to set it a few lower, than capacity 2295104 just in case.
# Must be at least one lower than the actual size for dd to do its thing.
# SECTORS=2295103
SECTORS=2295000

echo "Allocating image..."

dd if=/dev/zero "of=$2" bs=2048 "seek=$SECTORS" count=1

echo "Creating filesystem..."

mkudffs --blocksize=2048 $EXTRAARGS "$2"
mkdir "$2.mnt"

echo "Populating..."
mount -o rw,loop -t udf "$2" "$2.mnt"
cp -rvi "$1/"* "$2.mnt/"
echo "Unounting..."
umount "$2.mnt"
rmdir "$2.mnt"

echo "Done."

That was loosely based on a script I found on the Arch Linux site. But I didn’t like the original script, because it tried to do too much, wasted tons of time writing out 4.7GB of NULLs when it could have created a sparse file in an instant, and was interactive.

So there you have it. HD broadcast to playable on PS3, losslessly. Note that this info will also work equally well if you have a Bluray drive.

Don’t Buy Albatron

Hardware, MythTV

My MythTV box has an Albatron KM18GPro motherboard in it. Last week, the Ethernet port on it went dead. I can plug in a known good cable, and don’t even get LED activity on the port. Plug the same cable into any number of other machines, and it works fine. (This is repeatable across different cables and switches, too.)

So I submitted an online RMA request to Albatron. Despite their claim of “instant confirmation” when an e-mail address is supplied, after submitting the form, it says to call them if there’s no response after 48 hours. I also submitted the proof of purchase as they requested.

Since there wasn’t, I called them and got a voice mail box. Left a voice mail. It was never returned. I also e-mailed their RMA team. No response. I’m sure this has nothing to do with the fact that my warranty expires in a few weeks.

It’s now been days since I first contacted them. This is absolutely the worst hardware support I’ve ever received from any vendor — taking days to even *respond* to a problem. I recently had a problem crop up with my Sipura SPA-841 phone, and got a replacement on its way to me within 12 hours. That’s right, a *phone* got better service than this motherboard.

I’ll never buy from Albatron again.

Their site says “Capturing the spirit of the albatross.” I think they got it wrong. It should be “capturing the spirit of the weasel.”

HDTV Works! Woohoo!

MythTV

Finally, HDTV is completely working in my MythTV setup.

The pcHDTV 3000 card is working nicely with the DVB drivers.

My problem was playback performance. After switching to nVidia cards, this is fixed.

I got a nVidia Geforce 6800 card for my main workstation. Overkill for MythTV, yes, but it is a stellar performer with games. Absolutely great. I can run my games at 1600×1200 to take advantage of the native resolution of my flat panel, and they all work great — no framerate problems at all.

Very nice.

HDTV Working with MythTV!

MythTV

Some time back, I bought a pcHDTV HD-3000 card to use with MythTV. For various reasons, I never got around to getting it set up. Until now.

First, a quick background. HDTV is a means to transmit television signals digitally. In the USA, a standard called ATSC is used for over-the-air transmissions. ATSC transmissions can come in several different resolutions, ranging from no better than a basic analog signal to 1080i (1920×1080, interlaced).

To get HDTV to work with MythTV, there are several components that must be working first: HDTV playback in the frontend, HDTV kernel driver support, and HDTV tuner setup in the MythTV backend. I’ll talk about each of these in turn.

Click here to read more…

HDTV Playback

I was initially surprised at how much of an issue this is, when when you pause to think about it, it makes sense. HDTV pictures can be at up to 1920×1080 (1080i) or 1280×720 (720p) or 960×540 (540p). DVD pictures are typically 720×480. That means that a HDTV frame can have up to 6 times the number of pixels as a DVD frame. Bitrates for HDTV broadcasts are far higher than for DVD broadcasts as well.

This all combines to make a situation that can be extremely challenging even for systems with powerful CPUs and video cards.

In my own case, the frontend I was trying to configure is an Athlon64 system with a Radeon 9600 video card. Yet it still couldn’t keep up with even a 720p HDTV signal. I tried both the standard XFree86 radeon drivers, as well as the ATI proprietary fglrx ones, and none worked.

However, X.org CVS (NOT the latest release, I tried it!) has an extension to the radeon driver that permits DMA transfers for Xv. I grabbed this release, and with Steve’s intstructions and the wiki instructions, got it installed. (Note: I would advise against the lndir approach, and also make sure you put the #defines at the *TOP* of the file.) My custom lines were:


#define ProjectRoot /usr/local/X11R6.8
#define NothingOutsideProjectRoot YES
#define HasFreetype2 YES
#define HasFontconfig YES
#define DefaultGcc2i386Opt -O2 -march=athlon-xp -mmmx -mfpmath=sse -msse -msse2 -m3dnow

Restart kdm and test out a few things and… SUCCESS! HDTV playback works.

Now, on to:

HDTV Kernel Driver Support

For a pcHDTV card, there are two different kernel drivers you can use. One set uses the Video4Linux system for everything. The other uses the Linux Digital Video Broadcasting (DVB) infrastructure. This latter one is generally considered to be better and the way of the future, but it’s not very well documented.

The support for the pcHDTV card is in the mainline kernel as of 2.6.12-rc2. I downloaded and built 2.6.12-rc4.

There are a few surprises. First, in the video4linux config area (NOT the DVB area!), you must enable CONFIG_VIDEO_CX88 and CONFIG_VIDEO_CX88_DVB. I also enabled CONFIG_VIDEO_BT848 because some online resources said to, but I believe that is for the HD-2000, not the HD-3000.

Then, in the DVB area, you must enable CONFIG_DVB and CONFIG_DVB_CORE. HD-2000 owners, I think you want CONFIG_DVB_BT8XX. For the HD-3000, I set CONFIG_DVB_OR51132. HD-2000 owners will want CONFIG_DVB_OR51211. If you’re not using IVTV, you’ll also want to enable CONFIG_VIDEO_{TUNER,TVEEPROM}.

Also, you’ll want to follow the firmware download instructions. On a Debian system, you’ll want to put this into /usr/lib/hotplug/firmware.

Boot into your kernel. Now, you’re ready for:

HDTV Tuner Setup in MythTV Backend

This step configures MythTV to record from the HDTV tuner. I am using MythTV 0.18.

But before you begin, you’ll need to recompile MythTV. I’m using Matt’s debs, so it’s a simple matter of rebuilding from source.

The problem is that the files in /usr/include/linux on a default Debian install don’t include the latest DVB api. Workaround: move /usr/include/linux out of the way, symlink it to /include/linux, rebuild mythtv, restore /usr/include/linux, and install new mythtv and libmyth debs.

Next, go to labs.zap2it.com and make sure you have the digital channels selected and analog ones deselected.

Now, you are ready to run mythtv-setup. Make sure you kill the backend before you do this. You’ll want to add a new input source, DVB type, card 0. In input connections, connect your zap2it broadcast setup to this card. Then do a channel scan in the channel setup. Exit, run mythtv-filldatabase, then go back in. You’ll see duplicates for each channel. Note the XML id in the newly-created ones, copy it to the ones you created with the auto scan, then delete the new ones with mythweb. Finally, run mythtv-filldatabase again. The DVB setup HOWTO helped me with this.

Now you’re ready to enjoy HDTV!

MythTV update

MythTV, Reviews

I’ve received two e-mails from people asking for an update on the MythTV project. So here goes.

First, concerning the issue with the Biostar motherboard: the Biostar motherboard was bad. The TV out on it was physicially inoperative. It did not include the TV out in the box, so I had to buy that and the SPDIF out from Biostar. The SPDIF out was coax only, no optical, and the TV out was broken. I returned the motherboard and purchased an Albatron KM18g Pro 2.0 motherboard instead. The Albatron is nice. The TV out adapter is included and works. The SPDIF adapter is still extra, but instead of one port, has three ports: coax in, coax out, and optical out. My only complaint is that the GPU has a cooling fan that sounds like its about to die.

On the digital video (DVI output) side, I purchased an ATI Radeon 9200 card for $64.50. Together with the Gefen cable, I get crystal-clear output to my TV. While the picture is sharper than anything else, the colors are just not quite right somehow. After doing some checking, I am led to believe that this is because the TV does not auto-correct digital sources the way it does analog. I’m still looking into it. The ATI card is nice, and has a TV out too, so I could always fall back to that if need be. I am using the Free Software DRI drivers for the ATI card.

I’ve had a few problems with the MythTV unit overheating. We put it in a new entertainment center, and occasionally it will do a thermal shutdown to prevent damage. I ordered a nice SilenX power supply to try to move more air through there, and at a quieter volume. I also ordered one of their fans to mount on the entertainment center itself for ventilation.

The high-definition TV has been nice overall, but has exhibited discoloration in one corner. A repairman should be out this week to check it out.

Sweet Digital Goodness

MythTV, TV

Now that we have a HDTV unit and MythTV, it occured to me that there had to be a better way to get video to the HDTV than the standard S-Video cable I’d been using. And sure enough, there is.

Our TV has a HDMI input, which is pin-compatible with a digital DVI output on standard PC video cards. Slick. So, with the aid of a Gefen DVI to HDMI cable and a Radeon 9600 card stolen from my main PC, I got a pure digital picture on the TV. Wow. It’s nice. Beats S-Video cables handily.

Moreover, the TV supports EDID, the technology that lets PC monitors tell the PC what video modes they support. My TV reported 1920×540, 720×480, and 640×480 modes. Again, nice. And the ATI fglrx driver reports all the information you need to generate the appropriate ModeLine for it. Even slicker.

I’m going to buy a Radeon 9200 card for the MythTV unit so I can get the 9600 back for my desktop.

The only problem: the display is now so sharp that MPEG compression artifacts are more noticable and annoying than before. Guess I’ll have to bump up my bitrates in MythTV. Sigh.

Here are some links I found useful:

We\’ve gone HDTV

Hardware, MythTV, TV

I’ve been talking a lot about MythTV lately, and here’s a tangential topic: we bought our first HDTV unit last weekend. It’s a widescreen 30″ CRT Philips 30PW8402 unit. We don’t yet have hi-def video sources (save for DVD), but I’ve still gotta say: wow. The picture is so much better than our Sony TV (and not just because that Sony’s picture tube was dying, either!). There’s some stuff in 16:9 even in standard def, and that can be zoomed in upon. Very slick.

And DVDs are stunning on this thing. We’re very happy with it all and are glad we opted for a HDTV instead of a standard def one.

One of my concerns was about traditional stuff — would it all be “squashed” by the wide screen? Turns out no — the remote has a picture size button, that alternates between 4:3, zoomed 14:9, zoomed 16:9, widescreen, and superwidescreen modes. The zoomed modes are used when the black letterboxing at the top and bottom of a standard screen are part of the signal (such as widescreen movies broadcast in standard definition). The widescreen modes are used with true widescreen signals, such as from a DVD player.

Our DVD player already had component video out, so a few cables, and a quick check of the setup menu to tell it we had a 16:9 unit, and wow — stunning results. We’re really enjoying it.

My next project is to get a DVI to HDMI cable so I can hook it up to the MythTV unit digitally. This will get us the best possible quality, and I can use a Linux-based DVD player to send a pure digital signal to the TV.

Quick MythTV update

MythTV

My MythTV system is now mostly complete. My Biostar motherboard had a defective TV out, so it was sent in and now the TV out is working. I’ll post a more complete update in the next few days. In short, Biostar = bad, Albatron = good.

Also, had a fun adventure stringing some cat5 cable to the part of our house where our TV is. After sawing through the floor to get to the previously-inaccessible crawlspace, we found, among other things, a working wire stripper, pipes connected to nothing, and lots and lots of spider webs.

And my latest book project is just about complete. I’ll have some news there too in a day or two.