I’ve long had a problem. Back on January 20, I took the day off work to watch the inauguration of Barack Obama. I saved the HD video recordings MythTV made of the day (off the ATSC broadcast), intending to eventually save them somehow. I hadn’t quite figured out how until recently, so there they sat: 9 hours of video taking up about 60GB of space on my disk.

MythTV includes a program called mytharchive that will helpfully transcode your files and burn a DVD from them. But it helpfully will transcode the beautiful 1920x1080i down to DVD resolution of — on a good day — 720×480. I couldn’t have that.

Background

My playback devices might be a PC, the MythTV, or the PlayStation 3. Of these, I figured the PS3 was going to be hardest to accommodate.

ATSC (HD) broadcasts in the United States are an MPEG Transport Stream (TS). Things are a bit complicated, because there may be errors in the TS due to reception problems, the resolution and aspect ratio may change multiple times (for instance, down to SD for certain commercials). And, I learned that some ATSC broadcasts are actually 1920×1088 because the vertical resolution has to be a multiple of a certain number, and those bottom 8 pixels shouldn’t be displayed.

Adding to the complexity, one file was 7 hours of video and about 50GB itself. I was going to have to do quite some splitting to get it onto 4.7GB DVD+Rs. I also didn’t want to re-encode any video, both for quality and for time reasons.

Attempts

So, I set out to try and figure out how to do this. My first approach was the sledgehammer one: split(1). split takes a large file and splits it on byte or line boundaries. It has no knowledge of MPEG files, so it may split them in the middle of frames. I figured that, worst case, I could always use cat to reassemble the file later.

Surprisingly, both mplayer and xine could play back any of these files, but the PS3 would only play back the first part. I remembered this as an option if all else failed.

Next, I tried avidemux. Quite the capable program — and I thought I could use it to cut my file into DVD-sized bits. But I couldn’t get it to let me copy the valid MPEG TS into another MPEG TS — it kept complaining of incompatible formats, but wouldn’t tell me in what way they were incompatible. I could get it to transcode to MPEG4, and produce a result that worked on the PS3, but that wasn’t really what I was after.

Then, I tried mpgsplit. Didn’t recognize the MPEG TS as a valid file, and even when I used a different tool to convert to MPEG PS, acted all suspicious as if it bought the MPEG from a shady character on a grungy street corner.

dvbcut

I eventually wound up using dvbcut to split up the ATSC (DVB) recordings. It understood the files natively and did exactly what I wanted. Well, *almost* exactly what I wanted. It has no command-line interface and didn’t have a way to split by filesize, but I calculated that about 35 minutes of the NBC broadcast and 56 minutes of the PBS broadcast could fit on a single DVD+R.

It worked very, very nicely. The resulting files tested out well on both the PS3 and the Linux box.

So after that, I wrote up an index.txt file to add to each disc and a little shell scripting later and I had a directory for each disc. I started burning them with growisofs. Discs 1 and 2 burned well, but then I got an error like this:


File disc06/VIDEO/0930-inaug-ksnw-06.mpg is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.


Eeeeepp. So apparently the ISO 9660 filesystem can’t represent files bigger than 4GB. My files on disc 1 had represented multiple different programs, and stayed under that limit; and disc 2’s file was surprisingly just a few KB short. But the rest of them weren’t. I didn’t want to have to go back and re-split the data to be under 4GB. I also didn’t want to waste 700MB per disc, or to have to make someone change video files every 15 minutes.

So I decided to investigate UDF, the filesystem behind Blu-Ray discs. mkisofs couldn’t make a pure UDF, only a hybrid 9660/UDF disc that risked compatibility issues with big files. There is a mkudffs, but it doesn’t take a directory of its own. So I wrote a script to do it. Note that this script may fail with dotfiles or files with spaces in them:

#!/bin/bash

set -e

if [ ! -d "$1" -o -e "$2" -o -z "$2" ]; then
   echo "Syntax: $0 srcdir destimage [volid]"
   echo "destimage must not exist"
   exit 5
fi

if [ "`id -u`" != "0" ]; then
   echo "This program must run as root."
fi

EXTRAARGS=""
if [ ! -z "$3" ]; then
   EXTRAARGS="--vid=$3"
fi

# Get capacities at http://en.wikipedia.org/wiki/DVD+R as of 9/27/2009

SECSIZE=2048

# I'm going to set it a few lower, than capacity 2295104 just in case.
# Must be at least one lower than the actual size for dd to do its thing.
# SECTORS=2295103
SECTORS=2295000

echo "Allocating image..."

dd if=/dev/zero "of=$2" bs=2048 "seek=$SECTORS" count=1

echo "Creating filesystem..."

mkudffs --blocksize=2048 $EXTRAARGS "$2"
mkdir "$2.mnt"

echo "Populating..."
mount -o rw,loop -t udf "$2" "$2.mnt"
cp -rvi "$1/"* "$2.mnt/"
echo "Unounting..."
umount "$2.mnt"
rmdir "$2.mnt"

echo "Done."

That was loosely based on a script I found on the Arch Linux site. But I didn’t like the original script, because it tried to do too much, wasted tons of time writing out 4.7GB of NULLs when it could have created a sparse file in an instant, and was interactive.

So there you have it. HD broadcast to playable on PS3, losslessly. Note that this info will also work equally well if you have a Bluray drive.