It’s time to change locations for my server. I’ve been with CoreNetworks for almost 5 years. They provide a good value, with fully dedicated servers a lot cheaper than most places, and good support to boot — targeting people that can handle root on their own box. I’m switching, though, to Hetzner Online (of Germany), primarily because I am needing more than 1GB RAM, and they can provide a 2GB box for less than I pay now for 1GB. I asked for feedback on Google+, and got positive feedback. Today, I migrated from one machine to another, copying a dozen or two GB of data, and only had about 15 minutes downtime.

I was able to do this without console access to the source machine, though console access wasn’t really required for either. This will work with any basic Linux install on the destination, or it could be prepared directly from a rescue environment.

Before I explain how I did it, I thought it was interesting to think of the different places that the machine that was known as complete.org has lived over the years. Here’s a map (click for detail):

Here’s the general process.

First, well in advance of the move, crank the TTL on the domains way down. This way, old IPs won’t be cached for very long once the system moves.

Next, start rsyncing data from the old machine to the new. Do not yet shut down daemons on the old. Shut down as much as you can on the new. You’ll want to focus on static data, such as /home. /usr is a good candidate as well. /var if you are selective — databases may be a good candidate, or may not. The idea is to seed the destination with data so that when we do the “real” rsync, most of the data will be there; it will have to tidy things up after daemons are stopped, and update some things, but the bulk of the work should be done.

Next, start preparing some exclude lists for the final rsync, which will copy an entire machine to the other. You will likely want to exclude files such as /etc/fstab, /etc/network/interfaces, /etc/resolv.conf, /etc/hosts, /etc/default/grub, /sys/*, /proc/*, /dev/*, and the like. I used rsync -v -P -a -H -A -X -S –numeric-ids –delete-after.

Now, start editing config files for the new IP, but do it on the old server (these will be synced over to the new one). Start with the bind config files, and touch anything else that needs it — maybe Apache configs, whatever.

Next, get ready to do the final sync. In an ideal environment, we’d just shut both machines down to single-user mode, but that’s not going to quite cut it here. Use ps and shut down all daemons except sshd and udev on the destination. Yes, including even syslog. This is to prevent anything actively accessing the disk during the rsync, and also to prevent any issues with clients accessing server daemons that aren’t ready yet.

Now, on the source machine, reload bind. This will start answering DNS queries with the new machine’s IP, and should also propagate the changes to your secondary DNS. Next, kill off all the daemons on the source machine, except for ssh and bind. You might want to set /etc/nologin to prevent regular users from logging in on ssh, if you have them. Now rsync things over to the new machine. Do any final tweaking over there (merging in /etc/hosts maybe, dealing with the udev rules.d persistent net thing, etc.). Reboot the new server and you should be up and running.