Moving a FreeBSD Installation to a New Laptop

Some time ago, I wrote in “One of the reasons why X11 rocks” that I had to move my FreeBSD installation from my dying Toshiba Satellite U-220 laptop to a new laptop, one that I bought in a pretty short notice from Plaisio Computers. This article is, as I promised back then, a description of how I migrated my old laptop files to the new system.

Preparing for the Migration

One of the important details of performing the migration was that the new laptop comes with a 2-year warranty; a warranty which I did not really feel like invalidating after merely a few hours of purchasing it. This means that I had to move everything to the disk of the new laptop without opening the case of the new system at all. I usually keep at least one USB-attachable enclosure for 2.5″ laptop disks around, so this particular requirement was pretty easy to fulfill. I could use the USB enclosure to connect the internal disk of my old laptop to the new system without opening the case at all.

Booting from the USB-attached disk is a trivial task with many modern laptops, but I didn’t want to boot into the filesystem and use dump(8) & restore(8) on a “live” file system. To avoid that, I grabbed one of the FreeSBIE LIVE CD-ROM disks I keep around. Booting from the LIVE CD-ROM would leave both hard disks “clean”, and I could use even the plain BSD tar(1) or cpio(1) utilities to copy the files from the source to the target disk.

Booting Into FreeSBIE

My every-day environment is what a lot of people would call “Spartan”: just a bunch of terminals, a few Firefox or Emacs windows, and maybe an instant messaging client. As a result, I don’t really need a full blown XFCE4 environment to copy a few files around. This is why I like FreeSBIE for this sort of work. The boot loader of FreeSBIE is tunable enough to fire up just a very basic X11 session, with a preconfigured, bloat-free session of the fluxbox window manager.

Here’s how the default fluxbox session of FreeSBIE looks after booting into X11:

Setting Up the X11 Environment of FreeSBIE

I feel a bit uncomfortable with the default xterm fonts in FreeSBIE, and my eyes hurt if I stare for a very long time to a window with bright white background, so I spent a few minutes to fine-tune the default desktop environment.

I tweaked a bit the default colors of xterm, using the xrdb(1) utility:

% xrdb -merge - << EOF
XTerm*background: black
XTerm*foreground: #a8a8a8
EOF

I also changed the default font of xterm to “Lucida Sans Typewriter” at 12 points. I don’t really like the default “fixed” font of X11. Over the years I have grown to absolutely adore Lucida Sans, so whenever I am about to spend any significant amount of time staring at a terminal window, I usually switch fonts:

% xrdb -merge - << EOF
XTerm*font: lucidasanstypewriter-12
EOF

After these minor changes, I closed the xterm window and opened a new one. Seeing my favorite colors was a bit refreshing.

Preparing the Target Disk

Before copying anything over, I had to prepare the internal laptop disk, and mount both the source (USB) and target (internal) disks.

Laptops from Plaisio Computers come with a preinstalled image of Windows® Vista™ or Windows™ XP. I have no need for this sort of operating system, and I refuse to use unfree software when I have a choice, so preparing the disk was very easy. I wiped it out and created a new set of FreeBSD slices.

NOTE: The term “partition” is used in two different contexts when disks are described in the BSD world. MS-DOS® and vendor documentation often use the term “partition” when they refer to PC BIOS-style partitions. Back when BSD ran on non-PC hardware, the term “partition” was used for the sub-divisions that BSD used internally for its own filesystem organization.

FreeBSD can install on modern PC hardware and it supports two styles of disk organization. One of them is called “dangerously dedicated mode”, and it directly sub-divides a raw disk device in BSD-style partitions, without any PC-style partitions. The second mode of installation uses one or more PC BIOS-style partitions, and further subdivides each one of these to BSD-style partitions.

The use of the term “partition” to refer to both PC BIOS-style partitions and BSD-style partitions can be very confusing, so the FreeBSD Documentation Team has opted for a reasonable compromise when writing FreeBSD documentation: we call the PC BIOS-style partitions with a different name: “slices”. In an effort to avoid this confusion between the two different uses of “partition”, I will be using the term “slice” to refer to BIOS style partitions from now on, and the term “partition” to refer to the BSD-style subdivisions of a single “slice”.

To start with a clean slice table, without any cruft left over by the existing Windows installation, I used the dd(1) utility to overwrite the initial sectors of the internal laptop disk with zeros:

freesbie# dd if=/dev/zero of=/dev/ad0 bs=32768 count=10

Then I installed a new slice table with only one FreeBSD partition, covering the entire disk:

freesbie# fdisk -BI /dev/ad0

This slice was later resized to 120 GB (the full size of the disk is 250 GB, but I didn’t need all of that space; at least not just yet). To resize the slice I used fdisk(8) one more time, with the -u option:

freesbie# fdisk -u /dev/ad0

After going through the prompts of fdisk(8) I saved the changes and let fdisk(8) write everything back to the disk. The slice table of the internal laptop disk looks, even to this day, a couple of months after the initial installation, like this:

kobe# fdisk /dev/ad0
******* Working on device /dev/ad0 *******
parameters extracted from in-core disklabel are:
cylinders=484521 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=484521 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 63, size 251658225 (122879 Meg), flag 80 (active)
        beg: cyl 0/ head 1/ sector 1;
        end: cyl 828/ head 15/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
kobe#

Copying the files from the source disk to the target disks can be done with several ways in BSD. If there are only two BSD partitions involved in the process and they have precisely the same size, one can use even simple tools like dd(1) to shuffle data around. For anything more complicated, I usually prefer tools like tar(1), cpio(1) or a combination of dump(8) and restore(8). This way I can mount the source and target partitions in practically any combination and fine-tune the copying of the files in more flexible ways, i.e. to selectively copy only part of the source file tree, or to pre-mount the filesystems in a way that joins splits or otherwise reorganizes the source files by forcing them to end up in specific mount points.

In the particular case of moving all the files off my old laptop, I wanted to copy files from a single source partition (the /dev/da0s1a partition of the USB disk) to at least two targets partitions. The old laptop had only one, large root filesystem. I wanted to split this large partition to a root filesystem and a /home partition, for my personal files.

I chose to split the 120 GB fdisk partition to the following parts:

• A 30 GB root filesystem. This should be adequate for a full-blown X11 desktop, including several large packages like X11 itself, Firefox, GNU Emacs and a collection of compilers, debuggers and other every-day tools.
• A 6 GB swap partition. The laptop now has 2 GB of memory, but I may upgrade it to 4 GB later. Being able to save a full memory dump in the swap area may be useful for debugging kernel problems, so I wanted the partition to be large enough to hold a full dump and have enough free space to boot. With the price of disk space these days, 6 GB seemed “ok”.
• The rest of the first 120 GB fdisk partition should be mounted as a /home filesystem.

The bsdlabel(8) utility is the standard tool for splitting an fdisk slice to partitions. To split my 120 GB slice to the 3 partitions (root, swap and home), I started by installing a new “label” on the disk, with only one large ‘a’ partition covering the entire /dev/ad0s1 slice:

freesbie# bsdlabel -w -B /dev/ad0s1

Then I fired up an editor to manually specify the 3 partitions that the slice should eventually have:

freesbie# setenv EDITOR vi
freesbie# bsdlabel -e /dev/ad0s1

This popped up a vi(1) session, with the original label of the ad0s1 partition:

# /dev/ad0s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a: 251658209      16    unused        0     0
  c: 251658225       0    unused        0     0         # "raw" part, don't edit
~
~
~
~
/tmp/EdDk.G2iVHvNHNB: unmodified: line 1

Calculating partition sizes by counting sector numbers can be a bit tedious. It’s not rocket science, but it is a bit boring, and it involves a lot of stuff that may be automated sufficiently well by the FreeBSD version of bsdlabel(8). One of the nice features of FreeBSD bsdlabel(8) is that it can accept a partition layout of the form

# /dev/ad0s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:       30G      16    4.2BSD        0     0
  b:        6G       *    swap
  c: 251658225       0    unused        0     0         # "raw" part, don't edit
  e:         *       *    4.2BSD        0     0

and it can automagically calculate the missing bits of the disk layout. This is exactly what I typed in the vi(1) buffer that bsdlabel(8) spawned. I saved the new disk partition layout, exited vi(1). bsdlabel(8) filled-in the missing bits, run a few sanity checks on the manual changes I had made, and finally installed the new label on the ad0s1 slice.

The partitions are not auto-formatted when bsdlabel(8) installs a new slice label. So the next step was to run newfs(8) on them:

freesbie# newfs -L ROOT /dev/ad0s1a
freesbie# newfs -L HOME /dev/ad0s1e

Mounting the Source and Target Partitions

With the target partitions configured and formatted, I was ready to move on to the next step of the migration process: to mount the source and target partitions.

There was only one source partition in the old disk. A 70+ GB root filesystem, visible as the /dev/da0s1a partition of the USB-attached disk.

The target partitions were those I created on the internal laptop disk: the /dev/ad0s1a partition would be the target root filesystem, and the /dev/ad0s1e partition would be the target /home filesystem. The “/etc/fstab” file of the target root filesystem should also mention /dev/ad0s1b as the swap device.

I started by creating two new, empty mount points in the /mnt directory of the FreeSBIE LIVE CD-ROM session:

freesbie# mkdir /mnt/source /mnt/target

Then I mounted the source partition as read-only, and the target partitions as read-write:

freesbie# mount -o ro /dev/da0s1a /mnt/source
freesbie# mount /dev/ad0s1a /mnt/target
freesbie# mkdir /mnt/target/home
freesbie# mkdir /dev/ad0s1e /mnt/target/home

Note that I had to manually create the mount point for /dev/ad0s1e because the filesystem of /dev/ad0s1a (the target root) was empty; it had just been newfs-ed.

The extra pre-caution of mounting the source filesystem(s) as read-only is not strictly necessary, but it may help if a mistyped rm(1) command tries to remove the wrong files, or if you fail to notice the current working directory and you try to extract files in the wrong place.

One of the greatest advantages of starting the system with a LIVE CD-ROM becomes apparent at this point:

All the special filesystems of both the source and the target disks are unmounted. There is no /dev filesystem with device nodes that you have to exclude when you copy the files of the source tree. There is no special /proc filesystem that you have to exclude either. All the mount points for these special filesystems are there, but that’s all; they are merely there… as empty directories, which are safe to copy around.

Selecting What to (Not) Copy

Normally, at this point, one may just blindly copy everything and the kitchen sink from the old disk. The title of this article is, after all, how to migrate an existing installation… all of it… including those random photographs from Napoli where you were caught singing a local adaptation of “Hotel California” with satyrical words about Camping Solfatara and the crazy Napolitan drivers!

Having a brand new disk, with entirely empty filesystems, neatly arranged in a new layout, is also, however, a very good chance to clean up a few things. Especially if you already have most of the important bits of the old disk in backup copies elsewhere. The old disk isn’t going away either, so this is a chance that was far too tempting to miss.

This is why I started going through the /mnt/source hierarchy and keeping a list of directories that I would exclude from the copying process. I was planning to use tar(1) to copy the files, so the list of directories to avoid copying was kept in a simple text file called “/tmp/xlist” in the /tmp directory of the LIVE CD-ROM session.

The excluded directories were listed using relative pathnames under the /mnt/source tree, so that I would later be able to run something like:

freesbie# cd /mnt
freesbie# tar -C source --exclude foo/bar -cf - . | tar -C target -xvf -

NOTE: If the source tree includes more than one filesystem, there are a few directories that are excellent candidates for the “/tmp/xlist” list of excluded pathnames. All “.snap” directories of the source filesystems should probably be excluded. Especially if you have a different filesystem layout in the target tree. Even if you don’t, copying a few dozen snapshot files from the source tree to the target tree is not a very good idea.

Copying the Files

When I finished preparing the “/tmp/xlist” exclude list, I was ready to run the actual copying commands.

Typing several dozens of –exclude “foo/bar” options and carefully quoting the special characters — like spaces, TABS, parentheses and so on — isn’t really my idea of fun. So, instead of manually passing each excluded pathname from “/tmp/xlist“, I used the -X option of BSD tar(1) and let it read the excluded pathnames itself.

The copying itself was done by typing:

freesbie# cd /mnt
freesbie# tar -C source -X /tmp/xlist -cf - . | tar -C target -xvf -

At this point, I left the laptop merrily crunching its way through my files, and took a break. Copying several GB of data over USB was bound to take a while…

Target Filesystem Tuning

(Several minutes later…)

When tar(1) is done copying the files over to the target/ filesystem tree, there are a few final “tuning” steps that should be performed, to make sure that the new installation is bootable and that it works as expected.

Restoring File Permissions and Flags With mtree(8)

The tar(1) utility is an indispensable tool, but it doesn’t really mirror everything from the source file tree to the target file tree. Some of the things that are not easy to copy with tar(1) are filesystem access lists, special file flags like “sappnd” or “schg“.

Restoring the special permissions and file flags of the “base system” is easy in FreeBSD. We can use the mtree(8) utility and the predefined “specification files” of the /etc/mtree directory to do it. I restored the permissions of my new “base system” in /mnt/target by running something like this:

freesbie# ( cd /mnt/target ; \
    mtree -deU -f /mnt/target/etc/mtree/BSD.root.dist )
freesbie# ( cd /mnt/target/usr ; \
    mtree -deU -f /mnt/target/etc/mtree/BSD.usr.dist )
freesbie# ( cd /mnt/target/usr/include ; \
    mtree -deU -f /mnt/target/etc/mtree/BSD.include.dist )
freesbie# ( cd /mnt/target/usr/local ; \
    mtree -deU -f /mnt/target/etc/mtree/BSD.local.dist )
freesbie# ( cd /mnt/target/var ; \
    mtree -deU -f /mnt/target/etc/mtree/BSD.var.dist )

Checking /etc/fstab

If you have made changes to the partition layout of the old installation (as I did), the “/etc/fstab” file of the target file tree should be updated. I didn’t have a separate /home partition before, so I went ahead and added an entry for the new /home partition to the “/mnt/target/etc/fstab” file:

# Device     Mountpoint  FStype  Options  Dump  Pass#
/dev/ad0s1e  /home       ufs     rw       2     2

While editing the new fstab file I verified that the swap partition was correct too, saved the file and exited.

Booting Into the New Installation

The new, target partitions were now ready to test. I unmounted everything:

freesbie# umount /mnt/target/home
freesbie# umount /mnt/target
freesbie# umount /mnt/source

Then I detached the USB enclosure of the source disk, I rebooted the FreeSBIE LIVE CD-ROM session, and waited for the new installation to come up. Fortunately enough, it did!

This article was typed in the very same laptop, almost a couple of months after the files were copied. I’ve been using this installation of FreeBSD several hours every day, and I am grateful to all the great people who wrote the tools described in this article.

Without the excellent tools that are part of the FreeBSD system, it would take a lot of time to move my files over. Without FreeSBIE, or another LIVE CD-ROM image, I would have to boot into the FreeBSD installation that I wanted to copy, and this may be slightly inconvenient. Finally, without the detailed, thorough and integrated documentation of the tools of the FreeBSD base system, I would have to remember all the tool options by heart; something that may be a bit tough to pull off when one is hard pressed to finish the file move as fast as possible.

To everyone who has ever contributed to any of the magnificent tools that are part of the systems described here: THANK YOU :-)