I am way overdue upgrading my primary work machine to the latest Ubuntu release. The biggest thing that has been holding me back (besides finding the time) is the fear of losing some file or system config which will take ages to recreate. Often when I upgrade a system like this, I simply install a new hard drive and keep the old drive around in case I need to dig up some file I don’t have handy in any other location. But this is a laptop and I am already nearly out of free space. What I’m going to do this time is use Squashfs to create a compressed, read-only filesystem inside a file which can be placed on an external drive.

Considerations

There are a number of different things I could do to save a copy of my files:

• Use rsync to create a remote directory copy
• Use cp to create a remote directory copy
• Use tar to create a remote file copy
• Use tar + gzip (or bzip2) to create a remote compressed file copy
• Create a ext2/3/4 filesystem inside a remote file using the loopback device
• Use one of the many backup utilities available for linux

Why did I choose Squashfs? Well, since I’m going to be copying the majority of a running filesystem, any of the methods which create a remote directory copy will be filled with thousands of files and directories. I don’t really like that kind of mess unless the external disk is dedicated. Tar+gzip gets me a single file and compression too, but it’s a pain to search as the whole thing would need to be untarred before use. Creating a read/write filesystem in a file is really useful, but I’m happy to trade write ability for compression. I don’t really need a backup utility since this is a one time operation.

Squashfs is in widespread use as the primary filesystem for many live CD linux distributions, and the linux kernel has included support since version 2.6.29. On Ubuntu, the user level tools can be installed via the squashfs-tools package.

Squash It

Enough talk, time to put this into practice. First, we need to verify that our kernel has the squashfs module available:

$ modinfo squashfs
filename:       /lib/modules/2.6.28-19-generic/kernel/ubuntu/squashfs/squashfs.ko
license:        GPL
author:         Phillip Lougher <phillip@lougher.demon.co.uk>
description:    squashfs 3.2-r2-CVS, a compressed read-only filesystem
srcversion:     7626F567E1FBB5EDCFF9F67
depends:
vermagic:       2.6.28-19-generic SMP mod_unload modversions 586

Good, we’ve got the kernel module. Now we need to install the userland tools.

$ sudo apt-get install squashfs-tools

Now let’s create a little squashfs file to verify that everything works. I’m going to use one the linux kernel source directories as a test.

$ mksquashfs /usr/src/linux-headers-2.6.28-19 test.sqsh
Parallel mksquashfs: Using 2 processors
Creating little endian 3.1 filesystem on test.sqsh, block size 131072.
[============================================================================================] 8848/8848 100%
Exportable Little endian filesystem, data block size 131072, compressed data, compressed metadata, compressed fragments, duplicates are removed
Filesystem size 8196.05 Kbytes (8.00 Mbytes)
        23.39% of uncompressed filesystem size (35034.82 Kbytes)
Inode table size 94740 bytes (92.52 Kbytes)
        29.11% of uncompressed inode table size (325410 bytes)
Directory table size 89099 bytes (87.01 Kbytes)
        57.22% of uncompressed directory table size (155701 bytes)
Number of duplicate files found 202
Number of inodes 10353
Number of files 8842
Number of fragments 284
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1511
Number of uids 1
        root (0)
Number of gids 0

That was easy. Now let’s mount the file.

$ mkdir ~/squash_test
$ sudo mount -t squashfs -o loop test.sqsh ~/squash_test

Let’s compare the contents to the original source:

$ ls -l ~/squash_test
total 59
drwxr-xr-x 24 root root   358 2010-09-20 09:20 arch/
drwxr-xr-x  2 root root    57 2010-09-20 09:20 block/
drwxr-xr-x  3 root root    50 2010-09-20 09:20 crypto/
drwxr-xr-x 19 root root   223 2010-09-20 09:20 Documentation/
drwxr-xr-x 83 root root   889 2010-09-20 09:20 drivers/
drwxr-xr-x  2 root root    25 2010-09-20 09:20 firmware/
drwxr-xr-x 62 root root   653 2010-09-20 09:20 fs/
drwxr-xr-x 27 root root   351 2010-09-20 09:20 include/
drwxr-xr-x  2 root root    37 2010-09-20 09:20 init/
drwxr-xr-x  2 root root    25 2010-09-20 09:20 ipc/
-rw-r--r--  1 root root  2430 2008-12-24 15:26 Kbuild
drwxr-xr-x  6 root root   134 2010-09-20 09:20 kernel/
drwxr-xr-x  6 root root   131 2010-09-20 09:20 lib/
-rw-r--r--  1 root root 57730 2010-09-16 07:13 Makefile
drwxr-xr-x  2 root root    37 2010-09-20 09:20 mm/
drwxr-xr-x 45 root root   477 2010-09-20 09:20 net/
drwxr-xr-x  6 root root    89 2010-09-20 09:20 samples/
drwxr-xr-x 10 root root  1057 2010-09-20 09:20 scripts/
drwxr-xr-x  6 root root    81 2010-09-20 09:20 security/
drwxr-xr-x 20 root root   205 2010-09-20 09:20 sound/
drwxr-xr-x 21 root root   257 2010-09-20 09:20 ubuntu/
drwxr-xr-x  2 root root    37 2010-09-20 09:20 usr/

$ ls -l /usr/src/linux-headers-2.6.28-19
total 148
drwxr-xr-x 24 root root  4096 2010-09-20 09:20 arch/
drwxr-xr-x  2 root root  4096 2010-09-20 09:20 block/
drwxr-xr-x  3 root root  4096 2010-09-20 09:20 crypto/
drwxr-xr-x 19 root root  4096 2010-09-20 09:20 Documentation/
drwxr-xr-x 83 root root  4096 2010-09-20 09:20 drivers/
drwxr-xr-x  2 root root  4096 2010-09-20 09:20 firmware/
drwxr-xr-x 62 root root  4096 2010-09-20 09:20 fs/
drwxr-xr-x 27 root root  4096 2010-09-20 09:20 include/
drwxr-xr-x  2 root root  4096 2010-09-20 09:20 init/
drwxr-xr-x  2 root root  4096 2010-09-20 09:20 ipc/
-rw-r--r--  1 root root  2430 2008-12-24 15:26 Kbuild
drwxr-xr-x  6 root root  4096 2010-09-20 09:20 kernel/
drwxr-xr-x  6 root root  4096 2010-09-20 09:20 lib/
-rw-r--r--  1 root root 57730 2010-09-16 07:13 Makefile
drwxr-xr-x  2 root root  4096 2010-09-20 09:20 mm/
drwxr-xr-x 45 root root  4096 2010-09-20 09:20 net/
drwxr-xr-x  6 root root  4096 2010-09-20 09:20 samples/
drwxr-xr-x 10 root root  4096 2010-09-20 09:20 scripts/
drwxr-xr-x  6 root root  4096 2010-09-20 09:20 security/
drwxr-xr-x 20 root root  4096 2010-09-20 09:20 sound/
drwxr-xr-x 21 root root  4096 2010-09-20 09:20 ubuntu/
drwxr-xr-x  2 root root  4096 2010-09-20 09:20 usr/

Perfect! How much space did we save?

$ ls -lh ~/test.sqsh
-rwx------ 1 bstump bstump 8.1M 2010-10-18 15:34 /home/bstump/test.sqsh*

$ du -hs /usr/src/linux-headers-2.6.28-19
64M     /usr/src/linux-headers-2.6.28-19

In this example, the source directory is about 64 MB versus only 8 MB for the compressed version. That’s 8:1 compression. However, the linux kernel is highly compressible source code. Typical compression for mixed files is about 2:1.

Unmount the file when we’re done.

$ sudo umount ~/squash_test

Now I’m ready to back up my primary file system to a single archive file in preparation for a system update.