Every now and then I would like to use the data on a backup drive with a program that modifies the data. Obviously I would not want the backup to be modified. The standard approach would be to copy the data from the backup drive to a temporary location, use the data, and, once done, delete it again. This works well for small amounts of data, but copying the files will take ages for a double digit gigabyte dataset. The solution: An Overlay File System!
With an overlay file system, Linux offers a convenient way to map a file system to another directory. All reads will go to the original file system, while any modifications are put into an overlay. This is a kind of a ‘copy-on-write’ approach that happens transparently in the background. And once done, the overlay file system can be un-mounted, and the copy-on-write data can be deleted. This approach is used by Docker and other container systems to have an immutable base image that is mapped into a container, where it can be modified. All modifications, however, end up in an overlay and are discarded once the container is removed from the system. Next time a container is instantiated from an image, everything starts again from scratch. But one doesn’t need a container to use an overlay file system, here are the shell commands to do it in a ‘normal’ Linux environment:
# Create directories for the mount and overlay
mkdir /media/tmp
mkdir /media/changes
mkdir /media/work-dir
mkdir /media/overlay
# Mount a disk, preferably read only (-o ro), just to make sure :)
#
sudo -s
mount -o ro /dev/sdXX /media/tmp
mount -t overlay overlay -o lowerdir=/media/tmp,upperdir=/media/changes/,workdir=/media/work-dir/ /media/overlay
# Done, use /media/overlay to read AND write data!
And once done, the overlay file system can be unmounted again and the temporary data can be deleted:
umount /media/overlay
umount /media/tmp
rm -rf /media/tmp
rm -rf /media/changes
rm -rf /media/work-dir
rm -rf /media/overlay
Pretty powerful stuff! For more details, have a look here.