Last year, I analyzed why some of my backup hard disks had become so slow. It turned out that this was due to Shingled Magnetic Recording (SMR). In particular, a number of my 8 TB 3.5″ disk drives were using this recording technology and as they were becoming full anyway, I took them out of service and replaced them with larger drives that do not use SMR. However, these drives were still pretty much new, so I decided to put them into good use for a scenario in which speed does not matter that much: Background backups with Borg backup of virtual machine images that are streamed over the network and hence are slow to be backed up anyway.
Also, I decided to further backup a number local data heaps of which incremental backups might be useful. In total, this was about 2 TB of data initially. Over the year, however, incremental data on the backup drive started to pile up, as I never removed old backups and cleaned up the Borg data store. This is an action called ‘pruning’ in Borg parlance. So at the end of the year, the 2 TB of data had become 4.2 TB and it was time to clean up a bit as I didn’t need all differential backups anymore.
One of the reasons I don’t prune data from the Borg archives after every backup is that pruning is a time intensive process, and I rather prefer to do it periodically, so the backup processes finish as quickly as possible. But this comes at a price. The recent full year pruning took over 24h to complete in the background. From the 4.2 TB that was allocated, I was back down to 2.6 TB at the end of the process. In other words, it was quite worth the effort!