Another thing I have to do every now and then is to clone a partition to a new drive, e.g. for backup purposes or when upgrading. Depending on the amount of data to transfer, this can be quite a lengthy procedure. Copying 2 TB worth of data from one drive to another, for example, can take quite some time. rsync is a great tool to do this but it seems to be rather slow in practice. So far, I can’t get more than 120 MB/s out of it, even if both source and destination are local SSDs. Furthermore, when copying very small files, the transfer rate plunges into a back hole. So in practice with my mix of data, the whole operation usually takes half a day at least. So perhaps there’s a better way? How about partclone?
Why Not Rsync?
Rsync is great but I guess it checks a lot in the background before and during the transfer of a file, so it is rather slow and CPU limited. Partclone, on the other hand, at least as far as I imagine it, should be the exact opposite, as it copies used blocks of data and doesn’t care at all about individual files. This should be much faster in practice and make use of the available transfer rates of source and destination drives.
Partclone With An Image File
So I gave this a try in practice and used partclone to create a 35 GB image file of a partition which I then copied to the internal Samsung 970 Evo Plus NVMe SSD in my Lenovo X13 notebook. In my previous tests, I got a read and write data rates of around 1.6 GB/s with several simultaneous copy commands. Also previous tests have shown that I can also transfer data to this drive at a rate beyond 800 MB/s. So this would be the lower limit I would expect partclone to match when writing the contents of my image file to an external NVMe SSD connected over USB 3.2.
When I ran partclone, however, I was rather disappointed. What I got was a sustained throughput of around 150 MB/s with a high load on a single CPU. This is rather strange because even ‘cp‘ does much better in the same scenario with a single CPU core. Copying the 35 GB to /dev/null resulted in a speed of 600 MB/s. I ran partclone several times, with hours in between to let the external SSD ‘settle’, but always ended up at the same speed. I’m really puzzled, as reading a huge 30 GB file and writing blocks of data should not involve a huge computational overhead.
In practice, partclone would still speed-up data transfers, as the data rate does not fall into a black hole when a large number of small files are transferred. However, 150 MB/s is half an order of magnitude away from what the I/O hardware and SSDs are capable of in my notebook as shown in previous posts.
Partclone Drive to Drive
But hold on, perhaps partclone performs better if I go for my real scenario and copy a partition directly between two SSDs? I didn’t do this as first as the original SSD in combination with my USB-C/SATA adapter is limited to a read data rate of 220 MB/s with cat and cp. So in a further attempt, I used partclone to transfer the partition directly from the SSD/SATA/USB drive to the SSD/NVMe/USB drive. To my surprise I got a throughput of 220 MB/s, i.e. exactly the maximum the hardware is capable of!? O.k., so it looks like using an image file as source for partclone was the limitation. But since my source drive is ‘slow’, I can’t really say where the real limit would be from a CPU point of view. Anyway, 220 MB/s is way faster than what I could do with rsync if I wanted to copy a complete partition, so I count this as a success for now.
And Then, There’s UAS
I was about to wrap up this post when I had another idea: The SATA to USB adapter I used at first is a rather old model and I faintly remembered that some time ago, I bought two additional ones that support the USB Attached SCSI (UAS) protocol, which is supposed to make things a lot faster. So I gave these adapters a try in the same configuration as above and was very positively surprised: With one of the two UAS capable USB to SATA adapters, I got a sustained transfer speed of 300 MB/s. With the second adapter, the sustained transfer rate over long periods of the 35 GB was almost 400 MB/s, but it emitted a high pitched noise throughout the operation. Also, every now and then, the data transfer would stop for a few seconds. In the end, the overall speed was also in the range of 300 MB/s, or around 18 GB/min. All right, now we are talking! Both adapters are USB 3.0 devices which limits data throughput in the most ideal case to 500 MB/s. So the second adapter got pretty close. There now seem to be USB 3.1 Gen 2 to SATA adapters available that are advertised as supporting USB with 10 Gbit/s. Hm, one of those might push the limits even further.
To be continued.