[kwlug-disc] Transferring large backup files over the network

[peter_melse at gto.net]

 There is also Syncthing. cross platform, I'm about to try it on 15tb 
 between windows storage spaces, and a zfs on linux pool, and a netapp 
 via nfs.

 I'll let you all know how it goes

 ~Peter

 On Fri, 24 May 2019 00:47:01 -0400, Remi Gauvin <remi at georgianit.com> 
 wrote:
> On 2019-05-24 12:30 a.m., Paul Nijjar via kwlug-disc wrote:
>
>>> The other key point that was mentioned, but I re-iterate since it 
>>> is
>>> important, is to copy files as
>>> a whole, and not bother to checksum them.
>>
>> Even for large (20GB) files with only small changes? I thought this 
>> is
>> exactly why you would want the checksum/diff algorithm. even if you
>> have a relatively fast network.
>
> Not unless your network speed is significantly slower than your disk 
> speed.
>
> If rsync has to create a delta, first, the complete file has to be 
> read
> on both source *and* destination, differences calculated, then the 
> file
> has to be completely written on the destination, (unless using
> --inplace, but that has it's own special considerations.)
>
> If you're on a Gigabit network copying from/to a hard drives it would
> have been faster just to copy the whole file from the get go.