[kwlug-disc] btrfs/zfs for backups

[Paul Nijjar]

I am officially unhappy with Microsoft DFS replication. Earlier this
year I set up a couple of Windows Server 2008 R2 boxes to synchronize
big backup files over a wireless link, and it does not work well at
all. So now I am re-evaluating, and once again thinking of using some
Free Software filesystems for my job. 

I was thinking of scrapping the DFS replication and trying to use
rsync on Windows instead, but as far as I can tell all binaries of
rsync on 64-bit Windows are terrible. 

My inclination is to go with Debian/Ubuntu fileservers that
synchronize backup files (one way). Once upon a time somebody (I think
it was Chris Irwin) got me all excited by talking about btrfs send and
receive, that would allow you to send snapshots of incremental changes
to remote hosts via SSH. That sounds exciting, but The Internet (or at
least this StackExchange post:
http://serverfault.com/questions/285878/is-btrfs-production-ready ) it
seems that I should not consider this, and should go with ZFS instead
(which also has this functionality).

So here are my options: 
- Try btrfs on Debian/Ubuntu and hope it is mature enough to work for
  my use case
- Try ZFS on Linux the way Lori does
- Try ZFS on FreeBSD or some other OS where it is native
- Find some way to get large, effective filesync working on the
  Windows servers I have already built (ideally with FLOSS)

Here is some information about the infrastructure: 
- The fileserver will consist of a bunch of Samba shares 
- Symantec BackupExec (yes, I know) will write big backup files to
  these shares. Most of the files are 4GB large, but there are a few
  files that are almost 100GB large.
- The two servers are connected via a wireless link that effectively
  runs at 100Mbit
- The backup storage media are Western digital Green drives (sorry
  Cedric)
- The servers themselves are nothing special: 64-bit intel
  workstations with 2-4GB of RAM. 
- These are backup files, so data integrity is important
- We can assume there are 100s of GB of data backed up each week,
  although I am not sure whether this means hundreds of files are
  changing each week. (This could be the case; BackupExec has a habit
  of doing things in the most inconvenient way possible.) 

I am interested in hearing about how well btrfs works for the btrfs
send/receive scenario I am thinking about, and any advice
strengthening/contradicting the StackExchange opinion. If people are
using ZFS (in particular ZFS on Linux) with zfs send/receive in this
manner then I am interested in that information as well. If people
have other options (such as an effective rsync option for Windows 64
bit) then feel free to chime in. I am more interested in experiences
than speculation. 

- Paul 

-- 
http://pnijjar.freeshell.org