[kwlug-disc] btrfs/zfs for backups

[Chris Irwin]

On 12/02/2014 10:57 PM, L.D. Paniak wrote:
> On 12/02/2014 10:27 PM, Chris Irwin wrote:
>> On 12/02/2014 08:44 PM, Paul Nijjar wrote:
>>> My inclination is to go with Debian/Ubuntu fileservers that
>>> synchronize backup files (one way). Once upon a time somebody (I think
>>> it was Chris Irwin) got me all excited by talking about btrfs send and
>>> receive, that would allow you to send snapshots of incremental changes
>>> to remote hosts via SSH.
>> Once upon a time somebody (possibly this "Chris Irwin" guy) said he'd
>> do a talk about using btrfs, too.
>>
>> What months are open at this point? Keep in mind that winter adds a
>> "depending on weather" for my attendance, since I need to come from
>> (and back to) London.
>>
> Looks like March and onward are available.  Updates on the state of
> btrfs are always welcome.

I'm good with March.

Paul: I'll write up a summary blurb and send it to you tomorrow.

> I would avoid deduplication in ZFS unless you have plenty of $$ for
> server RAM (hundreds of GB).  Performance is generally abysmal on dedup
> systems if one skimps on RAM.  Given the price of hard drives at the
> moment, I find it difficult to recommend either deduplication or
> compression for ZFS:  more spindles will increase your performance anyway.

The difference with btrfs is that deduplication isn't live, but a task 
you'd schedule (just like a scrub). I believe this also means memory 
requirements are lessened, but I don't have figures on that.

I have unanswered questions regarding dedup support: Do snapshots 
change, hold references to the non-deduped data (thus not lessening your 
disk usage), or simply invalidate themselves. All options are bad, 
depending on your needs. This is why I'm planning on testing it (how 
would we live without VMs).

Ultimately, I'm of the same opinion on compression and dedup support: 
It's really not worth it versus the cost of another disk. I only 
mentioned it because multiple static large full backup images is just 
about the only scenario I'd find dedup actually useful (especially since 
a backup server presumably won't have disk activity during the day, and 
can run a dedup scan when workstations are not backing up). That 
usefulness also depends on backupexec's format.

-- 
Chris Irwin
e: <chris at chrisirwin.ca>
w: http://chrisirwin.ca