[kwlug-disc] btrfs(raid5) or ext4/mdadm(raid5) ?

[B. S.]

----- Original Message -----
> From: William Park <opengeometry at yahoo.ca>
> To: kwlug-disc at kwlug.org
> Cc: 
> Sent: Thursday, June 4, 2015 1:02 AM
> Subject: Re: [kwlug-disc] btrfs(raid5) or ext4/mdadm(raid5) ?
> 
> On Thu, Jun 04, 2015 at 04:00:52AM +0000, B. S. wrote:
>>  btrfs uses mdadm / raid5 in exactly the same way ext# uses mdadm / raid5.
> 
> You can set it up that way, I guess.  But, I was looking to use raid5
> feature that is built into BTRFS, eg.
>     mkfs.btrfs -d raid5 /dev/sd{e,f,g}
> which eliminates the need for MDADM.
> 
>> 
>>  {RAID5 = data availability, not the same thing as data reliability.
>>  (At home, lose your drives and you're going to be all over it, beating
>>  upon it until it's done. So I question whether the complexity of RAID
>>  brings anything to that party.)}
> 
> For the array in question with bad sectors, I need size greater than a
> single disks.  So, raid5.

That's my point - not necessarily. e.g. btrfs 'pools'. Give each disk its own btrfs fs, tell btrfs they're both part of the same pool, you're good to go. There are similar mechanisms for other fs, again nothing to do with raid5 / mdadm.

It is arguable that unless you have a single file spanning a single disk, disk spanning is not needed. It's also arguable that spanning a disk brings an avoidable level of complexity and risk. (Let alone irritation of trying to remember the special cases when something goes wrong.)

Please note, I'm not talking the enterprise here. Nor am I talking about a SQL or e-mail server. (But if you have a greater than 4TB sql or mail server at home ... well, we're also in a different playing field.)

>>  Last time I was in your situation, disks getting questionable ... I
>>  bought a replacement disk of such size to replace all and some growth.
>>  I moved the data over, 'converting' from ext4 to btrfs in the 
> process,
>>  and haven't looked back since. Once done I redeployed the questionable
>>  disk, with btrfs, and also have not looked back since. For my own
>>  sanity, too, I no longer have fs that cross disk boundaries. I just ln
>>  -s things like /data upon growth.
> 
> I did this long ago, but gave up due to maintenance.  I may need to
> reconsider.  Thanks for reminder.

Regardless of the mechanism used, partitions filling up are going to bite you eventually. All you can do is pick your form of poison. (I agree with you, learning fewer ways to do the same thing is better.) So if that link is to a btrfs fs, or you have btrfs pool the fs - I take your point that at least it's all btrfs.

One way or another, mdadm is involved. Whether btrfs does it for you, or you indirect via lvm (or zfs?), when stuff happens, you're going to want to be able to investigate via mdadm controls. K.I.S.S. (and one excelling utility per function) would seem to indicate managing mdadm and btrfs as separate beasties.

Given btrfs checksumming, it is arguable that RAID5 is not necessary / counter-productive, vis a vis, say, RAID1 across 2 physical systems, sync'ed or snapshotted via btrfs. (You want backups off the local physical system, so you've got at least two of these disk 'arrays' around, anyways.) And doing so avoids the 'which disk in the RAID5 degraded, K.I.S.S. giving you one drive, one problem, to focus upon. For a worst case, re-ln to the snapshot / equivalent across the net, for less than that beat the problem disk to death until resolved, and back to your regular day before the very irritating interruption of the disk going wonky.

RAID5 / backups are just the data. It's arguable that one should have 'mirrored' hardware to hand for all the non-data glitches that are going to occur. CPU failure? Either swap CPUs or put the drives in the other box. One stays productive in the mean time, and time is bought to deal with the glitch of the moment. Avoiding showstoppers, unless you happen to be in a time and place where you can focus on said showstopper. But at least you're not forced into that at the time by the showstopper.