[kwlug-disc] Filesystems for backups
William Park
opengeometry at yahoo.ca
Wed Aug 7 19:26:34 EDT 2019
I'm using btrfs on leftover hard disks as backup.--William
Sent from Yahoo Mail on Android
On Wed, Aug 7, 2019 at 5:49 PM, Paul Nijjar via kwlug-disc<kwlug-disc at kwlug.org> wrote: I have a bunch of drives (mostly 1TB and 3TB) and one of those 24-bay
disk array shelves I was writing about before. I want to have a big
contiguous pool of storage onto which I can store backups. Currently I
need something like 15-20TB of storage to feel comfortable, but this
may grow over time. I will be backing up using gigabit ethernet (which
might make restores interesting in an emergency, but oh well).
Here is my dream setup:
- One big storage pool
- Able to serve up SMB shares for inferior operating systems to use as
backup targets
- Able to add new drives as I need them without worrying about drive
sizes too much
- Able to replace small drives in the array with bigger ones and use
the additional capacity easily
- Able to use a lot of the space on the drives (so full redundancy is
not critical)
- If a drive dies I don't lose everything (but can afford to lose what
was on that one drive)
- Well supported, preferably on Ubuntu
- (Relatively) easy for others to maintain and use
- Good power usage?
I do not think I can achieve the dream. Here is what I have thought
of/looked into so far:
ZFS with FreeNAS or Ubuntu (using ZFS on Linux), raidz
------------------------------------------------------
- Allows a fair amount of storage without full redundancy?
- Does NOT allow me to add new drives easily. I can only add entire
sets that match the existing raidz array, turning RAID5 into RAID50?
- If a drive dies I can rebuild (with the usual RAID5 caveats)
- ZFS on FreeNAS might be preferable to Ubuntu, but I might still go
with Ubuntu
- Good protection against corruption
- Maybe difficult for others to administer?
ZFS with FreeNAS/Ubuntu, raid10 (or mdadm)
------------------------------------------
- Easier to add new drives as long as they are in pairs
(Do the pairs have to be matched to previous pairs?)
- Full redundancy which wastes space
- If a drive dies I can rebuild it
- Maybe it is hard/impossible to remove a pair of drives from the set
ZFS with FreeNAS/Ubuntu, raid0 (or mdadm)
-----------------------------------------
- Maximize storage
- But if one drive dies I lose everything
- Maybe I can easily add to the array?
- I don't think I can take anything out ever
Plain LVM on Ubuntu
-------------------
- Maximize storage
- Have one contiguous pool
- What happens if one drive dies? Do I lose everything?
Individual drives
------------------
- I lose having a big contiguous pool for backups, which sucks.
- Easy to add and remove drives, though
In all cases I think I can serve up SMB 2/3 shares for inferior
operating systems. Maybe I can do iSCSI something, but I do not know
whether I should care about this.
Are there other options I am missing or should consider?
I presume (from talking to Lori) that ZFS is the thing I should lean
towards for data integrity reasons, but I am open to people telling me
that is overkill and I should use something else instead. It feels as
if ZFS has the momentum in the storage filesystem wars, anyways (as
compared to btrfs/xfs, or ReFS on Windows).
A bunch of you are computer/storage geniuses. What do you do? What
should I do?
- Paul
--
http://pnijjar.freeshell.org
https://off-topic.kwlug.org/watcamp
_______________________________________________
kwlug-disc mailing list
kwlug-disc at kwlug.org
http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20190807/38db51b3/attachment.htm>
-------------- next part --------------
I have a bunch of drives (mostly 1TB and 3TB) and one of those 24-bay
disk array shelves I was writing about before. I want to have a big
contiguous pool of storage onto which I can store backups. Currently I
need something like 15-20TB of storage to feel comfortable, but this
may grow over time. I will be backing up using gigabit ethernet (which
might make restores interesting in an emergency, but oh well).
Here is my dream setup:
- One big storage pool
- Able to serve up SMB shares for inferior operating systems to use as
backup targets
- Able to add new drives as I need them without worrying about drive
sizes too much
- Able to replace small drives in the array with bigger ones and use
the additional capacity easily
- Able to use a lot of the space on the drives (so full redundancy is
not critical)
- If a drive dies I don't lose everything (but can afford to lose what
was on that one drive)
- Well supported, preferably on Ubuntu
- (Relatively) easy for others to maintain and use
- Good power usage?
I do not think I can achieve the dream. Here is what I have thought
of/looked into so far:
ZFS with FreeNAS or Ubuntu (using ZFS on Linux), raidz
------------------------------------------------------
- Allows a fair amount of storage without full redundancy?
- Does NOT allow me to add new drives easily. I can only add entire
sets that match the existing raidz array, turning RAID5 into RAID50?
- If a drive dies I can rebuild (with the usual RAID5 caveats)
- ZFS on FreeNAS might be preferable to Ubuntu, but I might still go
with Ubuntu
- Good protection against corruption
- Maybe difficult for others to administer?
ZFS with FreeNAS/Ubuntu, raid10 (or mdadm)
------------------------------------------
- Easier to add new drives as long as they are in pairs
(Do the pairs have to be matched to previous pairs?)
- Full redundancy which wastes space
- If a drive dies I can rebuild it
- Maybe it is hard/impossible to remove a pair of drives from the set
ZFS with FreeNAS/Ubuntu, raid0 (or mdadm)
-----------------------------------------
- Maximize storage
- But if one drive dies I lose everything
- Maybe I can easily add to the array?
- I don't think I can take anything out ever
Plain LVM on Ubuntu
-------------------
- Maximize storage
- Have one contiguous pool
- What happens if one drive dies? Do I lose everything?
Individual drives
------------------
- I lose having a big contiguous pool for backups, which sucks.
- Easy to add and remove drives, though
In all cases I think I can serve up SMB 2/3 shares for inferior
operating systems. Maybe I can do iSCSI something, but I do not know
whether I should care about this.
Are there other options I am missing or should consider?
I presume (from talking to Lori) that ZFS is the thing I should lean
towards for data integrity reasons, but I am open to people telling me
that is overkill and I should use something else instead. It feels as
if ZFS has the momentum in the storage filesystem wars, anyways (as
compared to btrfs/xfs, or ReFS on Windows).
A bunch of you are computer/storage geniuses. What do you do? What
should I do?
- Paul
--
http://pnijjar.freeshell.org
https://off-topic.kwlug.org/watcamp
_______________________________________________
kwlug-disc mailing list
kwlug-disc at kwlug.org
http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
More information about the kwlug-disc
mailing list