[kwlug-disc] Interesting hardware: ARM, Ceph, server-per-hdd

[Mark Steffen]

https://ceph.io/geen-categorie/500-osd-ceph-cluster/

This is older but at least they had the sense to use ECC memory to 
prevent data corruption (ZFS also recommends this).  I know for homelab 
or less critical data small amounts of corruption may be acceptable but 
if you are investing a lot of money into a Ceph cluster you're probably 
going to want to ensure your data is not easily corrupted.  BlueStore 
does have checksumming and stuff, but if the data is corrupted in memory 
before it's first written to the primary OSD and then replicated, that 
checksum won't help because the checksum will be of the corrupted data 
anyway.

ODroid has a board that might make a good ceph OSD node: 
https://www.hardkernel.com/shop/odroid-hc4-oled/ but again, would be 
nice to have ECC and two ethernet ports capable of 2.5gbps or a single 
port capable of 5gbps (even with a 10gbps interface these little SOCs 
likely won't push past 5gbps anyway)

--
Mark Steffen | Mobile/Signal/Telegram +1 226 600 0464
__

On 2020-12-28 12:30 a.m., Mikalai Birukou via kwlug-disc wrote:
>
>> Cool appliance for sure.
>> But trying to find the pricing of it online using some Google dorking
>> turns up nothing.
>> Not a good sign. It's probably an ARM and a leg.
>
> These guys are from Taiwan? First model 200 was with ARM on 32 bits, 
> 2GB RAM per node. Never model 400 is on ARM 64 bits with 4GB RAM per 
> node. It doesn't seem like a fluke. I was walking around net, and one 
> of the resellers, from who I found this, had configuration that said 
> around $5,990 for base, i.e. without hdds. When you press on 
> configure, it shows Base Config price on top of which is reseller is 
> pushing its pricey hdds :) . 
> https://store.avantek.co.uk/mars-400-ceph-storage-appliance.html
>
> There is an additional licensing for their UI to Ceph via browser, if 
> you need it.
>
> With, currently at newegg.ca, $559 for 14TB, WD Red Plus, already CMR 
> drive, server plus drives work out to $93.4 per TB. Only external 
> switches and cables are needed for whole esthetics.
>
> My feeling is that this server better be used strictly for OSDs, while 
> plenty of common hardware can work as other parts of Ceph complex. 
> Price and performance wise.
>
> With 14TB HDD sizes, rebuilding is a chore. May be it better to have 
> dedicated cpu, buses and lan?
>
>>>> Interesting use of many ARM servers:
>>>> https://www.ambedded.com.tw/en/product/ceph-storage-appliance.html
>>>>
>>>> Its a one hdd per ARM server board model, with eight in 1U.
>>> And there is even a presentation way back from 2017! In Boston:
>>>
>>> https://www.youtube.com/watch?v=4rY3yrE2ysQ&feature=emb_logo
>
>
> _______________________________________________
> kwlug-disc mailing list
> kwlug-disc at kwlug.org
> https://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org