Advice On a New Setup
-
Hi, i've been reading for last 3 weeks or so about sam-sd (open storage) but i can't decide which is best for my situation, and i would really appreciate your opinions on this setup as a (veeam backup, file archive and a lightly accessed file server) on site and an identical mirrored one off-site next building (1gb fiber)
Current Setup
2 old ibm x3650-7979 with raid 10 (6x 2tb wd black), please dont judge, it was already here when i started working here and we're moving forward :thumbs_up_medium-light_skin_tone:
but now it is so slow with backups and restores, so power hungry, raid is doing funny things and corruptions, low storageGoal
Run a 'proper' storage server OS
Upgrade storage capacity
distributed file system between the two serversthought of Dell R510 (Acctually, it is the only one beside r710 with 3.5 we can find as a refurbished and more than 6 hd bays)
single 6 core cpu
32g Ram
Raid H700
12 (empty bay)What i could find in my county is these hard drives
Seagate IronWolf NAS 4tb, 6tb, 8tb, 10tb
Western Digital Gold 4tb, 6tb, 8tb, 10tb, 12tb
Western Digital Red Pro 8tb
Western Digital Red 4tb, 6tb, 8tbwhat we can afford is 4tb, 6tb or 8tb of any model, unfortunately the increase in price between 8tb to 10tb or 12tb is out of our reach,
i would like your opinions on
1- Which first choice and second choice drive model to buy,
2- Which RAID setup if i want at least double the current setup capacity with the availability to expand the array later with just adding more drives and if it is OBR10, is it expandable on this setup or just the OBR6
3- Which linux (Virtualized) OS to use? im familiar with most of them
4- Which tool to use to mirror the two servers and distribute the load , can i use gluster or ceph or it is not worth it and just use DRDBthanks in advanced guys.
-
@GodfatherX64 said in Advice On a New Setup:
What i could find in my county is these hard drives
Seagate IronWolf NAS 4tb, 6tb, 8tb, 10tb
Western Digital Gold 4tb, 6tb, 8tb, 10tb, 12tb
Western Digital Red Pro 8tb
Western Digital Red 4tb, 6tb, 8tb1- Which first choice and second choice drive model to buy,
You likely want to rule out Seagate due to reliability concerns. See @CCWTech for info on why to avoid Seagate.
Now between Red, RedPro, and Gold. Red you likely want to rule out because those are 5400 RPM Green drives with TLER blocking removed. They are okay to use, but their speed alone is likely a bit concern for you in the real world. Find for storing archives, but they are insanely slow overall.
So that leaves Red Pro and Gold. Between these two, you have only the 8TB in the Red Pro, so almost certainly you will be going with the Gold just for availability reasons. Either is fine.
-
@GodfatherX64 said in Advice On a New Setup:
2- Which RAID setup if i want at least double the current setup capacity with the availability to expand the array later with just adding more drives and if it is OBR10, is it expandable on this setup or just the OBR6
OBR10 is almost never expandable. OBR6 generally is. OBR6 will be far less safe, especially as your array size gets bigger and bigger. And write speeds will be much lower. But it gives you more capacity at a lower price point.
-
@GodfatherX64 said in Advice On a New Setup:
3- Which linux (Virtualized) OS to use? im familiar with most of them
Most any are fine. I tend to use Fedora these days. Suse is best for the most storage options and updates, but that's not a big thing anymore. Ubuntu is fine.
-
@GodfatherX64 said in Advice On a New Setup:
4- Which tool to use to mirror the two servers and distribute the load , can i use gluster or ceph or it is not worth it and just use DRDB
This isn't something you would do. No one does this, not even in super high performance scenarios. Things like DRBD, Gluster, and CEPH are for use on the raw storage, not for use on top of your RAID.
If you needed clustering here (trust me, you don't) then you'd do it lower in the stack with Starwind, not up on top in the VM.
Keep in mind even big shops when they talk about clustering for file servers it is for failover, not load balancing.
-
@scottalanmiller said in Advice On a New Setup:
@GodfatherX64 said in Advice On a New Setup:
2- Which RAID setup if i want at least double the current setup capacity with the availability to expand the array later with just adding more drives and if it is OBR10, is it expandable on this setup or just the OBR6
OBR10 is almost never expandable. OBR6 generally is. OBR6 will be far less safe, especially as your array size gets bigger and bigger. And write speeds will be much lower. But it gives you more capacity at a lower price point.
You don't want do expand TBs of data. I've been through three of them, where one failed. The first took over a month, which was a big RAID 5... and afterwards a drive failed, but rebuilt again fine. I can only imagine a similar RAID 6 taking longer. The second was a few TB RAID 10 SSD array, which didn't take long and was fine. The third was a large RAID 10 which took forever and failed.
Either plan ahead capacity wise, or have a way to offload your data somewhere safe and do a complete RAID rebuild instead of expansion.
-
@Obsolesce said in Advice On a New Setup:
@scottalanmiller said in Advice On a New Setup:
@GodfatherX64 said in Advice On a New Setup:
2- Which RAID setup if i want at least double the current setup capacity with the availability to expand the array later with just adding more drives and if it is OBR10, is it expandable on this setup or just the OBR6
OBR10 is almost never expandable. OBR6 generally is. OBR6 will be far less safe, especially as your array size gets bigger and bigger. And write speeds will be much lower. But it gives you more capacity at a lower price point.
You don't want do expand TBs of data. I've been through three of them, where one failed. The first took over a month, which was a big RAID 5... and afterwards a drive failed, but rebuilt again fine. I can only imagine a similar RAID 6 taking longer. The second was a few TB RAID 10 SSD array, which didn't take long and was fine. The third was a large RAID 10 which took forever and failed.
Either plan ahead capacity wise, or have a way to offload your data somewhere safe and do a complete RAID rebuild instead of expansion.
Agreed, RAID growth is a form of induced failure and carries huge risks. It's almost never worth it. And even less worth it if you plan for it from the beginning. It's a bad fallback situation, not something to do intentionally from the beginning.
-
I'd say ditch the whole idea of file servers, especially if they are lightly used and move to cloud storage. $10 per user per month gets you unlimited storage from Google. You would only need local storage for Veeam backups, in one location, since you have gigabit fiber between sites. And for that you could just get Supermicro server, Asrock rack has some nice options too, or even build one. Synology or similar will work fine as a target for backups too.
-
@marcinozga said in Advice On a New Setup:
I'd say ditch the whole idea of file servers, especially if they are lightly used and move to cloud storage. $10 per user per month gets you unlimited storage from Google.
You sure that's the pricing?
-
On GSuite, they claim unlimited if you have six users or more, 1TB limit if you don't.
-
That's my account, G Suite business for $10, 1 user. They don't enforce 5 users minimum, but any business could afford 5 users.
-
I had to buy 4 drives few days ago, black friday sale, 4TB WD Red for $100 each, because all 10 of my drives are failing. That would have paid for 3 years and 4 months of G Suite, and with power cost it'd probably give me 4 years. Unfortunately I still need few TBs of local storage, but I just cannot justify the cost.
-
@marcinozga said in Advice On a New Setup:
That's my account, G Suite business for $10, 1 user. They don't enforce 5 users minimum, but any business could afford 5 users.
Odd that they price it so insanely at odds with their other plan.
-
(Ab)use it while you can
-
@marcinozga said in Advice On a New Setup:
I had to buy 4 drives few days ago, black friday sale, 4TB WD Red for $100 each, because all 10 of my drives are failing. That would have paid for 3 years and 4 months of G Suite, and with power cost it'd probably give me 4 years. Unfortunately I still need few TBs of local storage, but I just cannot justify the cost.
That works for one user, but for a business, it gets costly quickly. We certainly don't spend $120/year for storage for our users. We are a tiny business, but even so, we would be $2,280 per year for storage. Sure, if we used several PB of data, that would be hard to beat, but who does that?
In reality, we use under 1TB. So that cost is absurd. Even if we used a lot more, four drives of 4TB each in RAID 10 is 8TB for just $400 in drives. Add a small server to run NextCloud and maybe we spend $1,400. Let's say we go with a collocation facility to put it in. That's another $900. That's only $2,300 for the first year, and $900 for each additional year.
Over five years, we'd spend $5,900 instead of $11,400. And that's assuming we don't have any virtualization or shared computing at all, that's getting a dedicated server, dedicated drives, and dedicated colocation for that one box, and assuming we need 8TB of storage. For what we really need, we could easily make do with two 2TB drives taking that $400 cost down to just $165.
Now imagine a larger business with 40 or 80 or even 200 users. The expected per user storage needs would plummet while the cost for Google storage would continue to grow exponentially.
So it is anything but a slam dunk. For one or two users, that stuff is hard to beat. But for even a pretty small firm, it can be insanely expensive and limiting.
-
@marcinozga said in Advice On a New Setup:
(Ab)use it while you can
Really just shows their other plan as being crazy. Both seem to gouge on the pricing, though.
-
We just lost a customer who did a SAM-SD install a few months ago. They put in a server, I think it was about $6,000. It had over 48TB of storage on it. It's a lot of storage, but they have over 140 users.
With G Suite, it would have cost them $16,800 for that same storage per year. So in a five year window the one used in house would remain at $6,000 while the G Suite approach would balloon to $84,000. A difference of $78,000 in a real world SAM-SD scenario that we just went through.
That's not even considering that they don't have a network connection that could handle the traffic to have G Suite do the hosting. That's a separate, but very important issue. They need file transfer speeds at GigE and decently reliable. Their Internet (they are in the US) isn't very fast and definitely not stable.
-
If you had 6x2TB in RAID10 that's only 6TB of actual storage. That's very modest storage requirements.
A simple RAID1 with two 12TB drives would double that. WD Ultrastar (former HGST Ultrastar) for instance.I'm sure you can find other brands of hard drives regardless of what country you are in. You just need to find the right suppliers.
You handle storage expansion by adding a new array of disks.
Regardless of what drives you want even 5400 rpm drives will saturate a gigabit link. I have used WD Red and they were fine but I would look for drives with a 5 year warranty, not 3 years as WD Red have. The price difference is not that big.
Personally I wouldn't waste money on hardware raid but you need some kind of host adapter to access the drives. Software raid is very fast and RAID1 takes no cpu to speak of. With refurbished stuff you have to take what you can find though.
-
I'd also look for something newer than R510. Basically you want Ivy bridge or newer Xeon CPUs because there is a pretty big step down in power usage when Intel started with the 22nm process.
R510 and R710 are too old IMHO. R720 will work though but with the E5-2600 V2 series processor for lower power consumption.
-
wow, thanks guys for your informative opinions
so
@scottalanmiller , WD Gold, OBR10 for performance, fedora (vm) with xfs maybe,@scottalanmiller @Obsolesce , ok , no RAID expansion, plan from the beginning , got it.
@marcinozga , unfortunately, like @scottalanmiller said it will be more costly than local, and the gigabit fiber is between 2 buildings , not the internet connection
@Pete-S , thanks for the 12tb suggestion, but i want performance too if i went for 10gb lan
and unfortunately, i couldn't find any 3.5 servers except these 2, r710, r510
