Our New Scale Cluster Arrives Tomorrow
-
@scottalanmiller said:
@anonymous said:
So what's Scale using? I assume the hardware is all dell boxes? I assume there is a VSAN? What hypervisor are they using?
Not VSAN, this is a bit more advanced than that. It is a RAIN storage cluster using proprietary block mirroring. So no need for SAN links between nodes, the storage talks over 10GigE backplane directly node to node to handle storage replication. The VMs are on a local filesystem, not a remote one.
Hypervisor is KVM.
Hardware is all Dell. This is the newest hardware yet, so no one has it but us, it is R430 nodes.
I'm assuming it's proprietary? A shared backplane between three + hosts and 2 plus sets of disk? Is it, or is it not using a 10 GbE switch (or two) to connect all of the equipment?
AKA could someone buy off the shelf Dell equipment and replicate this? -
@anonymous said:
@scottalanmiller I assume there mega expensive?
Not really. Ours is a big more expensive as we are not on the entry level model and we are going up to six nodes, three of them being pure SSD. So ours is not indicative. But when you consider the hardware cost of three high end R430 nodes, 10GigE cards and such that you would need if building your own or not and that you get support it's pretty competitive. Quite hard to build your own system to compete with it, in fact. Trying to match the reliability and storage efficiency and performance would be hard. You'd likely have to overbuild significantly. If you don't need HA or don't need anywhere near the "scale", obviously you would be overbuying. But once you get into their range, they are extremely cost effective.
-
@Dashrender said:
AKA could someone buy off the shelf Dell equipment and replicate this?
All of the hardware is off of the shelf. It's all Dell line items that could be ordered from Dell directly. Except for the Scale faceplates, of course.
-
@Dashrender said:
Is it, or is it not using a 10 GbE switch (or two) to connect all of the equipment?10GigE Dell switch on ours for the backplane. It's a dedicated node communications channel, not open to the LAN, which is also 10GigE.
-
In a standard setup, each node has 2x 10GigE links to the bankplane and 2x 10GigE links to the LAN.
-
-
@scottalanmiller said:
In a standard setup, each node has 2x 10GigE links to the bankplane and 10x 10GigE links to the LAN.
Each node has 10, count them 10 links to the LAN? You need 30 10GigE ports - damnation! that's a huge chunk of change!
-
Sorry, slipped into binary there. 2 in decimal.
-
@scottalanmiller said:
Sorry, slipped into binary there. 2 in decimal.
ROFL - OK.. that makes more sense. I was thinking, my god, what would you need 100 GB network access for, per host.
2 I'm guessing, mostly for redundancy.
-
@Dashrender Yes, it is active/passive.
-
@scottalanmiller said:
@Dashrender said:
Is it, or is it not using a 10 GbE switch (or two) to connect all of the equipment?10GigE Dell switch on ours for the backplane. It's a dedicated node communications channel, not open to the LAN, which is also 10GigE.
Is there a redundant switch in the backplane?
-
@coliver said:
@scottalanmiller said:
@Dashrender said:
Is it, or is it not using a 10 GbE switch (or two) to connect all of the equipment?10GigE Dell switch on ours for the backplane. It's a dedicated node communications channel, not open to the LAN, which is also 10GigE.
Is there a redundant switch in the backplane?
Not at the moment on ours, but normally yes of course.
-
Beginning more formal testing with the cluster now. Running through Citrix documentation on it. Time to move past the "fun" testing into more serious stuff. LOL
-
So much activity, this thing is really filling up with VMs quickly!
-
We are getting more opportunity to run this cluster through its paces as I start working on the Linux Admin Guide on it.
-
Just found out that all of the gear is, indeed, moving to a US-based datacenter. Which one, I don't know yet, but hopefully will find out this week.
-
@scottalanmiller said:
Just found out that all of the gear is, indeed, moving to a US-based datacenter. Which one, I don't know yet, but hopefully will find out this week.
Too bad you don't want it UK based, I could have got you a deal here
-
@NattNatt said:
Too bad you don't want it UK based, I could have got you a deal here
What kind of facilities do you have over there?
-
@scottalanmiller said:
@NattNatt said:
Too bad you don't want it UK based, I could have got you a deal here
What kind of facilities do you have over there?
From co-lo to full hosting, we have several options available with different benefits etc - including live-scalable type environments etc
-
@NattNatt said:
From co-lo to full hosting, we have several options available with different benefits etc - including live-scalable type environments etc
We have so much gear that we could easily populate a second facility, too bad it is so costly to ship gear from the US to the UK.
