New Infrastructure to Replace Scale Cluster
-
@dyasny Wow, wow....no Hosted Engine? How come everyone keeps pushing HE?
Why no HE? -
@FATeknollogee because it doesn't scale. For a small setup it will work (because you don't want to waste a machine on it), but at scale you will keep getting hit by problems. Remember, the engine runs two postgres databases, both under stress, as well as a java based engine, which is also a resource hog (it's java after all). Add the fact it's doing a lot of network traffic polling all those hypervisors and getting a lot of data about everything they do every 2 seconds, and you have a VM that is doing a LOT.
For a few hypervisors, it will not be a huge issue, but drive that up to a point and you end up in a world of hurt. So for anything large-ish and where reliability is important, just avoid HE.
-
@dyasny The 300 host install you mentioned in the other thread is non-HE?
-
@FATeknollogee absolutely. Pretty much every setup with over 20 hosts I've ever built, wasn't using HE.
-
@dyasny What do you lose without HE?
Without HE does it become a manual setup where one can't use Cockpit to setup? -
@FATeknollogee Yes, it's a simple setup where you run ovirt-engine-setup and it asks you a few questions in the command line. For ease of management, I usually deploy it in a standalone VM on a separate machine. This way, if I need more resources, I can stop the machine, give it some more cores/ram or move it's disk to a faster storage, and start it up again. BAcking it all up is as simple as copying the VM disk.
-
@dyasny What are you deploying as a standalone VM? I thought you said no HE?
-
@FATeknollogee just a regular libvirt/KVM usually. If there is a multivendor virt environment, I install the engine in the second setup (vmware/hyper-v) and often the vCenter is installed in RHV
-
@dyasny Next time I do HE, I think I'll install it as a separate VM instead of the vm inside a vm approach.
-
@FATeknollogee that really depends on your cluster size. If you can afford to dedicate a separate host to it, then why not. Besides scalability, your main benefit will be not having to deal with all the hosted-engine clustering overhead. It really makes life simpler
-
@dyasny said in New Infrastructure to Replace Scale Cluster:
@FATeknollogee that really depends on your cluster size. If you can afford to dedicate a separate host to it, then why not.
You mean a separate host where the HE vm lives on?
-
@FATeknollogee said in New Infrastructure to Replace Scale Cluster:
@dyasny said in New Infrastructure to Replace Scale Cluster:
@FATeknollogee that really depends on your cluster size. If you can afford to dedicate a separate host to it, then why not.
You mean a separate host where the HE vm lives on?
Yes, it's your choice whether to do it in a VM though, it can be on baremetal
-
@dyasny said in New Infrastructure to Replace Scale Cluster:
@FATeknollogee said in New Infrastructure to Replace Scale Cluster:
@dyasny said in New Infrastructure to Replace Scale Cluster:
@FATeknollogee that really depends on your cluster size. If you can afford to dedicate a separate host to it, then why not.
You mean a separate host where the HE vm lives on?
Yes, it's your choice whether to do it in a VM though, it can be on baremetal
Is doing it in a vm bad? That would be my choice unless there is some compelling reason to do it baremetal.
-
@FATeknollogee doing it in a VM is convenient. You can always move that VM to another host, you can easily back it up by copying it's disk and domxml, you can even easily set it up as an HA cluster with pacemaker protecting the libvirt service. Databases though, feel more convenient on baremetal, so if you're going to build something with hundreds of hosts, I'd suggest you invest in the engine host as well.
-
@dyasny so to clarify for me, as i'm fighting a headache.
This design is similar to that of ESXi with vsphere.In that you should have 3 physical hosts, and 1 of which is installed with the vSphere service.
Correct?
-
@DustinB3403 no, in this particular setup, you have two options. The original one would be to go hyperconverged, installing both the storage and hypervisors services on all 3 hosts, and to also deploy the engine (vsphere equivalent) as a VM in the setup (that's called self hosted engine).
The better option, IMO, is to use two hosts as hypervisors, and the third - pack with disks, and use as the storage device (NFS or iSCSI). And also install the engine on it, as a VM or on baremetal - doesn't matter.
You will have less hypervisors, true, but having a storage service on the hypervisors is a resource drain, so you don't actually lose as much in terms of resources. And you gain a proper storage server, less management headache, and a setup that can scale nicely if you decide to add hypervisors or buy a real SAN. Performance will also be better, and you might even end up with more available disk space, because you will not have to keep 3 replicas of every byte like gluster/ceph require you to do.
-
@dyasny said in New Infrastructure to Replace Scale Cluster:
@DustinB3403 no, in this particular setup, you have two options. The original one would be to go hyperconverged, installing both the storage and hypervisors services on all 3 hosts, and to also deploy the engine (vsphere equivalent) as a VM in the setup (that's called self hosted engine).
The better option, IMO, is to use two hosts as hypervisors, and the third - pack with disks, and use as the storage device (NFS or iSCSI). And also install the engine on it, as a VM or on baremetal - doesn't matter.
You will have less hypervisors, true, but having a storage service on the hypervisors is a resource drain, so you don't actually lose as much in terms of resources. And you gain a proper storage server, less management headache, and a setup that can scale nicely if you decide to add hypervisors or buy a real SAN. Performance will also be better, and you might even end up with more available disk space, because you will not have to keep 3 replicas of every byte like gluster/ceph require you to do.
Isn't that an IPOD though?
-
@Dashrender said in New Infrastructure to Replace Scale Cluster:
Isn't that an IPOD though?
It's a server, not an ipod. If you mean SPOF, then yes, if the entire server just dies, you lose the cluster. Obviously, the server itself can be backed up, clustered, and installed with redundant components to avoid that. It's all a matter of balancing between budget, the admin's level or paranoia, and the desire to have a reliable setup without working too hard. I really like the latter, and in the long run this approach has always served me and my customers very well.
These 3 factors are up to the OP of course. All I'm trying to say with my suggestion is that you're not saving anything by hyperconverging, because you ARE making the setup much more complex, with many more moving parts that require configuration, tuning, updates and server resources than you would have by just sticking to the KISS principle.
-
IPOD = Inverted Pyramid of Doom.
This a term that Scott Allen Miller coined ages ago.1 - SAN
2 - Switches
2+ - serversThe general belief is that the SAN is 'so good' it will fail less frequently than the other components. Now - in your setup it might be as good as the servers, since it's not one of those manufactured typical SANs, but it's still just a server.
The general idea around these parts you don't go to centralized storage until you have at least 4 hypervisor hosts, otherwise putting storage locally is often much less expensive and less risky.
And of course, we haven't even touched on HA.
-
@dyasny said in New Infrastructure to Replace Scale Cluster:
These 3 factors are up to the OP of course. All I'm trying to say with my suggestion is that you're not saving anything by hyperconverging, because you ARE making the setup much more complex, with many more moving parts that require configuration, tuning, updates and server resources than you would have by just sticking to the KISS principle.
I definitely like the simpler solution - local storage for local VMs. If you need to move VMs between hosts for maintenance - fine - make sure you have enough resources to do just that, then move them. But moving to shared storage at two compute nodes and basically wasting the compute of the third node doesn't make sense to me, not to mention putting yourself in an IPOD situation - where if you loose the disk, you loose both compute nodes.
