@Dashrender said:
At this point I also have to ask myself - are you saying these things just to rial me up, like JaredBusch seemed to be to Scott the other day?
Nah, if I wanted to do that, I would do other things. Got better things to do with my time than just rile you up. 
I've dealt with this kind of mentality with bosses before. Really, you gotta break out of their mold and start breaking it down into a much more concise set of requirements.
Your cloud based EHR is your most important application, and if they are saying that nothing else matters then its time to ignore anything with onsite resources. That shit can be set it in the colo cage and is not the problem here. You need bandwidth to the cloud but cannot abide by any downtime. Since it's public traffic, there is no need for MPLS, fancy fiber loops between sites, or even the VPN solution I mentioned with Peplink. Your sites don't care if they can talk with each other, so any interoffice communication is pointless and unnecessary. You need interwebs and you need it to be working when you need it working. It sounds as though bandwidth is a pointless consideration, you can probably get away with 1.5Mbps loops for the amount of traffic you are probably generating.
There are two ways to handle this. One way is to add an additional high quality loop into the mix. This can be from a third party vendor, so you wouldn't have any layer 3 issues upstream, like if Cogent decided to stop routing traffic from one ISP. This is gonna cost you money because quality loops are not cheap. But this doesn't eliminate any layer 2 issues, like a cut fiber or central office burning to the ground. Even if you went with different mediums, like cable plant versus fiber versus traditional telco, you will still have layer 2 potential issues. The setup would be fairly simple, even if you didn't have fancy load balancers behind it. Worst thing that would happen would be you reload your ASA or whatever firewall you have with a config for the other ISP.
The other way is to go cheap but go many. Swarm the problem with super cheap pipes from multiple locations. A mix of commodity pipes with MVNOs and any other wireless vendor you can find. Sure, one pipe might go down, but odds are the Sprint, AT&T, and Verizon towers won't and you would still have other pipes to fall back on. This is a much more complex setup, but could be much cheaper for what you have now or move to with a second quality pipe. Failover would require more complex equipment, like the Peplink, but would keep downtime to a minimal.
You need clear directives from the bosses. If they are gonna shoot down anything that cost more than their current setup or focus and spend money on things that don't matter, then this is an exercise in futility because I wouldn't dare guarantee a cheaper solution because I know what it will entail. If you are requiring more bandwidth, the costs are going to increase no matter what. If ANY other items come in, like local services, then internet services are only a small part of the issue here. Then it's time to take a look at all your services, the business workflow, and what can be done to streamline everything and start doing things right. Moving to a colo with better pipes locally would really help.
