@scottalanmiller said in Proxmox install for use with a ceph cluster:

We could have told you that. True hardware RAID cannot be bypassed, it's physically in the path, if you cut it out, the drives have to vanish.

Some RAID controllers have a way of running pass through. For the 3108 based cards (Example P730) there is a pass through. Note historically it's been "pretty damn buggy", and it took a lot of joint engineering to get it stable enough for our purposes (don't even dare try it with the 2208 based 6Gbps Avago parts). Now this is kinda moot as everyone's using 3008 pure HBA firmware parts (We stopped certifying RAID controllers from Broadcom for pass through) and the other thing that's making it moot is NVMe running "proper" talks directly to the PCI-E bus. There are "Tri-Mode" RAID controllers that can raid NVMe but I just don't see a point. You bottleneck throughput pretty hard pretty fast.

@scottalanmiller said in Proxmox install for use with a ceph cluster:

Because it HAS to default to SOMETHING. If you wanted anything, you'd have selected it. So they default to what is safest and most common. Why pay for a hardware controller if you didn't have a use for it? The key features of a hardware controller are disabled with R0.

HPE doesn't sell an actual pass through HBA. All their HBA devices are duel use parts. (To be fair, the equivalent line from Broadcom/Avago like the 3008 in theory can be sold with a RAID 1 no cache support, but plenty of OEM's like Dell sell Pure HBA firmware (Sometimes called The IT firmware) such as the HBA 330. Now the Gen9's might of still offered controllers from both ODMs (by Gen10 though that was gone, and I'm pretty sure it was a lot earlier like gen7 where HPE last used Avago parts).

The reality is that even an automotive engineer would still go through the same buying process because:

1. It’s how the channel works.
2. Someone who designs automotive chassis isn’t going to track who has the most cup holders of the smoothest ride.

A SE still has value even with a hardened IT engineer as an SE ca. Provide insight into a niche said practitioner doesn’t spend 40 hours a week on.

@JaredBusch said in Router/firewall recommendations for small branch office:

If you network is down to outside factors you don’t get in trouble for 911 calls not completing. That has never been a thing. POTS goes down all the time.

In theory POTS is more reliable for 911 address lookup. In reality if I'm calling 911 in an office it's likely going to be from my cell phone assuming service.

Nothing stops you from getting a SIM card modem backup for the PBX, or for IP using a SD-WAN solution that bridges in cellular networks to cover normal circuit outages.

@RojoLoco said in Internet connection sharing:

I can't imagine Comcast allowing this, even on a biz class connection. And I feel like asking them about it would raise a huge red flag.

At worst. If you change for it, you likely become legally a telecom provider. Get ready for weird tax’s, logging requirements.

At best, get ready for the other business to get malware, and Comcast’s abuse department shutting down your connection, or the FBI knocking on your door when someone gets involved In something sketchy...

@Pete-S said in Dell PERC H740 with SSDs?:

Only go with HDDs when you don't have any other choice. SSDs will be much faster. A single SSD has IOPs in the several tens of thousands range while a HDD can only do a couple of hundred.

The ultra low end .1DWPD drives can have terrible sustained write latency. I'm not convinced spinning drives might win in this category.

Read intensive SATA SSDs are getting close to the same price as 10K SAS. If you want to save money don't buy SAS SSDs, because you don't need them.

The cheap ones fall over once you exceed the limited SLC or DRAM buffer. Beware using these for large DB copy jobs etc.

If you want to save even more money don't buy drives from Dell

This is... problematic. I've seen a server OEM run into an issue between their HBA/RAID controller nad a given drive, and get the firmware patch only released for their make/model of the drives, and not the general-purpose regular ones. Also, good luck getting the out of band to properly provide SMART. Also unless you are buying something that someone validated, good luck with HotAdd support, or having to use one-off solutions like Intel VMD when doing a pass through drives.

For SSDs look for Samsung and Intel enterprise drives

Going to point out that Intel shares fabs with Micron.

Samsung PM883 or Intel S4510 in your case

Ehhh, be careful here. The PM8xx is the bargain basement cheap Samsung SATA devices. Samsung in the earlier versions of this series had some... interesting firmware bugs so make sure you patch your drives.

The Intel 4510 with newer firmware should work around the previous performance problems on the last generation of these (The S37xx was a nightmare on low QD writes), but again, this isn't the fast drive. This is the cheaper one. You normally have something act as a write buffer to shield these drives from aggressive writes.

@Pete-S said in Dell PERC H740 with SSDs?:

I think not. The H330 (LSI SAS 3008 controller) will push hundreds of thousands of IOPs. The bottleneck will be your SSDs.

The H330 is the crap HBA that has a cut-down queue depth. If memory serves, it's maximum queue depth is equal to a single SAS drive (256 commands). It's still better than the garbage tier H300 from the last generation.

The HBA 330 is the version that has full queue depth.
Note, for high-performance setups, going all NVMe is easier (no need for an HBA).

@biggen said in RAID rebuild times 16TB drive:

I guess I was skeptical I had correct what @Pete-S said because I've seen so many reports that its taken days/weeks to rebuild [insert whatever size] TB Raid 6 arrays in the past. But I guess that was because those systems weren't just idle. There was still IOPS on those arrays AND a possible CPU/cache bottleneck.

Was the drive full? Smarter new RAID rebuild systems don't rebuild empty LBAs. Every enterprise storage array system has done this with rebuilds for the last 20 years...

@scottalanmiller said in RAID rebuild times 16TB drive:

Even on bare metal, we normally see a lot of bottlenecks. But normally because almost no one can make their arrays go idle during a rebuild cycle. If they could, they'd not need the rebuild in the first place, typically.

Our engineers put in a default "reserve 80% of max throughput for production" IO schedular QoS system, so at saturation rebuilds only get 20% so they don't murder production IO. (note rebuilds can use 100% if the bandwidth is there for the taking).

@scottalanmiller said in RAID rebuild times 16TB drive:

Its a system, not an IO, bottleneck typically. Especially with RAID 6. Its math that runs on a single thread.

Distributed storage systems with per object raid FTW here. If I have every VMDK running it's own rebuild process (vSAN) or every individual LUN/CPG (how Compellent or 3PAR do it) then a given drive failing is a giant party across all of the drives in the cluster/system. (Also how the fancy erasure code array systems run this).

@bnrstnr said in What Are You Doing Right Now:

Laptop shopping... Why are ASUS laptops so much cheaper than Dell/HP?? Anybody have a higher end model (ZenBook, VivoBook, ProArt) to compare?

If my mac has something wrong with it, I just have Apple send someone to the house to fix it. Can't get that with ASUS.

@black3dynamite said in Reconsidering ProxMox:

It’s supports multiple storage types like nfs or cifs

Obligatory - https://blog.fosketts.net/2012/02/16/cifs-smb/

(Saying CIFS is my pet peeve).

@scottalanmiller said in Reconsidering ProxMox:

Not ideal, but with compression and dedupe, not the problem it seems to be. And it is far less risky that continuous partials that always depend on rehydration.

2TB of data and a large backup window? Not a huge deal.

50TB of data, and 100% 24/7 workloads? Not going to fly.

@wrx7m said in VMware Host Cannot Connect from vSphere Client:

Could also be an issue if a vendor-specific ISO was used to initially install ESXi. I ran into a similar problem with Dell; I had to get the Dell ISO from their downloads for the specific server model/service tag.

For 6.0 this is fine (you'll use this to get ASNC drivers). You can also add the dell VIB depot to get them this way also.
For 6.7 Dell has (thankfully) stopped shipping ASYNC drivers and moved 100% to inbox. It's all for 3rd party management VIBs. Honestly I've seen 3rd party VIBs be a culprit for updates before. I'd make sure you are updating the BIOS/FIrmware also when updating ESXi (People forget this sometimes).

@Dashrender said in ESXi 6.7 Troubleshooting:

by making it a card - it just seems like a way to charge more money, but yeah.. that's what I'm talking about.. Nice to see it's an option.

"By putting it on the motherboard, it just seems like a way to charge everyone who doesn't use for it

An H330 is going to cost you about the same, and wastes 2 drive slots.
FWIW VxRAIL uses BOSS for boot as of 14Gen so there is a decent volume of them in production (For 13Gen they used a high endurance SATADOM device).

@pmoncho said in ESXi 6.7 Troubleshooting:

I have not seen any M.2 on dell servers yet

All 14G servers AFAIK support adding in a "BOSS" card that has 2 M.2 slots.
https://i.dell.com/sites/doccontent/shared-content/data-sheets/en/Documents/Dell-PowerEdge-Boot-Optimized-Storage-Solution.pdf

@scottalanmiller said in Veeam Company Announcement:

@nadnerB said in Veeam Company Announcement:

@scottalanmiller said in Veeam Company Announcement:

@StorageNinja said in Veeam Company Announcement:

Google does all the #$@% time. (Basically to destroy potential competitors). Precog antitrust actions have killed Inbox, and tons of great products.

Yeah, but Google doesn't buy companies like this. Google buys little unheard of things, not market leaders.

Ahem, Fitbit

Fitbit isn't a market leader. It's not tiny, but it's the largest, non-innovative company in the space. It's just a big, but basic, watch builder. They have a big brand name, but they don't lead the market in sales (they are in third, maybe), and they don't build their own tech. Both bigger brands don't just make their own hardware, they make their own processors!

Serious runners use Garmin. Smart watch leaders are apple and samsung.
Google wanted to pay for a device that would generate location data?

@pmoncho said in ESXi 6.7 Troubleshooting:

I get it, although, to even sell an H330 on a Dell PowerEdge R840 is like selling standard breaks on a Audi A6 and making the driver request anti-lock breaks.

The H330 is at least 10x better than the dumpster fire that was the H300 (256 vs. like 25 QD). Still no cache, bastard megaraid on a HBA blah blah blah is all true.

@pmoncho said in ESXi 6.7 Troubleshooting:

You really only need a 4GB USB key but I like to use 8GB or higher as they are still cheap for a higher quality USB key.

If your doing embedded installs PLEASE get at least 32GB. Your going to have issues with a crash dump partition either now or in the future with 4/8GB at some point.

@travisdh1 said in ESXi 6.7 Troubleshooting:

Does your Dell have the SD card reader? If so, it runs the SD cards as a RAID1 array.

SD Card raid controller are "Kinda raid" there's no patrol read, there's no scrubbing, and it they have ugly habbit of not always flagging failure correctly (forcing you to rip out the bad card).