@scottalanmiller said in Building your own lab:

@Pete-S said in Building your own lab:

@VoIP_n00b said in Building your own lab:

Enjoy your $500+ power bill

Depends on what you pay per kWh and what you put in the nodes.

We have similar servers at work and they draw about 55W per node with VMs running but almost no load. I think that would be a typical power profile in a lab setup.

So let's say 250W for four servers. That's 6 kWh per day and 180 kWh per month. If you pay 14 cents per kWh that's $25 per month in electricity.

And if you have it at home maybe you don't need all nodes running all the time.

Right, you can easily use just two, but get the experience of the hardware.

That's a good point. You could potentially populate just two nodes if you wanted for starters. Then when you need more, you could pick up some more CPUs and memory and populate the others.

A minimum config could be two nodes with just 1 CPU and 32GB of RAM each. That would be...$700.

First off, I'm a heavy xcp-ng user but don't use Xen Orchestra so I don't know how it is setup.

But why not an iso file instead of an xva? It's much smaller and more flexible.

Debian/ubuntu iso bundled with a preseed file will create a self installing system with just a 350MB file size.

You can run a script on the hypervisor that creates the vm and pulls down the iso (if it's not there), and starts the VM which then installs the OS and XO.

@DustinB3403 said in Looking for a NC host for an Open Source project:

@Pete-S said in Looking for a NC host for an Open Source project:

First off, I'm a heavy xcp-ng user but don't use Xen Orchestra so I don't know how it is setup.

But why not an iso file instead of an xva? It's much smaller and more flexible.

Debian/ubuntu iso bundled with a preseed file will create a self installing system with just a 350MB file size.

You can run a script on the hypervisor that creates the vm and pulls down the iso (if it's not there), and starts the VM which then installs the OS and XO.

Because needing an ISO would require people to either have XCP-NG Center installed or know how to build a VM via the command line.

What this provides is a direct import of a working management interface from a root shell on the hypervisor.

The user is downloading a script anyway to get started, bash -c "$(curl -s http://xoa.io/deploy)".

So it doesn't matter to the user if the script needs to create a vm with xe vm-install or if it will do it with xe vm-import.

I understand that you want to copy the method the Xen Orchestra guys have chosen. Personally I think it's suboptimal, even if their xoa.xva file is "only" roughly 800 MB in size.

@Dashrender said in Help setting up routing:

Static assignment 125.25.25.1 - 172.16.16.200.11
Static assignment 125.25.25.2 - 172.16.16.200.12
Static assignment 125.25.25.3 - 172.16.16.200.13
Static assignment 125.25.25.4 - 172.16.16.200.14
Static assignment 125.25.25.5 - 172.16.16.200.15
Static assignment 125.25.25.6 - 172.16.16.200.x (all others)

So you mean 1:1 NAT on the first 5 public IPs and 1:many on the .6 IP?

BTW, network is 125.25.25.0/29 and not 125.25.25.6/29.

@marcinozga said in NAS for Plex use... Again:

It really depends how much storage and CPU you need. If you need lots of storage, nothing beats unlimited, and I think only G Suite Business is viable option. I know lots of people host Plex with Hertzner, Vultr is probably attractive option too.

I had a look and it looks like you need 5 minimum users on G Suit Business to get unlimited TBs. If it's $12 per month then that becomes $60 per month. Under a five year period that's $3600.

Not saying it's the same thing but for the exact same money you can buy 9 x 16TB enterprise drives with 5 year warranty. That's about 100 TB of actual storage.

Using Drive makes sense in your case but if someone only needs say 10-15 TB, I'm not sure it does. And 10 TB may not sound like a lot but if we are talking about H.264 video it's more than 3000 movies/5000 episodes. Even if you binge watch 5 hours a day, every day, it will take about 3 years to get through it.

@JasGot said in Sending Secure E-Mail?:

The dept is engaged in a grant program with the State Department of Environment…, which requires us to include our banking information on every reimbursement application.

Come to think of it, banking information is not really sensitive info, is it? If you send an invoice to anyone, they have your banking information.

The only risk here is a man-in-the-middle attack where banking information is changed on the application while it's being submitted. So that the money is transferred into another account.

So do the company send all their invoices and ordinary mail containing banking info by registered mail in locked containers, so it is secure from end to end?

If not, then email isn't any less secure.

@EddieJennings said in Redoing Home Network:

@jmoore said in Redoing Home Network:

@EddieJennings said in Redoing Home Network:

@jmoore said in Redoing Home Network:

@Dashrender said in Redoing Home Network:

I've with JB - You should save the money and get an ER-4. The processor is the same.

POE can be done in the switches, so no need for that in the router.
The ER-4 is nearly half the ER-6.

I already ordered the pieces. Thanks for your input though. I needed a router with 4 ports for my 4 rooms plus the incoming port. I plan to use and learn everything about it.

So each room will be its own network?

Each room will have an 8-port switch because everyone has a lot of devices.

And since those switches will be going to a unique port on your router, they'll each be unique subnets (networks).

It's a good idea to have small switches in each room but normally you would do it like this:

If you wanted to have different networks inside the infrastructure, you'd use VLANs.
If you wanted to use PoE devices like security cameras, IP phones etc, you'd do PoE on the access switches.
If you'd want high performance the core switch would have 10 gigabit ports.
APs could also be connected in different places, for instance on the access switches.

@scottalanmiller said in Installing Laravel on Ubuntu 20.04:

@Pete-S said in Installing Laravel on Ubuntu 20.04:

@Pete-S said in Installing Laravel on Ubuntu 20.04:

@scottalanmiller said in Installing Laravel on Ubuntu 20.04:

@Pete-S said in Installing Laravel on Ubuntu 20.04:

OK, if you are not running apache or nginx, you should install the php-cli package instead.

So that seems to get installed anyway as a dependency on its own.

Yes, it does. But by using the php package and not php-cli, you probably got apache installed on your system as well - by dependencies.

You could find out by running: apt list --installed | grep apache

Or systemctl status apache2 to see if it's running.

Even if it was, Laravel uses Artisan's server.

I'm guessing they are invoking php's built-in webserver.

Regardless, the point is that if you swap php to php-cli in your install guide you don't get apache and other stuff you don't need.

@JasGot said in How to start taking a company to Microsoft 365 based operations.:

Any thoughts on what to think about and how to accomplish the goal?

Well, going "cloud based" is not a goal, that's a means to an end.
It would be interesting to know the goal. Not only interesting but crucial.

@JasGot said in How to start taking a company to Microsoft 365 based operations.:

@Pete-S said in How to start taking a company to Microsoft 365 based operations.:

@JasGot said in How to start taking a company to Microsoft 365 based operations.:

Any thoughts on what to think about and how to accomplish the goal?

Well, going "cloud based" is not a goal, that's a means to an end.
It would be interesting to know the goal. Not only interesting but crucial.

1. Their goal is to do away with any local (onsite) IT costs, including support. Not something I am in favor of, since we are an IT support company

2. because everyone says "cloud" is the way to go.

Well, they still going to have offsite costs and support needs. Who is going to setup O365 for them? Who will add users? Who will setup authentication for them so they can use their O365 login credentials in app xyz? Who will help them when they can't get it to work?

And of course they still going to have windows clients.

I wouldn't be surprised if their support need increases.

@scottalanmiller said in When Does It Stop Even Being IT: Buyers vs Doers:

@IRJ said in When Does It Stop Even Being IT: Buyers vs Doers:

It will cost you more support hours and you will get less productivity.

You say this, but we do this every day and this is absolutely false. Again, I'm not hypothesizing or just trying to push a point, we literally can lower our support pricing for companies doing this because it costs so much less to support. And that's when we aren't the ones buying it.

The amount of additional time needed to deal with the constant breaks and bugs in MS Office is costly. When you are the one paying for the time on tickets, you pay attention. When you pay for the time managing the licensing, you pay attention.

It's easy in the trenches, especially once you are a server engineer, to forget how many tickets and management time is going in to fixing registry breaks, account problems, licensing decisions, application incompatibility, upgrades, etc. for a product. But when you are looking at the tickets over thousands of users, it gets really obvious just all of the ways that a product is costly more to support than another.

We have right now one Windows PC that is royally screwed up after having had O365 locally installed with Microsoft login. I'm not even going to bother looking at it. Just reinstall OS + everything and start over. Which takes much, much, much longer than installing for instance ubuntu + libreoffice.

@IRJ said in When Does It Stop Even Being IT: Buyers vs Doers:

@scottalanmiller said in When Does It Stop Even Being IT: Buyers vs Doers:

@IRJ said in When Does It Stop Even Being IT: Buyers vs Doers:

You proved my point with this thread. Imagine if you just ponied up the money for Office 365 and didnt have to spend time doing all this dumb bullshit making it harder for users to collaborate.

It's funny how you call "more efficient with no problems at all", "all this dumb shit." We've had zero issues so far with users who have moved over. Zero. You act like you are confident it's crippling our businesses. Do you know the last time that we received a collaboration document in an MS Office format? Like... I can't even remember. Yes, once in a while we get CVs in that format. But you seem to think that LibreOffice doesn't work with those file formats. But it does, just fine.

You are basing your arguments on the theory that we are running into problems with LibreOffice and not with MS Office. But my point was, that that's the opposite of what's happening. Nearly every customer with MS Office is having it break on them and can't collaborate. None of the ones with LibreOffice are having that.

So this whole line of thinking makes no sense.

Are you Pete S. ? I assumed you were and you answered me like you were him

You have to realize we have clients. All my stupid posts on this forum doesn't reflect how we run our internal IT at all.

@biggen said in NVMe and RAID?:

I was playing around on the Dell configuration website building out an Epyc 2 socket machine with an NVMe backplane. What I noticed is there is no RAID availability for this configuration. How is this handled then if I wanted to put in two identical NVMe U.2 drives and mirror them? Is hardware RAID not an option for this configuration? Is this left to the OS you choose now?

We spec'd a handful of those Epyc 2 Dells with NVMe last year for a hyperconverged cluster.

Intel has VROC which is md raid (software raid) behind the scenes but that doesn't work on AMD CPUs. And you need BIOS support etc.

But I know people who put in 8 NVMe drives and run standard md raid with massive performance numbers.

Blind swap is not a big deal. You can fix that with a simple cron job. If the array is degraded and you put in a new drive in the old slot with the same or larger capacity it will automatically start a rebuild.

@PhlipElder said in NVMe and RAID?:

@scottalanmiller said in NVMe and RAID?:

@Pete-S said in NVMe and RAID?:

If you do a fileserver like this, skip the hypervisor completely and run it on bare metal. You'll lose at ton of performance otherwise.

Agreed. This is one of those rare exceptions.

I'm not sure about this claim? Maybe ten years ago.

The above solution I mentioned has the workloads virtualized. We've had no issues saturating a setup with IOPS or throughput by utilizing virtual machines.

It's all in the system configuration, OS tuning, and fabric putting it all together. Much like setting up a 6.2L boosted application, there's a lot of pieces to the puzzle.

EDIT: As a qualifier, we're an all Microsoft house. No VMware here.

We're not talking about any fabric because we are talking about local NVMe storage. Data goes straight from the drive over the PCIe bus directly to the CPU.

For high performance I/O workloads the difference between virtualized and bare metal has increased, not decreased, because the amount of I/O you can generate has increased.

When everyone was running spinners and SAS, you couldn't generate enough I/O for the small overhead that virtualizing added to matter. A few percent at most.

As NVMe drives becomes faster and faster and CPUs have more and more PCIe lanes it's not difficult to generate massive amount of I/O. Then every little added overhead for each I/O operation will become more and more noticeable. That's because the overhead becomes a larger percentage of the time, as the total time for the I/O operation becomes shorter.

That's why the bare metal cloud market has had massive growth the last three years or so. There is simply no way to compete with bare metal performance.

Typical bare metal server instances that for instance Oracle offers, runs on all NVMe flash local storage. They put 9 NVMe drives on each server. With high performance NVMe drives that's almost 20 Gigabyte of data per second.

@biggen said in NVMe and RAID?:

Chatting with Dell, they don't offer any of their Epyc servers with any 40Gbe offerings. They only go up to dual 25Gbe. They offer HDR100 Infiniband and Fibre channel, but these are pretty foreign to me and I don't even know if they can be used.

It's totally random what Dell offers and what they don't.

They have the Intel XL710-T2L which is dual port 10 GbE but not the XL710-QDA2 which is the dual port 10/40 GbE. It's the same driver and everything.

You could of course buy the network card anywhere you'd like and plug it in.

40 GbE is actually 4x10 GbE internally inside the interface. That's why 10 GbE switches have 40 GbE uplinks ports.

The interface is called QSFP+ as in Quad SFP+ (SFP+ being 10GbE, SFP being 1GbE)

And 25 GbE is the upgrade of the 10 GbE. Interface is called SFP28. Same physical dimensions as SFP+.
And 25 GbE switches have 100 GbE uplinks, because 100GbE is 4x25 GbE internally. And the interface is called QSFP28.

@taurex said in NVMe and RAID?:

@Pete-S I'd stay away from the 7xx Intel NICs, I heard lots of bad things on different IT forums how they play up. The Mellanox NICs would be my first choice for anything with RDMA support.

I just picked it because that is what Dell sells. It's a simple card, no RDMA, but I don't think RDMA is needed in a fileserver application like this with huge files.

I'm surprised to hear that people have problems with it because it's been around for 5-6 years something now and Intel have newer cards as well. You would kind of assume they've worked out the kinks by now.

Anyway, it more a proof-of-concept at this point. You got to have some numbers to play with to see if it's economically feasible for the customer. What you end up will depend on the budget and what the needs actually are. Switches are also a big cost when it comes to 10GbE and faster.

And yes, Mellanox is good stuff.

@PhlipElder said in NVMe and RAID?:

@biggen said in NVMe and RAID?:

The SFF-8654 to dual SFF-8643 is a bit of a unicorn isn't it? Heck, the SFF-8654 isn't even listed in the SAS wiki.

They are now. Finding them was a real challenge. And even then, we need to order them in bulk.

We may put a few up for sale for folks doing custom builds since they are so hard to find.

We have plans for them.

It could be good to know that Broadcom/LSI have them. They're used on Broadcoms Tri-Mode storage adapters (SAS/SATA/NVMe).

I think it's this model you want: 05-60002-00

Fanvil is readily available in many regions and the problem people have had is not with the hardware. It's firmware upgrades that breaks things, intermittent problems, provisioning stops working, DSS keys that stops working etc. All related to software.

Consensus among those that deploy both Yealink and Fanvil has been to get Yealink if possible as Fanvil causes a lot more support issues over time.

But hopefully Fanvil is getting better so it will be interesting to see what you guys think in a year or so.

So the tech said something inaccurate, so what? That happens just about everyday in every company on earth.

BTW, the title on the thread is misleading. AVImark didn't state anything.