No way to create larger than 2TB virtual disk with Xen or XCP-NG?
-
I just popped a 6TB drive into my new xcp-ng host and added it as a storage repo. I then wanted to create one large 6TB virtual disk since I only need it to be used by one specific VM. Come to find out, you can't. You have to break up the virtual disk to no more than 2TB in size. So I'd have to create three virtual disks and then attach all three to the VM in question. Then inside the VM I'd use the specific OS tools to span the disk (e.g. LVM, etc...) so I could realize the entire size and not have to mess with smaller chunks (e.g. multiple drive letters).
Is there no way to just pass the whole disk right to a VM? With KVM, you can simply create a disk element that defines whatever storage you want to pass through and then attach it. Is there no similar functionality with Xen (xcp-ng)?
-
2TB limit is because of ext3.
-
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
2TB limit is because of ext3.
Gotcha. No way to simply pass the disk through?
-
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
2TB limit is because of ext3.
Gotcha. No way to simply pass the disk through?
I’m not sure. It’s been a while since I used XenServer/XCP-ng.
There is a away to use ext4 to get passed the 2TB limit but it’s considered experimental.
-
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
2TB limit is because of ext3.
Gotcha. No way to simply pass the disk through?
No, and you wouldn't want to anyways as you'd be bypassing the hypervisor. The thing that manages the host hardware...
-
@DustinB3403 said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
2TB limit is because of ext3.
Gotcha. No way to simply pass the disk through?
No, and you wouldn't want to anyways as you'd be bypassing the hypervisor. The thing that manages the host hardware...
Do you recalled if there is any virtual disk size limitation when selecting LVM based (thick provisioned)?
-
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@DustinB3403 said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
2TB limit is because of ext3.
Gotcha. No way to simply pass the disk through?
No, and you wouldn't want to anyways as you'd be bypassing the hypervisor. The thing that manages the host hardware...
Do you recalled if there is any virtual disk size limitation when selecting LVM based (thick provisioned)?
It's the same limitation. 2Tb minus 4 GB per vhd.
-
I do recall the xcp-ng team working on getting zsf into xcp-ng thought
-
@DustinB3403 said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
I do recall the xcp-ng team working on getting zsf into xcp-ng thought
I think you mean ZFS.
It's already in the release of 8.0, but this doesn't address the 2TB limit AFAIK.
-
@Danp said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@DustinB3403 said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
I do recall the xcp-ng team working on getting zsf into xcp-ng thought
I think you mean ZFS.
It's already in the release of 8.0, but this doesn't address the 2TB limit AFAIK.
Mobile but yes
-
There must be some other limitation than just “ext3” restricting us to 2TB. According to the wiki, ext3 can use a max file system size between 4TiB and 32TiB depending on the block size.
-
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
There must be some other limitation than just “ext3” restricting us to 2TB. According to the wiki, ext3 can use a max file system size between 4TiB and 32TiB depending on the block size.
Thin provisioning uses vhd and that has a 2TB limitation.
-
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
There must be some other limitation than just “ext3” restricting us to 2TB. According to the wiki, ext3 can use a max file system size between 4TiB and 32TiB depending on the block size.
Thin provisioning uses vhd and that has a 2TB limitation.
Ok. Thanks for that. I may play with that experimental ext4 drivers.
-
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
There must be some other limitation than just “ext3” restricting us to 2TB. According to the wiki, ext3 can use a max file system size between 4TiB and 32TiB depending on the block size.
Right, but it's files, not filesystems that are the limitation.
-
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@black3dynamite said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
There must be some other limitation than just “ext3” restricting us to 2TB. According to the wiki, ext3 can use a max file system size between 4TiB and 32TiB depending on the block size.
Thin provisioning uses vhd and that has a 2TB limitation.
Ok. Thanks for that. I may play with that experimental ext4 drivers.
Why they don't use XFS we just can't understand.
-
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
Is there no way to just pass the whole disk right to a VM?
Yes, there is. You need to edit a config file on the host. Then the disk will show up as removable storage in xenserver/xcp-ng and you can assign it to the VM you want.
-
Well I went ahead and used the "Experimental" EXT4/XFS drivers. They work fine from what I can tell. I actually used XFS and not EXT4. I haven't installed an OS on EXT4 in years.
But you are still limited to 2TB for the VHD size no matter the underlying file system. So I had to create multiple 2TB vhds with xcp-ng for that SR, attach those vhds to the VM, and then span the volumes in the VM guest to create one large fat drive. It's not quite an elegant approach but I guess it works nonetheless.
One thing I don't like is xcp-ng doesn't make it very easy to figure out how to use all the available space on the SR when creating the vhds. For example, after I added my 6TB SR, I needed to break that up into 3 vhds. So I created two 2TB vhds and needed one more to use the remaining space. However, you have to do some math here to figure out how much space you have left as I noticed xcp-ng will let you overprovision the SR to your heart's content. I could have created a dozen 2TB vhds on the single 6TB storage repo if I wanted to. Also, when creating vhds, it asks you how large you want the vhd to be in GB (Gigabyte) but after you create it, its shows you the size in TiB (Tebibytes).
-
@Pete-S said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
Is there no way to just pass the whole disk right to a VM?
Yes, there is. You need to edit a config file on the host. Then the disk will show up as removable storage in xenserver/xcp-ng and you can assign it to the VM you want.
BTW, it's the block device that you will pass from the hypervisor to the guest. So it can be any size and any file system that the guest supports.
It also works with drives that are already formatted, for instance if you insert an NTFS drive full of files to your xenserver host you can pass it to a windows guest VM.
-
@Pete-S
Pete, I actually found your post (at least I think it was from you, I was tired last night) from a year ago or so on here about doing exactly that. I may test that method out as well. -
@biggen said in No way to create larger than 2TB virtual disk with Xen or XCP-NG?:
Well I went ahead and used the "Experimental" EXT4/XFS drivers. They work fine from what I can tell. I actually used XFS and not EXT4. I haven't installed an OS on EXT4 in years.
Hey awesome, that's way better.
