With VMware vSphere 7 and Tanzu, there is a new way of using storage. In fact, we have an additional way of using storage now within vSphere so I would like to take time and explain the basic storage concepts in K8s, vSAN, and vSphere Virtual Volumes (vVols).
Many new IT admins that learn VMware technology must face a decision point and storage is one of the areas that every admin must know pretty well, to make wise decisions. Especially with vSphere and Kubernetes, there are some areas that we’ll cover.
VMware vSphere and Kubernetes (K8s)
VMware calls it Cloud Native Storage (CNS) and it is a component that is an extension of the vCenter server management. Through the CNS you implement the provisioning and lifecycle operations for persistent volumes.
vSphere with Kubernetes supports three types of storage.
- Ephemeral virtual disks –this storage is temporary. Virtual disk stores objects such as logs or other temporary data. Once the pod does not exist any longer, the disk is gone too. However, this type of disk persists across restarts. Each pod only has one disk.
- Container Image virtual disks – This disk has the software that is to be run. When the pod is deleted, the virtual disks are detached.
- Persistent volume virtual disks – Certain K8s workloads needs persistent storage to save some data that are independent of the pod. The Persistent volumes objects are backed by First Class Disks (also called Improved Virtual Disk). This First-Class Disk is identified by UUIDs, that are valid even if the disk is moved elsewhere or snapshotted.
The persistent volumes are usually used for stateful applications. vSphere 7 supports dynamic and static provisioning of persistent volumes.
With Dynamic Provisioning the storage does not need to be pre-provisioned and persistent volumes can be created on-demand.
With Static provisioning, you are able to use an existing storage object and make it available to the cluster. When you provision a static persistent volume, you basically manually create a virtual disk to use as backing storage for the persistent volume. Only Tanzu Kubernetes clusters support static provisioning.
Now, things get a bit more complex, at least from a terminology perspective. But it’s just a new terminology, that’s all.
vSphere CNS-CSI: Container Storage Interface (CSI) – This component provides an interface to container orchestrators such as Kubernetes on a Supervisor Namespace. The vSphere CNS-CSI is in communication directly with CNS control plane and all the storage provisioning requests that come from vSphere Pods and Tanzu Kubernetes cluster on the namespace
Paravirtual CSI (pvCSI) – this is the vSphere CNS-CSI driver that has been modified for Tanzu Kubernetes clusters. The pvCSI is inside the Tanzu and manages all the storage requests from the Tanzu clusters. It is better to look at these graphics from VMware to understand the different blocks and how they interact with each other.
vSphere CNS-CSI and pvCSI
VMware VSAN
If you’re new to VSAN, it’s a hyperconverged storage system that uses local SSD/HDD to create a pool of shared storage as a single datastore that is available for all host within vSAN cluster.
You need a minimum of 3 disks to be part of a vSphere cluster and enabled for vSAN. Each ESXi host has a minimum of 1 flash cache disk and 1 spinning or 1 flash capacity disk. There is a maximum of 7 capacity disks that can exist in a single disk group, and up to 5 disk groups can exist per host.
vSAN is object-based storage that uses policies to enable features needed to protect your VMs.
You can use policies to enable multiple copies of data (raid1 etc), performance throttling, or stripe requirements.
Image from the lab shows a single disk group per host composed from a cache and capacity disk.
VMware VSAN disk group and cache and capacity disks
VMware vVols
vVols is quite a different kind of storage compared to traditional types of storage you know where you would carve storage out into LUNs and then you would create datastores on them.
The storage administrator has a key role in meetings with the virtualization administrators. In fact, decisions had to be made in advance, about storage schemas and layouts.
The traditional datastore layout is also hurting because of another problem. And that is that you create a bottleneck per-datastore.
When you have multiple VMs that are stored and executed on the same datastore and if they have different IOPs required different things, you can of course use Storage IO control, but this is a per-datastore level option.
With vVols, things are different. It has a granular control that helps you to bring storage functionality to the needs of individual VMs. vVols map virtual disks and different pieces, such as clones, snapshots, and replicas, directly to objects called virtual volumes, on a storage array.
This helps vSphere to offload tasks such as cloning and snapshots to the storage array. You save some CPU cycles by doing that. And because you are creating individual volumes for each virtual disk you have the possibility to apply policies at a very granular level. You can control the performance via policies much easier.
vVols creates a minimum of three virtual volumes, the data-vVol (virtual disk), config-vVol (config, log, and descriptor files), and swap-vVol (swap file created for VM memory pages). You can also let it create more, but this depends on the features you’re using. Those can be features such as snapshots or read-cache etc.
vVols start by creating a Storage Container on the storage array. The storage container is a pool of raw storage that the array is making available to vSphere. Once done, you register the storage provider with vSphere.
You can then create datastores in vCenter and create storage policies for them. All you need to do next is to deploy VMs to the vVols.
Let’s find some pictures of vVols from VMware.
VMware vVols and storage array
Final words
We just scratched the surface, but I hope you enjoyed the post. We haven’t talked about the more traditional storage protocols such as NFS or iSCSI, but those exist in vSphere since the beginning. VMware vSphere continues the innovation and there is certainly more to come.
vSphere and basic storage concepts in K8s, vSAN, and vSphere Virtual Volumes (vVols) are also part of the VCP7-DCV certification exam. A free community guide, where you can find other lessons, is available here: VCP7-DCV Study Guide – VCP-DCV 2021 Certification.
