Introduction
A couple of days ago, I decided to re-distribute VM resource shares. I, basically, wanted several VMs to get some more resource without compromising their latency. For that purpose, I played around with Storage I/O Control parameters a bit. And, you know, I decided to look at things more globally. Actually, here’s how I decided to take a deeper dive into I/O filtering. In today’s article, I’m going to tell you about the VMware vSphere APIs for I/O Filtering (VAIO) framework providing the direct access to the to the VM I/O stream. I shed light on how to enable those filters, how they work, and why you need them.
What I/O filters are and how they work
Introduced in vSphere 6.0 U1, VAIO filtering allows users to intercept and manipulate virtual machine I/O regardless of the underlying storage topology. The feature is an alternative to the unsupported kernel-level methods used before to access open-ended data services. The framework utilizes VAIO filter driver installed on VMware ESXi hosts as the vSphere Installation Bundle package. In this way, you do not need any additional software to add filters. You can enable I/O Filtering in a virtual machine itself while creating or cloning it, or you can just apply filters to the already existing VMs. If the VM has several disks, you can apply multiple I/O filters to it.
Both synchronous and asynchronous replication modes can be used with I/O filter. Note that there’s a small thing about synchronous replication: it induces additional loads to the disk. In this way, disk latency may be slightly higher than if you were using asynchronous replication.
Here’s how I/O Filtering works:
With I/O Filtering enabled, all I/O requests are transmitted through three layers (they are named “worlds” in most of VMware documents):
- User World – here, I/O filters are implemented
- Kernel world – after implementation in the user world, I/O filter gets processed in the kernel word
- Physical device – once settings are applied, I/O reaches its destination – physical device
The I/O path with an I/O filter enabled looks like as I described in the scheme above:
- VM sends the request to the vSCSI
- I/O filter goes to the vSCSI backend
- Afterward, the request goes to the file system layer
- Next, the request goes to the file device layer
- VAIO framework sends the request back to the I/O filter
- Eventually, I/O filter lets the I/O request to reach the physical device
Creating and configuring I/O Filter
In this article, I set up VM Storage Policies in vSphere Web Client 8.0
In order to apply I/O Filter to the virtual machine, create a VM storage policy in the Policies and Profiles vCenter menu:
Here are the parameters I used for the policy creation:
Once you click Finish, you can find the recently added policy on the VM Storage Policies list.
Now, you can change VM storage policy to IO Filtering:
While creating a VM new storage policy, on the Policy Structure step, you can set up two storage policies: VM Encryption and Storage I/O Control. Let’s take a closer look at them.
Encryption
As it comes from the policy name, VM Encryption allows encrypting a particular VM on the fly. This VM storage policy allows using third-party software to encrypt and decrypt data streams from the VM that has the I/O filter enabled on it. Encryption occurs in the User World, so all data is sent across the wire being already encrypted. Looks pretty secure, doesn’t it?
This VM storage policy allows using third-party software to encrypt and decrypt data streams from the VM that has the I/O filter enabled on it. Encryption occurs in the User World, so all data is sent across the wire being already encrypted. Looks pretty secure, doesn’t it?
Note that there’s the small thing about vSphere Virtual Machine Encryption: data cannot be deduplicated or compressed. This may be a kinda of a pain in all-flash environments where both deduplication and compression are used for space saving. Also, VMware warns against changing the bundled VM Encryption sample storage policy. Instead, you should clone the policy and edit the clone.
Storage I/O Control
Storage I/O Control (SIOC) is an I/O queue-throttling mechanism that prevents any VM from monopolizing the datastore by leveling out all I/O requests that datastore receives. Note that it is disabled by default, so the administrator has to enable it manually on each datastore. Once SIOC is enabled, ESXi hosts monitor the datastore latency. If the latency exceeds a defined threshold, SIOC dynamically adjusts the I/O resources allocated to each VM, ensuring that high-priority VMs receive preferential access, preventing performance degradation, and maintaining service level agreements.
This feature is crucial for large environments where it is vital to ensure that all VMs can get a fair share of resources without compromising latency. SIOC prioritizes I/O requests based on the shares assigned to each VM, not necessarily ensuring that mission-critical VMs’ performance is unaffected, but rather managed according to configured policies. This ensures that your mission-critical VMs are *prioritized* based on their assigned shares, improving performance during contention.
Conclusion
In this article, we’ve taken a detailed look at the VMware vSphere APIs for I/O Filtering and explored its fundamental concepts. In vSAN environments, VM Storage Policies unlock an even broader spectrum of settings and capabilities. For instance, you can directly enable features like deduplication, fault tolerance levels (FTT), and force provisioning directly within the storage policy. Clearly, there’s still a wealth of information to uncover about the intricacies and advanced applications of I/O filtering, and I look forward to exploring those in future discussions.
