VMware has recently released VMware vSphere APIs for I/O Filtering (VAIO), a handy document that highlights the VAIO I/O Filtering technology operational principles. Here, we shed light on them briefly and discuss how VAIO can be implemented in a production environment.
VAIO is a technology and an API providing the direct access to the virtual machines’ guest OS I/O Stream. VAIO is already employed in partner products for maintaining various tasks (i.e., caching write-back and write-through). VAIO is based on the Storage Policy Based Management framework, dedicated to managing virtual machines storages and set the storage rules.
Technically, this framework is implemented by VAIO filter driver that is installed on a VMware ESXi host as a VIB package. The filter eliminates the need for the dedicated software installation on the virtual machine guest OS.
Let’s take a look at the screenshot above which describes data transfer from a virtual machine to the physical storage device. According to the image, there are two VAIO-related components of data transfer. The first one, VAIO framework, works on VMkernel level. This component determines the filter policies. The second component, I/O Filter, represents the software working whithin the User World level of the ESXi host. This filter, once it has performed its function (i.e., data caching or replication), sends data to a physical storage device for its writing.
The architecture described above provides I/O Filter driver with an access to the I/O Stream without significant impact on the speed of SCSI commands execution. Additionally, the I/O Filter provides access to the I/O Stream for both writing (data transfer from the OS to the storage) and reading (when a storage acknowledges receipt of the data). This feature comes in handy in replication where it is essential to get acknowledgments for data storing to the disk.
By the way, SCSI commands get to the I/O Filter right after they pass SCSI-emulation layer, vSCSI. This feature makes VAIO compatible with any storage type: VMFS, NFS, SCSI, and vSAN. Commands interception before their sending to the network storage stack ensures data consistency and integrity.
After having a careful look at the architecture of I/O Stream, it becomes obvious that I/O Filter driver exists on the User World level, thus, the filter does not impact the ESXi server functioning. The main SCSI commands stack, which is located in the kernel, keeps on working even though VAIO components fail.
I/O Filter works on the basis of specific VM’s VMDK. This provides various features for Storage Policy Based Management framework that, in its turn, can be applied to specific virtual machines and services.
All the mentioned above VMware VAIO advantages make the framework suitable for accomplishing the following tasks:
- Encryption enables the third-party software to encrypt and decipher the data streams from a virtual machine on the fly. As a result, the storage receives the already encrypted data. The inherent flexibility of VAIO enables the solution to process the data stream going from not only the entire virtual machine but also a single application.
- De-duplication is already employed in VMware Virtual SAN. Data that passes through the driver is deduplicated in real time and then sent to storage, allowing to reduce the occupied disk space. This technique is already available for VMware partners.
- Tiering allows multi-level data storing based on its importance. Data is kept on storages with different availability and performance characteristics. VAIO enables to analyze data and determine its destination storage.
- Analytics enables to inspect data stream from a particular virtual machine directly and build different solutions including, for example, caching systems (i.e., such solutions as PrimaryIO which can be connected to the hypervisor directly). It also allows detecting data from particular applications in the stream or using write-back caching.
VAIO framework is compatible with VMware vSphere 6.0 Update 1 and higher supports vMotion and VMware DRS clusters. In order to enable live migration, VAIO should be available on the target host, while to support VMware DRS clusters the framework should be accessible to all hosts.
Let’s take a closer look at real-life VAIO implementation. To start working with VAIO, you should install I/O Filter as a VIB package that, in its turn, presents functioning of a partner’s product for VMware vSphere.
The filter is usually supplied with SPBM policy example (similar to the ones for vSAN and VVols). The policy can be assigned to virtual machines on the VMDK level.
This policy can be assigned during the virtual machine creation in the Select storage section as it is shown in the image below (SSD caching policy in this case):
The policy that comes with the filter usually includes Common Rule, the set of rules implementing Data Services (filter core functionality). Common Rules are set by adding an object in Storage Policy Components (Common Rules section).
Here, we can set either the Write Through or Write Back caching type in the Common Rules Section while creating a new Storage Policy:
By default, these components are installed together with the integral vSphere filters and other products, which, in their turn, supplement I/O Filters’ and components’ lists. Thankfully to the Storage Policy Components, new I/O Filter features can be included in any of the already assigned VM Storage Policies.
We find it pretty convenient because a virtual machine or VMDK can have different filters components added regardless of having only one storage policy assigned to them.
The list of I/O Filters from different vendors can be found in VMware Compatibility Guide tab on the following page: https://www.vmware.com/resources/compatibility/search.php?deviceCategory=vaio.
According to the image above, there are very few partners utilizing VAIO framework for their solutions so far. Mainly, the framework is applied for replication and caching. In the meantime, VMware promotes VAIO actively, hence, there may appear new I/O-Filters-based products for solving different tasks.
