Organizations today face tremendous cybersecurity and compliance challenges. As businesses battle to keep data private, protected, and aligned with compliance framework requirements, data encryption is required. While many companies understand the benefits of encrypting their data, encryption technologies can be challenging to implement and require many “moving parts” as part of the solution. Today’s modern virtualization workloads also benefit from data encryption and can significantly bolster the security and compliance of business-critical workloads.
VMware vSphere can implement vSAN Encryption, VM encryption, and virtual TPMs (vTPM) for workloads for quite some time. However, these technologies have previously required third-party key management solutions to implement. This requirement changed in vSphere 7 Update 2 with the introduction of the vSphere native key provider. So how is it implemented, and what are the best practices?
What is the vSphere Native Key Provider (NKP)?
Until VMware vSphere 7 Update 2, customers who wanted to implement encryption in their vSphere environment had to use a third-party key solution to implement encryption. This type of technology is commonly referred to as a key management server. You can take a look at a list of certified third-party key management server solutions here: vi_kms_guide.pdf (vmware.com).
However, based on much customer feedback and what VMware saw in the field with the success of customers implementing encryption technologies, the requirement for external technologies for many customers hindered implementing data encryption in their VMware vSphere environments. Therefore, with the release of VMware vSphere 7 Update 2, VMware has introduced a new Native Key Provider (NKP) that removes needing an external key provider to implement basic encryption technologies.
With a standard key provider, external to your vSphere environment, vCenter Server contacts the external key server, pull keys and then distributes the keys to the ESXi servers. In a vSphere Trust Authority configuration, trusted ESXi hosts contact and fetch the keys directly from the key server.
However, with the introduction of the vSphere Native Key Provider, organizations now have a built-in way to natively generate encryption keys for use with data encryption in vSphere. With the new vSphere Native Key Provider, the key mechanism is built directly into vCenter Server. The way it works is vCenter generates what VMware calls a Key Derivation Key (KDK). Then, this key is pushed to all ESXi hosts within a vSphere cluster.
The NKP helps ease the logistics of encrypting data and some of the headaches when using an external key provider, such as ensuring access to the external key provider at all times and high availability configurations. With the NKP, organizations are protected from the “chicken-egg” scenario with an external key provider. Unlike External Key Providers, the NKP provides key persistence on the ESXi hosts.
ESXi hosts use the Key Derivation Key (KDK) mentioned earlier to generate data encryption keys to enable data encryption capabilities. Also, the ESXi hosts store and cache these within a local TPM device, if one is available. If the VCSA goes offline and the cluster goes dark, the vSAN DS will come back online as the ESXi hosts pull encryption keys out of TPMs for the native key manager.
The Native Key Provider enables ESXi hosts to provision vTPM, vSAN encryption, and VM encryption.
A Native Key Provider configured on VCSA
vTPM
In the physical server hardware world, Trusted Platform Modules (TPMs) are discrete security devices that allow storing sensitive cryptographic information in a way that prevents tampering. A TPM chip is essentially a secure crypto-processor that carries out cryptographic operations. TPM devices are designed with many different physical security mechanisms.
It helps make the TPM resistant to tampering by a threat actor looking to change or alter the security keys associated with the device. TPM devices are used for system integrity measurements, and host attestation, ensuring the boot code (firmware and OS components) are validated as secure and not compromised.
Adding a vTPM device to a virtual machine in VMware vSphere
The virtual TPM (vTPM) represents a physical TPM 2.0 device inside software. It allows virtual machines to have a software-based TPM 2.0 device, allowing the virtual machine to store sensitive cryptographic information inside the vTPM. Using a vTPM device is required for modern operating systems like Windows 11 since it now checks for the presence of a TPM device during installations and upgrades.
One of the requirements for using the vTPM virtual hardware is a key provider in your vSphere environment. The vSphere Native Key Provider allows adding the vTPM since VM encryption is required to add the vTPM.
vSAN Data at Rest Encryption
Using the vSphere Native Key Provider, organizations can encrypt data at rest that resides in the vSAN datastore. Once all other processing is finished on the data (deduplication, etc.), the data is encrypted. The data at rest encryption protects data from being compromised by removing cluster storage devices.
Data at Rest Encryption in VMware vSAN
Virtual Machine Encryption
VMware virtual machine encryption provides end-to-end encryption of virtual machines. It is a great way to protect sensitive workloads that house secure data. In addition, access to encryption keys can be made conditional to the ESXi host being in a trusted state.
Virtual machine encryption in VMware vSphere
vSphere Native Key Provider limitations
Depending on an organization’s use case and encryption needs, there are limitations to the Native Key Provider to note. These include:
- Works only with VMware infrastructure products
- Does not provide external interoperability, KMIP support, hardware security modules, or other feature
If your organization requires any of these features or functionality for non-VMware products and components, you will need to install a traditional, third-party key server.
VMware vSphere Native Key Provider Best Practices
When it comes to data encryption, designing and making sure you can access your encryption keys is necessary when considering business continuity. When used, the VMware vSphere Native Key Provider is this centralized store for encryption keys, and best practices should apply. What best practices need considered when using the VMware vSphere Native Key Provider? Let’s look at the following:
- Initial backup and regular backups of the NKP
- Protecting vCenter Server application-level backup with encryption
- Enterprise Plus licensing to unlock all NKP features
- Make sure physical ESXi hosts have TPMs installed
- Don’t mix encryption technologies
1. Initial backup and regular backups of the NKP
Before you can place the vSphere Native Key Provider into production, you must backup the NKP and download the private key to store it safely for “break glass” scenarios. As you can see below, you must backup the NKP key before you can place it in production.
Backing up the vSphere Native Key Provider before placing it in production
It is also a great idea to configure the VCSA application-level backups as once you have enabled the NKP and initially performed the backup, the application-level backup also backs up the NKP.
2. Protecting vCenter Server application-level backup with encryption
As a secondary consideration for your VCSA application-level backups, you will want to encrypt your VCSA backups. Since the VCSA backups contain a backup of your NKP, these backups also need to be encrypted as an attacker who gains access to these backups can also gain access to the NKP encryption key.
In the configuration of your VCSA backups, you will see the option to enter an encryption password. Backup encryption will ensure the backup files created are encrypted, including the NKP configuration and key.
Encrypting VCSA application-level backups
3. Enterprise Plus licensing to unlock all NKP features
One detail of the NKP to consider that we are listing as a “best practice” is to consider your vSphere licensing. While the NKP is included in all editions of vSphere, to use vSphere Native Key Provider for vSphere Virtual Machine Encryption, you must purchase the VMware vSphere Enterprise Plus Edition.
While it is likely many organizations will already be running Enterprise Plus Edition, it is undoubtedly a consideration for using the NKP if an organization desires to use virtual machine encryption and is only running vSphere Standard licensing.
4. Make sure physical ESXi hosts have TPMs installed
To be clear, the vSphere Native Key Provider does NOT require the physical ESXi host have a TPM 2.0 module installed. Therefore, you can implement the NKP without these installed in your ESXi hosts. However, using ESXi hosts with TPMs installed with the NKP is a best practice. Why?
As we mentioned earlier, the vCenter Server appliance generates a Key Derivation Key (KDK) and seeds the ESXi hosts with this key. The ESXi hosts create the encryption keys used for the encryption processes needed for virtual machine encryption, vSAN encryption, etc., from this key.
The encryption key is stored in the local TPM device if the ESXi host has a TPM installed. It means that even if the ESXi host loses contact with the VCSA, it can still pull the encryption keys from the TPM. In thinking about a disaster scenario where you have lost the VCSA and need to power on your VMs, you need access to the keys to perform these actions. The local TPM ensures customers maintain uninterrupted access to the NKP encryption keys during failure scenarios.
5. Don’t mix encryption technologies
One might think with the ease of configuring and using the vSphere Native Key Provider, configuring and using all available encryption technologies would be a good idea. For example, using vSAN data at rest encryption with virtual machine encryption might seem more secure.
However, there are two reasons you would not want to do this, as based on the VMware KB, Understanding vSAN Datastore Encryption vs. VMcrypt Encryption (2148947):
It will cause vSAN deduplication for doubly encrypted VMs to essentially be turned off – As the encrypted data written by virtual machine encryption appears to be random, it does not deduplicate well. If using virtual machine encryption with vSAN deduplication, expect deduplication efficiency to approach zero for encrypted VMs
It will lead to elevated CPU on your ESXi hosts – Dual encryption will significantly increase CPU overhead as now data is encrypted (and decrypted) twice.
Final Notes
The new vSphere Native Key Provider is an excellent option for organizations looking to implement basic encryption needs in their VMware vSphere environment, including vTPM, virtual machine encryption, and vSAN encryption. However, as mentioned, the NKP is limited to VMware-only environments and does not provide advanced features and functionality such as KMIP support and hardware security modules.
Best practices when using the NKP include backing up your NKP regularly, securing these backups with encryption, considering your licensing, installing TPMs in the physical ESXi hosts, and not mixing encryption technologies. You can learn more about the vSphere Native Key Provider here:
Configuring and Managing vSphere Native Key Provider
