VMware snapshots are important part of vSphere infrastructure. Using snapshots is very flexible way of being able to go back in time and revert changes. VMware snapshots are used by admins, developers and other IT team members who are not all VMware specialists. As such it might be a good idea to learn some good practices about snapshots and this technology, to get the most out of it.
VMware storage technology has evolved over time, including snapshots. vSphere 6.5 uses SEsparse as a default format for all delta disks on the VMFS 6 formatted datastores. The SESparse stands for “space efficient” and supports space reclamation, which previous formats did not. The blocks deleted within VM are marked, then used by the hypervisor which issues a command within the SEsparse layer to unmap (to free) those blocks and save space on a datastore.
If you still have VMFS 5 formatted datastores in your environment and still planning to use snapshots, you might consider upgrading to VMFS 6 in order to benefit those enhancements.
You can create a snapshot of a VM when it is powered on, off or suspended. VMware snapshot stores the complete state and data of a virtual machine whenever a snapshot is created. It means that you can easily go back in time with the point-in-time saved state of the VM.
However, with each snapshot, there are delta files which can grow to the same size as the original base disk file.
That’s why the provisioned storage size of a VM increases by an amount up to the original size of the VM multiplied by the number of snapshots on the virtual machine.
Do not use snapshots as backups
This is the number one thing not to do. I guess everyone has heard this at least once, but who knows. While it might be tempting to save your work in snapshots instead of creating a regular backup job via your backup software. However, there are some risks.
The snapshot file is only a change log of the original virtual disk, it creates a place holder disk, virtual_machine-00000x-delta.vmdk, to store data changes since the time the snapshot was created. If the base disks are deleted, the snapshot files are not enough to restore a virtual machine.
There is a maximum of 32 snapshots supported in a chain so imagine you have 32 snapshots and the chain breaks in the middle. Then you have problem. For best performance and best security always use only 2 to 3 snapshots.
Do not use a snapshot for more than 72 hours. The snapshot files continue to grow as you keep using the VM. This can cause the snapshot storage location to run out of space and impact the system performance.
By default, the storage location is within the same folder as the VM’s files, so if you’re not cautious and do not have enough space on your datastore to accommodate those grows, your datastore will simply fills up and your VMs will be suspended.
Backup software creates snapshots too
When using a third-party backup software, ensure that snapshots are deleted after a successful backup. In the past, there has been many backup vendors having problems with storage APIs and the problems resulted many snapshots created (and not deleted) after successful or failed backup jobs.
Note: Sometimes snapshots taken by third party backup software (through API) may not even appear in the Snapshot Manager, so you’ll have to manually check for snapshots from time to time. Either use a command line or manually run some free tools, such as RVTools which shows all snapshots created within your organization.
RVTools shows snapshots and size on datastore
But there are many other software tools which detects snapshots and even many modern backup vendors integrate snapshot detection (and deletion) at the end of backup job.
That’s the case of for example Veeam Backup and replication which checks the datastore to discover orphaned snapshot file. Veeam has a built-in process called “Snapshot hunter”.
The Snapshot Hunter is started as a separate process scheduled within every job session. If there are some orphan/phantom snapshots discovered, the Veeam Backup Service schedules the snapshot consolidation, which means that the VMware Snapshot Consolidate method is executed. It uses the same mechanism that VMware vSphere uses for VMs with the “Needs Consolidation status”.
When looking for a backup software, make sure to ensure that snapshots are handled properly.
How many snapshots to keep per VM?
When working with a team where many users has access to a possibility to create a temporary snapshot, it can go wild. Devops environments where many developers maintain their temporary work and use snapshots intensively is just one of many examples.
When multiple users have multiple snapshots on multiple VMs, storage performance might decline and more storage needs to be provisioned.
However, there is a way to control the number of snapshots allowed per VM. There is also VMware’s best practice for the number of VM snapshots allowed, which is 32 at maximum, but you should never go that high. I’d limit this to 2-3 snapshots maximum.
It has to be done at per-vm level and you can do it by editing the VM’s configuration. After clicking on “Edit Configuration” button, advanced configuration parameters > Go to “snapshot.maxSnapshots” > “Value” column, change the default setting to 3. This will limit the number of maximum snapshots to three.
Limit the Maximum snapshots number to three via advanced VM configuration
This way you can also disable snapshots completely. Just enter “0” to the field and nobody will be able to snapshot the VM. (including your backup software). So actually, to stay safe and still be able to backup your VM, you should enter “1” and have a possibility to take at least one snapshot.
Snapshot management via UI of vSphere client
This is the easiest way to manage snapshots. The snapshots tree is displayed when you open the snapshot manager.
Right click a VM > Snapshots > Manage Snapshots.
Manage Snapshots via vSphere client
You can delete particular snapshot in a snapshot tree by first selecting it and then hit the Delete button. You can also Delete All snapshots there.
Always 3-2-1 backup rule
As a datacenter admin you should always follow 3-2-1 backup rule. It means that you should always keep multiple copies of data in multiple locations on different types of media. This allows you to mitigate a risk of losing both production data and backup data.
The “3-2-1 rule” simply put is that you store at least “3” copies of your production data on “2” different types of media and at least “1” copy is stored off-site In the remote location (remote datacenter or cloud).
Final Words
Best strategy with snapshots is probably regular check of your infrastructure via utilities such as RVTools or other tool allowing you to see which VM has some snapshots. Of course, there is also the option of using the vSphere PowerCLI cmdlets.
