Virtual workload migration with VMware

It is quite surprising how many companies migrate virtual workload nowadays. There could be different reasons – expiring contract with the service provider, move to the new office, consolidation of resources to minimize Capex and Opex, but in every case, companies need efficient and flexible V2V migration solution.

Very often people think of migration in terms of moving data from one place to another. However, migration is much more than that. For instance, you would probably want VMs to keep the same IP addresses after the migration. Which means you need to consider stretching VLANs between the source and target locations in case you cannot migrate all VMs in particular subnet in one go. You must plan a network cutover too.

You might need to work with applications owner to understand how this migration impacts the services provided by the VMs you migrate. Alternatively, this can be an infrastructure led migration with minimal disruptions to services and minimal involvement of the applications teams.

Network cutover is not only about moving subnet and its default gateway to the new site. This may also include migration of the security policies, migration of the load balancers used the by VMs being migrated, etc.

On top of that migration may require reconfiguration of your automation tools, backup policies, monitoring systems, you name it.

As you can see, the moving VM data to the new environment is just the tip of the iceberg in the migration strategy, and that’s we are going to be reviewing today.

In this post, we will review the most popular migration approaches, we will have a look at their pros and cons and will check what free alternatives are available.

Cross-vCenter vMotion

vSphere 6.0 and later allows you to migrate virtual machines between vCenter Server instances. This was very popular feature request and people were heard. Initially, VMware introduced cross-vCenter vMotion in Web Client and between vCenters in Enhanced Linked Mode, but a bit later it was updated to allow vMotion between vCenters in different SSO domain.

Pros:

• Minimal impact on the running applications. Usually, there are 1-2 pings lost only when you vMotion VM between two datacenters. Though, you may have the longer impact during the network cutover.
• No application team involvement. As there are no changes from the VM/Application/Service perspective there is no need to go through dozens of workshops with application owners. The change is absolutely transparent for them and for the users.
• Migration can (and should) be automated. William Lam has produced the script that uses vSphere API for cross-vCenter vMotion. You can customize it further to run, for instance, some pre-migration checks and use CSV files to feed the data into the script.
• It is possible to vMotion to the newer version of vSphere. For instance, you can move the virtual machine from the vSphere 6.0 to 6.5.
• No additional software to be installed for the migration. You may need to change vMotion and Provisioning VMK interface configurations.
• As long as you have the correct license at both sites this feature is free

Cons

• You cannot pre-stage data. This limits the throughput of migration in terms of VMs per hour. There could be situations when parts of the multi-tier applications may temporarily run in different datacenters during the migration period, which can result in higher network latency and higher inter-sites link utilization.
• Rollback isn’t fast. Since the data is moved between two locations in case of the rollback it will have to be copied back.
• While it is possible to run XVC vMotion from vSphere 6 to vSphere 6.5 there is no roll back option except the restore from backup.
• It is not possible to test the what-if scenario of the migration.

Requirements:

• Source and Destination vSphere must be at least version 6 or later. William Lam wrote a very nice summary on different combinations of supported scenarios…
• Stretched VLAN has to be configured for the VMs being migrated
• Source and Target Distributed vSwitches must be the same versions. So, make sure to select version 6.0.0 when you create DVS at the target site. You can always upgrade it to 6.5.0 once the migration is complete.
• Enterprise Plus license is required for XVC and Long-Distance vMotion
• An RTT (round-trip time) latency of 150 milliseconds or less between hosts.
• vMotion VMK interfaces have to be placed on the separate network stacks. This will allow you to route vMotion traffic between sites. Normally, vMotion subnets are not routable and contained within the same cluster or datacentre. Please note that you cannot change TCP/IP stack for existing vMotion VMK interfaces. Instead, these VMK interfaces will have to be deleted first and new vMotion interfaces should be created on the vMotion network stack.
• All snapshots should be consolidated into base disk prior to the migration to avoid copying cold data via management or provisioning networks.

Site Recovery Manager

Site Recovery Manager is more famous as a disaster recovery software, but it also proved to be a very useful migration tool.

I am not sure whether it makes sense to purchase SRM solely for migration purpose, but if you already have SRM license that it makes a perfect use case for the migration.

Pros:

• No dependency on storage replication as SRM works just fine with vSphere Replication.
• Ability to pre-stage the data prior to the migration. Essentially, vSphere Replication can copy all VMs in advance to the target site and on the migration date, it will only run final delta sync of the data.
• Bulk migration in short time. That is because you can pre-stage VMs prior to the move.
• Ability to run pre-migration test to simulate the move of the virtual machines in a bubble without any impact on the production VMs.
• Simplifies rollback. You will only need to shutdown VMs at the target site and power on original VMs at the source site.
• Initial replication can be more efficient using replication seeds. You can physically move VMDK files to the target site in case it is not practical to copy data over the network.
• vSphere Replication compresses the traffic which results in more efficient usage of the link.

Cons:

• There is an outage during the cutover. The length of the application outage depends on many factors, e.g. the size of the application, the number of VMs in the Recovery Plan, post-migration tasks.
• Even though the outage can be short it can still impact the application if the power on sequence is not correct. Therefore, it is required to understand each application dependencies.

Requirements:

• vCenter and SRM servers have to be the same version on both sites
• Specific network ports to be opened in the firewall between sites
• Dedicated network for vSphere Replication in case you don’t want copy data across management network due to bandwidth limitations or security restrictions
• Stretched VLAN. SRM can renumber IP addresses of the virtual machines as part of the migration, but in practice, it isn’t an easy task as many applications have hard coded dependencies based on IP addresses.

vSphere replication

If the budget is tight or you have no intentions to use SRM for Disaster Recovery you can still use vSphere Replication, though you will lack orchestration features of the SRM.

Pros:

• As long as you have vSphere Essential Plus Edition license vSphere Replications is free.
• Most of the tasks of the migration workflow can be automated with PowerCLI, e.g. the script can shut down VMs at the source site, disconnect their NICs. After a cutover to a target site, you can leverage script to upgrade VMs’ hardware, update VMware Tools and power on VMs in the right order.

Cons:

• There is no vSphere Replication API which limits automation of migration. For instance, the configuration of VM replication and VM failover must be done per each VM. Having 20-30 VMs to move is not a big deal, but when we talk about hundreds or thousands VMs there is no quick way to migrate the workload with vSphere Replication.

The requirements are pretty much the same you would have for SRM, which covered in the previous section.

VMware Converter

This tool has been known for years as a software for P2V migrations, but in certain cases you can use for V2V as well.

Pros:

• It is free
• Data can be pre-staged in advance to make the migration quick
• Ability migrate to vSphere from other hypervisors
• Minimal impact during migration

Cons:

• Converter uses in-guest agent, hence not all operating systems are supported
• Inconsistent performance.
• google

There are a couple more migration options which probably are not suitable for bulk migrations, but these can be used to migrate few VMs for free.

OVF export/import

OVF is Open Virtualization Format (OVF) is an open standard for packaging and distributing virtual appliances.

Using OVF you can export VMs into zipped files, copy them to external storage, ship them to the destination and import into vSphere.

Pros:

• Virtual Machine data is compressed during the export to minimize the data footprint
• Ability to move a large amount of data at once. It all comes down to the size of the external media.
• Suitable for migration when there is no network connection between the source and target sites or security policies don’t allow traffic between environments.

Cons:

• Longer downtime for VMs as you can’t bring up them until they are physically delivered to the target site
• The risk of the data breach during migration as a result of theft. You may consider copying exported VMs onto encrypted storage if these servers contain sensitive information

Another migration alternative is to copy VMs between hosts. It is very handy workaround when there is no time to setup vSphere Replication, or it is not possible due to different versions of the source and target environments.

Pros:

• Easy to use – one command line
• No changes required, apart from enabling opening SSH client ports on the source

Cons:

• Copy performance is not great unless you run multiple simultaneous copy sessions
• VM must be powered off during the entire copy process

Depending on the size of the environment you could either use a small excel worksheet to prepare for the migration or end up with several comprehensive documents defining migration strategy.

There are plenty of different commercial products from other vendors which can deliver the same functionality for the migration, but if you already run vSphere and probably SRM look no further. You are already well equipped.

Related materials: 

• Encryption of VMware vSphere 6.5 virtual machines and vMotion migrations. And their performance
• V2V Converter Overview