In part of my job, I audit some Hyper-V clusters to remediate issues such as Live Migration. Most of the time, Live Migration part is not well configured. For example, the wrong network is selected or authentication is left to CredSSP. In this topic, I’ll show you how I configure Live Migration in Hyper-V clusters (S2D or not).
Choose Kerberos authentication
By default, Hyper-V hosts use CredSSP to authenticate with member of the cluster to run Live Migration. Microsoft chooses CredSSP by default because there is no further configuration to apply in order that Live Migration works. However, CredSSP has two main issues regarding Kerberos:
- CredSSP is less secure than Kerberos
CredSSP doesn’t allow to pass credential in more than one hop. So, if you want to run a Live Migration, you must login on the node which hosts the VM you want to migrate.
For this reason, I prefer to spend time on Kerberos configuration than staying in CredSSP authentication. In order to Kerberos works, you have to configure constrained delegation in Active Directory. For example, in my 2-Node S2D cluster, I have the following configuration in each AD computer object:
Each computer account can present delegated credentials to other nodes in the cluster for the service CIFS and Microsoft Virtual System Migration Service. For two nodes, you can do it manually but if you have 40 nodes or more, it will be endless. So I have written a script for that:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
$Nodes = "node01","node02","node03","node04","etc...." $DomainName = "mydomain.local" Foreach ($Node in $Nodes){ Write-Host "-------- $Node ------" Foreach ($VMHost in $Nodes){ If ($Node -notlike $VMHost){ Write-Host " -> $VMHost" Get-ADComputer $Node | Set-ADObject -Add @{"msDS-AllowedToDelegateTo"="Microsoft Virtual System Migration Service/$($VMHost).$($DomainName)", "cifs/$($VMHost).$DomainName","Microsoft Virtual System Migration Service/$($VMHost)","cifs/$($VMHost)"} } } } |
Thank to this script, the constrained delegation is set for each node in the cluster. Moreover, if you add another node in the cluster, you can run again this script without impact.
Then you can set Kerberos authentication in each node:
You can also do it by using a script:
|
1 2 3 4 5 6 7 |
$Nodes = Get-ClusterNode -cluster "MyClusterName" | Select -Expand Name Enable-VMMigration –Computername $Nodes -ErrorAction Stop| Out-Null Set-VMHost –Computername $Nodes ` –VirtualMachineMigrationAuthenticationType Kerberos |
Performance options and simultaneous number of migrations
Usually, I configure performance options to SMB with 10GB network adapters and Compression with 1GB network adapters. If you have RDMA network adapters, you should use SMB to benefit SMB Direct. To configure these options with PowerShell you can run the following script:
|
1 2 3 4 5 |
$Nodes = Get-ClusterNode -cluster "MyClusterName" | Select -Expand Name Set-VMHost –Computername $Nodes ` -VirtualMachineMigrationPerformanceOption SMB |
Regarding the simultaneous number of Live Migration, it depends on the network adapter bandwidth. You should try to increase the number and verify that the Live migration not uses all bandwidth. Usually I set 4 simultaneous Live Migration on 1GB/s network adapters and 8 on 10GB/s. This setting depends also on the network topology you have deployed: dedicated live-migration NICs or converged network. To configure the number of simultaneous Live Migration you can use the following script:
|
1 2 3 4 5 |
$Nodes = Get-ClusterNode -cluster "MyClusterName" | Select -Expand Name Set-VMHost –Computername $Nodes ` -MaximumVirtualMachineMigrations 8 |
Configure Live-Migration in one shot
We have seen several times the cmdlet Set-VMHost for various settings. You can configure Live Migration settings on all hosts in one shot by using the following script:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
$Nodes = Get-ClusterNode -cluster "MyClusterName" | Select -Expand Name Enable-VMMigration –Computername $Nodes -ErrorAction Stop| Out-Null Set-VMHost –Computername $Nodes ` –VirtualMachineMigrationAuthenticationType Kerberos ` -VirtualMachineMigrationPerformanceOption SMB ` -MaximumVirtualMachineMigrations 8 ` -MaximumStorageMigrations 8 ` -ErrorAction Stop | Out-Null |
Configure the network used by Live Migration
In a cluster, the Live Migration network is set from the Failover Clustering console. Right click on Networks and choose Live Migration Settings. Select the network(s) you want to use as Live Migration.
I have taken this screenshot from a 2-node S2D cluster where I have only two 10GB/s network adapters per node. So, I have converged all networks. I have one vNIC for management and two vNICs for storage in each node. Because Live Migration and S2D use SMB protocol, I can’t make the difference from the QoS configuration. So, I choose to use the cluster network for S2D, heartbeat and Live Migration. This is why I choose only the cluster network.
Conclusion
Live Migration configuration is not really tricky and you should spend times on settings. There is nothing more frustrating than waiting a long time to place into maintenance a node because Live Migration has poor performance. In this topic, you can retrieve scripts which help you to automate the Live Migration configuration. The values that I propose in this topic are indicative and you should test your Live Migration before running into production.
