StarWind Virtual SAN: Configuration Guide for VMware vSphere [ESXi], VSAN Deployed as a VM (CVM) using GUI

Configuring Networks

Configure network interfaces on each node to make sure that Synchronization and iSCSI/StarWind heartbeat interfaces are in different subnets and connected physically according to the network diagram above. All actions below should be applied to each ESXi server.
NOTE: Virtual Machine Port Group should be created for both iSCSI/ StarWind Heartbeat and the Synchronization vSwitches. VMKernel port should be created only for iSCSI traffic. Static IP addresses should be assigned to VMKernel ports.
NOTE: It is recommended to set MTU to 9000 on vSwitches and VMKernel ports for iSCSI and Synchronization traffic. Additionally, vMotion can be enabled on VMKernel ports.

1. Using the VMware ESXi web console, create two standard vSwitches: one for the iSCSI/ StarWind Heartbeat channel (vSwitch1) and the other one for the Synchronization channel (vSwitch2).

2. Create a VMKernel port for the iSCSI/ StarWind Heartbeat channel.

3. Add a Virtual Machine Port Groups on the vSwitch for iSCSI traffic (vSwtich1) and on the vSwitch for Synchronization traffic (vSwitch2).

4. Repeat steps 1-3 for any other links intended for Synchronization and iSCSI/Heartbeat traffic on ESXi hosts.

Deploying StarWind Virtual SAN for vSphere

1. Download zip archive that contains StarWind Virtual SAN for vSphere: https://www.starwindsoftware.com/starwind-virtual-san#download
2. Extract virtual machine files from the downloaded archive.
3. Deploy a virtual machine on each ESXi host using the “Create/Register VM” button. Select “Deploy a virtual machine from an OVF or OVA file” in the Select creation type section and press Next.

4. Specify the name for the virtual machine with StarWind VSAN, drag and drop the extracted files to the wizard, and press Next.

5. Specify the location for the StarWind Virtual SAN VM and press Next.

6. Read and accept the license agreements by pressing on “I agree” button. Press Next to continue.

7. Choose the proper networks for the VM by their purpose and other options.

8. Review the settings and click Finish to start the deployment process.

12. Repeat all the steps from this section on the other ESXi hosts.

NOTE: In some cases, it’s recommended to reserve memory for StarWind VSAN VM.

NOTE: When using StarWind with the synchronous replication feature inside of a Virtual Machine, it is recommended not to make backups and/or snapshots of the Virtual Machine with the StarWind VSAN service installed, as this could pause the StarWind Virtual Machine. Pausing the Virtual Machines while the StarWind VSAN service in under load may lead to split-brain issues in synchronous replication devices, thus to data corruption.

Configuring StarWind VMs Startup/Shutdown

1. Setup the VMs startup policy on both ESXi hosts from Manage -> System tab in the ESXi web console. In the appeared window, check Yes to enable the option and choose the stop action as Shut down. Click Save to proceed.

2. To configure a VM autostart, right-click on the VM, navigate to Autostart and click Enable.

3. Complete the actions above on StarWind VM located on all ESXi hosts.

4. Start the virtual machines on all ESXi hosts.

Configuring StarWind Virtual SAN VM settings

By default, the StarWind Virtual SAN virtual machine receives an IP address automatically via DHCP. It is recommended to create a DHCP reservation and set a static IP address for this VM. In order to access StarWind Virtual SAN VM from the local network, the virtual machine must have access to the network. In case there is no DHCP server, the connection to the VM can be established using the VMware console and static IP address can be configured manually.

1. Open a web browser and enter the IP address of the VM, which it had received via DHCP (or had it assigned manually), and log in to StarWind Virtual SAN for vSphere using the following default credentials:

Username: user
Password: rds123RDS
NOTE:  Make sure to tick Reuse my password for privileged tasks check box.

2. After the successful login, on the left sidebar, click Accounts.

3. Select a user and click Set Password.


4. On the left sidebar, click Networking.

Here, the Management IP address of the StarWind Virtual SAN Virtual Machine, as well as IP addresses for iSCSI and Synchronization networks can be configured.
In case the Network interface is inactive, click on the interface, turn it on, and set it to “Connect automatically“.

5. Click on Automatic (DHCP) to set the IP address (DNS and gateway – for Management).

6. The result should look like on the picture below:

NOTE: It is recommended to set MTU to 9000 on interfaces, dedicated for iSCSI and Synchronization traffic. Change Automatic to 9000, if required.

6. Alternatively, log in to the VM via the VMware console and assign a static IP address by editing the configuration file of the interface located by the following path: /etc/sysconfig/network-scripts

7. Open the file, corresponding to the Management interface  using text editor, for example:
sudo nano /etc/sysconfig/network-scripts/ifcfg-ens192

8. Edit the file:

Change the line BOOTPROTO=dhcp to: BOOTPROTO=static

Add the IP settings needed to the file:
IPADDR=192.168.12.10
NETMASK=255.255.255.0
GATEWAY=192.168.12.1
DNS1=192.168.1.1
By default, the Management link should have an ens192 interface name. The configuration file should look as follows

9. Restart interface using the following cmdlet: sudo ifdown ens192 ,  sudo ifup ens192 or restart the VM.

10. Change the Host Name from the System tab by clicking on it

11. Change System time and NTP settings if required

12.  Repeat the steps above on each StarWind VSAN VM.

Configuring StarWind Management Console

1. Install StarWind Management Console on a workstation with Windows OS (Windows 7 or higher, Windows Server 2008 R2 and higher) using the installator available here.
NOTE: StarWind Management Console and PowerShell Management Library components are required.

2. Select the appropriate option to apply the StarWind License key.
Once the appropriate license key has been received, it should be applied to StarWind Virtual SAN service via Management Console or PowerShell.

3. Open StarWind Management Console and click Add Server.


4. Type the IP address of the StarWind Virtual SAN in the pop-up window and click OK.

5. Select the server and click Connect.

6. Click Apply Key… on the pop-up window.

7. Select Load license from file and click the Load button.

8. Select the appropriate license key.
As an alternative, PowerShell can be used. Open StarWind InstallLicense.ps1 script with PowerShell ISE as administrator. It can be found here:
C:\Program Files\StarWind Software\StarWind\StarWindX\Samples\powershell\InstallLicense.ps1
Type the IP address of StarWind Virtual SAN VM and credentials of StarWind Virtual SAN service (defaults login: root, password: starwind).
Add the path to the license key.

9. After the license key is applied, StarWind devices can be created.
NOTE: In order to manage StarWind Virtual SAN service (e.g. create ImageFile devices, VTL devices, etc.), StarWind Management Console can be used.

Configuring StarWind Storage on Top of Hardware RAID

1. Add a new virtual disk to the StarWind Virtual SAN VM by editing its settings. Make sure it is Thick Provisioned Eager Zeroed. Virtual Disk should be located on the datastore provided by hardware RAID.

NOTE: Alternatively, the disk can be added to StarWind VSAN VM as RDM. The link to VMware documentation is below:
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-4236E44E-E11F-4EDD-8CC0-12BA664BB811.html

NOTE: If a separate RAID controller is available, it can be used as dedicated storage for StarWind VM, and RAID controller can be added to StarWind VM as a PCI device. In this case RAID volume will be available as a virtual disk in the Drives section in the Web console. Follow the instructions in the section below on how to add RAID controller as PCI device to StarWind VM.

2. Login to StarWind VSAN VM web console and find in the Storage section under Drives the Virtual Disk that was recently added and choose it.

3. The added disk does not have any partitions and filesystem. Press Create partition table and press Format afterward to create the partition and format it.

NOTE: It is not necessary to overwrite data while creating partition. 

4. Create the XFS partition. Specify the name and erase option. The mount point should be as following: /mnt/%yourdiskname% . Click Format. To enable OS boot when mount point is missing (e.g., hardware failure), add nofail as a boot option. 

5. On the storage page of the disk, navigate to the Filesystem tab. Click Mount.

6. Using StarWind Management Console, connect to StarWind Virtual SAN VM and configure storage pool (default storage for StarWind devices) by clicking Yes.

7. Select the disk which was recently mounted.

Configuring StarWind Storage on Top of Software RAID

Make sure that the prerequisites for deploying Software RAID with StarWind Virtual SAN are met:

• The ESXi hosts have all the drives connected through HBA or RAID controller in HBA mode
• StarWind Virtual SAN for vSphere VM is deployed on the ESXi server and turned off
• StarWind Virtual SAN VM is installed on a separate storage device available to the ESXi host (e.g. SSD, HDD etc.)
• HBA or RAID controller can be added via a DirectPath I/O passthrough device to a StarWind Virtual SAN VM without affecting ESXi host work

PCI Device Configuration

1. Login to the ESXi host where StarWind Virtual SAN VM is installed.

2. In the Navigator, go to Manage, and in the Hardware tab, select PCI Devices.

3. Locate the HBA/RAID Controller of the ESXi host. Check the box on the appropriate PCI device. Click Toggle passthrough.

4. Restart ESXi host to make PCI device available to VMs.

5. Right-click on the StarWind Virtual SAN VM to Edit Settings.

7. Click ADD NEW DEVICE. Select PCI Device.

8. Add HBA/RAID Controller to the VM. Reserve memory for the StarWind Virtual Machine. Click OK.

9. Boot StarWind Virtual SAN VM.

10. Repeat steps 1-8 for all hosts where StarWind Virtual SAN for vSphere is deployed.

11. Login to StarWind Virtual SAN VM via IP. The default credentials:
Login: user
Password: rds123RDS
NOTE: Please make sure that the default password is changed.

12. Go to the Storage page. The Drives section shows the drives connected to HBA/RAID Controller (if available). For each disk, create partition table. 

13. Click “+” in the RAID Devices section to create Software RAID. (In the current example, RAID 10 will be created with 4 HDD drives). The RAID configuration depends on the number of disks, chunk size, and array level are shown in the table below:

StarWind Software RAID recommended settings can be found here:
https://knowledgebase.starwindsoftware.com/guidance/recommended-raid-settings-for-hdd-and-ssd-disks/

14. Select the drives to add to the array.

15. After the synchronization is finished, find the RAID array created. Press Create partition table and press Format afterward to create the partition and format it.

NOTE: It is not necessary to overwrite data while creating a partition.

16. Create the XFS partition. Mount point should be as follows: /mnt/%yourdiskname% . Select the Custom mounting option and type noatime. To enable OS boot when mount point is missing (e.g., hardware failure), add nofail as a boot option.  Click Format.

17. On the storage page of the disk, navigate to the Filesystem tab. Click Mount.

18. Connect to StarWind Virtual SAN from StarWind Management Console or from Web Console. Click Yes.

19. Select the disk recently mounted.

Synchronous “Two-Way” replication

1. Right-click the recently created device and select Replication Manager from the shortcut menu.

2. Select the Add Replica button in the top menu.

3. Select Synchronous “Two-Way” replication as a replication mode.

4. Specify a partner Host name or IP address and Port Number.

Heartbeat

1. Select Failover Strategy.

2. Select Create new Partner Device and click Next.

3. Select a partner device Location and click Next.

4. Select Synchronization Journal Strategy and click Next.
NOTE: There are several options – RAM-based journal (default) and Disk-based journal with failure and continuous strategy, that allow to avoid full synchronization cases.

RAM-based (default) synchronization journal is placed in RAM. Synchronization with RAM journal provides good I/O performance in any scenario. Full synchronization could occur in the cases described in this KB: https://knowledgebase.starwindsoftware.com/explanation/reasons-why-full-synchronization-may-start/

Disk-based journal placed on a separate disk from StarWind devices. It allows to avoid full synchronization for the devices where it’s configured even when StarWind service is being stopped on all nodes.
Disk-based synchronization journal should be placed on a separate, preferably faster disk from StarWind devices. SSDs and NVMe disks are recommended as the device performance is defined by the disk speed, where the journal is located. For example, it can be placed on the OS boot volume.
It is required to allocate 2 MB of disk space for the synchronization journal per 1 TB of HA device size with a disk-based journal configured and 2-way replication and 4MB per 1 TB of HA device size for 3-way replication.

Failure journal – provides good I/O performance, as a RAM-based journal, while all device nodes are in a healthy synchronized state. If a device on one node went into a not synchronized state, the disk-based journal activates and a performance drop could occur as the device performance is defined by the disk speed, where the journal is located. Fast synchronization is not guaranteed in all cases. For example, if a simultaneous hard reset of all nodes occurs, full synchronization will occur.

Continuous journal – guarantees fast synchronization and data consistency in all cases. Although, this strategy has the worst I/O performance, because of frequent write operations to the journal, located on the disk, where the journal is located.

5. Click Change Network Settings.

6. Specify the interfaces for Synchronization and Heartbeat Channels. Click OK and then click Next.

7. In Select Partner Device Initialization Mode, select Synchronize from existing Device and click Next.

8. Click Create Replica. Click Finish to close the wizard.
The successfully added device appears in StarWind Management Console.

9. Follow the same procedure for the creation of other virtual disks that will be used as storage repositories.

Node Majority

There are two ways to configure Witness for 2-nodes StarWind HA device, created with Node Majority Failover Strategy: File Share (SMB) as Witness and additional server as Witness Node.

– Creating HA device with File SHare(SMB) as Witness:

SMB Witness is a file, located on SMB share, which can be accessed by both nodes and help them to eliminate the split-brain issue in case of synchronization connection interruption between the nodes. To set up the SMB file share as a Witness for 2-nodes HA device with Node Majority Failover Strategy, perform the actions, described on this page:
https://www.starwindsoftware.com/help/ConfiguringFileShareSMBasWitness.html

– Creating HA device with Witness Node:

1. Select the Node Majority failover strategy and click Next.

2. Choose Create new Partner Device and click Next.

3. Specify the partner device Location and modify the target name if necessary. Click Next. Select Synchronization Journal strategy and location and click Next.

4. In Network Options for Replication, press the Change network settings button and select the synchronization channel for the HA device.

5. In Specify Interfaces for Synchronization Channels, select the checkboxes with the appropriate networks and click OK. Then click Next.

6. Select Synchronize from existing Device as the partner device initialization mode.

7. Press the Create Replica button and close the wizard.

8. The added devices will appear in StarWind Management Console.
Repeat the steps above to create other virtual disks if necessary.

Adding Witness Node

Witness node can be configured on a separate host or as a virtual machine in a cloud. It requires StarWind Virtual SAN service installed on it.
NOTE: Since the device created in this guide is replicated between 2 active nodes with the Node Majority failover strategy, a Witness node must be added to it.

1. Open StarWind Management Console, right-click on the Servers field and press the Add Server button. Add a new StarWind Server which will be used as the Witness node and click OK.

2. Right-click on the HA device with the configured Node Majority failover policy and select Replication Manager and press the Add Replica button.

3. Select Witness Node.

4. Specify the Witness node Host Name or IP address. The default Port Number is 3261.

5. In Partner Device Setup, specify the Witness device Location. Optionally, modify the target name by clicking the appropriate button.

6. In Network Options for Replication, select the synchronization channel with the Witness node by clicking the Change Network Settings button.

7. Specify the interface for Synchronization and Heartbeat and click OK.

8. Click Create Replica and then close the wizard.

9. Repeat the steps above to create other virtual disks if necessary.
NOTE: To extend an Image File or a StarWind HA device to the required size, please check the article below:

https://knowledgebase.starwindsoftware.com/maintenance/how-to-extend-image-file-or-high-availability-device/

Adding Discover Portals

1. To connect the previously created devices to the ESXi host, click on the Storage -> Adapters -> Software iSCSI and in the appeared window choose the Enabled option to enable Software iSCSI storage adapter. Push the Save configuration button.

2. In the Configure iSCSI window, under Dynamic Targets, click on the Add dynamic target button to specify iSCSI interfaces.

3. Enter the iSCSI IP addresses of all StarWind nodes for the iSCSI traffic.

Confirm the actions by pressing Save configuration.

4. The result should look like in the image below.

5. Click on the Rescan button to rescan storage.

6. Now, the previously created StarWind devices are visible to the system.

7. Repeat all the steps from this section on the other ESXi host, specifying corresponding IP addresses for the iSCSI subnet.

Creating Datastores

1. Open the Storage tab on one of your hosts and click on New Datastore.

2. Specify the Datastore name, select the previously discovered StarWind device, and click Next.

3. Enter datastore size and click Next.

4. Verify the settings and click Finish.

5. Add another Datastore (DS2) in the same way but select the second device for the second datastore.

6. Verify that your storages (DS1, DS2) are connected to both hosts. Otherwise, rescan the storage adapter.

NOTE:  Path Selection Policy changing for Datastores from Most Recently Used (VMware) to Round Robin (VMware) is added into the Rescan Script, and this action is performed automatically. For checking and changing this parameter manually, the hosts should be connected to vCenter.
Multipathing configuration can be checked only from vCenter. To check it, click the Configure button, choose the Storage Devices tab, select the device, and click the Edit Multipathing button.