Unable to delete files of a removed device

[AHC_Admin]

In order to delete a device and all of its files, is a restart of the StarWind service required? I ask as I am unable to delete the IBV files because they are reportedly open by the service.

If a restart of the service is required, should I shut down all virtual machines that are using other devices as part of their Hyper-V cluster shared volumes on the same SAN server?

We are using StarWind version 6.0.5437


Kind regards
AHC_Admin

[Max (staff)]

Hi,
Usually service restart isn't required and the files can be removed right away.
However, the files can remain locked for some time while StarWind is flushing their cache to the disk in the background mode.

If you're restarting the service the VMs, running on the StarWind devices have to be turned off.
If the cluster quorum is served by StarWind then all the VMs need to be turned off or an alternative quorum has to be configured.

[Ironwolf]

ImageFile locked CPU.PNG (50.6 KiB) Viewed 6368 times

Starwind version 8.0.7509, (single node)

I am experiencing the same thing

Drive configured:
32 GB
512MB write-through cache
L2 2GB, Write-Back

When copying a 20GB VM into the LUN, everything works fine if it is the only thing going on, if we are using other LUN’s during this process, the Copy will fail and one CPU core will be locked at 100%, and the LUN becomes inaccessible.

At this point we can still use other LUN’s but the LUN we are moving the VM too will now be inaccessible.

Deleting the LUN does not release the files (cannot be deleted) and does not release the CPU usage, though the LUN is removed from the Management Console

Creating replacement LUN leaves the new image files in a (non-active state)

We 1st encountered this when we moved a 800GB VM, the 2nd attempt was the 20GB VM, both required a reboot of the server to release the CPU core and files.

[Max (staff)]

I think it is a different issue with similar symptoms.
Is there any chance we could get the diagnostic information from the system to identify what went wrong?

[Ironwolf]

I have uploaded the Log files, they have rotated very frequently during the locked core, hopefully their is something usable in them

As for the Dump file, we have already reboot the system, will try to recreate the locked CPU core and send you that dump file as well.

[Max (staff)]

That would be really great since the most valuable information on the issue will be in that dump.
Please upload the dump and the logs once you reproduce the issue and I'll get it to our R&D immediately.

[Ironwolf]

I recently uploaded the log files and memory dump.

I had been having problems with moving large amounts of data, as in VM move storage. This time around we were running stable VM's for weeks not moving storage. At some point the LUNs which had L2's started slowing down and showing signs of instability. A week later VM's with out the L2 cache started experiencing similar issues, the more the LUN's with L2's were some how as a side effect, effecting the ones without the L2's but no where near a dramatic.

Also on the CPU core that became locked, stayed locked even after the computer running Hyper-V was shutdown, and were no current connections. Additionally the logs spiraled out of control and were continuing even with the Hyper-V shutdown.

Hope this helps

[Anatoly (staff)]

Quick questions:
What is the amount of the L2 cache?
What is the mode of L2 cache that you are running? Write-Back or Write-Through?

[Max (staff)]

Hi IronWolf!
Since last time you've reproduced this issue we have fixed multiple issues leading to the device lock-up.
So the version we currently have on the website may just fix all the issues you had.
Is there any chance you could install the most recent version and then try to reproduce the issue?

[Ironwolf]

7509 -> 7852(now loaded)

Haven't had a chance to re-visit this issue, Just updated the system today...

For the L2 sizes they were about 5%, 1 to 8 GB in size depending on the LUN usage/size
They were all configured for write-back

-=-=-

WoW, only been on the server about 1 hour, we have noticed huge improvements, only a few LUNs atm (IMG, write-through ram cache, no L2), but so far throughput is a bit over double of what it was.

Also noticed this version seem no longer locked to a single core of execution and work load is now spreading over a single chip, still no NUMA support though, at lease on IMG files that is, this may be the brunt of the Throughput improvement but its made a huge improvement so far.

Hopeful the CPU usage will expand to LSFS/DeDup as well (none loaded for testing atm)

-=-=-=-

Planning on testing next week, will let you know

[Max (staff)]

Glad to see you're enjoying the improvements!
BTW: We're working on the NUMA improvements too. Stay tuned.
And don't forget to tell us if the update fixed the original issue you had!