LSFS volume grew, filled hdd, now datastore EMPTY??

MichelZ · Wed Mar 25, 2015 1:35 pm

An update from me: Support ran a Tool yesterday and then needed to reconvene for the next day to review the output.
I'm currently waiting (since 35 minutes) for the engineer to show up for our scheduled session

It just feels that we are not taken seriously here

kkuszek · Wed Mar 25, 2015 1:42 pm

I don't believe that's the case. It's a serious issue threatening their primary product. They have to be aware of the severity of the state of v8. I think they are not staffed to handle support requests in a timely manner, I have only ever talked to 2 engineers and most of my problems were addressed days/week+ out.
I also think that v8 was not thoroughly tested before it was rushed to market. Too many basic needs unfinished, unimplemented, or just broken.
I did purchase the product, but am now on free because of the problems we had. Unfortunately we have some of our data hostage here waiting on them to make good. If they can not recover lost customer data I am sure that would not bode well for future sales.

MichelZ · Wed Mar 25, 2015 1:48 pm

So why am I waiting now for 50 minutes (I had an appointment) without anyone telling me anything?
I have e-mailed the engineer and haven't got anything back yet... It is kinda annoying.

kkuszek · Wed Mar 25, 2015 1:51 pm

I am willing to bet their phone is ringing off the hook right now, I bet you will find they have another customer on the line from an earlier call.

I would agree it would be professionally courteous if they at least e-mailed you or someone let you know.

Romsla · Thu Mar 26, 2015 7:48 am

I feel sad for you guys.

Could you at least share the server workload type? You must have a very specific workload on your VMs.

Unless your vm guests are low on RAM and swap like hell, I can't imagine a workload that would lead to 8TB writes for 2TB systems such as yours in a short time.

kkuszek · Thu Mar 26, 2015 12:20 pm

For me I setup Starwind about 4 months ago and I haven't even migrated all of our production servers to it! Not only that, it was 1 of 3 Starwind installations dividing up the write load. Our backup incremental's are only around 100gb on a bad day so we are not pounding away with data transfers. performance was so Aweful we couldn't if we wanted to. Veem wasn't breaking 20mb at 3am with nobody on and completely idle servers

Fri Mar 27, 2015 11:09 am

Gentlemen,

I want to apologize once again that you are facing this issue. I hear you very loud and clear, and I totally understand how you feel. I was in your shoes once, I don`t want to feel it again. Believe me, support and R&D team are doing their best to get to the bottom of the issue and get the proper tool to recover the data.

Once we will have it, we will make sure that this tool will be available for everybody who will need it.

kkuszek · Thu Apr 02, 2015 1:40 pm

Almost weekly check-in post. No progress then I am guessing?
Thread has been quiet

MichelZ · Thu Apr 02, 2015 1:48 pm

I delivered a 570 GB logfile from a tool end of last week. I guess it takes a while to look through that (yes, that's 1/2 TB)

kkuszek · Thu Apr 02, 2015 3:50 pm

Thanks for letting me know.

For reference also, I have sw 8.0.7509 thin datastores on my test host I have completely migrated off over a month ago. These datastores are consuming 3tb of space on disk. I never had more than a few hundred gigs on that datastore. I am decommissioning both of them now to reclaim that space for that particular machine. I don't know how much longer I can sit on the other 2 host stores unused so I hope something comes of this.

Wed Apr 08, 2015 9:42 am

Hello Kkuszek!

We have released new build (7852), where the uncontrollable growth of LSFS devices were fixed.
Though you have to keep in mind that if you create 10GB LSFS device it will require 30GB of actual disk space.
It is 3 times bigger due to LSFS device logs all the write and erase operations with data.

You can always download our lated build by following this link:
https://www.starwindsoftware.com/customer-page

kkuszek · Wed Apr 08, 2015 12:34 pm

Hello Oles, thank you for the reply.
I did see the notification of this via e-mail. I will wait and see other results over much time. I believe requiring triple space is not very intuitive and likely should be built into the software with multiple safeguards, warnings, and perhaps visualizations.

That being said, I no longer run any of my production data on Starwind datastores. I appreciate you posting but my concern is disaster recovery still.

Is the log playback tool still in development, and is it a reasonable expectation still to see this created or now that the issue is resolved is it not a fruitful pursuit for Starwind?

Sat Apr 11, 2015 3:04 pm

There's a way to make 1:1 space requirement but in this case performance would suffer. Also 1:3 is maximum space used for working set and it would eventually shrink back to 1:1 if left untouched.

Yes.

kkuszek wrote:Hello Oles, thank you for the reply.
I did see the notification of this via e-mail. I will wait and see other results over much time. I believe requiring triple space is not very intuitive and likely should be built into the software with multiple safeguards, warnings, and perhaps visualizations.

That being said, I no longer run any of my production data on Starwind datastores. I appreciate you posting but my concern is disaster recovery still.

Is the log playback tool still in development, and is it a reasonable expectation still to see this created or now that the issue is resolved is it not a fruitful pursuit for Starwind?

kkuszek · Mon May 04, 2015 2:55 pm

hate to be a pest but hate to have unrecoverable data,

TTT

MichelZ · Mon May 04, 2015 3:07 pm

Unfortunately, we settled for dataloss and gave up on recovery