- Joined
- Oct 8, 2012
- Messages
- 28,275
- Reputation
- 10,118
- Reaction score
- 85,054
- Points
- 0
- Website
- www.autoflower.org
Hey guys, here's what happened:
We installed a chat add-on to try it out (and it worked great while it was running,) but it proved to be rather resource intensive for our server hardware (which admittedly is a bit older.) The speed slow down was in part because the majority of the server memory was being used up (not allowing free memory to do other things, like process requests from clicks on the site.)
So we disabled it and did a restart of the server. Do you guys know Murphy? From "Murphy's Law?" He showed up. After the restart, data in one of the tables got corrupted on one of the hard drives (which is why we had to run a restore, which is why we lost a day or two of data.) After the restore, it seemed like things were working well for an hour, then it bogged down again and started throwing errors. We had every techie on our admin team and the data center where it's hosted looking at it; it turns out our aging hard drive had just aged enough (and was dying.) The data center deemed it was a hardware failure, we went ahead and replaced ALL HARD DRIVES with brand new drives, and we've got the new restore up as of just a few minutes ago.
We sincerely apologize for the downtime and for the loss of the past 1-2 days of data; the hard drive failing was not an expected thing and we're already in deep discussion on beefing up our hardware requirements.
Thank you everyone for all the messages, PM's, e-mails, Facebook messages, and everything else! Really shows a lot of great support from the community and we appreciate all of you!
We installed a chat add-on to try it out (and it worked great while it was running,) but it proved to be rather resource intensive for our server hardware (which admittedly is a bit older.) The speed slow down was in part because the majority of the server memory was being used up (not allowing free memory to do other things, like process requests from clicks on the site.)
So we disabled it and did a restart of the server. Do you guys know Murphy? From "Murphy's Law?" He showed up. After the restart, data in one of the tables got corrupted on one of the hard drives (which is why we had to run a restore, which is why we lost a day or two of data.) After the restore, it seemed like things were working well for an hour, then it bogged down again and started throwing errors. We had every techie on our admin team and the data center where it's hosted looking at it; it turns out our aging hard drive had just aged enough (and was dying.) The data center deemed it was a hardware failure, we went ahead and replaced ALL HARD DRIVES with brand new drives, and we've got the new restore up as of just a few minutes ago.
We sincerely apologize for the downtime and for the loss of the past 1-2 days of data; the hard drive failing was not an expected thing and we're already in deep discussion on beefing up our hardware requirements.
Thank you everyone for all the messages, PM's, e-mails, Facebook messages, and everything else! Really shows a lot of great support from the community and we appreciate all of you!
