No idea what happened, I couldn't get on EC until just now, it kept saying it was down. Tech issue or EC-side problem?
There were some database issues. I don't know any more details, but Chip will let everyone know if there's anything more to tell.
Three separate problems. The data center where our servers are housed had a power problem, which took nearly all of the servers in the data center (including ours) offline. When they solved the power problem, the power failure had caused many file corruptions in the servers that had to manually be corrected. Then, once the file corruptions were fixed, there were issues with connections to our database server. All in, I think we were down for about 8 hours. This is extremely unusual for our provider, who has the reputation for being one of the most reliable web hosts in the business. Hopefully that's the end of it.
I got worried that it was a problem on my end since my net has been having problems for a while. Thankfully it's back up and let's hope it stays that way. I was experiencing forum withdrawls.
Being a tech guy myself - a power outage? In a datacenter? Does not compute. I've seen a catastrophic power issue in a datacenter exactly once in over 10 years of doing this. Trust me, it's no fun. Planned powerdowns are no fun, either - but that's another story Hopefully this was an isolated incident, sounds like it was
They had a massive failure with their uninterruptable power supply grid. Apparently a software update propagated through the network of UPSs, and cause them to believe there was a large power spike that wasn't really there, so the whole network shut down, but the auto-shift to generator power didn't occur for some reason. This data center is huge on redundancy, and they are now investigating why the in-place redundancy didn't work as planned. That outage lasted less than half an hour; it was bringing up all the routers and servers that ungracefully shut down that took all the time.
The site is still going in and out for me. Something on my end? I've reset my router and cleared my cookies and cache.