Outage and redundancy

dreamobjects

#1

This morning’s outage is still going on for hours. a huge part of our business, is for our customer’s to be able to stream audio, that is stored in dreamobjects. we are now thinking of switching to Amazon, as we are about to launch, and this would have been devastating. We can’t afford this to happen even once a year. How often does this happen? I know dreamobjects came out in 2012, so it’s pretty new.

what can we do for redundancy for our customers to stream their audio? do we have to use dreamobjects and Amazon? we were wondering how we would do that. we are also thinking of just using Amazon.


#2

Our apologies for the issues with DreamObjects this morning. Our team is working on getting this issue resolved as soon as possible and when we do we will update dreamhoststatus.com. We have our cloud team plus others from support contracts working on the issue, and once it is resolved we will be revising our hardware and systems to reduce the possibility of such an error re-occurring or taking a longer time to resolve.
[hr]
I just now received word that traffic is again being served by DreamObjects, so you should be able to host the files you need. We are continuing backend work on the system to improve it and get it back to a fully normal state.


#3

how long will it take for these new upgrades? I’m very worried about this. I saw you had an outage in June, that was a pretty long one. How many outages has there been this year? is there anything we can do for redundancy?


#4

We are expecting the hardware that caused this issue to be fixed and replaced within the next couple hours from spares we have on hand. After that, we are continuing a long process of upgrading to the newer versions of Ceph, fully enabling deep scrubbing for data consistency, and adding additional capacity to the cluster. We are not expecting outages with the addition of new hardware to grow the cluster. Next year there are also plans to add other data storage locations to diversify and grow. I understand how disruptive this outage can be but we are putting in all efforts to make sure this product retains all data stored in it, and is as high availability as possible.


#5

DreamObjects server seems to be out again!


#6

working again… must have been a temp glitch