Downtime? Error Loading Buckets For Some Users

dreamobjects

#1

This morning I cannot access buckets and public objects return

503 Service Unavailable
No server is available to handle this request.
[hr]
Okay this seems resolved now 15-20 minutes later.

I guess that’s why the beta label, not quite ready for primetime.

But I thought the idea is the entire network cannot go down at once if there is replication.


#2

We are seeing some intermittent issues related to Ceph’s RADOS gateway software. This is the software that provides the API connectivity via S3 and Swift protocols. Any activity in the panel is actually using the S3 API through these gateways. We’ve been doing a lot of testing and benchmarking on these machines to identify their maximum load capability and will be making the necessary changes when we have the results. I apologize for these problems but I appreciate you taking the time to report them.

As for replication, this is for your data. Your data is stored redundantly, with at least three copies, spread across the storage cluster. Ceph’s CRUSH algorithm ensures data is protected from a failed disk or even an entire rack of equipment going down. Any network issues can still cause trouble accessing your data. We do have redundant networking systems in place as well, but this particular issue is a software issue.


#3

I’m not sure if I’m experiencing the same problem but whenever I launch the Objects Viewer from the Panel and click on my only bucket I’m getting “Network Error. Try again later”. I’m seeing this for a week now. Can you confirm that this problem is known to you?
This is not a big deal for me since I can still access my data via Cyberduck etc. but I am glad you guys are working on this. Thanks for your effort.


#4

Axel, that sounds more as though there may be something unusual about your bucket that’s causing the Bucket Viewer to get confused — since you’re still able to access the bucket through Cyberduck, the data is clearly still available. Please contact DreamHost Support with details so we can investigate this issue!


#5

Done. Thanks!