We are seeing some intermittent issues related to Ceph's RADOS gateway software. This is the software that provides the API connectivity via S3 and Swift protocols. Any activity in the panel is actually using the S3 API through these gateways. We've been doing a lot of testing and benchmarking on these machines to identify their maximum load capability and will be making the necessary changes when we have the results. I apologize for these problems but I appreciate you taking the time to report them.
As for replication, this is for your data. Your data is stored redundantly, with at least three copies, spread across the storage cluster. Ceph's CRUSH algorithm ensures data is protected from a failed disk or even an entire rack of equipment going down. Any network issues can still cause trouble accessing your data. We do have redundant networking systems in place as well, but this particular issue is a software issue.