Objects don't store!

dreamobjects

#1

Today starting just afternoon my usual uploads stopped working. About everyday I store hundreds of images in the DreamObject cloud. These are jpegs that are viewable online by my users. Today one session worked Ok. Since 12:22 PM 15 more sessions have uploaded without error but the images are not found like this:
https://objects.dreamhost.com/argentum/ARU/phac/ttn34/thumbnails/IMG__002.jpg
I’ve re-run uploading this session a few times. I get no errors and it seems to take the usual amount of time. But there is nothing in the cloud.
Anybodey else having a problem today?
-mort

additional:
I’m using the perl module Amazon::S3. error reporting seems a bit scratchy. All I get is “Quota Exceeded.” I don’t know how or what that is all about. I’m pretty sure my billing is up to date. I’m not a huge user, currently using about 3 TB. Anybody know what it means?


#2

I just checked my buckets. Everything seems to be okay on my end.


#3

mort, please contact DreamHost Support.


#4

Have done, thanks.
#6884749 and #6884913


#5

This still isn’t fixed.
I first complained yesterday at 4PM.
I have not heard anything from support.
I’m feeling pretty crappy.


#6

I’m geting very slow times for PUT and DEL operations.


#7

My problem was related to a quota on number of objects per bucket. Support tells me that there is a limit of 10 million objects per bucket. I started 2 new buckets and can upload to them fine. Has anybody heard of this before? How can I know how many objects I have in a bucket?
-mort
[hr]

Me to. Starting at 5:30 AM uploads have been intermittently slow for me.
Today to noon, the graph’s maximum is 100 seconds
-mort


#8

I got this from support today:

[quote]Hello,

I wanted to update you and let you know that we have removed any quotas
from your account. It is important though that you work to re-organize
the data and prevent putting 10’s of millions of objects into a single
bucket. Due to limitations with the Ceph system the metadata can grow so
large and be in a single file (not sharded) that it can cause disruptions
in service to DreamObjects, and so we have to suggest to customers, and
in extreme situations put on quotas to avoid disruption of service. If
you can shard your data, distributing it between multiple buckets, it
would help to prevent stability issues with DreamObjects as well as make
your access faster.

We are awaiting later versions of Ceph that will have sharding for
metadata built in so that it can handle larger number of objects.

Thanks![/quote]

But last February in a post titled
Is there a recommended max number of objects per bucket?
one reply said

[quote]I’m not aware of any recommended practices for this. Compared to a real file system, where it does make sense to have subfolders of subfolders to ease in browsing, listing files and preventing reaching files-per-directory limits, DreamObjects is different.

Every file you upload, doesn’t go into a directory but the name of the file has the full path in it instead. So one could imagine it as 200,000 files in the / directory, and they just happen to have some paths in their file names, so when you browse a specific directory in a S3 client, it searches all files for that path in their name to display them like they are in a directory.

So in general, do what is easier for you for organizing your data and you should be fine.[/quote]

Times change.
-mort