Dreamobjects not compatible anymore with awscli?

dreamobjects

#1

With the last version of awscli (on jessie) I cannot upload anything anymore.
Download works.

But sync upload, and cp to bucket results in:

A client error (AccessDenied) occurred when calling the PutObject operation: Unknown

What happened?


#2

can you be more specific as to which version of awscli you’re using?

 $ aws --version

will do. Thank you.


#3

aws-cli/1.4.2 Python/3.4.2 Linux/4.1.19+
[hr]
Your question about the version got me warned.
It turned out that Debian Jessie has a deprecated version of awscli.
I had uninstalled from apt-get and then used PIP to get the last version, now it is working.
Thank you.


#4

Glad to hear that that helped. There is a configuration change needed on newer awscli versions for the signature version, which is detailed in this article if you run into any further issues.


#5

Hello Justin.
It has been very difficult to deal with awscli on a fresh debian jessie install.

I just installed via pip install.
aws --version is:
aws-cli/1.10.20 Python/2.7.9 Linux/4.1.19+ botocore/1.4.11

But when I try to create a bucket:

aws --endpoint-url http://objects.dreamhost.com s3 mb s3://apriltest2016

I get:

make_bucket failed: s3://apriltest2016/ Parameter validation failed:
Invalid type for parameter CreateBucketConfiguration.LocationConstraint, value: None, type: <type ‘NoneType’>, valid types: <type ‘basestring’>

I am able to ls, cp and sync stuff to my buckets.

But the worst of it all is that awscli is reading wrong characters from the files in the buckets. It is like a locale mess.
File with names like 201 - foo bar.test , as are in my macbook, and as I upload then via cyberduck, looks like 201%-%foo_%bar.test inside the linux machine using awscli. This did not happned with my old machine with debian wheezy before.

[hr]
This is the aws --version output of my old linux machine were everything works just fine:
aws-cli/1.8.6 Python/2.7.3 Linux/3.18.0-trunk-rpi

raspbian wheezy.


#6

After uninstalling evertything, manually instaling all the python3 deps, and reinstallig aws-cli I end up with something that:

  • can create buckets
  • reads the files with the correct characters
    But cannot upload.

I got the error:
Unknown Signature Version: .

And yes I already did
aws configure set default.s3.signature_version “”


#7

FWIW I cannot replicate the issue… tried now again, with an older version of awscli and after an update. Latest version I have is

reed@laminato:~/workspace/dh$ aws --version
aws-cli/1.10.20 Python/2.7.10 Linux/4.2.0-35-generic botocore/1.4.11

and this is the attempt at uploading:

reed@laminato:~/workspace/dh$ aws --endpoint-url http://objects.dreamhost.com s3 cp test.txt s3://stefphone/test.txt
upload: ./test.txt to s3://stefphone/test.txt

Have you tried from a clean virtualenv?


#8

I created a virtualenv and then “pip install awscli”.

I end up with a aws cli that works!

But awscli inside the virtualenv seems to behave with wrong locale.

The filenames are like: wrong%%%charactes%$%$%.ext

and the sync job wants to delete everything I already downloaded and download those files again (thanks --dryrun!).

This locale error does not happens with the awscli I got from apt-get install awscli. Unfurtunatelly the awscli from apt-get cannot upload stuff.

So I am between a rock and a hard place. Good locale+upload error versus. Bad-locale+all the functions.

What am I missing ?


#9

I just gave it up completely now that I discovered the tool rclone, and I am ashame for not dicovering it before, I hope awscli and python go to another planet.
Rclone is amazing , and fast.
Please always support rclone, make sure Dreamhost’sDreamobjects is rclone best friend.
See ya! And Thank you very much


#10

It looks like rclone works with Ceph so it will work with DreamObjects - http://rclone.org/s3/#ceph:4678b1b3e85584748062cdb14e1461d6

The endpoint in the configuration will be https://objects.dreamhost.com


#11

I works so good, It comes with DreamOjbects settings inside its wizard dialog.
My suggestion would be for you the incldue rclone inside the DreamOjbects Wiki, it is probably the best tool for DreamObjects command line oparation, ever, and has got binaries that require no extra libraries for all the major plataforms.


#12

Glad to hear it’s working well for you!

Thanks for the info. We’ll write up a configuration guide and put it in our knowledge base.