ROOT Emptied my files


#1

I am so mad right now, that I’ll have to take my time not offending anyone here.

I run sickmydollars.com at dreamhost, and yesterday my customers started to complain about that they could not reach the video-files they had bought access to.

We checked everything, bandwidth, disk-space, even put up a support ticker, when one of my guys found out that someone from Dreamhost had tampered with our files.

Now, the Dreamhost support team is nearly unreachable, but our lawyers are not.

Someone has destroyed our files, and that is someone who has root access (because the file-flags and dir-flags that is present can only be changed by someone who has root access to the server, which we have not)

Now, we’re loosing aprox $2000 every day the system is down, and you don’t need to be a math scientist to understand that this is a major problem that must be solved right now.

Now I have to stop writing, because I could write something i would regret tomorrow.

So SUPPORT - Don’t send me standarized mail, SOLVE the problem and get back to me ASAP!!!


#2

Have you checked the backup folder .snapshot for a backup?

I should add by the way that this is a customer to customer forum only. You need to address your support question to support.


Norm


#3

where is the backup?

I know that it is a c2c forum, but Support has a 24h respond time, and time is money.


#4

cd .snapshot from your web root directory.


Norm


#5

there is no .snapshot at the webroot directory nor is it anywhere else on the server.


#6

You need to type cd .snaphot. It does not show up in any listings.


Norm


#7

I found it, but read this:

THE FILES IS GONE FROM THE BACKUP - ALL THE BACKUPS

This is sabotage, what should I do?

Some log files has also been emptied


#8

You will have to contact support in that case if your hourly backups are not there.


Norm


#9

This sounds like someone hacked your site, not somehthing that dreamhost would do. The logs from the ‘logs’ directory off root are only http logs. Actions from your user or root taken through SSH (how suport would connect to your server) or FTP would not be listed there. However, someone using a unsecure script on your site to hack into your account and deface things would want to clear those access logs, as his IP would be sotred there.

So while you’re still without a site and loosing money, you may want to consider the possibility that it’s not Dreamhost’s fault.

–Matttail
art.googlies.net - personal website


#10

I’m not accusing anyone, but someone had root access, and we do NOT have root access, wa cannot give a category root ownership, as the files are currently owned by root.

there should be backups, but the backups are empty too.


#11

I am also not accusing anyone. And I understand that it seems root has modified your files. No customer at dreamhost has root access (even dedicated cusomters normally only have superuser access).

I still fail to see why dreamhost would clear out logs that wouldn’t store any information about their actions, enless you’re saying that all of the logs have simply been deleted.

My point was only to sigguest that it might have been the actions of a hacker, and I still think that’s a possiblity, rather than dreamhost. I hope that your issue is resolved quickly, I know it must be beyond frustrating.

–Matttail
art.googlies.net - personal website


#12

file1.wmv Size 0 Date: 12.05.2006 12:54 -rwxrwxrwx

the files has been edited one by one, the clock tells me that this has been going on systemathicly since 01:30 untill 12:54
one by one the file has been emptied, and the ownership of the file got changed to root.

if it’s a hacker, then dreamhost has a problem, because the hacker has root access.


#13

Sorry for asking, but were the deleted files of an illegal nature? (ie: questionable copyright). I believe DreamHost retains the right to delete such files from their servers when/if they are discovered.

Mark


Save [color=#CC0000]$50[/color] on DreamHost hosting using promo code [color=#CC0000]SAVEMONEY[/color] ( Click for promo code details )


#14

We have the copyrights of all material, legal documents, and everything is legally ok.

If what you say is true, then it could be Dreamhost who has edited (emptied) the mediafiles, but I don’t understand why the index.html file is emptied as well.

I’ve still has not heard from anyone at dreamhost regarding those problems we had, even though i’ve posted a support ticket, and faxed them.

If I would be so stupid that I host illegal copyrighted material, then I would not post it on the forum, but when it’s videos lisenced to us for thousands of dollars, I really want my files to be stored safely, and not tampered with.


#15

Well, I was right. The files where tampered with by Dreamhost.

Quote dreamhost support:

I am terribly sorry - we were trying to balance out our data storage on the central file servers but it looks like there was a failure in one of the moves which resulted in the data not getting copied over correctly. I fixed the permissions so that you’ll have access to the files and have asked our admin team to restore (but if you prefer you can just upload your own local files if you have copies).

I might add, that the problem never got solved by dreamhost, because we where using seven hours to re-upload files, and it took dreamhost more than 36 hours just to respond to our request.

that aside, I have 624 unanswered questions from my customers who want to know why the video they rented from us coulden’t be shown. Some want their money back, some want to rent free videos to compensate their loss, and in money, well we lost quite an amount in rental-income and even more time wasted for our technical staff who where currently working on some other products, and had to put that aside to work out the data-loss issue, and re-locate files etc.

I understand that foo happends, but yet it’s frustrating not to get a quick respond when something like this is happening.


#16

I am glad that file wise everything looks to be being resolved.

I just wonder though, and please do not take this the wrong way, that with so much cash involved that using a low-priced shared server with all of the attendant problems is not the best way to go. If it were me I would tend towards using a dedicated server or two with a backup server for alternate serving of your customers files. Perhaps using a service where the uptime can be guaranteed as part of the server cost.

At least DreamHost admitted to a mistake and put their hands up where some companies would deny any failure with their dying breath.


Norm


#17

I started this business two years ago with two empty hands and $10 in my pocket.

Today we’re one of the fastest growing game & multimedia companies in Norway.

I still manage the business the same way I did two years ago, with low cost hosting, and high value income, because that gives us more to invest in projects that generate more income.

And, I don’t like spending money on stuff I don’t currently need.


#18

That is fair enough I suppose.

[quote]And, I don’t like spending money on stuff I don’t currently need.
[/quote]
I can understand that.


Norm


#19

Sounds like you need dedicated support. Something you won’t find on DH shared services. Your server is shared with hundreds of other people. Your data is at risk from normal users. Don’t know if you know permissions well enough, but that file above was set at 777. That means anybody on the server can delete it, or just empty it with a simple “echo > file1.wmv.”

Anyway, if you’re site is truly that important, then you should look for something with dedicated support. Specifically ones you can call up and talk to somebody in person. Just my opinion.


yerba# rm -rf /etc
yerba#


#20

hold on a second.
We have dedicated servers to handle our database and web requests
we only use dreamhost to store large mediafiles

We can discuss the business aspect from a to z for days, but the issue was files that suddently had no data in them.

The problem is solved, and it was in fact a dreamhost failure that happend, and dreamhost has apoligized and their apology was accepted.

The file with the 777 was an example file of how the emptied files where after they had been tampered with.