My website was hacked - figuring it out


#1

So, quite to my amusement, my dreamhost website was hacked: www.krisreed.com . No real harm was done. The site had nothing important on it–just a CMS that I am programming (of which the most recent version is on my hard drive). However, I am quite interested in figuring out how he hacked it, in order to prevent it in the future. My questions are at the very bottom if you would like to peruse them first.

Note
There is a nice page on the wiki: http://wiki.dreamhost.com/Troubleshooting_Hacked_Sites about what to do if your site was hacked

The name given on the hacked page is Cyb3erking, which, from a google search, seems to be a Turkish hacker. Now, my goal is to figure out how he hacked my site. I am putting this up here for anyone to add their 2 cents in.

So, here are the facts:
The only file that seems to have been affected is the index.php file, the content of which were completely replaced with the string “By Cyb3rking”.
The “modified” property of the file still says the 20th of October, which I believe corresponds to when I uploaded it.

Now, my ftp password was really weak, so it is quite possible that he cracked the password on it. However, I looked up the access log, and I got this:

(executed the command “last -i | grep kristoph” while SSHed in)

kristoph pts/0 98.206.92.143 Sun Nov 1 20:25 still logged in
kristoph ftpd6501 98.206.92.143 Sun Nov 1 18:31 gone - no logout
kristoph ftpd6446 98.206.92.143 Sun Nov 1 18:31 gone - no logout
kristoph ftpd26494 98.206.92.143 Sun Nov 1 09:59 gone - no logout

(the pts/0 entry is the ssh login I was doing to read the log file)

Notice that all of the entries are from my IP address, so 18:31 UTC-8 must be when I was FTPing in to see how the hack was done.

I also checked the HTTP access logs, and I have this (portion of text from my log file, stored in ~/logs/krisreed.com/http/:

66.249.67.196 - - [01/Nov/2009:01:32:02 -0800] “GET / HTTP/1.1” 200 277 “-” "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
67.202.60.234 - - [01/Nov/2009:01:48:59 -0800] “GET / HTTP/1.1” 200 214 “-” "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20090926 Iceweasel/3.5.3 (Debian-3.5.3-1)"
67.202.60.234 - - [01/Nov/2009:01:48:59 -0800] “GET / HTTP/1.1” 200 214 “-” "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20090926 Iceweasel/3.5.3 (Debian-3.5.3-1)"
66.249.67.196 - - [01/Nov/2009:09:03:36 -0800] “GET /robots.txt HTTP/1.1” 404 494 “-” "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.67.196 - - [01/Nov/2009:09:03:36 -0800] “GET /layout/layout.css.php HTTP/1.1” 200 3847 “-” "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.67.196 - - [01/Nov/2009:09:04:41 -0800] “GET /cjcms/editors/cjcms/editors/editors/xinha/XinhaCore.js HTTP/1.1” 404 513 “-” "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.67.196 - - [01/Nov/2009:12:12:07 -0800] “GET /cjcms/spages/ HTTP/1.1” 200 561 “-” "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.67.196 - - [01/Nov/2009:13:17:00 -0800] “GET /cjcms/cjcommon/ HTTP/1.1” 200 627 “-” "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
98.206.92.143 - - [01/Nov/2009:18:31:07 -0800] “GET / HTTP/1.1” 200 277 “-” "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3) Gecko/20090920 Firefox/3.5.3 (Swiftfox)"
98.206.92.143 - - [01/Nov/2009:18:31:07 -0800] “GET /favicon.ico HTTP/1.1” 200 270 “-” "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3) Gecko/20090920 Firefox/3.5.3 (Swiftfox)"
98.206.92.143 - - [01/Nov/2009:18:31:11 -0800] “GET / HTTP/1.1” 200 277 “-” “Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.3) Gecko/20090920 Firefox/3.5.3 (Swiftfox)”

Now, there are two hits at 1:48 UTC-8 from a 67.202.60.234 (which is in Turkey). I am assuming this is him. He apparently uses Iceweasel (and more-OpenSource version of Firefox). All the other entries for that day (up until I checked it at 18:31 UTC-8, everything afterwards is ommited) are either googlebot or myself (the “Swiftfox” entries).

So, basically, besides the two HTTP log hits, I don’t have a record of him doing anything (besides loading my page twice). I would expect an FTP login if he did it via FTP, or an HTTP log of more meddlesome activities if he had compromised my CMS. Neither are found.

Questions:
Could he have edited either the FTP or the HTTP log files to cover his trails?
How else could he have done this hack?
Is there anything else that I can check for?

Of course, any other insights, thoughts, or questions are appreciated.


#2

It looks as though the attacker managed to find an improperly secured rich text editor in your CMS. There are some older hits from a Turkish IP (78.172.180.110) in your archived logs for 2009-10-30 that indicate that they used the ImageManager plugin of one editor to upload a PHP script, which they then ran. I’ve placed a copy of that log file in your home directory for your perusal.

FWIW, users can’t edit their own web log files for exactly this reason.


#3

Pardon the bump, but I must say thank you very much!


#4

I’m replying to your older post instead of starting a new thread, because I ran into something similar recently.

Five-plus years at DH. Paid for the next four-plus years.

On 12/26 I spotted odd activity on one of my low-traffic subdomains which appeared to have started on 12/25.

It seems that I had a script dropped into the ‘images’ subdirectory on one of my subdomains back on 12/1, but it wasn’t /triggered/ until an HTTP POST request on 12/25.

The script was a double-encoded PHP script. The original would pretty much let you POST to the site whatever you wanted to do through a shell_exec.

Once the main script was triggered, it created a second script in the web root which wound up being the target of what looked like random documents from a news aggregator which then wrote those pages to a new hidden subdirectory on my site.

Other than taking up space, /those/ new files didn’t /seem/ to be much of an issue. Perhaps they were meant to mask a more threatening file in their midst.

Still, despite the fact that the initial script was so dangerous, I’m not really worried about it, per se. Its behavior was fairly easy to follow, once discovered.

I /am/ worried about being unable to identify the injection vector.

The HTTP logs (I have my own copies) show a GET request for the file placed in the ‘images’ directory at the same time as the timestamp on the file. No difference that I could see.

I’d expect a second or more delay - at least - if the file was written locally, then requested by some remote site.

This happened in mid-afternoon, then in two other subdomains a half-hour or so later, then a wave of about a dozen file requests/drops two hours later.

Some of the subdomains are dormant and the /only/ logged activity was the HTTP GET request on 12/01 for the files were dropped apparently at that time.

I have no packages on that domain. No apparently linked activity in the HTTP logs.

After swapping mail for a couple days, tech support basically told me to take two aspirin and ‘good luck finding that.’

I am so cheesed off by tech support - their lack of support was bad, but their lack of /concern/ may be even worse - that I’m now looking for new hosting.

I have no problem if it turns out to be something vulnerable that I put on my site, but the inability to know where to look for a remedy - to prevent a repeat - is ridiculous.


#5

Work backwards through your logs from there to see if you can find a clue.

Search known exploits if using a downloadable script on the affected domain (plug-ins, etc.).

Maximum Cash Discount on any plan with MAXCASH

How To Install PHP.INI / ionCube on DreamHost


#6

I’m no security expert, but is it possible to exploit a hole on a website hosted in /home/user/example1.com to place a file into the /home/user/example2.com directory?

-Scott


#7

Yes, this is entirely possible as both sites run “as”, and with the permissions of, the same user.

This is a very common occurrence, and “smarter” trespassers will do often do this. Only defacing “example2.com” and leaving “example1.com” alone serves to obfuscate their attack vector, and they are counting on all your research into what happened focusing on the domain where you notice the signs of the attack and overlooking the actual security flaw in the other site.

When doing forensics on an attack, you have to consider and inspect all code running under the user in whose directory an attack is in evidence, irrespective of the website showing the damage.

–rlparker
–DreamHost Tech Support


#8

Going back through my Apache logs for a month previous to the initial file drop shows no references to the script or the source IPs involved.

Here are what seem to be the earliest incursion events, with the previous and following requests included.

The HTTP POST request is where the script (a double-encoded PHP-generated web form) starts doing stuff. I’m trying to identify how the scripts aaa.php and giles_cary.php found their way in that subdirectory.

72.14.199.166 - - [01/Dec/2009:15:30:48 -0800] "GET /site/rss HTTP/1.1" 200 4597 "-" "Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=6609005421341191915)" [color=#CC0000]208.113.230.28[/color] - - [01/Dec/2009:15:54:15 -0800] "GET /Collections/1633BChapters/[b]aaa.php[/b] HTTP/1.1" 200 297 "http://google.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" 208.113.230.28 - - [01/Dec/2009:15:54:16 -0800] "GET /Collections/1633BChapters/giles_cary.php HTTP/1.1" 200 297 "http://google.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" [color=#0000CC]72.32.68.170[/color] - - [01/Dec/2009:15:54:16 -0800] "POST /Collections/1633BChapters/aaa.php HTTP/1.1" 200 193 "-" "-" 67.195.114.226 - - [01/Dec/2009:16:31:11 -0800] "GET /site/archivebook/ShadowofSaganami/PARAMETERS/ HTTP/1.0" 200 3940 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" [color=#CC0000]208.113.230.28[/color] is another Dreamhost box.
[color=#0000CC]72.32.68.170[/color] is a rackspace box, jpgonline.com

So, basically, the first reference at 15:54:15 is what seems to be the first reference to the aaa.php script which suceeds, the next line with the giles_cary.php script is more typical of what I’ve found. In other subdomains in my ‘images’ subdirectory.

The two scripts seem to have the exact same timestamps as the initial HTTP GET requests.

That subdomain is running a site written with a framework system called Codeigniter (as are a couple of other subdomains).