Using htaccess to block referrer spam and hotlinks


#1

This is a two-part question I’ve done alot of searching and experimenting on and would really appreciate some answers I can’t seem to find.

I’m using Movable Type 3.2 and get a lot of referrer spam, (this also affects my other, non-blog site to a lesser degree) comments and trackbacks are disabled and the files for these are either renamed or deleted.

I was searching for a way to block this spam, and the most common approach was using htaccess. I put together a piece that looks like this:

[code]RewriteEngine On #only include this line once to enable the rewriting engine

deny from 84.174.72.102
deny from 217.185.183.70
deny from 205.234.145.222

RewriteCond %{HTTP_REFERER} ^http://(www.)?.(-|.)?debt(-|.).$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.(-|.)?consolidation(-|.).$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.(-|.)?equity(-|.).$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.(-|.)?poker(-|.).$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?.(-|.)?casino(-|.).$ [OR]

(etc … snip)

RewriteCond %{HTTP_REFERER} ^http://(www.)?slamhost.$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?spielepsychatrie.
$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?superface.$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www.)?texasholdem.
$ [OR]

(etc … )

RewriteRule .* - [F,L]
[/code]Originally I tried including a write-to-log piece I picked up during my searching,

[code] # Set an environment variable “BAD_GUY” so we can send their logs to a different file

Be sure to leave out OR on the last RewriteCond, or your RewriteRule will -always- be

executed in some versions of Apache

RewriteRule ^(.*) %{HTTP_REFERER} [R=301,E=BAD_GUY:1,L]

Because we have set E=BAD_GUY above, we can do this in our log file:

CustomLog /var/log/apache/access.log combined env=!BAD_GUY
CustomLog /var/log/apache/access_bad.log combined env=BAD_GUY
[/code]but this gave me a ‘500 internal server error’ (when viewing supernaut.info) when the .htaccess (permissions set to 644 rw-r–r--) file was placed in the supernaut.info folder, (as well as I’m not sure of the correct file path for CustomLog). Without that last piece everything seems to run ok, but I don’t really know how to check or if it will even make a difference. Any ideas on how to improve this, confirm it’s blocking, and include the logging to a separate file.

The second part is relating to hotlinking, which I get a bit of.

Currently I’m using this:

RewriteEngine on RewriteCond %{HTTP_REFERER} !^$ RewriteCond %{HTTP_REFERER} !^https?://(www\.)?supernaut.info/?.*$ [NC] RewriteCond %{HTTP_REFERER} !^https?://(www\.)?francesdath.info/.*$ [NC] RewriteRule \.(gif|jpg|jpeg|png|mp3|mpg|avi|mov|pdf|swf|tif|tiff)$ - [F,NC] My images are arranged by year-month folders in the images/ folder, eg: /images/06apr/foo.jpg. The main question here is that from my experience (and htaccess stuff is not really something I spend much time contemplating), this .htaccess file needs to be in the folder with the image files to work, i.e. in images/06apr/ rather than the next level up, images/ which only contains folders. Is this correct?

Thanks heaps for any help

Frances