RewriteCond %{HTTP_REFERER} not working .htaccess


#1

Hi All,
I am hoping someone can help me, as I am totally stumped. I wrote support and they said this was a “programming” issue and couldn’t help me.

Anyway, I am trying to block referral visits from ebay to my site.

I placed the following in my .htaccess file (per the article in the wiki)
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.co.uk [NC]
RewriteRule .* - [F]

And when I go to ebay and click on the link pointing to my site, it comes right up. (And yes, the referring link is cgi.ebay.com (or .co.uk).

All of the rewrites in my htaccess file are working correctly, so I have absolutely no idea what the issue is. I have cleared my cache and tried it from a completely different computer, yet my site still comes up when clicking on the referring link.

Any ideas/help/advice is greatly appreciated.

Thanks
Morty


#2

How about changing it to (see additions in red):

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.com/[color=#CC0000].[/color] [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.co.uk[color=#CC0000].
[/color] [NC]
RewriteRule .* - [F]

The only other difference I see between yours and the ones I use on a few domains would be that I have RewriteBase / after RewriteEngine On.


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#3

Still not working :frowning: This is really starting to drive me crazy.

I put in exactly what you had, and after clearing my cache, still able to get to the website.

Then based on your statement about RewriteBase, I tried this but I am not sure if it is what you meant.

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.com/.* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.co.uk.* [NC]
RewriteRule .* - [F]

Thanks!
Morty


#4

Semi-educated guessing here. How about change from:

RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.com/.* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.co.uk.* [NC]
RewriteRule .* - [F]

to (red):

RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.com[color=#CC0000][/color].* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.co.uk[color=#CC0000][/color].* [NC]
RewriteRule [color=#CC0000]^[/color].* - [F]

Did you check error logs for clues? What are you trying to stop, if you wouldn’t mind giving a little more detail?

You’re not using a referer blocker by any chance? :slight_smile:

They recently hired more support help.
DreamHost Customers wiki And Forum


#5

thanks anonymous,
That didn’t work either :frowning: Ugh, this should not be that difficult.

Basically someone on ebay is pointing to my site in their listings, and I don’t want to be associated with them.

Not sure what you mean by a refferral blocker, but I am not running anything, just trying to manually add it in my .htaccess file.

I have other Rewrite calls in my file, all of those work fine. So I have no idea what is going on…


#6

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://cgi.ebay.com/
RewriteRule /* http://www.yoursite.com/restricted_url.html [R,L]
RewriteCond %{HTTP_REFERER} ^http://cgi.ebay.co.uk/
RewriteRule /* http://www.yoursite.com/restricted_url.html [R,L]

That’s a minor tweak to a copy paste from the wiki. If that doesn’t work, I’m giving up for now (sorry).

You’d need to put up a restricted_url.html page or live with 404’s, I suppose. A page would deliver an appropriate message if you wanted.

A referer blocker is something that removes or modifies referers being sent by your browser. For example Firefox has a plugin and some firewalls can do it. A standard setting might be to always send a referer of the homepage of the site being requested from (i.e. your site in this case). If you (or someone else) were using one, it could prevent what you’re trying to accomplish.

They recently hired more support help.
DreamHost Customers wiki And Forum


#7

Tried it and it is not working :frowning:
(created the restricted_file.html and everything)

Uggghhhhh.

Thanks for your help…

Does anyone else have this problem?
Perhaps I am missing something basic?

BTW, I stripped out everything else in my .htaccess file and tested it again (to eliminate any possible conflicts) and it still didn’t work.

Morty


#8

You might look in your access log to see if something funny is going on to make the referer different than expected. Just a thought. Now I’m giving up again. :slight_smile: Somebody will come along and be the hero before too long. Then it’ll look obvious.

They recently hired more support help.
DreamHost Customers wiki And Forum


#9

I’m pretty useless with .htaccess (kchrist, where are you?), but you might be able to try a different approach like this:

RewriteEngine on RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)ebay(-|.).*$ [NC,OR] RewriteRule ^(.*)$ %1 [R=301,L]If I’ve worked this out correctly, I think it will block any referrer that contains “ebay” somewhere in the URL.


Simon Jessey | Keystone Websites
Save $97 on yearly plans with promo code [color=#CC0000]SCJESSEY97[/color]


#10

This brought the entire site down, giving me the following error message (even when accessing the URL directly)
The page isn’t redirecting properly

Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

  • This problem can sometimes be caused by disabling or refusing to accept
    cookies.

This is so frustrating.
This is a dyanmic php site, could that have anything to do with it? I tested blocking links from other sites that I control and I can’t block those either (so this eliminates anything along the lines of the ebay referral being blank).

Help :slight_smile:

Morty

PS> Thanks to everyone and their ideas so far, keep 'em coming, I am willing to try anything at this point!


#11

Checked the logs nothing funny there.

I switched the code that scjessey recommended (taking out the OR) to
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://(www.)?.(-|.)ebay(-|.).$ [color=#CC0000][NC][/color]
RewriteRule ^(.*)$ %1 [R=301,L]

this eliminated the sitewide error, but I could still link from ebay…


#12

I probably should have noted that I wasn’t trying to add a ., but rather “.*” to catch anything – in case there was something silly in a query string that he was getting past the check.


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#13

Give this a shot–but keep in mind that it might block more than you want it to.

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_REFERER} .ebay.(.) [NC]
RewriteRule .* - [F]

Here’s what that “should” do: Block anything with “ebay.” in it, no matter where it is (I left the dot to not block way more domains than necessary). But keep in mind that if that works, you would block names like freebay.com, ebay.someotherdomain.net, etc…

If that works, then we can start adding stuff back until it stops, to see where it’s going wrong.

Also, you should probably include the exact URL from where you’re clicking from the browser address bar (if it’s something you don’t mind revealing, or don’t care if we know your domain).


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#14

Thanks, but that didn’t work either.

At this point I would prefer not to share the URL’s.

However, the issue is not with the ebay domain itself, I can’t block any domains including my own.

Am I really the only person on dreamhost having this problem?

Thanks again,
Morty


#15

OK,
I have taken everything down to its simplest form, and tried blocking domains on a different site, hosted at the same DH server, and blocking doesn’t work on these sites either.

Here’s the test scenario:
From http://{snip}/sorry.htm
that link should be blocked at {snip}

I have to be missing something really really simple? right?

Thanks,
Morty


#16

These may seem like silly questions, but just to make sure it’s not something super simple being missed:

Is your browser set to hide referrer info?

Where is your .htaccess file?

If your domain is example.com, it should be in the /username/example.com directory.

Here’s a test for your browser. I just blocked discussion.dreamhost.com from seiler.dreamhosters.com.

So give this a click and see where it gets you.

I tested it and it works, so it should work for you. This is exactly what I put in the .htaccess file:

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_URI} /403test.html
RewriteCond %{HTTP_REFERER} .discussion.dreamhost.com. [NC]
RewriteRule .* - [F]

EDIT: Edited to change link to a specific URL so it will always work here as an example, without blocking all other links from the forum. Also updated .htaccess example to show the specific file being blocked from this site. Remove that line to block an entire domain.


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#17

Found it!!!

It is my Norton Firewall, blocking the referrer.
UGGGHHHHHHHH.

Thank you so much! I appreciate everyone’s help. I was about to lose my mind…

Now, is there anyway to get around firewall’s blocking referrers? If I find the IP and block that , would that work?

Thanks,
Morty

PS. thank you, thank you, thank you!!


#18

Looks like I got here a little late for the party.

Nope. Referers (sic) are strictly optional; user agents may or may not send them, as they choose, and proxy servers may strip them out at will. You could try also blocking anything with an empty referer string but this is guaranteed to block legitimate traffic as well.

What I’m curious about is why you want to prevent people from following links from Ebay to your site.


If you want useful replies, ask smart questions.


#19

Glad it worked out! :wink:

Well, you can block blank referrers, but then you block type in traffic and probably a few bots that shouldn’t be blocked.

RewriteCond %{HTTP_REFERER} ^$

I’d probably avoid doing that unless blocking ebay is more important than losing important traffic… and maybe even dropping from search engines.

You can block the IP like this, using 127.0.0.1 as an example:

RewriteCond %{REMOTE_ADDR} 127.0.0.1

You could shoot for either and hope the referrer isn’t blocked, or that multiple IPs aren’t being used:

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_REFERER} ^http://(cgi.)?ebay.co(.)$ [NC,OR]
RewriteCond %{REMOTE_ADDR} 127.0.0.1
RewriteRule .
- [F]

Note that I wildcarded eBay after “.co” so it will catch com or co.uk.


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#20

Ahh, that’s what I figured.

The reason I want to block the link is that someone is selling a turnkey script, and is saying that my site demo’s the script. Which it sort of does and sort of does not, as I have made some serious custom modifications to it. And I never agreed to have my site used as a “demo”.

Previous attempts to contact the person , have been futile, so I wanted to take the matter into my own hands.

Thanks!!!

Now, I need a nice stiff drink!!!

Morty