Is mail being filtered?


#1

Is Dreamhost applying any sort of filtering, ban list, or other blockage of mail to addresses hosted here, aside from the user-configurable spam filtering options? I’m asking because mail from some servers never seems to get through, including some used by my employer (not hosted at Dreamhost).

– Dan


#2

Yes.

There’s currently no kbase article documenting this, though I hope to add one soon. We did make an announcement before we started doing this sort of blocking.

Currently we block based on a few open proxy blocklists, as well as an internal list; there are some blocks based on the HELO string sent by the sender, and (a very few) header-based ones.

These are all fairly conservative checks. If any messages are being rejected, they will be rejected with a clear indication of why, which will result in a bounce message being returned to the sender. I’ll be happy to look at any bounce messages indicating that legitimate mail is being rejected.


#3

Are there any plans to block more agressively? Is there any reason to allow messages from open relay servers at all? Not too server savy, just hoping there’s some way to kill some more of the spam…


#4

Well, you can turn on spam filtering per mailbox on your domain. It’s the Razor option in your mail tab in the Panel.

It doesn’t filter all, but at least it takes a good chunk of spam away from my mailbox before I see it. Then I can let MozillaMail filter the rest, or manage by hand.

I feel that Razor used to work better than it does now, but it still works pretty well. It’s a start, and it doesn’t hurt to turn it on if you haven’t done that already. Look for other forum posts about Razor to learn more about this.


TorbenGB


#5

I have Razor turned on, but the problem is that I have copy of my incoming messages mailed to my cell phone (via a sms-service). And that happens before Razor-ing. So I end up getting lots and lots of mail notifications of spam in my cell phone.

SpamSieve takes care of spam filtering on the computer and does it well also.

I’m just thinking that if much spam originates from open relay servers, wouldn’t it be a good thing to block all open relay servers (and teach those legitimate malconfigured mailservers a lesson or two… :wink:

Just thinking aloud, not too server savy, as I said.


#6

Open relay spam is nowhere near as popular as it once was. We don’t currently block open relays on our main customer mail machines, but it’s mostly

We do try our best to block open HTTP proxies and trojanned Windows machines[1]; this is a somewhat difficult task for many reasons:

  1. Spammers like to rotate proxies, meaning that some end up staying “under the radar”, so to speak.

  2. In cases where compromised Windows boxes are being used, the spammers usually have fresh ones available.

  3. There are a LOT of these machines. Maintaining an accurate list of them is very difficult. Also, detecting this sort of activity often requires invasive port scanning and other intrusive testing, which many organizations and ISPs don’t like. In general, most proxy lists tend to only scan hosts which have connected to a mail server affiliated with the blocklist maintainer in some way - this way they can say “you sent us mail; so we have the right to scan you for security holes”.

  4. Blocking dynamically assigned IP space would probably help, but isn’t really an option for us at this time (too much risk of blocking legitimate mail).

We do use a few DNS blocklists that attempt to block this sort of abuse - cbl.abuseat.org (which is very effective), proxies.blackholes.easynet.nl (which is, unfortunately, stopping operation at the beginning of December). It’s going to be difficult to find a good replacement.

I know it’s hard to believe (considering how much makes it through), but I’d guess we actually do block aproximately 35-45% of incoming mail due to UBE restrictions, and that’s with very conservative blocks in place.

If someone reminds me, I’ll try to post a graph of incoming mail volume vs. rejected mail this week (unfortunately, our graphs don’t show which rejections are due to unknown user errors and other stuff like that, and which ones are due to UBE controls).

Oh yeah - and happy Thanksgiving, folks. I’d better get cooking soon.

[1] Not sure if most people are familiar with open proxies, as well as the recent rash of viruses that create proxies designed specifically for spammers to use, but here are some of the reasons open proxies are bad and the reasons we try to block them:

  1. Unsecured (“open”) HTTP CONNECT proxies are misconfigured web proxies (squid, socks, etc.) which can be used to open an arbitrary tcp connection (including an SMTP connection) to an outside host - without the source IP address being disclosed. Even with open relay spam, you can see the original point of origin, assuming it’s not an open proxy -> open relay spam. So these proxies open up all sorts of problems (not just spam related), since people can use proxies to make an anonymous connection from anywhere to anywhere. Since most proxy operators are unaware that they’re operating an unsecured proxy, it’s rare for one to find logs of proxy abusers’ actual IPs (although some people have done some interesting “honeypot” work in this area).

  2. Sort of related to #1 - proxies allow spammers to insert forged headers, causing misdirected complaints, fooling spam tracking and / or identification services, and confusing recipients.

For example, I got a complaint about the following message yesterday:
Received: from source ([68.50.172.88]) by exprod5mx95.postini.com ([12.158.34.245]) with SMTP;
Wed, 26 Nov 2003 13:09:08 PST
Received: from [66.33.193.195] by pcp699323pcs.hyatsv01.md.comcast.net id <5321649-56884>; Thu, 27
Nov 2003 01:02:44 +0400

Now 66.33.193.195 is one of our IPs (one which isn’t currently assigned to anything). This received line is completely forged, though. The actual origin is the 68.50.172.88 IP.

For more information on proxy abuse, check out:
http://www.kb.cert.org/vuls/id/150227


#7

If you feel comfortable enough disabling the DH filters (the actual .procmailrc will stay in place), you could reverse the order so that the filtering happens first.


#8

Anyone checked out spam filtering in Mail on Panther yet? It has an option to ‘follow my ISPs advice on what junk mail is’ or something to that extent. What it basically does, in addition to its own filtering, if it find the header ‘X-Spam-Flag’ in the headers, that might have been added in by the ISP filtering, then it will class it as junk.

Is X-Spam-Flag associated with SpamAssasin?

  • wil

#9

Yep. We don’t currently do this scanning, but if you setup SA on your own, you could have it follow that.

Of course, since if you were using SA on our system at this time, you’d be using procmail, you could simply setup a procmail recipe to direct suspected spam to a different IMAP folder, which would probably be easier.

I will say that SA 2.60 is working pretty well for me - I finally got tired of doing the constant training of bogofilter, and using both together was kind of a pain. The version that’s installed on the mail servers is pretty old and crusty. I know I said I wasn’t going to do this, but I am seriously considering installing the backported package of 2.60; this would require some announcement and coordination in advance, however, because some of the options and features have changed with the new version.

I use a pretty high threshold (6 required hits to mark something as spam, and 9 hits required to automatically move it to my spam folder).


#10

You’re right. It would be easier to setup a .procmail rule to filter out spam to a IMAP directory. Thinking about this more; it would be neat to filter spam from the 20 or so accounts we have into a single collective spam box which I can check every so often. That way, our end users, theoretically should see no spam in their box or know that spam exists. If you get what I’m getting at …

We have a number of senior people working here that often get quite upset by some rather vulgar and graphic spam. By not allowing them even access, or to know about, the catch-all spam mailbox then this could be the answer I’m looking for to keep them all happy.

Hmm. Sorry, I’m rambling on about thoughts in my head here.

Is SpamAssasin installed on the servers just not setup? Or would I need to install my own local copy. Indeed I would to get the latest version if you don’t decided to backport, right?

  • wil

#11

It’s installed, but the current version is old and crusty. It’s not the kind of thing we’re likely to upgrade often, so I would suggest maintaining your own copy, especially if you’re using it with a number of users.

Maintaining a central spam box would be somewhat tricky if you’re running it on our system. Also, note that it will only work with “real” users, not with m123456 type (mail-only) users. If you want to do something like that, you’re probably better off either running your own mail server, or setting up a local server to pull the mail using fetchmail or getmail, and doing the processing there.


#12

This is interesting. If other users (that would be me) also wanted to run this better-version SpamAssassin, how would we actually go about doing that? I’m happy using the control panel, and uploading my own files and such. But installing a mail filtering program on the server, and configuring it … that’s unknown territory for users without *nix experience (me again). Is there a guide to follow somewhere?


TorbenGB


#13

The SA documentation explains the process for building your own copy in /home/user.

It boils down to downloading and uncompressing the software, renaming its directory, and running a few commands to build it. There are some other steps you’d need to follow to actually start tagging mail.

It’s pretty easy to do, but overall not something I’d recommend without at least basic experience compiling / installing software.

Doing something like Wil’s suggesting (for multiple users) is a bit more tricky.