SpamAssassin

software development

#1

Has anyone installed SpamAssassin on a dh server? How is it working? Worth the install?


#2

I’ve never used it on a Dreamhost server but I use it on our internal office mail gateway and I gotta say that I love it, catches 90% of my spam and labels it as spam and filters it to a spam folder. Only thing missing from it is catching all these Klez email viruses.

  • wil

#3

It’s installed (but not “officially supported”) on all of our mail machines. The example documentation and man pages may or may not be installed on your user machine, but can be installed easily.

Since it’s somewhat processor intensive, we haven’t made an official announcement about this. Since there aren’t that many people on this forum, I don’t mind mentioning it.

I can forward directions to people who are interested; they do assume that you are comfortable messing around with this stuff, and that you know how to edit text files etc.


#4

Is it setup on all Dreamhost mail accounts with some default minimal options or something? As I don’t seem to be recieving that many number of spam on my dreamhost account compared to a few others …

  • wil

#5

[quote]Is it setup on all Dreamhost mail accounts with some default minimal
options or something?

[/quote]

Nope. By default we perform no filtering on customer mail. We’d rather err on the side of caution where customer mail is concerned.

We’d like to be able to do some global tagging (but not blocking) of mail for spam, but we’d have to perform a lot of optimizations before we’d be able to process all of our incoming mail through spamassassin (remember that our largest group of email servers has about 22k users on it at the moment).

At some point, I’d like to integrate the per-user UCE patch for Postfix, which would allow customers to “opt-in” to different blocklists on a per-user basis. This would require some backend adjustments, but would be a good compromise between customers who want to agressively reject spam and those who don’t want to risk losing important business related mail.

I’m glad to hear you’re receiving less spam to your DH account than to others… my only guess would be that you’ve been more careful with addresses at domains that are hosted with us.


#6

Hm. The per user UCE control patch for Postfix looks good, Will. I’m sure a number of people would appreciate something like that.

I can understand and appreciate the reluctance to generally filter any incoming mail. That’s a good thing. However, I guess my only concern with the Postfix patch would be to educate users. It looks like quite a powerful tool and if someone’s quite not sure what they’re doing it could result in some lost emails.

A global tagging of spam email – now this idea I like. I know this would mean a big impact on the mail servers, and unfrotunately on the mail servers is somewhere where you try to avoid any unneccessary overheads – but if there was such a patch or a hack for postfix that would just add an extra header to spam email flagging it as spam so the user would have the choice of easily fileting out to a special mailbox on their side.

Hmmm… Sorry, I’m just wondering out aloud to myself here. :slight_smile:

Me, myself, I don’t see spam as much of a problem (thankfully) as I do have SpamAssasin installed on our local gateway here, but I don’t know how much of a problem spam email is for others.

  • wil

#7

We’ve had some discussions on global content filtering recently.

There was an interesting thead on the postfix-users mailing list that had some good suggestions on how to optimize performance with Spamassassin…

These threads (as well as some threads that Jeff forwarded from the SA-Talk list) were interesting (follow the ‘next in thread’ links to view the next message).

http://marc.theaimsgroup.com/?l=postfix-users&m=102407222401993&w=2
http://marc.theaimsgroup.com/?l=postfix-users&m=102616301303570&w=2

This comment (from the SA-Talk list; too lazy to google for a link right now) was also interesting:

[quote]To use SpamAssassin in a production environment, we do a number of
optimisations. We patched Net::DNS to remove the use of $& , which
speeds up all SA regexp matches by orders of magnitude.

[/quote]

So most likely it would have to be run as a Postfix content-filter, make no changes to the body (spam report in the headers), and we’d have to skip checks that are more epensive.

Other (good) suggestions were to run spamd on a different machine from the mail machines. However the content-filter itself needs to be very robust, and failure tolerant. I am leaning towards Amavis at this point (well one of the many Amavises that are around right now), even though I have very little experience with any of the Amavis variants (we use RAV for virus scanning here in the office).

Anyway an interesting idea, and we’re certainly weighing our options. Jeff can attest to the fact that I was very hesitant to even consider something like this. I still have some reservations about Spamassassin for such a large, production level project, even though I find it very effective for keeping spam out of my inbox.

Ok - back to sleep - kinda tied one on last night, and I keep trying to enter vi commands (hitting escape and what not) in this browser text box (not sure if the two are related)… Sick, I tell you.


#8

This all sounds very interesting, Will, and would certainly be yet-another-added-value service for Dreamhost customers.

I’ll take a ganda over some of those links laters.

[quote]Ok - back to sleep - kinda tied one on last night, and I keep
trying to enter vi commands (hitting escape and what not)
in this browser text box (not sure if the two are related)…
Sick, I tell you.

[/quote]

So now you’re so desperate to keep up with my message count you’re staying up all night making sure you grab any new messages first?! Sheeeeeeeesh. :slight_smile:

  • wil

#9

I’m having major issues getting spamassassin to work – can you forward me some hints to getting it working?

Thanks,

Jonathan


#10

Sure… send me a message on here with your email address, or email support.

We’re working on getting official support for SA going, as well as possibly allowing customers to opt-in to individual blocklists (on a per-user basis)…

I’m also starting a DNS based blocklist for internal use, which will block some of the major offenders…


#11

jsalkoff(at)yahoo.com

tx


#12

hey, i’d love some tips, too…pretty new around here…perhaps SA is way too much overkill…

thanks

jack greenwood


#13

Send me your email address privately on here (or submit a support request) and I’ll get you DH-specific directions. There are also some hints at a link on one of the kbase articles…
http://donkin.org/bin/view/Main/SpamAssassin
also, the general procmail article at:
https://kbase.newdream.net/index.cgi?area=2626
may be of help.

Note that SA is broken on the web machines, but it’s not necessary for it to work properly there. It is installed (and works) on the mail machines.


#14

What does broken on the Web machines mean? I am using it on frigga and it seams to be working fine. I would think that we really can’t use it on the mail machines anyway since we can’t login and create the necessary procmail settings.

Jeff


#15

The version installed on the web machines doesn’t work:
frigga% spamassassin
Can’t locate Pod/Usage.pm in @INC (@INC contains: /usr/lib/perl5/5.005/i386-linux /usr/lib/perl5/5.005 /usr/local/lib/site_perl/i386-linux /usr/local/lib/site_perl /usr/lib/perl5 .) at /usr/bin/spamassassin line 8.
BEGIN failed–compilation aborted at /usr/bin/spamassassin line 8.

Since it’s run on the mail machine and not on the web machine, it works anyway.

[quote]I would think that we really can’t use it on the mail machines anyway since
we can’t login and create the necessary procmail settings.

[/quote]

Wrong… the mail machines mount the same home directory as the web machine - so even though you can’t login, you can make the necessary changes.

If you were actually using it on frigga, it wouldn’t process any of the mail coming into your domain - just mail coming directly to frigga.


#16

Ok, I think I understand now. The shell accounts as well as the mail accounts are mounted on both the mail and web server. Since the mail server is actually doing the mail handling thats why it works.

Jeff


#17

Right - we use a central filer for data storage, so your home directory is the same on both machines.


#18

I’ve been using and loving SpamAssassin for at least a couple months now. One thing that doesn’t seem to work for me, though, is the whitelist. I have a few thoughts about this:

  1. Here’s what my /.spamassassin/user_prefs file looks like (the whitelist portion):

Whitelist and blacklist addresses are now file-glob-style patterns, so

"friend@somewhere.com", “@isp.com", or ".domain.net” will all work.

whitelist_from someone@somewhere.com

whitelist_from ADDRESS_THAT_ALWAYS_ENDS_UP_IN_SPAM_FOLDER

  1. How does auto-whitelisting work?

use spamassassin -Pa if you want to use auto-whitelisting

  1. I have my own spam block list in a recipe file that is called BEFORE spamassassin is called, so I’m guessing certain addresses from particular friends are being caught by this for some reason, although I have no idea why. The messages are NOT being labeled with the same text that SpamAssassin uses for messages, so that’s the only thing I can figure.

  2. Here’s the end of my .procmailrc file:

#INCLUDE FILES
#=============
#INCLUDERC=$PMDIR/rc.inbox
INCLUDERC=$PMDIR/rc.autoresponders
INCLUDERC=$PMDIR/rc.lists
INCLUDERC=$PMDIR/rc.spam
INCLUDERC=$PMDIR/rc.toblocks
INCLUDERC=$PMDIR/rc.spamassassin
~

Messages that fall through all your procmail recipes are delivered

to your default INBOX (to find out yours, see step 2 above)

:0
$HOME/Maildir/

I’m not sure why that tilda (~) is there after the last include. Should it be? It may have inadvertently ended up in that file.

Thanks,
Gabe


#19
  1. I don’t use the whitelist at all, but looking at some other employees’ user_prefs files, that looks like the right syntax.

  2. Auto whitelisting works by remembering the score of previous messages from an address; thus if someone sends you lots of un-spammy messages and then one spammy looking one, the high score will be mitigated by the whitelist adjustment; similarly, if someone sends you lots of spammy email, they’ll be scored up higher. That’s my understanding of it, at least.

I honestly rarely have to whitelist people, but I filter 96% of my mail before it even gets to Spamassassin. Only unmoderated lists and stuff that’s going to my inbox goes through SA.

  1. coughLOGGINGcough.

In other words, turn on procmail logging, and see why messages from those people are being filtered. If they’re not being tagged by SA, this is almost definitely your problem. I hope that you’re filtering them to a separate folder and not to /dev/null? Be very careful when sending stuff to the great bit-bucket in the sky. I try to use very specific recipes when I killfile people to avoid accidents.

  1. The tilde shouldn’t be there (perhaps you cut and paste from a vi window)?

#20

Hi Will-

Thanks for the reply and suggestions. I’ve removed the tilda and have turned on autowhitelist to see if it helps. The only messages I send to /dev/null are from specific spammers when their messages leak into my inbox - once is enough! I don’t fliter to /dev/null on subject - only FROM addresses.

I do have logging enabled and that doesn’t give me any clues as to why a particular friend’s messages are consistently dumped into my spam box:

From USER@usa.net Mon Oct 14 19:36:42 2002
Subject: Re: [RE: finally]
Folder: .spam/new/1034649402.31858_2.ludo

Here’s my one clue, though: If I “fake” an email from this friend using her address and send to myself, it does NOT go into spam box. So I’m wondering if my own spam recipe is somehow picking up hidden header info from usa.net? I’ve double-checked that this address, nor any of the words in subject line, are filtered on. Could it be the [ ] that show up in reply? I don’t think so, but you never know…

Thanks,
Gabe