Yet another DH spam filter? Subject: ***DHSPAM***


#1

Anyone else finding legit incoming mail being Subject-tagged DHSPAM (e.g. below)? (Not due to either Razor or Junk Filter AFAICT.) And found any way of disabling this unwlecome defacement?

Subject: DHSPAM [snip]
X-DH-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at deathwish
X-Spam-Status: Yes, hits=0.2 tagged_above=-999.0 required=0.0
tests=NO_REAL_NAME
X-Spam-Level:
X-Spam-Flag: YES


#2

I have… Isn’t that what the DH spam filter does to messages that are above a trigger score for tagging, but not above the higher number for filtering? That’s configurable in the spam filtering section of the control panel.

– Dan


#3

[quote]Isn’t that what the DH spam filter does to messages

[/quote]

You mean the Junk Filter? The following suggest JF is not responsible for this marking:

a) The marking is ***DHSPAM***but according to the JF config panel, JF marking is SPAM
b) The marking says “tagged_above=-999.0” but the JF config “Tag Level: 0”

Mind you, having never got JF to operate properly, perhaps I’m not qualified to determine when it is operating not properly! So if anyone can tell me whether this marking does look like the work of JF, please do say! Thanks.


#4

I’m sure that SPAM comes from the Junk Filter. I see it (rarely) and I use the JF and not Razor, so it can’t come from anywhere else.

IIRC, there was a different amount of * during the beta phase. Don’t be too picky about the amount, as long as it’s clearly labeled.

TorbenGB
Try out DreamHost with a free WebIDPrices, options


#5

[quote]I’m sure that SPAM comes from the Junk Filter.

[/quote]

Meaning the DHSPAM that I’m seeing does not?


#6

I meant DHSPAM, I just wasn’t paying attention. Sorry, my bad.

TorbenGB
Try out DreamHost with a free WebIDPrices, options


#7

Ah, so DH can’t get a simple thing like the tagging right. :frowning:

Thanks Torben.


#8

That’s from the junk filter; I’d suggest adjusting the threshold.


#9

Thanks. I’d suggest getting the tag right.


#10

It’s a typo in the configuration page itself. Messages will still be tagged with “DHSPAM”, so if you’re filtering based on that, you can safely filter based on DHSPAM.

(Note - I fixed this typo)

If you’re objecting to the fact that subject line tagging is being done at all, recognize that the object here isn’t to create the most configurable, most flexible, most effective filter ever - the goal is to create something that works pretty well and significantly reduces the amount of spam that its users get. I don’t personally like adding anything to the subject line, but it’s something that a lot of people seem to be used to. It’s possible that we’ll add options to enable / disable subject line tagging at some point.


#11

Also, I believe the reason for “-999” instead of “0” is that setting it to “0” disables it entirely. Are you trying to do some sort of whitelist-only scheme or something, or what are your reasons for setting it to tag everything 0 and above as spam?


#12

[quote]It’s a typo in the configuration page itself.

[/quote]

I’m amazed it got past beta testing.

[quote](Note - I fixed this typo)

[/quote]

Thanks.

[quote]If you’re objecting to the fact that subject line tagging is being done at all

[/quote]

I wasn’t (sorry to be unclear, Will). I was objecting to the fact the some option/program I hadn’t asked for was defacing the Subject.

But since you mention it, yes it is objectionable that JF tags the subject line whether this is wanted or not, because this is known to cause problems such as false positive when the recipient replies, broken threading etc. Retained subject is not the preserve of:

[quote]the most configurable, most flexible, most effective filter ever

[/quote]

but a basic requirement which for example you have implemented for Razor, so it is disappointing to read only


#13

[quote]the reason for “-999” instead of “0” is that setting it to “0” disables it entirely

[/quote]

Urk! The config page says the Tagging value that disables is not 0 but 999. Are you sure, Will?

[quote]what are your reasons for setting it to tag everything 0 and above as spam?

[/quote]

Having found JF failing to trap anything, it was an extreme measure to confirm JF was applying SpamAssassin at all. It worked.

Now I stand a chance of tracing the real-world failures, but it is a struggle Will particularly because the docs are so vague as to make it impossible to tell precisely what success/failure /is/. For example:

The tag level is the numerical score required to identify a message as being spam.

Meaning higher score == spam (my guess from previous experience), or lower score == SPAM?

If I succeed in discovering how it works, I’ll be happy to write the docs for you! :wink:

Chris (BA Hons Cantab Comp. Sci.)


#14

re: the -999 thing - I was wrong. I looked into it some more, and that’s just because it does add the header markup to all emails… the actual subject line tagging and setting “Yes” in X-Spam-Status is still based on your tag level - i.e., if you had a message with a negative score, it wouldn’t be marked as spam with a tag-level of 0.

[quote]Having found JF failing to trap anything, it was an extreme measure to confirm JF was applying SpamAssassin at all.

[/quote]

The filter always adds markup to the email, so you can confirm that it’s applying it without adjusting the score.

Higher score is more spammy. The score shows up with the asterisks in the X-Spam-Level: header (easier to filter based on in procmail or with client-side filters), and in the X-Spam-Status level as a numeric value.

I find that somewhere between 4 and 7 is probably a good level to consider spam. My setup’s a little different than the junk filter setup, but I tend to consider stuff at 8 or 10 or above definitely spam (though this is with the bayes stuff enabled, and training miscategorized mail, so it is probably a little more effective) and anything above 5 as “probably spam”.


#15

[quote]re: the -999 thing - I was wrong.

[/quote]

OK, no problem! Thanks for letting me know.

[quote]The filter always adds markup to the email,

[/quote]

So I’d hoped, but in the absense of any documentation to that effect I concluded the unmarked mails I got must have been negatives. Now I’ll try and work out what made the JF overlook them. Thanks.

[quote]Higher score is more spammy. The score shows up with the asterisks in the X-Spam-Level: header
and in the X-Spam-Status level as a numeric value.

[/quote]

Good info. But especially as you’ve configured SA way off the defaults, even better would be to have that info in the KB!

Thanks, Will.


#16

By unmarked do you mean spam that wasn’t marked as such, or that there were NO headers at all added by the scanner? /Any/ email message processed by the junk filter will have headers showing that (as well as some extra Received lines showing it go through the various machines and smtpd instances on the junkmail machines).

[quote]But especially as you’ve configured SA way off the defaults, even better would be to have that info in the KB!

[/quote]

What have we configured way differently from the defaults? That particular info is exactly the same as with stock SpamAssassin.

We haven’t made many tweaks or changes at all, though there are some differences in the way that amavisd-new deals with the actual scanning part (since it just loads the Perl modules directly rather than feeding it through the “spamassassin” program). Actually adjusting the threshold is something the user can control.


#17

[quote]By unmarked do you mean spam that wasn’t marked as such, or that there were NO headers at all
added by the scanner?

[/quote]

The latter - no X-Spam- fields.

[quote]What have we configured way differently from the defaults?

[/quote]

e.g. there’s no X-Spam-Flag

[quote]That particular info is exactly the same as with stock SpamAssassin.

[/quote]

Apologies if I’m wrong, but I thought X-Spam-Flag was the primary markup of stock Apache SA 3.0. Which “stock SA” are you using?


#18

[quote]Which “stock SA” are you using?

[/quote]

On my systems, 3.0.0-rc4 and 3.02, on OpenBSD and Debian, one from source and one from the backports.org packages.

With the junk filter, just as with “stock” SpamAssassin, X-Spam-Flag: YES is set if a message is marked as spam. I get this flag set on messages which our spam filter marks as spam as well. The “X-Spam-Report:” is omitted in our system.

Example (from a domain that has the junk filter enabled):
X-DH-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at deathwish
X-Spam-Status: Yes, hits=6.2 tagged_above=-999.0 required=5.0
tests=HTML_10_20, HTML_MESSAGE, MIME_BOUND_NEXTPART,
MIME_HTML_MOSTLY, MPART_ALT_DIFF, MSGID_FROM_MTA_HEADER, RCVD_BY_IP,
RCVD_HELO_IP_MISMATCH, RCVD_IN_BL_SPAMCOP_NET, RCVD_NUMERIC_HELO
X-Spam-Level: ******
X-Spam-Flag: YES


#19

The positives I’ve used for testing are blacklisted, and none have got X-Spam-Flag. Even though JF quarantines all (excepting other failures). If that’s by design, then I can’t see how a user is ever going to be able to get segregation at the email client rather than at the server.


#20

Quarantined mail wouldn’t have the tags.

That’s why there are separate thresholds for tagging and quarantining.