.procmailrc proper use


#1

I’m not sure I’ve got .procmailrc working correctly.
I am trying to enable the .procmailrc file as suggested by Kevin Hatfield in the wiki. I also want to use John Paulette’s suggestions for clamassasin.

A few things that aren’t clear from the wiki about .procmailrc:

a. I see the Maildir directory and the three subfolders. Where are my actual mail documents?

b. I see these x-headers in my full headers:
X-DH-Virus-Scanned: Debian amavisd-new at madmax.dreamhost.com
X-Spam-Score: -
X-Spam-Level:
X-Spam-Status: No, score=x tagged_above=-999 required=999 WHITELISTED tests=[]
The spam x-header might be right, but the virus scan looks like something dreamhost has installed, rather than the clamassassin x-headers suggested by John Paulette’s article:
http://jhcore.com/2007/03/10/clamav-email-checking-on-a-shared-host/

c. Do I need to disable the Dreamhost filters in order for .procmailrc to work properly?

d. The default place to create folders is within the inbox, not on the same level as the inbox. In webmail, do I create the folders in the inbox or on the same level as the inbox? If I don’t create the .Junk folder and the .virus folder in webmail, will they be created automatically when .procmailrc processes it for the first time?

e. I just saw the creation of a folder at the same hierarchical level as the inbox called ‘Junk Folder’ that I didn’t put there. Did DH spamassassin put it there, or did mine from the .procmailrc?

f. I’ve got a site I’m about to transfer to DH. I get a lot of spam. I cannot open the webmail account to whitelist anyone until the DNS has propagated. I don’t want to lose any real mail, but I don’t want tons of spam either. All the email addresses are in place on the DH control panel. All the web pages are ready to go. The only thing left to do is change the DNS. I would appreciate any suggestions as to how best to approach email so I don’t lose anything important during the transfer.

I’d like to put an x-header in every email that I get showing me that it has gone through a particular version of the .procmailrc as updated by me from Kevin Hatfield’s original example. I’m not sure how to do that, and would appreciate the correct suggestion on how to do that.

These are probably silly questions, but they may help clear up some things for other people like me who know just enough to be dangerous to themselves.

Cheers,

Chris


#2

a. Mail is initially put in the new directory, but it moves to cur once it’s read (eg if you use webmail or IMAP).

b. It looks like DH does its own virus scanning if you use the standard Junk mail filters.

c. I don’t think so. The standard filters happen before procmail gets involved. In fact, you could turn off the DH quarantine and use procmail to send spam to your own junk folder.

d. I don’t think you can create folders from the shell. I use webmail to do this, but IMAP also works.

e. No clue.

f. I believe you can access webmail before DNS has propagated from webmail.dreamhost.com and entering a full email address as the user name. It’s not possible to “flip a switch” and get the DNS to change instantaneously so there will be a period of time where mail will go to the old hosting company.

g. You can use the formail tool to add headers to the message. I’m not much of a procmail hacker, so I can’t give you an example off the top of my head.

Larry


#3

The Maildir is a structure is defined in http://www.qmail.org/man/man5/maildir.html and the version used in DreamHost is the one expanded by http://www.courier-mta.org/imap/README.maildirquota.html

Quick and dirty: ~/Maildir holds your inbox. It uses 3 directories: new for messages recently arrived and were never read by anything (were thing applies to software rather than people… that is, if your mail program (Thunderbird, Outlook, whatever) did a scan of your inbox, even if only to display the message subject/from/date, then the message is moved away from new and into cur. Don’t pay attention to tmp since it is only used temporarily (and very briefly) by the software delivering messages into your maildir.
There may be some files and directories within your ~/Maildir called couriersomething… these are usually handled by the IMAP server and you shouldn’t mess with them (at least until you do understand how they are used)… however all (most?) of these files are plain text files that you can look into if you are curious.

I didn’t read the article you refer but dreamhost has its own version of SpamAssassin installed as described in http://wiki.dreamhost.com/Junk_Mail.
You probably have junk mail filtering enabled in your domain (see https://panel.dreamhost.com/index.cgi?tree=mail.junk).

Nope… you can have as many spamassassins working on the same message as you want (can). Only be sure to use different headers in each instance, since you can’t modify dreamhost’s spamassassin, use a different header name for your installation… I guess you should also read http://wiki.dreamhost.com/SpamAssasin.

I don’t think you are able to create folders outside (i.e. at the same level as the inbox), at least not with the webmail provided by dreamhost… I just tried and it didn’t let me do so… I also tried to do it via IMAP (with Thunderbird) and, although I didn’t get any error message, the folder didn’t get created. You can only create folders within other folders (starting with inbox).

As for the autocreation of the Junk/Trash folders… I think that Sent, Trash and Drafts are autocreated the first time you enter webmail (that is, squirrelmail creates them)… I think other imap clients also auto-create some folders (I think Thunderbird only auto-creates Trash).

I think (but I’m not sure) that if you name a non-existant folder in a rule in procmail, the folder is created when (and if) the rule is actually executed (e.g. by a message hitting it). BUT… the folder won’t be subscribed to… that is, it won’t show in your webmail or imap client folder list… you’ll have to manually subscribe to it via you webmail folders settings or the subscribe option in your imap client (the subscription is server based, so subscribing in one of them is enough so that can be seen in every client).

Junk Mail’ is a special case… please read http://wiki.dreamhost.com/Junk_Mail … it is not a folder that you can see other than by the specially modified webmail client that dreamhost gives you at webmail.[/i]yourdomain.com[/i]. You won’t be able to see it as a folder within your ~/Maildir since it actually resides (I think) in dreamhost’s anti-spam server farm and you get to sneak it via webmail in order to delete or let pass thru the quarantined messages that dh’s anti-spam identified as spam and captured (you have to configure this… keep reading http://wiki.dreamhost.com/Junk_Mail :slight_smile: .

Dreamhost doesn’t delete your mail until you tell it to… IIRC, the default settings when you create an address within a junk-mail protected domain, it is configured to not block or tag anything. If you want to stop most spam, you may want to configure each mailbox junk settings to block mail at 4 or 5 and to keep quarantined messages for 30 days -the maximum allowed (you do this from the webmail spamassassin configuration options).

Mmmhhh I know very little procmail (though I have it working for my parents and wife, I myself use maildrop)… a quick google on this suggests me (though I really, really can be very wrong about this) that this is not possible to do within procmail… you’d have to pipe your message to a script to do this, but I think that not being done within procmail defeats what you actually want to do… maybe some procmail wizard will be able to help you on this…


[color=#0000CC]Mariano Absatz[/color] - “[color=#6600CC]el baby[/color]”

[color=#00CC00]Message transmitted on 100% recycled electrons[/color]


#4

Mariano,

Thanks for the very thorough explanation. You’ve given me a handle on everything, except my current Maildir situation.
I like qmail, and have had previous experience with it.

I must be missing something…

I see this:
[mountaindew]$ pwd
/home/myUserName/Maildir
[mountaindew]$ find *
cur
new
new/1184227206.V13I23afad6.mountaindew.dreamhost.com
new/1184313613.V13I282670c.mountaindew.dreamhost.com
new/1184400008.V13I1de28d6.mountaindew.dreamhost.com
new/1184486406.V13I7809eca.mountaindew.dreamhost.com

If I do a '[mountaindew]$ more 1184486406.V13I7809eca.mountaindew.dreamhost.com, I can see that the email in new is actually a cron alert to the system user ‘myUserName.’ Okay, thats fine.

But with qmail, I’d expect to see something like this:
[mountaindew]$ pwd
/home/myUserName/Maildir
[mountaindew]$ find *
cur
new
new/1184227206.V13I23afad6.mountaindew.dreamhost.com
new/1184313613.V13I282670c.mountaindew.dreamhost.com
new/1184400008.V13I1de28d6.mountaindew.dreamhost.com
new/1184486406.V13I7809eca.mountaindew.dreamhost.com
mydomain.com
mydomain.com/cur
mydomain.com/new
mydomain.com/tmp
mydomain.com/chris
mydomain.com/chris/cur
mydomain.com/chris/cur/1184227206.V13I23afad6.mydomain.com
mydomain.com/chris/cur/1184400008.V13I1de28d6.mydomain.com
mydomain.com/chris/new
mydomain.com/chris/tmp

It may not be exactly what you’d see in qmail, but I expect some sort of folderized hierarchy of what’s actually in the Mailboxes of my separate users for the website ‘mydomain.com.’ I don’t see that. Regardless of the number of mailboxes I create in the DH admin interface, I only have Maildir with three subfolders cur, new and tmp, with only a few cron alerts – even though I can access them from webmail.

What am I missing here?
Where’s my mail for chris@mydomain.com?

Cheers,

Chris


#5

Thanks Larry,

my concern is that I cannot see email accounts or their relevant emails in what I’m used to seeing in qmail’s preferred Maildir format.

I would like to include an x-header in an existing incoming email as proof that the email has gone through the .procmailrc process. Isn’t formmail used for creating new emails from data gathered from a web form?

Thanks for the awesome tip about accessing webmail.dreamhost.com! This is exactly what I needed so I can import my whitelist and blacklist prior to DNS propagation.

Cheers,

Chris


#6

Each mailbox has its own Maildir with the corresponding structure. So you won’t see all the mailboxes for a domain together in one place. Also, DH sets the permissions on the Maildir directory such that only the owner can access messages in Maildir.

The command line tool is called formail, with only 1 ‘m’. You can do man formail from a shell to find out more. If you want to add a header to all messages, you can try adding this to .procmailrc:

:0f
| /usr/bin/formail -i “X-Tag: True”

Larry


#7

Larry,

Thanks again for the x-tag suggestion. Excellent.

With regards to mailboxes: I’m just looking for any mailbox. They do not exist anywhere within my user’s home. In other instances where I’ve used qmail, they were at least nicely organized in a hierarchical folder situation somewhere. I cannot find them at all. Where are they?

This is all I see at user root:

[mountaindew]$ ls -la total 152 drwxr-x--x 17 myUserName myUserGroup 4096 Jul 12 18:30 . drwxr-xr-x 77 root 1000 4096 Jul 15 16:41 .. -rw-r--r-- 1 myUserName myUserGroup 260 Nov 27 1999 .alias -rw------- 1 myUserName myUserGroup 6189 Jul 15 18:22 .bash_history -rw-r--r-- 1 myUserName myUserGroup 226 Jul 11 11:16 .bash_profile -rw-r--r-- 1 myUserName myUserGroup 55 Nov 27 1999 .bashrc -rw-r--r-- 1 myUserName myUserGroup 417 Nov 8 2000 .cshrc -rw------- 1 myUserName myUserGroup 6 Jul 11 14:29 .nano_history -rw-rw-r-- 1 myUserName myUserGroup 64825 Jul 11 10:29 .procmailrc drwx------ 5 myUserName myUserGroup 4096 Jul 10 11:17 Maildir drwxr-xr-x 16 myUserName myUserGroup 4096 Jul 12 18:30 mydomain.org drwxr-xr-x 2 myUserName myUserGroup 4096 Jul 10 18:20 admin drwxrwxr-x 17 myUserName myUserGroup 4096 Jul 11 09:57 clamav-0.91 drwxr-xr-x 2 myUserName myUserGroup 4096 Jul 11 14:45 mydomain.com drwxr-xr-x 16 myUserName myUserGroup 4096 Jul 11 09:36 mydomain.us drwxrwxr-x 3 myUserName myUserGroup 4096 Jul 11 18:59 install_files drwxr-xr-x 5 myUserName myUserGroup 4096 Jul 11 15:28 ion.mydomain.org drwxrwxr-x 5 myUserName myUserGroup 4096 Jul 11 06:29 log-archive dr-xr-x--- 8 myUserName dhapache 4096 Jul 11 15:02 logs drwxr-xr-x 12 myUserName myUserGroup 4096 Jul 11 14:04 mydomain2.org drwxrwxr-x 8 myUserName myUserGroup 4096 Jul 11 10:09 packages drwxr-xr-x 3 myUserName myUserGroup 4096 Jul 11 06:57 mydomain2.com drwxrwxr-x 2 myUserName myUserGroup 4096 Jul 11 08:04 tmp drwxrwxr-x 2 myUserName myUserGroup 4096 Jul 11 18:40 webalizer Maildir includes only the three expected folders.
I have created multiple email addresses (mXXXXXXX-style) for multiple domains owned by this user. Shouldn’t they live somewhere inside home for this user?

Chris


#8

The messages for “myUserName” will be split between the Maildir/new and Maildir/cur directories. For example, send “myUserName” an email and then check Maildir/new; there will be a new file containing the message.

The messages for each mXXXXXXXX mailbox will be stored in a Maildir folder associated with that account. They won’t be found within the directory hierarchy of “myUserName”. Even though mXXXXXXX mailboxes don’t support shell login, these are in fact separate user accounts on the DH servers.

Larry


#9

Hi Larry,

So will mail for multiple mXXXXXX users show up in the same cur and new directories of Maildir associated with the shell/ftp user who owns the websites? Or is there a separate Maildir for each mXXXXXX user somewhere out of my reach? When I send a test email to chris@mydomain.com, I see it in the webmail, but I don’t see Maildir/cur or …/new for the associated ftp user and domain. Isn’t it true that .procmailrc can only work for the Maildir of email addresses that are in the home folder as the ftp/shell user?

How can .procmailrc work on an email file if it isn’t inside the home directory of the associated ftp/shell user where .procmailrc resides?

I don’t have an email address associated with any of my ftp/shell users in the DH interface. Only mXXXXXX users receive email. The only items that are coming up in my ftp-user’s Maildir are the nightly cron alerts.

Chris


#10

There is a separate Maildir for each mXXXXX user and as far as I know, it is totally out of your reach through a shell account.

You are correct that a given .procmailrc can only work for the specific user in whose directory it is located. This means that if you want to filter a user’s email via procmail, the user cannot be an mXXXXX “user”, but must be user created under the Users section of the DH panel.

(Actually, DH provides keyword filters for mXXXXX users, which are implemented with procmail, but they are limited in what they can do.)

I’ve generally created all mailboxes as full-fledged users, so that I can use my own version of spamassassin + procmail to filter email.

Larry


#11

Larry,

Thanks for your patience. It’s been very helpful.

I’ve thought I’d been all through the wiki, yet there it is, halfway through the email setup notes.

I can imply everything you’ve just illucidated after I’ve read through it carefully several times. I wish the differences and possiblities available for each type of user were more readily stated right at the beginning of the email section in the wiki.

Again, thank you.

Chris


#12

greenman,

Sorry for the delay… I’ve been a few days completely off-line.

Larry is right… every mail user has a separate Maildir in a separate Unix account… if the user is also a shell/ftp user, then that user can see his/her own Maildir within his/her Unix account (either via ssh or ftp).

However, if you only create the mailbox, then it resides in an m####### account elsewhere… my guess is that those are standard stripped down unix account but are only accessible from the mail clusters.

For what I’ve seen, the mail servers are separated from the web servers… your shell/ftp accounts reside on a web server. The mail servers mount the same shares as the web servers to have access to users’ mailboxes. My guess regarding the unix accounts on the mail servers (or whatever PAM method they use), is that they have replicated the user accounts of every web server associated to the mail cluster PLUS the m####### accounts (probably mounted from another share/filer).


[color=#0000CC]Mariano Absatz[/color] - “[color=#6600CC]el baby[/color]”

[color=#00CC00]Message transmitted on 100% recycled electrons[/color]


#13

I’ve got my email address set to a regular user, so I can see that the mail is coming into /home/userName/Maildir/…
[color=#0000CC][code][mountaindew]$ wget http://www.projectstuff.org/procmailrc
–19:21:26-- http://www.projectstuff.org/procmailrc
=> `procmailrc’
Resolving www.projectstuff.org… 208.97.171.234
Connecting to www.projectstuff.org[208.97.171.234]:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 64,673 [text/plain]

100%[====================================>] 64,673 --.–K/s

19:21:26 (38.38 MB/s) - `procmailrc’ saved [64673/64673]

[mountaindew]$ mv procmailrc ./.procmailrc
[mountaindew]$ more .procmailrc

PROCMAILRC PROVIDED BY KEVIN HATFIELD - KHATFIELD@PROJECTSTUFF.ORG

############################# Copyright 2007 ###################################

To install:

Upload the procmailrc file to your home directory.

Rename the file to .procmailrc

Feb 28, 2007

################################################################################
MAILDIR=$HOME/Maildir
SHELL=/bin/sh

… etc.

[mountaindew]$ find *
courierimapkeywords
courierimapsubscribed
courierimapuiddb
cur
cur/1185301433.V9I4978892.spaceymail-a4.dreamhost.com:2,S
cur/1185324087.V9I39f35a.spaceymail-a4.dreamhost.com:2,S
cur/1185304685.V9I4978895.spaceymail-a4.dreamhost.com:2,S
new
tmp
[mountaindew]$
[/code][/color]
I sent myself an email from another address with the words ‘Stock Report’ in the subject.

It did not go into my Junk folder. I’ve tried this several times and find that .procmailrc isn’t working at all.

I explicitly followed all the directions at this URL:
http://wiki.dreamhost.com/Procmailrc-spamfilter

I have not white-listed anyone.

Am I forgetting something? Have I done something wrong?

Cheers,

Chris


#14

Mmmhhh… it seems to me that the instructions in http://wiki.dreamhost.com/Procmailrc-spamfilter are missing a bit of information. As stated in http://wiki.dreamhost.com/SpamAssassin#.7E.2F.forward.postfix you have to create a file in your home directory named .forward.postfix in order to instruct the mail server to pass your messages thru Procmail.

This file should have only one line as the following one:
[color=#6600CC]"|/usr/bin/procmail -t" [/color]
(the quotes must be there)

It is important that this file has the permission bits set right. If you’re logged in via ssh the easiest way to do this is to simply issue the following command:

[color=#0000CC]chmod 0600 ${HOME}/.forward.postfix [/color]

HTH


[color=#0000CC]Mariano Absatz[/color] - “[color=#6600CC]el baby[/color]”

[color=#00CC00]Message transmitted on 100% recycled electrons[/color]


#15

el baby - You rock!

That’s got it.

Cheers,

Chris