Procmail is a general
purpose electronic mail filtering program available on most of
Monash's Unix servers (eg. silas). Anybody receiving
more than a couple of emails a day can greatly benefit from the
features provided.
This is intended as a guide to helping people get started with procmail at monash. Comments and queries should be directed to [email protected], or posted on the monash.unix newsgroup.
.forward.procmailrc file.forwardFirst of all, put the following in a file in your home directory
called .forward :
"|IFS=' '&&exec /usr/local/bin/procmail -f-||exit 75 #ajc"
Note that you should include the quotation marks in the file, and
that the ajc at the end should be replaced with your own
username (eg. jrsmi1).
Note : CC machines now share a common home directory, so you need to do two things :
Create a .forward file saying something like the
following :
[email protected]
This will tell any machine that uses your CC home directory
(currently ccds/ccas, the silas
cluster, CC linux accounts and fawltyds.
Then create a second file called .forward.silas-1
containing the procmail line above :
"|IFS=' '&&exec /usr/local/bin/procmail -f-||exit 75 #ajc"
Then cp .forward.silas-1 .forward.silas-2 (since
mail sent to silas could end up on either silas-1 or
silas-2, and you want to process it either way).
.procmailrcThe most basic .procmailrc file is a non-existent one -
your mail will be delivered to the mail spool as normal. A slightly
more useful example is one that files all incoming mail into a single
"mbox"-style folder under your home directory:
MAILDIR=$HOME/Mail :0: incoming
MAILDIR is a variable that tells procmail
what directory our mail folders are in.
The :0 means the start of a new recipe.
The second : tells procmail to use a lockfile
while executing the actions in this recipe - otherwise, if you
were to get 2 emails at once, they might both try and write to
the file at once (and your mailbox could get
corrupted).
The next line specifies a folder to file the mail into - in
this case, a file called incoming.
Perhaps you have a particular friend who sends you lots of mail.
Their email address is [email protected]. To file
all mail from this person into a folder, you could use the following
.procmailrc :
MAILDIR=$HOME/Mail :0: * ^From:.*[email protected] friend
The middle line specifies a condition that must be met for this action to occur - in this case, we want to match the "From" address on the incoming mail.
When matching things like this, we use ordinary regular
expressions. In the above example, the special character
^ matches the start of a line, and .*
matches zero or more of any character.
"mailing lists" in this sense are ones that you send a "subscribe" message to some address, and get mail from it. With mailing lists, one must look at the "To:" header, not the "From:" one. One example is Monash's "downtime" mailing list :
MAILDIR=$HOME/Mail :0: * ^TOdowntime downtime
^TO is a special token that matches on the
To:, Cc: and other relevant
headers. You have to have it in capitals, so ^To
or ^to won't work, for instance.
Sometimes you might have subscribed to a "digest" version of a
mailing list (you get an email, maybe once every couple of hours,
containing all the messages sent to the list within that time). You
can use formail (a companion program to procmail) to
split the digest into separate emails like so :
:0: * ^Subject:.*Blah.*Digest | formail +1 -ds >>blah
If we add up what we've done so far, we end up with this :
MAILDIR=$HOME/Mail :0: * ^TOdowntime downtime :0: * ^From:.*[email protected] friend :0: incoming
Note the ordering - if our friend happens to post to the downtime
list, we want it to be filed in the downtime filder. Anything else
falls through to the last recipe, which delivers the mail into the
incoming folder. If we don't have a "catch-all" recipe
like this, any leftover mail will simply be delivered into the mail
spool (as if procmail was never there).
If you use MH, or any of its "front-end" programs like ExMH, MH-E and such, you'll want to file mail away in an MH-style folder. An MH folder is a directory (each email is a separate, numbered file). If we wanted downtime mail saved this way, we'd do the following :
MAILDIR=$HOME/Mail :0: * ^TOdowntime downtime/.
Note that this doesn't update the "unseen" sequence that you may be using to determine which mail you haven't read yet. If you want that to happen, do this instead :
MAILDIR=$HOME/Mail RCVSTORE=/usr/lib/mh/rcvstore -create :0w:downtime.$LOCKEXT * ^TOdowntime | $RCVSTORE +downtime
Note that we have to specify a lockfile name, normally it
will use a lockfile called name.$LOCKEXT, where
name is grabbed from the folder name we're
delivering to. But since this action is piping the mail into a
program, procmail can't guess the name of the folder.
Note the w flag on the first line of the
recipe, too - this tells procmail that if the program we're
piping the mail to fails, then to fall through and try any other
actions in our .procmailrc - normally this would
mean the mail ends up being delivered in our mail spool/incoming
folder, which is better than not getting it at all...
You might find it advantageous to forward some mail to another site. You might want to forward certain emails (say, from a particular mailing list) to a friend (but also file it in a folder, of course) :
:0
* ^TOprocmail
{
:0c
! [email protected]
:0w:
procmail
}
Perhaps you have a permanent net feed at home, and any mail at home, you want forward at work, and vice-versa. This is a bit tricky, as we could end up with a mail loop if we're not careful. Here's how to do it :
:0c *!^TO(ajc|Andrew.Cosgriff)@.*monash.edu.au *!^(From|Sender):.*(ajc|Andrew.Cosgriff)@.*monash.edu.au *!^X-Loop: [email protected] | (formail -A"X-Loop: [email protected]" | \ /usr/sbin/sendmail -oi [email protected])
At uni, I'd then do :
:0c *!^[email protected] *!^(From|Sender):.*[email protected] *!^X-Loop: [email protected] | (formail -A"X-Loop: [email protected]" | \ /usr/sbin/sendmail -oi [email protected])
(making sure, of course, that sendmail lives in
/usr/sbin). The presence of the X-Loop header prevents
us from causing a mail loop by sending back mail that came from uni,
say.
comsat is the service used to notify you when new mail
arrives - you get a message written to your screen telling you who the
mesage was from, the subject, and a bit of the body of the message.
Many people find that this interrupts their work if they get told
about every piece of mail that arrives. By default, procmail will get
comsat to notify you of new mail. To turn this off, add the following
line to your .procmailrc :
COMSAT=no
Most shells (ksh, csh, zsh,
bash and tcsh, for instance) will check for mail
every so often, and tell you
You have new mail.
All of the above shells let you specify a list of mail folders to
check for new mail. For instance, try putting these in your shell
environment file (.cshrc, .profile, etc)
:
ksh, zsh and bash :
MAILPATH="/var/spool/mail/ajc:$HOME/Mail/incoming:$HOME/Mail/friend:$HOME/Mail/downtime"
csh and tcsh :
set mail=(/var/spool/mail/ajc ~/Mail/incoming ~/Mail/friend ~/Mail/downtime)
Read the relevant shell manual page for a few other things you can
do with these two variables (including customized messages and such).
If you're using MH-style folders, specifying the directory
(eg. $HOME/Mail/inbox ought to work.
You may find it useful to have procmail inform you via goofey if an
important email arrives. Something like the following would work,
placed near the top of your .procmailrc :
:0c * ^Subject:\/.* urgent.* | (echo "== mail received ==\n$MATCH" ) | /usr/local/bin/goofey --forget-fail -s cos
the c flag on the first line tells procmail
not to stop here - as well as following this rule (if the
conditions are ok), it'll make a second copy of the mail and
continue through the rest of your
.procmailrc
procmail isn't case-sensitive when it matches, so "urgent", "URGENT", "Urgent" will all match, for instance.
The \/ on the second line tells procmail to
put all text on the matched line on the right of the
\/, into a variable called MATCH - in this case, if
the subject line said "Subject: urgent help required", MATCH
would contain "urgent help required".
The --forget-fail option to goofey tells it
not to save the message if you're not there (this way, we only
get notified about the mail if we're actually logged
in).
Possible enhancements to this might be to show the "From:" header as well, and possibly even the first few lines of text from the body of the mail.
If you've ever posted much to Usenet, or are on a few mailing lists, you've probably had your email address found by people who enjoy sending out unsolicited advertisements about various crud you never want to hear about. You can filter out any spam like so :
SPAM=$MAILDIR/lib/spam.txt :0 * ? formail -ISubject | fgrep -iqsf $SPAM /dev/null
$MAILDIR/lib/spam.txt contains a list of hostnames for
sites that are known to send out spam. I used to have my list
available here, but it's way out of date.
These days, I've found that the above technique isn't particularly effective by itself, but that, after filtering out all mailing list emails into other folders, any mail left that isn't personally addressed to me must therefore be spam, and gets filtered into a "possible spam" folder which I check every few days.
[2002] Nowadays, it's even worse. I'm having success by using SpamAssassin and Vipul's Razor, though.
Sometimes you'll receive multiple copies of the same email,
sometimes for no apparent reason, apart from broken mail software
somewhere out there, and other times because you got it by personal
email as well as via a mailing list. In order to avoid this, you can
keep a cache recently received Message-ID headers, and throw away any
messages that have a message-id present in the cache. To do this,
simply add the following to your ,procmailrc, above any
recipes :
:0 Wh:dup | formail -D 8192 msgid.cache
If you receive a lot of mail, still get duplicates (ie. with the same message-id's), you might want to increse the cache size.
Some people like to use old mailreaders that don't deal well with MIME headers. If you read a lot of postmaster mail, for instance, you'll end up reading a lot of "bounce" messages that contain multiple MIME parts, and since they're all plain text, you probably just want to read the damn thing without having to stuff around. Kevin Lentin suggests the following solution :
:0 fbw: * ^TOpostmaster | formail "-R Content-Type: X-Content-Type:" >>postmaster
As an example of a "server"-like application you can implement with procmail, I used to make my PGP public key available via email at my previous workplace (we didn't have direct internet access, only email and news, batched up via UUCP) :
######################################## # # tiny PGP KEY SERVER # # If e-mail message has subject of "get pgp key" or # "send pgp key", then this will reply with my pgp key # and log the request in $MAILDIR/pgpkey-request.log # # Orignally by me (Andrew Cosgriff), MIME niceties added by # Richard Coleman ([email protected]). ###################################################### [email protected] FROMSIG="Andrew's Mail Gremlin <$MYADDR>" WHOAMI="Andrew J Cosgriff <$MYADDR>" PGPKEYLOG=$MAILDIR/pgpkey-request.log :0 c * ^Subject:.*(send|get) *pgp *key * !^X-Loop: $MYADDR { FROM=`formail -rt -xTo:` SUBJECT=`formail -xSubject:` :0 ch: pgp$LOCKEXT | (formail -rI"From: $FROMSIG" \ -I"Reply-To: $MYADDR" \ -I"Subject: PGP public key" \ -I"Mime-Version: 1.0" \ -I"Content-Type: application/pgp; format=keys-only" \ -I"Content-Description: PGP key of $WHOAMI" \ -I"Content-Transfer-Encoding: 7bit" \ -I"X-Loop: $MYADDR" ;\ sh -c "pgp -kxaf $MYADDR 2>/dev/null") \ | $SENDMAIL -oi -t :0 w: $PGPKEYLOG$LOCKEXT | echo "`date` : $FROM : $SUBJECT" >> $PGPKEYLOG }
A second example is a file server, which can serve files from a directory (and its subdirectories) :
[email protected] FROMSIG="Andrew's Mail Gremlin <$MYADDR>" WHOAMI="Andrew J Cosgriff <$MYADDR>" FILESERVLOG=$MAILDIR/fileserv-request.log ######################################## # # File Server # :0 * ^Subject: (get|send) file [0-9a-zA-Z] * !^X-Loop: [email protected] * !^Subject:.*Re: * !^FROM_DAEMON * !^Subject: (get|send) file .*[/.]\. { FROM=`formail -rt -xTo:` SUBJECT=`formail -xSubject:` :0ac:fileservlog.lock | egrep "^(Subject|From|Date):" >> $FILESERVLOG MAILDIR=$HOME/tmp/fileserv # reverse mailheader and extract name :0 fhw * ^Subject: (get|send) file \/[^ ]* | formail -rA "X-Loop: [email protected]" \ -rI"From: $FROMSIG" \ -I"Reply-To: $MYADDR" # the requested filename (see procmailrc(5)) FILE="$MATCH" :0 ah | cat - ./$FILE | $SENDMAIL -oi -t }
Note that if someone requests a file like /something or ../something, the request won't match the initial conditions (we do this deliberately, so people can't request files outside the particular directory we're using).
To use it, one would send email with a subject of "send file
blah" to be sent a file called blah - it's useful to
have a file called index and/or help, which provide
a list of files and instructions on retrieving them.
If, after modifying your .procmailrc, you suddenly stop receiving
any mail, you'll probably want to fix it rather quickly - to help, add
the following to your .procmailrc, near the top ;
LOGFILE=procmail.log VERBOSE=yes
which will log what procmail is trying to do. You should be able
to get enough information from the logfile to fix your problem. If
you still can't fix it, comment out the offending section, and seek
help - try asking on the local monash newsgroup monash.unix. If you read the
procmailex manual page, you'll see a scheme for caching the
last few emails received, so if you're trying new stuff, you don't
lose the email.
I've also installed proclint on silas,
which you can use to sanity check your .procmailrc. It
was written by Alan
Stebbens.
The procmail manual pages are well worth reading - try running :
man procmail man procmailrc man procmailex man procmailsc man formail
Infinite Ink's Processing Mail with Procmail page has a good list of pointers and examples.
Stephen van den Berg, the author of procmail, obviously deserves many thanks, as do the various people who provide useful help on the mailing list and/or have created useful online resources for procmail.
I'd also like to thank the following people for their input on this particular document :