"Linux Gazette...making Linux just a little more fun!"

Configuring procmail with The Dotfile Generator

By

In this article, I'll describe how to configure procmail using The Dotfile Generator (TDG for short). This will include:

How to sort mails coming from different mailing list
How to setup an auto reply filter, when you are on vacation
How to change some part of a letter, i.e. remove the signature
How to avoid that mail get lost!

It might be a good idea to download the program. To do that, please go to the home page of TDG, to find the nearest mirror. You might also be interesting in reading the article I've written about TDG for Linux Journal.

Starting TDG

To start TDG with the procmail module, type dotfile procmail. Now a window should appear like in figure 1. As you can see, the module is split up in three pages, the first two are very simple, so lets start with the page called General Setup. This page can be seen in figure 2.

Figure 1

Figure 2

On this page there are four things, to configure:

The directory to use as prefix for all file operations. This is just for ease, since all file operations may be with full prefix.
Your email address, will be used in preventing loop-backs.
Configuration of log files. These are very useful, when you wish to investigate where the different mail went. If you turn on abstract logging, you may find the program mailstat very useful. (See The log file below)
The search path, in which procmail may find the programs which it needs. Note this is only the programs, that you specify in filters etc.

Avoiding lost mail

Since procmail handles your incoming mail, the security is very important to this module. This means that you may backup your incoming mail in three different ways. To do this, go to the page called Backup. Part of it can be seen in figure 3.

figure 3

The first category of backup is to back up all incoming mail. The code, which must be generated to the procmailrc file for this, will be written as the very first line. This is to avoid that any errors in the generated procmail file will throw away any of your mails. This sort of backup is only a good idea when you at first start using the generated procmail file. The main drawback is that all incoming mail is saved in one file, and this file may become huge very fast.

The second method is to backup all incoming mail, which are delivered by procmail. This may be a good idea to use, to verify that mail are sorted into the right places.

The third method is to backup all mail, which makes it to your incoming mailbox. This mail are often mails, which do not come from a mailing list, and which are not junk mail to thrown away.

In the first method, you have to specify the full filename. This is because this method has to be 100% full prof. In the next two methods you may build the file names from the current date and time. This makes it possible to save this sort of mail to folders, for the current year/month/week etc. E.g. a folder called backup-delivered-1997-July

As an additional feature, you may keep the files as gziped files.

The backup of delivered mail may be specified for each individual recipe, or for all recipes at once. (see figure 4 check box 9) To learn how to use the FillOut elements, which configures the file to save to, please see the Dotfile Generator article in Linux Journal.

Setting up the recipes

In procmail a central concept is a recipe. A recipe is a set of conditions, and a set of actions. All the actions are executed, if all of the conditions is fulfilled. Below is a few examples of conditions:

The letter comes from [email protected]
The subject is subscribe
The size of the letter is greater than 1Mb
The letter contain the text ...

A list of actions may include:

Reply to the sender, that you are on holiday
forward the letter to another person
save the letter to a file
change some part of the letter (i.e. add a new header field, add some text to it etc.)

A procmail configuration is a sequence of recipes. When a letter arrive, each recipe is checked to see if all its conditions are fulfilled. If they are, the actions of the recipe is executed.

Procmail will finish testing recipes when one is matched, unless a flag is set to tell it, that this recipe should not stop the deliverment (see figure 4 check box 8). This means that the order of the recipes are important, since only the first recipe, which match will process the letter.

If none of the recipes are fulfilled, or if the ones which are fulfills have check box 8 in figure 4 set, the letter is delivered to the incoming mailbox as if the procmail filter haven't been there at all.

You configure the recipes on the page called ``Recipes''. This page can be seen in figure 4.

figure 4

What you see here is an ExtEntry. An ExtEntry is a widget, which repeats it elements as many times as necessary (just like a list box repeats the labels.) All what you see on this page, is one single recipe. To see a new recipe, you have to scroll the outer scroll bar (1). To add a new recipe, you have to press the button below the scroll bar.

As described above, a recipe is a set of conditions. This set is also represented with an ExtEntry (2). To scroll to another condition in a recipe, you have to use scroll bar (2), and to add a new condition, you have to use the button below scroll bar (2).

You may give each recipe a unique name, which will make it easier to find a given recipe. This name will also be written to the file with mail delivered by recipes (method 2 above), so you can see which recipe matched the actual letter. To give a recipe a name, use entry (3). At the right side of the entry, a button labeled Idx is located. This is a quick index to the outer ExtEntry (i.e. the recipes). If you press this button a list box will drop down, where you may select one of the recipes to scroll to, by its name.

The conditions of a recipe

The most common condition one wishes to set up, is that one of the header fields matches a given regular expression, or that the body of the letter matches a given regular expression. To explain how to do that, lets first see what a header may look like.

From [email protected]  Tue Jan 28 16:30:46 1997
Date: Tue, 28 Jan 1997 10:06:28 -0500 (EST)
From: Rick Troxel <[email protected]>
Subject: Re: Lynx as an external viewer for pine
In-reply-to: <[email protected]>
To: procmail mailing list <[email protected]>
Cc: "Robin S. Socha" <[email protected]>

The very first line of the letter is special. This line has been written by the program sending the letter (often called sendmail). This header field is often always the same for a given mailing list, so to sort mail from a mailing list, it might be a good idea to read the letter with an ordinary file reader (NOT a mail reader, as it will seldom show this line). And copy this information to the pattern field (figure 4, label (6) ). As the element to match, you have to select Sendmail from in entry (5).

Three special macros exists in procmail. These may be used, when matching header fields:

TO: This macro matches every destination specifications.
FROM_DAEMON: This should match when the letter comes from a daemon (which includes mailing lists). This is useful, to avoid creating a mail loop with some mailing list.
FROM_MAILER: An other regular expression, which matches that the letter comes from mail daemon.

To see what these macros stand for, please refer to the manual page called procmailrc.

There is a lot of header fields to chose between in the pull down menu (5), but if the one you wish to select isn't located there, you may type it yourself.

The check box (4) may be used to negate the condition, i.e. the pattern shall not match to fulfill the condition.

Regular expressions

So far, I have mentioned that you may type a regular expression in (6). In most cases, it may not be necessary to know anything about regular expressions, since the procmail module will take care of most of it for you. One thing may be worth to know anyway, and that is that you may match ``anything'' with .* This means that abc.*def will match anything which start with abc and ends with def, eg. abcdef or abcXXXXdef. To see a more detailed description of the set of regular expression, that procmail uses, you may press the button labeled Description

One common pitfall is to forget to match everything at the start of the line. I.E. If you wish to set up a regular expression for the From: field above, it is not enough to give the pattern: [email protected], since this is not at the start of the line, you have to tell procmail that every mail messages, which includes the text [email protected] is to be handled, I.E. insert .* in front of the email address.

Letting an external program decide whether a condition is fulfilled

A final way to set up a condition is by using an external program to verify some conditions. This is done by pressing button (7). This will bring up a window with a FillOut like the one you can see in figure 3. This time, however, the entry has been replaced with a text box. In this text box you may type some commands, which reads either the header or the body on standard input. These command may refer to some header fields from the letter. The line (separated with a newline) will be joined together with a separating semicolon. This makes each line a separate command.

Procmail will consider the condition fulfilled if the exit code from the program is 0 and not otherwise. This behavior may be changed with the check button (4) in figure 4.

Actions

The actions, that this module can handle is split up in six parts. These are described in detail below. To activate an action, you first have to select the check box, which is located next to it. This is, to make it clear which actions are enabled for a given recipe.

Predesigned filters

In the window which appear, when you press the button labeled Predesigned Filters, you may set up a filter. This filter may change the header fields, add new header fields and/or remove existing header fields.

On this page you will find one custom-made filter: Remove signatures. With this filter, you may specify a signature for each email address. If the text you specify is found (exactly!) it will be removed from the letter. My intention is that more custom-made filters will be added, as users send me their ideas and filters.

Handmade filters

If you wish to create your own filter, you have to go the page Handmade filters. On this page, you may send the header and/or the body of a letter through a command.

As an example, you may remove the header with the command cat - >> /dev/null, or add a message to the body of a message with the command echo This letter has been resent to you, by my procmail filter!; cat -

If only the filter actions is selected, the filter will change the letter permanently, i.e. the changes will have effect on the subsequent recipes (even on the delivered letter, if no recipe match!) This may be useful if you e.g. uses a mail reader, which does not support mime, and you have a filter, which can convert mime encoded text to 7 bit ascii.

If however one of the other actions are enabled, the changes will only have effect within this recipe!

The reply action

With the reply action, you may set up a reply mechanism, which sends a letter back to the sender, with a message you specify. One feature of this mechanism is that you may specify how often a reply should be send, you have the following possibilities.

Send a reply to each letter
Send a reply only once
Send only a reply if it is more than a given number of days since the last reply was send.

This is useful, if you leave on vacation, and wish to send a message that you will not read your letter at once.

The reply is only sent, if the letter does not come from a daemon, to avoid that you sent a reply to every message on a mailing list.

The forward action

With the forward action, you may forward letters to other email accounts.

The save to file action

With this action, you may save the letter to a file. The file name is specified with a FillOut widget, just like you specified the name of a file to save backups to. This time, however, you have two additional features: you may use the content of a header field, or you may use the output from a command. In figure 5, you can see how to select a header field to extract as part of the file name.

figure 5

Email addresses may be given in three ways:

real name <email>
email (real name)
email

If you select that the field is an email address, you may specify that you wish to extract the user name with or without the domain part.

Finally, you may pipe the header field though a command you specify yourself. This command may read the value of the header field on standard input, and write to standard output.

The pipe action

With the pipe action, you may specify a command, which takes care of the letter. This command may read the letter on standard input, and may not write anything (it will be ignored!).

The log file

The procmail file generated from TDG, contain lots of comments, which should make it easy for you to find a specific recipe, in case you wish to see what it look like.

Should something go wrong, you may turn on extended diagnostic, this will write additional lines to the log file, which lets you see what it does. To debug this, you have to read both the log file and the procmail file.

If you use the log abstract options, you will find the program mailstat very useful. This will tell you how many letters have been delivered where. One line in the output from the mailstat programs is fake, and that is the line, which says: /bin/false. This line is due to the way that the generated code look. When you wish to throw away a letter, in a way so you can explicit see that is has been dumped, you should deliver it to the file called /dev/null. Please note that you can only use the mailstat program, if extended diagnostic is turned off.

The End

Before procmail start filtering all your incoming mail, you have to add the following line to the file called ~/.forward:

"|IFS=' ' &&exec /usr/local/bin/procmail -f-||exit 75 #YOUR_USERNAME"

With the correct pathname for procmail, and your email address replaced with YOUR_USERNAME

Here's a few link, which you may find interesting:

The procmail mailing list: To subscribe to the procmail mailing list, send a letter to . Letters to it should be send to
Procmail Mini-Tutorial in Linux Gazette: In The Linux Gazette issue 14, Jim Dennis has written a mini howto on procmail
Procmail Mini-FAQ: Era Eriksson Has created an excellent FAQ for procmail, which may be worth visiting before you ask anything on the mailing list
The home page of the Dotfile Generator: At the home page of The Dotfile Generator, you may find additional information about the other modules, which TDG contains (at the moment this include: Bash, Elm, Emacs, ipfwadm, Fvwm1, Fvwm2, Tcsh and Rtin).