Antispam with listhelper

If you're using a mailman list on lists.gnu.org for your project(*), it's possible to configure mailman to automatically delete most spam. This is done by default for lists created through Savannah. The system is called listhelper, and its home page is http://listhelper.nongnu.org/.

Here is how to set it up (again, this is done by default):

  • On the General Options page:
  • add listhelper@gnu.org as a moderator (not administrator).
  • ensure admin_immed_notify=yes
  • ensure respond_to_post_requests=no (**)
  • On the Privacy Options > Sender Filters page:
  • ensure generic_nonmember_action=hold
  • (recommended) ensure default_member_moderation=yes ([**])

If you change these settings, we will assume you want to delete all the spam yourself and not use listhelper.

(*) Actually, the spam removal via listhelper@ is not tied to gnu.org in any way. A few people are using it for handling non-GNU mailman lists as well. If you'd like to do that, email us. On the other hand, the shared moderation help via listhelper-moderate requires a script and cron job on the target system.

(**) Actually, respond_to_post_requests has nothing to do with the listhelper system. But it's important to do, because it reduces backscatter from lists.gnu.org, which in turn decreases our chance of being blacklisted.

([**]) Setting default_member_moderation=yes protects against the spammers who know how to subscribe to a list and then send out their spam. We have seen these on gnu.org, although thankfully they are not yet widespread. The cost is manually approving new members' first message (only).

How it works, what list owners have to do

Without going into every detail, the result is that message that remain in the hold queue will be piped through a conservatively-configured spamassassin, and 95+% of spam will be automatically deleted after a short delay (usually measured in minutes, unless things get backlogged).

Real messages from non-members (or non-approved addresses) will remain in the hold queue, and need to be approved by a human. (We recommend adding real addresses to the "accept in the future" list when approving them.) Thus, you should pay attention to the once-per-day mailman notification when there are "pending message(s)", but ignore the once-per-message mailman notification.

The process does not affect messages from approved members and other approved addresses, which continue go through without delay.

Extensive suggestions and comments about many other Mailman configuration settings, based on experiences using listhelper over the years, are at http://listhelper.nongnu.org/mailmanconf.html. Many of those settings are made by default.

Please feel free to add other GNU lists, we don't need to be explicitly informed. And of course we're happy to answer any questions. And if you need help moderating the list, likewise let us know.

Shared moderation: lists @(non)gnu.org can use listhelper-moderate@gnu.org as an administrator or moderator to help spread the moderation load. (There is no difference between listhelper[-moderate] @gnu.org and @nongnu.org. We tend to use @gnu.org for GNU lists and @nongnu.org for non-GNU lists, but we aren't rigorous about it, and it makes no practical difference.)

On lists.gnu.org, there is a cron entry (for user http://freefriends.org/~listhelper/moderate, which lists the lists with recent pending messages. (There are many other cron entries which are a standard part of mailman installation.)

Inside listhelper

The system was set up by Bob Proulx, with kibitzing from Karl Berry. We all attempt to review both the spam and nonspam that goes through the system, to try to catch misclassified messages. (Very few real messages are classified as spam; the other direction is naturally somewhat less precise.)

We do not claim this is anything like an optimal or efficient system. But it is what we could do now, without requiring deep mail system infrastructure changes. Maybe someday a better system will be possible.

Nearly every list on lists.gnu.org is checked to some extent. The known exceptions are those which explicitly did not want any checking, such as fsfe-uk, bug-freedink, Aubrey's lists (slib|jacal|scm|wb)-[*], FSF member/campaign lists, etc. The exception list is given in the file ~list/etc/ignorelists.egrep on lists. We spend about 15 minutes a day x 2 volunteers dealing with the normal flow, on average.

Email-based trackers

The email-based trackers at http://debbugs.gnu.org (see the section in SavaneTasks for a bit more) are filtered via the debbugs front-end Mailman, but still go through listhelper and mailman if it passes that. Therefore the message approvals, etc., happen as usual.

When trying to re-vivify a message for one of the debbugs lists, look at http://debbugs.gnu.org/cgi-bin/mailman/admindb/debbugs-submit. It won't show up in the regular queue on lists.gnu.org.