MailSystem

See also ListServer.

The principal GNU (and FSF?) email server is lists.gnu.org, managed by FSF sysadmin. Savannah people and programs do not have root access.

For outgoing mail from Savannah, the internal host is used; see below.

Mailman

On lists.gnu.org, the usual per-list static Mailman aliases do not exist; instead, it looks at /var/lib/mailman/lists/listname/domains/ and checks for files that represent the domain of the list, such as gnu.org, nongnu.org, and the like.

Then, I don't really know:

  • there is some Exim code to take those files into account
  • currently (?), there is a replication script that copies all the list+domains information to mp.gnu.org - to it's probably checked there. jag said "it runs on the 42 of every hour". We can also run it through ~/mailman-export.

Archives

Archives are updated every 30 minutes (?) as of May 2011. They are located at 2 places:

  • private archives are managed by Mailman: /var/lib/mailman/archives/private/list_name.mbox/list_name.mbox (use mailman's arch command to manage them; look at the --wipe command)
  • public archives are managed by mharc: ~mharc/mbox and ~mharc/html (check ImportMailingListArchive).

Mailing list creation

I (Sylvain) basically installed sv_mailman on internal.sv.gnu.org, modified it to support mailman virtual hosts (and use the proper admin mail instead of username@sv.gnu.org). Then I wrote a fake /usr/sbin/newlist that connects to list@lists.gnu.org via SSH and call an helper script there (savannah_wrapper.pl). This happens via a Savannah public key in list@lists.gnu.org's authorized_keys with a command= restriction.

There is also an empty /usr/sbin/config_list on internal that does nothing; we don't want what Savane currently does there.

The cron job runs on internal every five minutes, file /etc/cron.d/sv. The script being run is named sv_mailman.

On lists.gnu.org, there is a script mycreate_list.pl which can be used to create lists by hand. It duplicates what savannah_wrapper.pl does for Savannah actions. (The scripts must be kept in sync by hand. They should be merged so there is only one.)

Valid sender

lists.gnu.org makes (used to make?) a series of checks that will reject mail from, say, kickino@savannah.gnu.org because this is apparently not a valid email. Email validation eventually falls-back against fencepost (Unix users and the shares /com/mailer/aliases).

To bypass those checks, we added aliases at fencepost for Savannah users that need to send mail (gatekpr, libcrsync, Savane's INVALID.NOREPLY, etc.). See also ListServer.

Antispam

If listhelper@gnu.org is listed as a moderator for a Mailman list, it wil use a conservatively-configured spamassassin to automatically (though asynchronously) remove spam. Details at ListHelperAntiSpam.

How mails are sent from within Savannah

The internal vserver handles email stuff for Savannah. All other systems delegate sending mail to internal.

A Savane cron job creates alias tables: /etc/aliases and /etc/email-addresses. This is used to map Unix user accounts to an outgoing email, based on the email' field in the user's Savane account. Alias tables are only updated oninternal'.

  • (clearly stale info as we don't own the dom0 any more, but leaving for what it may be worth) frontend, sftp, vcs-noshell _and_ the host/dom0: an exim4 system is configured (see infra/exim_forwarder.txt):
  • smarthost = 10.1.0.101
  • QUEUERUNNER='queueonly'

Debugging

Mail mostly happens from frontend. The PHP there would send the email. If you have a received message in hand, you can see the trail of servers in the Received: headers. It starts from PHP. It goes from there to /usr/sbin/sendmail (postfix on those systems) and then out to eggs.gnu.org, then to lists.gnu.org and out to the mailing list for delivery.

If there is an error in sending the PHP and Apache combination should log an error to the frontend:/var/log/apache2/error.log file.

All of the mail status should be logged to frontend:/var/log/mail.*. When a message is sent you should see the transaction logged there.

Logs are rotated. If you are looking back into the past you will need to look through the various dated log files to find the transaction, e.g.:

frontend:/var/log# ls -lhog mail.*
-rw-r----- 1    0 May  8 06:25 mail.err
-rw-r----- 1 1.3K May  7 06:25 mail.err-20160507.gz
-rw-r----- 1 1.7K May  7 06:31 mail.err-20160508
-rw-r----- 1 600K Jul  6 23:26 mail.info
-rw-r----- 1 122K Jun 12 06:25 mail.info-20160612.gz
-rw-r----- 1 114K Jun 19 06:25 mail.info-20160619.gz
-rw-r----- 1 117K Jun 26 06:25 mail.info-20160626.gz
-rw-r----- 1 1.2M Jul  3 06:25 mail.info-20160703
-rw-r----- 1 600K Jul  6 23:26 mail.log
-rw-r----- 1 122K Jun 12 06:25 mail.log-20160612.gz
-rw-r----- 1 114K Jun 19 06:25 mail.log-20160619.gz
-rw-r----- 1 117K Jun 26 06:25 mail.log-20160626.gz
-rw-r----- 1 1.2M Jul  3 06:25 mail.log-20160703
-rw-r----- 1    0 Jun 21 06:25 mail.warn
-rw-r----- 1  211 May 22 05:46 mail.warn-20160522.gz
-rw-r----- 1  208 May 28 10:13 mail.warn-20160529.gz
-rw-r----- 1  119 Jun 12 00:48 mail.warn-20160612.gz
-rw-r----- 1  114 Jun 20 12:23 mail.warn-20160621

You can see messages pending for delivery in the mail queue with the mailq command on frontend.

NOTE: On May 7th 2016 the mail system was changed from exim4 to postfix (see http://lists.gnu.org/archive/html/savannah-hackers-public/2016-05/msg00007.html, http://lists.gnu.org/archive/html/savannah-hackers-public/2016-06/msg00030.html).