taskSavannah Administration - Tasks: task #15599, Add bounce processing to Savannah

 
 

task #15599: Add bounce processing to Savannah

Submitter:  Ian Kelling <iank>
Submitted:  Sat 18 Apr 2020 11:48:49 PM UTC
   
 
Should Start On:  Sat 18 Apr 2020 12:00:00 AM UTC Should be Finished on:  Sat 18 Apr 2020 12:00:00 AM UTC
Category:  System Priority:  * 7 - High
Status:  Done Privacy:  Public
Assigned to:  rwp Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 01 Dec 2023 09:50:54 AM UTC, comment #2: 

It looks like frontend1 was simply missing an alias for www-data which caused it to try to deliver the bounce through eggs.  Because of course eggs has the MX record for *.gnu.org as a catch-all.  But in this case it was not intended.  It was just left off and forgotten.

Previously that alias was a standard part of the system but appears now to no longer be included.  I'll make sure it gets added to future systems.  I am actually quite shocked that it has gone this long before this was noticed.

I have added an alias for www-data to frontend1 and that should stop it from forwarding bounces to eggs.

Bob Proulx <rwp>
Site Administrator
Wed 29 Nov 2023 02:25:01 AM UTC, comment #1: 

original submission:

>
> I would make eggs deliver the bounces to a savannah machine,


Can you start with this? (by you, I mean any FSF sysadmins, since other people have no access to eggs TTBOMK)

Ineiev <ineiev>
Site Administrator
Sat 18 Apr 2020 11:48:49 PM UTC, original submission:  

frontend1.savannah.gnu.org is sending out email notifications through
eggs from www-data@savannah.gnu.org. Sometimes they bounce, and
currently we are just ignoring those bounces. A lot of those bounce
messages indicate that we should not be sending any more email to that
address, either its no longer a valid email or a few other reasons, and
continuing to do so hurts the sender reputation of eggs. We've been
having significant trouble with our server getting blacklisted over the
last year or so, so I'm giving this item a high priority.

I could also make a separate email server for Savannah, but it could
hurt the reputation of our ip range, or the parent domain, gnu.org, we
don't know exactly, email hosters are typically running some secret
system.

So, I'm hoping someone at savannah will Savannah implement some code to
start processing these bounces and disabling sending notifications, or
email entirely to that address if the bounce calls for it.

I would make eggs deliver the bounces to a savannah machine, then the
Savannah bounce processor would maybe run from a cronjob or directly
from the mta, look at the bounce and decide if the email address in the
bounce should be disabled then discard the bounce email. Ideally, the
Savannah web ui would have some way to indicate that the email is
disabled, so a user can reenable it in case they update their email
server/hosting so it becomes a valid email. In discourse, they call the
whole account disabled.

Many web applications that send notifications seem to have created their
own system for bounce processing. A few examples: GNU mailman disables
any email that bounces too many times within a certain period. Civicrm
searches for the reason within the body of the bounce, and tries to
classify it into categories, then has a separate threshold of # of
bounces within each category. Discourse does something
similar,https://meta.discourse.org/t/handling-bouncing-e-mails/45343. I've
found it was more reliable to alter the Civicrm way into a blacklist of
strings and ignore the rest. Anyways, the GNU Mailman way looks to be
the simplest to implement.

Some headers from an example message that bounceed:

X-Savane-Tracker: bugs
Subject: [bug #redacted] redacted

In my research, I found an advanced bounce parsing library in ruby or
perl, https://github.com/sisimai, but I'm guessing it would be pretty
easy to use any popular email parsing library in a high level language
to build your own totally sufficient bounce processing with a few hours
or days of effort. Also, I've found the GNU Mailman code to be fairly
readable, so it might be a good example.

Ian Kelling <iank>
Site Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rwp (Posted a comment)
  • -email is unavailable- added by ineiev (Posted a comment)
  • -email is unavailable- added by iank (Submitted the item)
  • -email is unavailable- added by iank
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-12-01 rwp StatusNeed Info Done
        Assigned toineiev rwp
        Open/ClosedOpen Closed
    2023-11-29 ineiev StatusNone Need Info
        Assigned toNone ineiev
    2022-05-25 ineiev Carbon-CopyRemoved fontdevelopment -
    2022-01-05 _289157 Carbon-Copy- Added fontdevelopment
    2020-04-18 iank Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code