bugSTUMP - Bugs: bug #62034, Filter more mail-specific headers...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #62034: Filter more mail-specific headers from submitted articles

Submitter:  Paul Schleck <pschleck>
Submitted:  Fri 11 Feb 2022 11:50:23 PM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None

Fri 09 Jun 2023 11:46:11 PM UTC, comment #1: 

Additional headers identified from further moderated newsgroup submissions:

            -I "x-ms-exchange-antispam-relay:" \                                                                                          
            -I "x-ms-exchange-antispam-message-data-chunkcount:" \                                                                        
            -I "x-ms-exchange-antispam-messagedata-0:" \                                                                                  
            -I "X-CrossPremisesHeadersFilteredBySendConnector:" \                                                                                                                                                                   
            -I "X-Proofpoint-GUID:" \                                                                                                     
            -I "X-Proofpoint-ORIG-GUID:" \                                                                                                                                                                                                                                                                  
            -I "X-OrganizationHeadersPreserved:"        \                                                                                                                                                                                                                                                
            -I "msip_labels:"                           \                                                                                 
            -I "X-Rspamd-Action:"                       \                                                                                 
            -I "X-Mailman-Approved-At:"                 \                                                                                 
            -I "X-SES-Outgoing:"                        \                                                                                 
            -I "Feedback-ID:"                           \                                                                                 

Paul Schleck <pschleck>
Fri 11 Feb 2022 11:50:23 PM UTC, original submission:  

Many mail headers in submitted articles have no informational value for Usenet, taking up unneeded article size, conflict with the need to replace/rewrite the contents to reflect moderator approval and processing, prevent a proper tracing/audit trail, or even result in posting errors and rejections when the articles containing them are sent to news servers.

The bin/processApproved script already filters some:

    echo Date: "$(date "+%a, %e %b %Y %H:%M:%S CST")"
    formail -b -f -a "Newsgroups: $NEWSGROUP" \
    -I Path: \
            -I X-Moderate-For:                          \
            -I Return-Path:                             \
            -I X-Mailer:                                \
    -I "Date:"                                  \
            -I "X-400-Received:"                        \
    -I Received: -I "From " \
    -a "Approved: $PMUSER_APPROVAL" \
            -I Lines:                                   \
            -I Cc:                                      \
    -I To: -I Status:                           \
    < "$TMPFILE"

Experiences with received e-mail over the years by moderation teams that use STUMP to moderator newsgroups which receive more general submitted content than just from news servers, including from mailing lists and RSS aggregators, motivated additions to the list as follows.  Of course, it might be better to reverse this to only allow certain headers, rather than enumerate a indefinitely large number of prohibited headers.  It might also be better to put either allowed or prohibited headers in a configuration file, particularly if other enhancements to STUMP, like removing dependencies on Procmail's formail command, are made.

 echo Date: `date "+%a, %e %b %Y %H:%M:%S %Z"`
    cat $MESSAGE                                        \
      | formail -f -a "Newsgroups: $NEWSGROUP"          \
            -I "ARC-Authentication-Results"             \
            -I "ARC-Message-Signature:"                 \
            -I "ARC-Seal:"                              \
            -I "Authentication-Results:"                \
            -I "Date:"                                  \
            -I "FLines:" \
            -I "Importance:"                            \
            -I "List-Unsubscribe:"                      \
            -I "Message-ID-Hash:"                       \
            -I "Priority:"                              \
            -I "Received-SPF:"                          \
            -I "Resent-From:"                           \
            -I "Resent-Message-Id:"                     \
            -I "Resent-To:"                             \
            -I "X-ASG-Debug-ID:"                        \
            -I "X-ASG-Orig-Subj:"                       \
            -I "X-Authenticated-Sender:" \
            -I "X-Authentication-Warning:"              \
            -I "X-Authority-Reason:" \
            -I "X-Automattic-Destination:" \
            -I "X-Automattic-Tracking:" \
            -I "X-Avast:"                               \
            -I "X-BWhitelist:" \
            -I "X-Barracuda-Apparent-Source-IP:"        \
            -I "X-Barracuda-BBL-IP:"                    \
            -I "X-Barracuda-BBL-Trusted-Forwarder:"     \
            -I "X-Barracuda-BWL-IP:"                    \
            -I "X-Barracuda-Bayes:"                     \
            -I "X-Barracuda-Connect:"                   \
            -I "X-Barracuda-Envelope-From:"             \
            -I "X-Barracuda-RBL-Trusted-Forwarder:"     \
            -I "X-Barracuda-Spam-Report:"               \
            -I "X-Barracuda-Spam-Score:"                \
            -I "X-Barracuda-Spam-Status:"               \
            -I "X-Barracuda-Start-Time:"                \
            -I "X-Barracuda-URL:"                       \
            -I "X-BeenThere:"                           \
            -I "X-Brightmail-Tracker:"                  \
            -I "X-CMAE-Envelope:" \
            -I "X-Cloudmark-Score:"                     \
            -I "X-Content-Filtered-By:" \
            -I "X-Country:" \
            -I "X-DCC-netfront-Metrics:"                \
            -I "X-Default-Received-SPF:"                \
            -I "X-Email-Count:" \
            -I "X-Exim-ID:" \
            -I "X-FEAS-DKIM:" \
            -I "X-Filter-ID:" \
            -I "X-FireEye:"                             \
            -I "X-Get-Message-Sender-Via:" \
            -I "X-Gm-Message-State:" \
            -I "X-Google-DKIM-Signature:" \
            -I "X-Google-Smtp-Source:" \
            -I "X-Hay-Net-MailScanner-From:"            \
            -I "X-Hay-Net-MailScanner-ID:"              \
            -I "X-Hay-Net-MailScanner-Information:"     \
            -I "X-Hay-Net-MailScanner-SpamCheck:"       \
            -I "X-Hay-Net-MailScanner:"                 \
            -I "X-IP-Stats:"                            \
            -I "X-IPAS-Result:"                         \
            -I "X-IronPort-AV:"                         \
            -I "X-IronPort-Anti-Spam-Filtered:"         \
            -I "X-Junkmail-IWF:"                        \
            -I "X-Junkmail-SD-Raw:"                     \
            -I "X-Junkmail-Status:"                     \
            -I "X-Junkmail-Whitelist:"                  \
            -I "X-LangGuess:"                           \
            -I "X-Local-Domain:" \
            -I "X-MS-Exchange-CrossTenant-AuthAs:" \
            -I "X-MS-Exchange-CrossTenant-AuthSource:" \
            -I "X-MS-Exchange-CrossTenant-Network-Message-Id:" \
            -I "X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg:" \
            -I "X-MS-Exchange-CrossTenant-fromentityheader:" \
            -I "X-MS-Exchange-CrossTenant-id:"          \
            -I "X-MS-Exchange-CrossTenant-mailboxtype:" \
            -I "X-MS-Exchange-CrossTenant-originalarrivaltime:" \
            -I "X-MS-Exchange-CrossTenant-rms-persistedconsumerorg:" \
            -I "X-MS-Exchange-CrossTenant-userprincipalname:" \
            -I "X-MS-Exchange-CrossTenantHeadersStamped:" \
            -I "X-MS-Exchange-Transport-CrossTenantHeadersStamped:" \
            -I "X-MS-Has-Attach:"                       \
            -I "X-MS-TNEF-Correlator:"                  \
            -I "X-MailFrom:"                            \
            -I "X-Mailman-Version:"                     \
            -I "X-Mailman-Rule-Hits:"                   \
            -I "X-Mailman-Rule-Misses:"                 \
            -I "X-Message-ID-Hash:"                     \
            -I "X-Mfilter:"                             \
            -I "X-MyRbl:"                               \
            -I "X-Not-Ascii:"                           \
            -I "X-NotAscii:" \
            -I "X-Notice:" \
            -I "X-OriginalArrivalTime:"                 \
            -I "X-Originating-IP:"                      \
            -I "X-OriginatorOrg:" \
            -I "X-OutGoing-Spam-Status:" \
            -I "X-PHP-Originating-Script:" \
            -I "X-PHP-Script:" \
            -I "X-Phrase:"                              \
            -I "X-Pobox-Delivery-ID:"                   \
            -I "X-Pobox-Loop-ID"                        \
            -I "X-Pobox-Pass"                           \
            -I "X-Probe:"                               \
            -I "X-Proofpoint-Spam-Reason:" \
            -I "X-Proofpoint-Virus-Version:" \
            -I "X-Provags-ID:"                          \
            -I "X-RZG-AUTH:" \
            -I "X-RZG-CLASS-ID:" \
            -I "X-Rcpt-To:"                             \
            -I "X-Received:"                            \
            -I "X-Recommended-Action:" \
            -I "X-Return-Path:"                         \
            -I "X-Source-Args:" \
            -I "X-Source-Auth:" \
            -I "X-Source-Cap:" \
            -I "X-Source-Dir:" \
            -I "X-Source-IP:" \
            -I "X-Source-L:" \
            -I "X-Source-Sender:" \
            -I "X-Source:" \
            -I "X-SpamContent:"                         \
            -I "X-SpamDetect:"                          \
            -I "X-SpamExperts-Domain:" \
            -I "X-SpamExperts-Outgoing-Class:" \
            -I "X-SpamExperts-Outgoing-Evidence:" \
            -I "X-SpamExperts-Username:" \
            -I "X-SpamUrl:" \
            -I "X-To-Not-Matched:"                      \
            -I "X-UI-Out-Filterresults:" \
            -I "X-VADE-STATUS:" \
            -I "X-Verify-SMTP:"                         \
            -I "X-Virus-Scanned:"                       \
            -I "X-Whitelist:"                           \
            -I "X-X-Encryption:"                        \
            -I "X-X-MyRbl:" \
            -I "X-X-Originating-IP:" \
            -I "X-X-SpamDetect:" \
            -I "X-Xfinity-VAAS:" \
            -I "X-Xfinity-VMeta:" \
            -I "X-listMember:"                          \
            -I "spamdiagnosticmetadata:"                \
            -I "spamdiagnosticoutput:"                  \
            -I "x-clientproxiedby:" \
            -I "x-eopattributedmessage:" \
            -I "x-exchange-antispam-report-cfa-test:"   \
            -I "x-exchange-antispam-report-test:"       \
            -I "x-forefront-antispam-report:"           \
            -I "x-forefront-prvs:"                      \
            -I "x-incomingheadercount:" \
            -I "x-incomingtopheadermarker:" \
            -I "x-mailer:" \
            -I "x-microsoft-antispam-message-info:" \
            -I "x-microsoft-antispam-prvs:"             \
            -I "x-microsoft-antispam:"                  \
            -I "x-microsoft-antispam:" \
            -I "x-microsoft-exchange-diagnostics:"      \
            -I "x-ms-exchange-antispam-messagedata:" \
            -I "x-ms-exchange-messagesentrepresentingtype:" \
            -I "x-ms-exchange-purlcount:" \
            -I "x-ms-exchange-senderadcheck:" \
            -I "x-ms-exchange-transport-forked:"        \
            -I "x-ms-office365-filtering-correlation-id:" \
            -I "x-ms-oob-tlc-oobclassifiers:" \
            -I "x-ms-publictraffictype:"                \
            -I "x-ms-traffictypediagnostic:" \
            -I "x-mx-publictraffictype:" \
            -I "x-tmn:" \
            -I Path:                                    \
            -I Received: -I "From "                     \
            -I "Delivered-To:"                          \
            -I "DomainKey-Signature:"                   \
            -I "Path:"                                  \
            -I "Thread-Index:"                          \
            -I "Thread-Topic:"                          \
            -I "X-400-Received:"                        \
            -I "X-Allowed-from:"                        \
            -I "X-Authority-Analysis:"                  \
            -I "X-DF-Seen-By:"                          \
            -I "X-Greylist:"                            \
            -I "X-MSMail-Priority:"                     \
            -I "X-MimeOLE:"                             \
            -I "X-Origin:"                              \
            -I "X-Original-To:"                         \
            -I "X-PMX-Version:"                         \
            -I "X-Postfilter:"                          \
            -I "X-Priority:"                            \
            -I "X-RBL-Warning:"                         \
            -I "X-Scanned-By:"                          \
            -I "X-Scanner:"                             \
            -I "X-Sender-Whitelisted:"                  \
            -I "X-Spam-Checker-Version:"                \
            -I "X-Spam-Flag:"                           \
            -I "X-Spam-Level:"                          \
            -I "X-Spam-Report"                          \
            -I "X-Spam-Score:"                          \
            -I "X-Spam-Status:"                         \
            -I "X-Spamd-Result:"                        \
            -I "X-Rspamd-Queue-Id:"                     \
            -I "X-Rspamd-Server:"                       \
            -I "X-Trace:"                               \
            -I "X-UID:"                                 \
            -I "X-Virus-Status:"                        \
            -I "X-X-Sender:"                            \
            -I "X-Spam-Relay-Country:"                  \
            -I Return-Path:                             \
            -I X-Mailer:                                \
            -I X-Moderate-For:                          \
            -i "NNTP-Posting-Date:"                     \
            -i "NNTP-Posting-Host:"                     \
            -i "X-Complaints-To:"                       \
            -i "Injection-Date:"                        \
            -i "Injection-Info:"                        \
            -i "X-Injection-Date:"                      \
            -i "X-Injection-Info:"                      \
            -a "Approved: $PMUSER_APPROVAL"             \
            -I Lines:                                   \
            -I Cc:                                      \
            -I To: -I Status:

Paul Schleck <pschleck>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pschleck (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code