bugClasspath Extensions - Bugs: bug #40378, SMTP client does not do dot...

 
 

bug #40378: SMTP client does not do dot stuffing

Submitter:  Michael Clase <mclase>
Submitted:  Sat 26 Oct 2013 03:09:34 AM UTC
   
 
Category:  mail Severity:  4 - Important
Item Group:  None Status:  In Progress
Privacy:  Public Assigned to:  conradpino
Open/Closed:  Open
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 29 Jan 2014 11:20:05 AM UTC, comment #17: 
Conrad T. Pino <conradpino>
Group Member
Wed 29 Jan 2014 09:13:18 AM UTC, comment #16: 

Replacing "int CR = 13;" or "int CR = 0x0d;" with "int CR = '\r';" gives other character set users a start, improves readability, and IMO harms nothing here.

Conrad T. Pino <conradpino>
Group Member
Wed 29 Jan 2014 09:02:02 AM UTC, comment #15: 

The definition of CR and LF in the relevant Internet Drafts is based on ASCII, so you can ignore EBCDIC.

Chris Burdess <dog>
Group administrator
Wed 29 Jan 2014 08:53:00 AM UTC, comment #14: 

Notice how the if below behaves in the "c = CR" case:

    if (last[0] == LF && last[1] == END && c == LF)

Above line used twice at Lines 68 & 92.

CR only: http://en.wikipedia.org/wiki/Newline#Representations

Let's avoid ASCII decimal constants to be portable across other character sets such as EBCDIC.

Conrad T. Pino <conradpino>
Group Member
Wed 29 Jan 2014 08:34:45 AM UTC, comment #13: 

If we handle LF, then we also handle CRLF by the same token. So the only issue is CR only.

Are there actually any platforms left that have a single CR and are capable of running Java 5? The only ones I can think of are 6502 and Commodore 8-bit machines, and Mac OS 9 and earlier, which are no longer supported by their vendors.

Chris Burdess <dog>
Group administrator
Wed 29 Jan 2014 08:26:43 AM UTC, comment #12: 

The gnu.inet.util.MessageOutputStream has these faults:

1. It fails "dot stuffing" when message newline sequences are not strict POSIX newline; it fails reliably for both CRLF and CR only sequences. Platform portability is critical.

2. The buffer write method does a buffer allocate & copy for every "dot" discovered which could be multiple times per invocation.

Conrad T. Pino <conradpino>
Group Member
Wed 29 Jan 2014 08:15:30 AM UTC, comment #11: 

Upon reviewing RFC5321 I was reminded "dot stuffing" shall occur only within the "DATA" command phase and at no other time.

Upon further project source code review I noted class gnu.inet.util.MessageOutputStream purpose is already "dot stuffing" and does so at the right time; it just does so poorly for both platform portability and performance reasons.

Collectively we leaped too soon and in the wrong direction but recovery is simple:

1. I propose reverting gnu.inet.smtp.SMTPConnection class to replace the gnu.inet.smtp.SMTPOutputStream with the gnu.inet.util.CRLFOutputStream class.

2. I propose revising gnu.inet.util.MessageOutputStream to use similar implementation previously proposed for gnu.inet.smtp.SMTPConnection class.

3. I propose deleting gnu.inet.smtp.SMTPOutputStream class.

Additional comments and patches shall follow shortly.

Conrad T. Pino <conradpino>
Group Member
Wed 29 Jan 2014 01:34:54 AM UTC, comment #10: 

The following patches are proposed for resolving comment #8 (below) the multiple lines in buffer issue.

https://savannah.gnu.org/patch/index.php?8296
https://savannah.gnu.org/patch/index.php?8297

Conrad T. Pino <conradpino>
Group Member
Wed 29 Jan 2014 12:36:24 AM UTC, comment #9: 

The method included below fails to double periods when buffer contains multiple lines which is likely when java.io.BufferedOutputStream is used.

class gnu.inet.smtp.SMTPOutputStream extends ...
{
...
    public void write(byte[] b, int off, int len)
        throws IOException
    {
        if (len > 0)
        {
            int b0 = b[off];
            if (atBOL && b0 == Period)
            {
                super.write(b0); // double dot at start of line
            }
            super.write(b, off, len);
        }
    }
}

Conrad T. Pino <conradpino>
Group Member
Tue 28 Jan 2014 09:33:07 PM UTC, comment #8: 
Conrad T. Pino <conradpino>
Group Member
Sun 26 Jan 2014 02:20:40 AM UTC, comment #7: 

SIMPLE MAIL TRANSFER PROTOCOL:
https://tools.ietf.org/html/rfc821 RFC821 obsoleted by
https://tools.ietf.org/html/rfc2821 RFC2821 obsoleted by
https://tools.ietf.org/html/rfc5321 RFC5321

Internet Message Format:
https://tools.ietf.org/html/rfc822 RFC822 obsoleted by
https://tools.ietf.org/html/rfc2822 RFC2822 obsoleted by
https://tools.ietf.org/html/rfc5322 RFC5322

RFC5321 and RFC5322 are suggested as the testing standards.

Conrad T. Pino <conradpino>
Group Member
Mon 28 Oct 2013 10:26:12 AM UTC, comment #6: 

We need to create a test RFC822 message that has a dot as the first character of a line, then verify that we can send this using the SMTP provider without error or corruption.

Chris Burdess <dog>
Group administrator
Sun 27 Oct 2013 08:39:15 PM UTC, comment #5: 

Oooohhh! You're fast; I saw your gnu.inet.smtp.SMTPOutputStream revision 1.1 commit.

If your "we need to test against inetlib HEAD" means you'd like help, I shall try but I'm not sure what something useful looks like at the moment.  Can you elaborate?

Conrad T. Pino <conradpino>
Group Member
Sun 27 Oct 2013 08:14:10 PM UTC, comment #4: 

Thanks for reporting this. The place it actually needs to be fixed in is not here but in the Classpath inetlib sources. I have put in a fix so we need to test against inetlib HEAD.

Chris Burdess <dog>
Group administrator
Sat 26 Oct 2013 03:09:34 AM UTC, comment #3: 

This item has been reassigned from the project Classpath Extensions support tracker to your tracker.

The original report is still available at support #107195

Following are the information included in the original report:

[field #0]                  Item ID: 107195
[field #1]                 Group ID: 139
[field #2]              Open/Closed: Open
[field #3]                 Severity: 3 - Normal
[field #4]                  Privacy: Public
[field #8]                         : Unknown support Field Display Type
[field #9]                 Category: None
[field #10]             Submitted by: mclase
[field #11]              Assigned to: None
[field #12]             Submitted on: Tue 05 Jan 2010 10:49:06 AM PST
[field #13]                  Summary: SMTP client does not do dot stuffing
[field #14]      Original Submission: According to the SMTP spec, a client should escape a dot character when it is the first character on a line by doubling it. See: http://tools.ietf.org/html/rfc2821#section-4.5.2 which refers to the obligation of the server to remove one of the dots.

The classpathx mail-1.1.2 package does not implement this properly. As a result, the initial dot on a line gets deleted by the receiving SMTP server.

This is particularly a problem when using Content-Transfer-Encoding: quoted-printable because this wraps lines to 76 characters using soft carriage returns; this may force dots to the beginning of a line. This encoding is used when the message has MIME-type text/html.
[field #16]               Item Group: None
[field #17]                   Status: None
[field #18]        Component Version: None
[field #19]         Operating System: GNU/Linux
[field #20]          Reproducibility: None
[field #21]               Size (loc): None
[field #22]            Fixed Release: None
[field #23]          Planned Release: None
[field #24]                   Effort: 0.00
[field #28]                 Priority: 5 - Normal
[field #31]         Percent Complete: 0%
[field #33]                  Release: None
[field #58]     Custom Select Box #1: None
[field #59]     Custom Select Box #2: None
[field #60]     Custom Select Box #3: None
[field #61]     Custom Select Box #4: None
[field #62]     Custom Select Box #5: None
[field #63]     Custom Select Box #6: None
[field #64]     Custom Select Box #7: None
[field #65]     Custom Select Box #8: None
[field #66]     Custom Select Box #9: None
[field #67]    Custom Select Box #10: None

Conrad T. Pino <conradpino>
Group Member
Sat 26 Oct 2013 03:05:04 AM UTC, comment #2: 

For other reasons I've read relevant RFC; IMO this really needs to happen.

Conrad T. Pino <conradpino>
Group Member
  Spam posted by anonymous
Sat 26 Oct 2013 03:09:34 AM UTC, original submission:  

According to the SMTP spec, a client should escape a dot character when it is the first character on a line by doubling it. See: http://tools.ietf.org/html/rfc2821#section-4.5.2 which refers to the obligation of the server to remove one of the dots.

The classpathx mail-1.1.2 package does not implement this properly. As a result, the initial dot on a line gets deleted by the receiving SMTP server.

This is particularly a problem when using Content-Transfer-Encoding: quoted-printable because this wraps lines to 76 characters using soft carriage returns; this may force dots to the beginning of a line. This encoding is used when the message has MIME-type text/html.

Michael Clase <mclase>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dog (Posted a comment)
  • -email is unavailable- added by conradpino (Posted a comment)
  • -email is unavailable- added by mclase (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-01-28 conradpino StatusReady For Test In Progress
        Assigned todog conradpino
    2013-10-28 dog StatusConfirmed Ready For Test
    2013-10-27 dog StatusNone Confirmed
        Assigned toNone dog
    2013-10-26 conradpino CategoryNone mail
        Severity3 - Normal 4 - Important
    2013-10-26 conradpino Reassign itemClasspath Extensions, sr #107195 Classpath Extensions, bug #40378

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code