bugClasspath Extensions - Bugs: bug #40378, SMTP client does not do dot...

 
 

bug #40378: SMTP client does not do dot stuffing

Submitted by:  Michael Clase <mclase>
Submitted on:  Sat 26 Oct 2013 03:09:34 AM UTC  
 
Category: mailSeverity: 4 - Important
Item Group: NoneStatus: In Progress
Privacy: PublicAssigned to: Conrad T. Pino <conradpino>
Open/Closed: Open

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 29 Jan 2014 11:20:05 AM UTC, comment #16:

This issue is resolved by these patches:

https://savannah.gnu.org/patch/?8296
https://savannah.gnu.org/patch/?8298
https://savannah.gnu.org/patch/?8299

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Wed 29 Jan 2014 09:13:18 AM UTC, comment #15:

Replacing "int CR = 13;" or "int CR = 0x0d;" with "int CR = '\r';" gives other character set users a start, improves readability, and IMO harms nothing here.

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Wed 29 Jan 2014 09:02:02 AM UTC, comment #14:

The definition of CR and LF in the relevant Internet Drafts is based on ASCII, so you can ignore EBCDIC.

Chris Burdess <dog>
Project Administrator
Wed 29 Jan 2014 08:53:00 AM UTC, comment #13:

Notice how the if below behaves in the "c = CR" case:

if (last[0] == LF && last[1] == END && c == LF)

Above line used twice at Lines 68 & 92.

CR only: http://en.wikipedia.org/wiki/Newline#Representations

Let's avoid ASCII decimal constants to be portable across other character sets such as EBCDIC.

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Wed 29 Jan 2014 08:34:45 AM UTC, comment #12:

If we handle LF, then we also handle CRLF by the same token. So the only issue is CR only.

Are there actually any platforms left that have a single CR and are capable of running Java 5? The only ones I can think of are 6502 and Commodore 8-bit machines, and Mac OS 9 and earlier, which are no longer supported by their vendors.

Chris Burdess <dog>
Project Administrator
Wed 29 Jan 2014 08:26:43 AM UTC, comment #11:

The gnu.inet.util.MessageOutputStream has these faults:

1. It fails "dot stuffing" when message newline sequences are not strict POSIX newline; it fails reliably for both CRLF and CR only sequences. Platform portability is critical.

2. The buffer write method does a buffer allocate & copy for every "dot" discovered which could be multiple times per invocation.

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Wed 29 Jan 2014 08:15:30 AM UTC, comment #10:

Upon reviewing RFC5321 I was reminded "dot stuffing" shall occur only within the "DATA" command phase and at no other time.

Upon further project source code review I noted class gnu.inet.util.MessageOutputStream purpose is already "dot stuffing" and does so at the right time; it just does so poorly for both platform portability and performance reasons.

Collectively we leaped too soon and in the wrong direction but recovery is simple:

1. I propose reverting gnu.inet.smtp.SMTPConnection class to replace the gnu.inet.smtp.SMTPOutputStream with the gnu.inet.util.CRLFOutputStream class.

2. I propose revising gnu.inet.util.MessageOutputStream to use similar implementation previously proposed for gnu.inet.smtp.SMTPConnection class.

3. I propose deleting gnu.inet.smtp.SMTPOutputStream class.

Additional comments and patches shall follow shortly.

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Wed 29 Jan 2014 01:34:54 AM UTC, comment #9:

The following patches are proposed for resolving comment #8 (below) the multiple lines in buffer issue.

https://savannah.gnu.org/patch/index.php?8296
https://savannah.gnu.org/patch/index.php?8297

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Wed 29 Jan 2014 12:36:24 AM UTC, comment #8:

The method included below fails to double periods when buffer contains multiple lines which is likely when java.io.BufferedOutputStream is used.

class gnu.inet.smtp.SMTPOutputStream extends ...
{
...
public void write(byte[] b, int off, int len)
throws IOException
{
if (len > 0)
{
int b0 = b[off];
if (atBOL && b0 == Period)
{
super.write(b0); // double dot at start of line
}
super.write(b, off, len);
}
}
}

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Tue 28 Jan 2014 09:33:07 PM UTC, comment #7:

https://savannah.gnu.org/patch/index.php?8291
https://savannah.gnu.org/patch/index.php?8294
https://savannah.gnu.org/patch/index.php?8295

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Sun 26 Jan 2014 02:20:40 AM UTC, comment #6:

SIMPLE MAIL TRANSFER PROTOCOL:
https://tools.ietf.org/html/rfc821 RFC821 obsoleted by
https://tools.ietf.org/html/rfc2821 RFC2821 obsoleted by
https://tools.ietf.org/html/rfc5321 RFC5321

Internet Message Format:
https://tools.ietf.org/html/rfc822 RFC822 obsoleted by
https://tools.ietf.org/html/rfc2822 RFC2822 obsoleted by
https://tools.ietf.org/html/rfc5322 RFC5322

RFC5321 and RFC5322 are suggested as the testing standards.

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Mon 28 Oct 2013 10:26:12 AM UTC, comment #5:

We need to create a test RFC822 message that has a dot as the first character of a line, then verify that we can send this using the SMTP provider without error or corruption.

Chris Burdess <dog>
Project Administrator
Sun 27 Oct 2013 08:39:15 PM UTC, comment #4:

Oooohhh! You're fast; I saw your gnu.inet.smtp.SMTPOutputStream revision 1.1 commit.

If your "we need to test against inetlib HEAD" means you'd like help, I shall try but I'm not sure what something useful looks like at the moment. Can you elaborate?

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Sun 27 Oct 2013 08:14:10 PM UTC, comment #3:

Thanks for reporting this. The place it actually needs to be fixed in is not here but in the Classpath inetlib sources. I have put in a fix so we need to test against inetlib HEAD.

Chris Burdess <dog>
Project Administrator
Sat 26 Oct 2013 03:09:34 AM UTC, comment #2:

This item has been reassigned from the project Classpath Extensions support tracker to your tracker.

The original report is still available at support #107195

Following are the information included in the original report:

[field #0] Item ID: 107195
[field #1] Group ID: 139
[field #2] Open/Closed: Open
[field #3] Severity: 3 - Normal
[field #4] Privacy: Public
[field #8] : Unknown support Field Display Type
[field #9] Category: None
[field #10] Submitted by: mclase
[field #11] Assigned to: None
[field #12] Submitted on: Tue 05 Jan 2010 10:49:06 AM PST
[field #13] Summary: SMTP client does not do dot stuffing
[field #14] Original Submission: According to the SMTP spec, a client should escape a dot character when it is the first character on a line by doubling it. See: http://tools.ietf.org/html/rfc2821#section-4.5.2 which refers to the obligation of the server to remove one of the dots.

The classpathx mail-1.1.2 package does not implement this properly. As a result, the initial dot on a line gets deleted by the receiving SMTP server.

This is particularly a problem when using Content-Transfer-Encoding: quoted-printable because this wraps lines to 76 characters using soft carriage returns; this may force dots to the beginning of a line. This encoding is used when the message has MIME-type text/html.
[field #16] Item Group: None
[field #17] Status: None
[field #18] Component Version: None
[field #19] Operating System: GNU/Linux
[field #20] Reproducibility: None
[field #21] Size (loc): None
[field #22] Fixed Release: None
[field #23] Planned Release: None
[field #24] Effort: 0.00
[field #28] Priority: 5 - Normal
[field #31] Percent Complete: 0%
[field #33] Release: None
[field #58] Custom Select Box #1: None
[field #59] Custom Select Box #2: None
[field #60] Custom Select Box #3: None
[field #61] Custom Select Box #4: None
[field #62] Custom Select Box #5: None
[field #63] Custom Select Box #6: None
[field #64] Custom Select Box #7: None
[field #65] Custom Select Box #8: None
[field #66] Custom Select Box #9: None
[field #67] Custom Select Box #10: None

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
Sat 26 Oct 2013 03:05:04 AM UTC, comment #1:

For other reasons I've read relevant RFC; IMO this really needs to happen.

Conrad T. Pino <conradpino>
Project MemberIn charge of this item.
 Spam posted by an anonymous
Sat 26 Oct 2013 03:09:34 AM UTC, original submission:

According to the SMTP spec, a client should escape a dot character when it is the first character on a line by doubling it. See: http://tools.ietf.org/html/rfc2821#section-4.5.2 which refers to the obligation of the server to remove one of the dots.

The classpathx mail-1.1.2 package does not implement this properly. As a result, the initial dot on a line gets deleted by the receiving SMTP server.

This is particularly a problem when using Content-Transfer-Encoding: quoted-printable because this wraps lines to 76 characters using soft carriage returns; this may force dots to the beginning of a line. This encoding is used when the message has MIME-type text/html.

Michael Clase <mclase>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by dog (Posted a comment)
  • -unavailable- added by conradpino (Posted a comment)
  • -unavailable- added by mclase (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 28 Jan 2014 09:33:07 PM UTCconradpinoStatusReady For Test=>In Progress
      Assigned todog=>conradpino
    Mon 28 Oct 2013 10:23:49 AM UTCdogStatusConfirmed=>Ready For Test
    Sun 27 Oct 2013 08:14:10 PM UTCdogStatusNone=>Confirmed
      Assigned toNone=>dog
    Sat 26 Oct 2013 03:14:21 AM UTCconradpinoCategoryNone=>mail
      Severity3 - Normal=>4 - Important
    Sat 26 Oct 2013 03:09:34 AM UTCconradpinoReassign itemClasspath Extensions, sr #107195=>Classpath Extensions, bug #40378

    Back to the top


    Powered by Savane 3.1-cleanup