bugGuile - Bugs: bug #31680, R6RS string literal intraline...

 
 

bug #31680: R6RS string literal intraline whitespace removal

Submitted by:  Göran Weinholt <weinholt>
Submitted on:  Wed 17 Nov 2010 12:15:37 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: None
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 21 Jan 2011 08:22:32 AM UTC, comment #9:

Fixed in git. You have to enable a reader option:

(read-enable 'hungry-eol-escapes)

Andy Wingo <wingo>
Project Administrator
Fri 21 Jan 2011 07:35:16 AM UTC, comment #8:

I looked at implementing this, and it seems that Guile does handle the sequence \ <LF> as being nothing; e.g.

"foo\
bar"
=> "foo bar"

This appears to have been the case since time immemorial, or 1996 at least. So we will probably have to add a reader option for this, unfortunately...

Andy Wingo <wingo>
Project Administrator
Fri 19 Nov 2010 05:57:33 PM UTC, comment #7:

> OK, why don't we implement the escape sequence,
> \<line ending><intraline whitespace>* -> nothing,
> then. Does that sound right to you, Mike?


Sounds great. I'll commit something this weekend if you
don't get to it first.

Mike Gran <mike121>
Project Member
Fri 19 Nov 2010 03:24:17 PM UTC, comment #6:

OK, why don't we implement the escape sequence, \<line ending><intraline whitespace>* -> nothing, then. Does that sound right to you, Mike?

Andy Wingo <wingo>
Project Administrator
Thu 18 Nov 2010 09:51:51 PM UTC, comment #5:

> I think the deal is that all (?) of the other escapes can be
> dealt with via the equivalent of a `case' expression. This
> one requires a property lookup. It's not as nice.


> Also note the thread at
> http://lists.r6rs.org/pipermail/r6rs-discuss/2010-November/006146.html.


> Is there a use case for allowing intraline spaces before
> the newline? Disallowing that would eliminate a state
> in the parser.


I don't think there is a valid use case for allowing that
initial intraline space. If (according to the discussion
in the R6RS list) the intention was to work around broken
editors, those bugs should be fixed with the editors, and
not become workarounds enshrined in a language spec.

Mike Gran <mike121>
Project Member
Thu 18 Nov 2010 08:24:00 PM UTC, comment #4:

On Thu 18 Nov 2010 20:45, Göran Weinholt <INVALID.NOREPLY@gnu.org> writes:

> (define modp-group1-p
> (string->number
> "FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD1
> 29024E088A67CC74020BBEA63B139B22514A08798E3404DD
> EF9519B3CD3A431B302B0A6DF25F14374FE1356D6D51C245
> E485B576625E7EC6F44C42E9A63A3620FFFFFFFFFFFFFFFF"
> 16))


Sounds legit.

> surely you should not need to check for U+205F and so on explictly,
> isn't it enough to see that they belong to the right
> char-general-category?


I think the deal is that all (?) of the other escapes can be dealt with
via the equivalent of a `case' expression. This one requires a property
lookup. It's not as nice.

Also note the thread at
http://lists.r6rs.org/pipermail/r6rs-discuss/2010-November/006146.html.

Is there a use case for allowing intraline spaces before the newline?
Disallowing that would eliminate a state in the parser.

Andy

Andy Wingo <wingo>
Project Administrator
Thu 18 Nov 2010 07:45:17 PM UTC, comment #3:

I've used it for multiline strings, like so:

(define modp-group1-p
(string->number
"FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD1\
29024E088A67CC74020BBEA63B139B22514A08798E3404DD\
EF9519B3CD3A431B302B0A6DF25F14374FE1356D6D51C245\
E485B576625E7EC6F44C42E9A63A3620FFFFFFFFFFFFFFFF"
16))

or like so:

"My name is Ozymandias, king of kings:\n\
Look on my works, ye Mighty, and despair!"

I think any implementation of multiline strings should be prepared to handle characters from other operating systems. But surely you should not need to check for U+205F and so on explictly, isn't it enough to see that they belong to the right char-general-category?

Göran Weinholt <weinholt>
Thu 18 Nov 2010 06:56:00 PM UTC, comment #2:

Interesting issue. I have no idea what that particular escape sequence is for. Göran, do you use it? What for?

Andy Wingo <wingo>
Project Administrator
Thu 18 Nov 2010 03:05:12 PM UTC, comment #1:

Yeah it is true that this isn't implemented and should be.

Ludo, Andy, I did in fact write an implementation of this in r6rs-strings tree, but, I never committed it to master.

The commit was cff6adf899dbc0336c7c017d52504f8138c89b3d and the part of the commit specific to this bug is all in read.c in scm_read_string and SCM_READ_SPACE_LINE_SPACE

I didn't commit it because, at the time, I took exception with the r6rs spec itself.

It states that \<intraline-whitespace><line-ending><intraline-whitespace> expands to nothing. And then it calls out that whitespace and line-ending are any characters in the Unicode whitespace and line ending families!

I guarantee that no one will ever put something like <U+005d "backslash"> <U+205F "medium mathematical space"> <U+2028 "line separator"> <U+2006 "6 per em space"> in the middle of a source code file. It will never, ever happen. But writing the code to allow for all those cases lead to an ugly patch.

Mike Gran <mike121>
Project Member
Wed 17 Nov 2010 12:15:37 PM UTC, original submission:

In R6RS subsection 4.2.7 there is this escape sequence:

\<intraline whitespace><line ending>
<intraline whitespace> : nothing

This escape sequence isn't working. Both the examples below should return "foobar":

GNU Guile 1.9.13.58-b98d5a
Copyright (C) 1995-2010 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (import (rnrs))
scheme@(guile-user)> (read (open-string-input-port "#!r6rs \"foo\\\n bar\""))
$1 = "foo bar"
scheme@(guile-user)> (read (open-string-input-port "#!r6rs \"foo\\ \n bar\""))
ERROR: In procedure scm_lreadr:
ERROR: #<unknown port>:1:10: illegal character in escape sequence: #\space

Göran Weinholt <weinholt>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by wingo (Posted a comment)
  • -unavailable- added by mike121 (Posted a comment)
  • -unavailable- added by weinholt (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 21 Jan 2011 08:22:32 AM UTCwingoStatusNone=>Fixed
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1