bugmake - Bugs: bug #35063, Suggest conventional $(%XX) syntax...

 
 

bug #35063: Suggest conventional $(%XX) syntax for special characters

Submitted by:  David Boyce <boyski>
Submitted on:  Wed 14 Dec 2011 02:27:53 AM UTC  
 
Severity: 3 - NormalItem Group: Enhancement
Status: NonePrivacy: Public
Assigned to: NoneOpen/Closed: Open
Component Version: 4.0Operating System: Any
Fixed Release: NoneTriage Status: Medium Effort

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 29 Jan 2012 10:07:13 AM UTC, comment #7:

I just suggested a more general solution which covers your issues too.

Eldar Abusalimov <abusalimov>
Sun 29 Jan 2012 03:15:37 AM UTC, comment #6:

I think the root of our disagreement is in the goals. This is not at all about adding url encode/decode capability to make; it's about a consistent, safe way of getting special characters past the make parser and I suggested url encoding (aka percent-encoding) as a well-known mechanism which is easy to implement. Another encoding would serve just as well.

In particular, there's no call for url encoding support as I see it. If you can make a case that there's a reason to support both then maybe your proposal would be better. But that's not the stated goal here.

Your question about 'Hello%2C%20world%21' is answered in the original proposal; if the variable reference begins with '$(%' then the string is subjected to standard url decoding, which is well defined to produce "Hello, World!".

WRT special functions I think the choices are pretty clear. Given that %3A is a colon:

$(call %3A) returns :
$(value %3A) returns :
$(origin %3A) returns default
$(flavor %3A) returns simple

In essence $(%2C) is defined to behave exactly as if the line

%2C := ,

had been seen (actually not quite, it would be in the list of predefined variables such that --no-builtin-variables would suppress the feature).

BTW see <http://www.cmcrossroads.com/ask-mr-make/8442-gnu-make-escaping-a-walk-on-the-wild-side> for some background on the problem.

David Boyce <boyski>
Sat 28 Jan 2012 08:06:17 PM UTC, comment #5:

Well, when saying "altering the basic syntax" I mean changing the semantics of '$(...)' notation, which currently stands for expanding a variable reference or invoking a built-in function.

Personally for me it is unclear what '$(%20)' or '$(.%20)' means. If we perceive it just as a variable reference, then what would '$(value %20)' or '$(call %20)' return? I guess, it still should be a space. Also, what's about computed references, like '$(%$(twenty))', or the one I told about in my previous comment? In such case the problem can't be solved only by the tokenizer and requires '%XX' special variables to be defined.

Finally, encoding/decoding multiple characters at once becomes a pain. For example, how do I properly unescape 'Hello%2C%20world%21' string (assuming it is computed somewhere, that is not literal)?

However, I agree, that Paul is to make the final decision.

Eldar Abusalimov <abusalimov>
Sat 28 Jan 2012 06:40:54 PM UTC, comment #4:

I don't agree with this, and I don't know what you mean about altering the basic syntax. The original proposal makes no change to syntax (although I see that I used the word incorrectly in the subject line), it adds a new range of built-in variables. But Paul will make the final call.

David Boyce <boyski>
Sat 28 Jan 2012 05:39:36 PM UTC, comment #3:

I would vote up for just adding two new functions, i.e. `url-encode` and `url-decode`, without altering the basic syntax. Both functions are simple enough to implement.

Thus, the suggested usage is the following:

The argument can be, of course, computed as usual:

Eldar Abusalimov <abusalimov>
Tue 17 Jan 2012 05:05:38 AM UTC, comment #2:

According to RFC 1738 (http://www.ietf.org/rfc/rfc1738.txt):

... octets may be encoded by a character triplet consisting of the character "%" followed by the two hexadecimal digits (from "0123456789ABCDEF") which forming the hexadecimal value of the octet. (The characters "abcdef" may also be used in hexadecimal encodings.)

Therefore I believe URL encoding aka percent encoding requires all three characters, and thus omitting a leading 0 would be disallowed by definition.

Though the unreserved set seems pretty inert wrt make, I still think this feature should not be limited to the restricted set. Its main value IMO is that while currently there's one trick for getting a space past the makefile parser and another for backslashes and another for colons etc, the percent-encoding scheme is regular and predictable. A user could easily write a script to convert a string to its make-percent-encoded equivalent. Having to consider each character separately to determine whether its a "may", "must", or "must not" would bring back the pain of special casing.

Although makefiles are of course not URLs, the practice of allowing any octet to be encoded has always been tolerated. From RFC 1738 again:

"... characters that are not required to be encoded (including alphanumerics) may be encoded ..."

WRT when the expansion is done, I'm not well enough versed in the intricacies to have an opinion.

David Boyce <boyski>
Mon 16 Jan 2012 08:01:47 PM UTC, comment #1:

A useful idea. FMI:

http://en.wikipedia.org/wiki/Percent-encoding

Note we already have a $% variable (for archive members) and their associated $(%D) and $(%F) (which are both legal hex values BTW).

So we will have to decide whether we'd prefer the "uncleanliness of $(.%<token>), or the potential backward-compatibility issue with $(%xx)... as long as we require two digits and don't allow omitting of the leading 0.

Also we should consider whether this encoding should be limited to the strict URL restricted character set, or whether it will extend to any legal UTF-8 character with code <128.

We'd need to decide when the expansion is done. Is it done using normal expansion rules? Or is it done very early by the makefile parser?

Paul D. Smith <psmith>
Project Administrator
Wed 14 Dec 2011 02:27:53 AM UTC, original submission:

This idea seems too obvious to not have been brought up before but I haven't seen it and it seems both helpful and easy to implement so here goes. There's a lot of clever trickery around for inserting special characters in make syntax, e.g. the famous

_empty :=
_space := $(_empty) $(_empty)

So why not support a special syntax for "predefined" variables which uses URL encoding? With this feature, a space would be $(%20) and a backslash $(%5C), etc. The "." namespace is already reserved for internal variables, e.g. $(.DEFAULT_GOAL), so the same could be done for the "%" namespace with a low likelihood of breaking anything. Or, to be perfectly safe, it could be placed in the "." namespace as in $(.%20).

Just to be clear, what I'm suggesting here would be a simple token replacement which requires no actual use of the symbol table. Whenever the sequence "$(%XX)" is encountered while doing variable evaluation it's replaced by its character equivalent.

The primary use would be for single characters but I see no reason url-encoded strings couldn't be allowed. As long as the variable string starts with % (or .%), the whole thing is replaced by its url-decoded equivalent.

David Boyce <boyski>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by abusalimov (Posted a comment)
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by boyski (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 16 Jan 2012 08:01:47 PM UTCpsmithTriage StatusNone=>Medium Effort

    Back to the top


    Powered by Savane 3.1-cleanup