mainThe GNU Bourne-Again SHell - Support: sr #110830, Do not pass through non-charset...


sr #110830: Do not pass through non-charset and non-printable chars unescaped

Submitter:  Cedric <manday>
Submitted:  Wed 01 Feb 2023 04:31:48 PM UTC
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  In Progress
Privacy:  Public Assigned to:  None
Open/Closed:  Open Operating System:  None
* Mandatory Fields

Add a New Comment Rich Markup

Jump to the original submission

Wed 01 Mar 2023 12:57:27 PM UTC, comment #6: 

Given what is currently the problem, I suppose "val & 1<<7 || 31 < val < 127" is not enough of an indicator. Generally and including possible cases of Unicode, I'd assume that the locale tells a program which encoding is used and thus which codepoints are valid "text".

Isn't there already a library/mechanism which simply answers the question whether a codepoint is text? How do programs like `ls` (at least my version of it) do it? (partly rhetoric question, if I get around to it I will check it myself)

Cedric <manday>
Mon 27 Feb 2023 07:38:30 PM UTC, comment #5: 

This behavior is controlled by readline's `output-meta' variable. If that's enabled, readline displays characters with the eighth bit set by directly sending them to the output stream.

You're suggesting that the behavior be changed so that there's some sort of `hybrid' case: printable characters with the eighth bit set are displayed as-is, while non-printable ones are displayed as an octal escape sequence.

Does this capture your request?

Chet Ramey <chet>
Group administrator
Sat 04 Feb 2023 07:13:05 PM UTC, comment #4: 

(concerning their presentation by a shell)

Cedric <manday>
Sat 04 Feb 2023 07:11:59 PM UTC, comment #3: 

What I meant was more in the spirit of "filenames should let the user discern files by distiguishable text" as opposed to "... discern files by different ways of how they make the terminal blink and move the cursor around", if you get my meaning. That's what I meant by their semantics are "textual".

Cedric <manday>
Sat 04 Feb 2023 06:08:11 PM UTC, comment #2: 

Before I look at this, I'll note your incorrect assumption: filenames are arbitrary byte strings that do not include '/' or NUL. There are no other semantics, textual or otherwise, associated with them.

Chet Ramey <chet>
Group administrator
Wed 01 Feb 2023 04:34:18 PM UTC, comment #1: 

I should clarify that `echo` was perhaps not the best example here, as it may be expected to print characters as is. The better (original) situation is Tab-Completion:


prints the control sequences onto the prompt and thereby makes in unusable.

Cedric <manday>
Wed 01 Feb 2023 04:31:48 PM UTC, original submission:  

Rationale: Some sources of text which are reproduced by bash's output can not be assumed well-formed or safe.

Notably, filenames may contain non-textual (and thus, semantically invalid, as filenames are supposed to be textual) characters which must not be printed back to the terminal as-is.

If there is no way to infer the set of textually valid characters with certainty (e.g. from $TERM), a conservative guess must be made as to which are non-textual characters (with other semantics than text, such as terminal control) and the latter must be escaped.

See for example the behaviour of some implementations of `ls` or `file` and compare that to bash:

$ touch 'ph+'$'\256''nom+'$'\277''ne'
$ ls
$ file ph*
ph+\256nom+\277ne:  empty
$ echo ph*

where in bash's output the invalid charaacters \256 and \277 are printed as-is and can affect the terminal (e.g. alacritty).

Cedric <manday>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by chet (Posted a comment)
  • -email is unavailable- added by manday (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.


    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-02-04 chet StatusNone In Progress

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code