mainThe GNU Bourne-Again SHell - Support: sr #110686, Excution of pasted text despite...

 
 

sr #110686: Excution of pasted text despite "bracketed paste protocol"

Submitter:  Holger <private_lock>
Submitted:  Mon 11 Jul 2022 07:57:58 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Wont Do
Privacy:  Public Assigned to:  None
Open/Closed:  Open Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 02 Aug 2022 09:48:43 PM UTC, comment #8: 

comment #7:

> > My idea is to transition into a better world, where bracketed paste would be
> > the only way of pasting.
>
> I'm simply not interested in this. It would not be useful, and would be extremely user-hostile. I can't see cat, or sed, or
> awk being changed to understand the bracketed-paste escape
> sequences or to read input a character at a time. I can't imagine
> a bigger insult to users than to tell them that suddenly they
> can't continue to use the workflow they're used to because the
> programs in it don't understand the new-style pasting, and there
> is no other option.


I acknowledge your point of view and don't see, how to proceed. Therefore, I suggest to close this bug as "Won't fix". Thank you for the time and effort to explain all this to me.

Holger <private_lock>
Tue 02 Aug 2022 07:11:16 PM UTC, comment #7: 


> That was not my concern - if a program acts poorly, we should report bugs
> against that program. But of course, it would be nice, if the underlying web
> of bash + Konsole could handle such cases gracefully - e.g. by discarding the
> rest of the half-eaten input. You'd not go to a restaurant and eat what the
> other guests left on their plates :D


OK. I didn't address this case again because I thought we had
already resolved it.

This is a non-starter from a shell perspective: there is no way
to distinguish such input from typeahead the user intended to
direct to bash. I'm not interested in adding an option to flush
input when calling readline; I don't think that's productive.

Any option to flush unconsumed input from the pty when a
foreground process exits would have to come from the terminal
emulator side, and it's very difficult to do there, since the
terminal emulator process is not the parent of the foreground
process in question and would not be notified of its
termination.

> My idea is to transition into a better world, where bracketed paste would be
> the only way of pasting.


I'm simply not interested in this. It would not be useful, and would be extremely user-hostile. I can't see cat, or sed, or
awk being changed to understand the bracketed-paste escape
sequences or to read input a character at a time. I can't imagine
a bigger insult to users than to tell them that suddenly they
can't continue to use the workflow they're used to because the
programs in it don't understand the new-style pasting, and there
is no other option.

> The protocol of bracketed paste and the evolution of it happens in the
> interface between Konsole and bash. Neither of you can change it onesided.


Surely you don't think Konsole is the only terminal emulator that
implements bracketed-paste mode. I don't even use Konsole.

Chet Ramey <chet>
Group administrator
Mon 01 Aug 2022 08:57:59 PM UTC, comment #6: 

English is not my first language ... but I try again :D

comment #5:

> a program that enables bracketed-paste mode but doesn't read all of its input before exiting.


That was not my concern - if a program acts poorly, we should report bugs against that program. But of course, it would be nice, if the underlying web of bash + Konsole could handle such cases gracefully - e.g. by discarding the rest of the half-eaten input. You'd not go to a restaurant and eat what the other guests left on their plates :D

> there's no solution that will be right all the time.


Full concurrence!

> If you're arguing against the entire concept of bracketed paste,


Nope, not at all against it - in fact, I'd like to extend it.

My idea is to transition into a better world, where bracketed paste would be the only way of pasting. The "strict mode" could simulate this and see how far you get, just like browsers were experimenting with https for many years until it got enough momentum to take over the web.

Maybe a convenience feature of blacklisting / whitelisting programs could help with starting to spread the use. Actually programs, that accept bracketed pastes are already advertising this, right? So this would be a starting-point of a whitelist.

> You'd have to ask the folks who implement terminal emulators, or
> who maintain libraries used by terminal emulators. I don't really
> pay attention to that.


Well, they closed my report immediately:
https://bugs.kde.org/show_bug.cgi?id=456373

The protocol of bracketed paste and the evolution of it happens in the interface between Konsole and bash. Neither of you can change it onesided.

Holger <private_lock>
Mon 01 Aug 2022 03:14:01 PM UTC, comment #5: 


> Yes, it removes the ambiguity of "pasting vs. typing" into bash directly by
> introducing a fresh inconsistency of treating pasted text differently, whether
> or not it is inserted before the current command exits.


This doesn't make sense, or it's phrased poorly. The trouble
scenario you're using, as I understand it, is a program that enables bracketed-paste mode but doesn't read all of its input
before exiting. That is indeed a problem, because there might
not be a program that understands bracketed paste available to
read that input, but it's a problem that a terminal emulator
will have to solve, and there's no solution that will be right
all the time.

The `issue' of whether or not text is pasted and not read before
a foreground command exits is not new.

If you're arguing against the entire concept of bracketed paste,
since that's what `introduces a fresh inconsistency', I'm not
interested in having that discussion.

> Then why not make an effort, to extend this protection to text pasted while
> another program is executing?


The current state of the art requires every program to be
modified.

> Just a little though experiment - would it be possible, to get a terminal in
> "strict" mode, that only accepts bracketed pastes? Programs, that don't
> implement the protocol will never see any input. And the paste will either be
> rejected (e.g. with a beep) or be delayed until some program offers, to accept
> pasted text in bracketed mode. They do register in a way, so that Konsole
> knows, to wrap the Escape sequences or not, right?


You'd have to ask the folks who implement terminal emulators, or
who maintain libraries used by terminal emulators. I don't really
pay attention to that.

If you want terminals to reject pastes to programs that don't
enabled bracketed paste mode, that seems like a pretty user-
hostile change, even if you make it a non-default option a user
has to enable.

> Question is, if this would gather enough momentum, to get programmers in
> rewriting their programs/commands to accept bracketed paste - or if such a
> feature would be disabled by default and never used again?


My guess is that programs that read their input character by
character and expect to read from a terminal might change, and
programs that simply read standard input will never change (and
should not).

> For the whole bracketed paste feature, I only learned about it, because it was
> "forced" on me be some default switch. Yes, I got confused by it at first and
> I would have loved, to get some information dialog and choice upfront, instead
> of just dumping it on me.


This sounds like something you should take up with your vendor.


Chet Ramey <chet>
Group administrator
Sun 31 Jul 2022 09:10:12 PM UTC, comment #4: 

comment #3:

> Bracketed-paste mode was invented to remove ambiguity: to signal
> that text is explicitly the result of a paste and not the result
> of typed input. Programs generally react to that as readline does,
> by disabling any special meaning attached to the characters.


Yes, it removes the ambiguity of "pasting vs. typing" into bash directly by introducing a fresh inconsistency of treating pasted text differently, whether or not it is inserted before the current command exits.

> So yes, there is a security consideration there: how can we
> paste something and ensure that pasting it will have no side
> effects?


Then why not make an effort, to extend this protection to text pasted while another program is executing?

Just a little though experiment - would it be possible, to get a terminal in "strict" mode, that only accepts bracketed pastes? Programs, that don't implement the protocol will never see any input. And the paste will either be rejected (e.g. with a beep) or be delayed until some program offers, to accept pasted text in bracketed mode. They do register in a way, so that Konsole knows, to wrap the Escape sequences or not, right?

Question is, if this would gather enough momentum, to get programmers in rewriting their programs/commands to accept bracketed paste - or if such a feature would be disabled by default and never used again?

For the whole bracketed paste feature, I only learned about it, because it was "forced" on me be some default switch. Yes, I got confused by it at first and I would have loved, to get some information dialog and choice upfront, instead of just dumping it on me. Nevertheless, I got fond of it and it already helped me when trying to paste a command and accidentally pasting the line-break as well, even though I intended to edit it further before execution.

Holger <private_lock>
Fri 15 Jul 2022 07:29:20 PM UTC, comment #3: 


> OK, I see, we shall not discard what someone typed while e.g.
> sleep 5 is executing - that's a valid point.


That's a user expectation. (And the expectation of someone who
types ahead while executing any program -- like glxgears --
that doesn't enable bracketed paste mode.)

> Only now I am confused, what was the purpose of inventing the
> "bracketed paste protocol"? Is it just meant to be a "best-
> effort-don't-shoot-your-foot" or is there some real security 
> consideration behind it?


Bracketed-paste mode was invented to remove ambiguity: to signal
that text is explicitly the result of a paste and not the result
of typed input. Programs generally react to that as readline does,
by disabling any special meaning attached to the characters.

So yes, there is a security consideration there: how can we
paste something and ensure that pasting it will have no side
effects?

But you're correct: programs have to explicitly enable it or
be prepared to understand the bracketed-paste prefix and suffix,
or both.

> So the ambiguity is here:
> a) run input accepting program -> text paste is expected to
> complete immediately


Not necessarily. Programs read as much input as they need to do
their work. POSIX explicitly specifies that the shell, for
example, only reads as much input as it needs to parse a complete
command, leaving other data on standard input for a program it
invokes.

However, if a program enables bracketed-paste mode, it should
read all of the text between the bracketed paste prefix and
suffix, to avoid confusing other programs with the escape
sequences.

And usually, it works as you expect: programs read and process
their input until they get an EOF.

> b) run non-input-program -> pasted text shall be retained, so
> it can be treated as if pasted directly into bash WITHOUT
> automatic execution, prompting for another press of Enter. The
> pasted text becomes indistinguishable from any other keyboard-
> input.


No. This is true of bracketed paste mode (and there are people
who have complained about that), but in general a newline that
appears in the input buffer is treated as a newline and has
whatever context-dependent meaning a newline has. That means if
the pasted text contains a newline, bash will read a newline and
execute the command it's reading (the most common case).

> How I understand it, Konsole as the GUI needs to finish the
> paste as it happens, and is forced to decide on a paste method
> (bracketed or not) BEFORE the program exits. There might even
> come a second paste after this - so waiting is not an option.


Konsole needs to decide, preferably via a user setting, how the
pasted text appears to the process on the other side of the pty.
If the user (or program, like bash), has enabled bracketed paste
mode, Konsole has to send the bracketed paste prefix, the text
to be pasted, and the bracketed paste suffix into the pty input
queue. If there's no bracketed paste mode in effect, it just
stuffs the text to be pasted into the queue.

Konsole doesn't know whether a program that is reading from the
other side of the pty exits -- that program is not a child of
Konsole, and, unless the program that exits is the shell, which
is, it won't know. If the shell exits, Konsole will be notified,
and can react appropriately by destroying the window, in which
case none of this matters.

So Konsole's primary responsibility here is to make the pasted text appear on the slave side of the pty's input queue.

> The critical moment is, when bash returns to power and has a
> look at what is waiting in the buffer, potentially pasted or
> typed. How hard would it be, to again put this in a linebuffer
> and ask the user to confirm again by pressing Enter or further > editing the line, as if he did not paste a line-break / press
> Enter before the blocking fg-program exited?


That would defeat user expectations. Users who do not use
bracketed paste mode expect newlines in pasted text to be treated
as newlines. Some intentionally disable bracketed paste mode
because it doesn't do this. Bash doesn't need to be second-
guessing the user like that: if you aren't using bracketed paste
mode and you don't want text you paste to be immediately accepted
and potentially executed, don't put a trailing newline in it.

> In my opinion, this would streamline the experience of pasting
> with an additional Enter (irrespective of program running or
> not) - but at the same time force an additional Enter when
> typing commands while a program is running.


It would do exactly the opposite. It would insert an extra step
that would make a process that has worked one way for a long time
more cumbersome.

> Also I marvel, if
> someone is relying on the current behavior e.g. by sending fake
> key-events asynchronously not waiting for the individual
> commands to finish. But then, shouldn't they just submit their
> input as a script file?


This happens more often than you think. To the user, they are not
`fake' key events: they are real keys that they pressed, and they
expect those key presses to be honored. And since this is
primarily an issue with interactive shells, retroactively making
something a script is not an attractive option.

Chet Ramey <chet>
Group administrator
Thu 14 Jul 2022 11:58:30 PM UTC, comment #2: 

OK, I see, we shall not discard what someone typed while e.g. sleep 5 is executing - that's a valid point.

Only now I am confused, what was the purpose of inventing the "bracketed paste protocol"? Is it just meant to be a "best-effort-don't-shoot-your-foot" or is there some real security consideration behind it?

So the ambiguity is here:
a) run input accepting program -> text paste is expected to complete immediately
b) run non-input-program -> pasted text shall be retained, so it can be treated as if pasted directly into bash WITHOUT automatic execution, prompting for another press of Enter. The pasted text becomes indistinguishable from any other keyboard-input.

How I understand it, Konsole as the GUI needs to finish the paste as it happens, and is forced to decide on a paste method (bracketed or not) BEFORE the program exits. There might even come a second paste after this - so waiting is not an option.

The critical moment is, when bash returns to power and has a look at what is waiting in the buffer, potentially pasted or typed. How hard would it be, to again put this in a linebuffer and ask the user to confirm again by pressing Enter or further editing the line, as if he did not paste a line-break / press Enter before the blocking fg-program exited?

In my opinion, this would streamline the experience of pasting with an additional Enter (irrespective of program running or not) - but at the same time force an additional Enter when typing commands while a program is running. Also I marvel, if someone is relying on the current behavior e.g. by sending fake key-events asynchronously not waiting for the individual commands to finish. But then, shouldn't they just submit their input as a script file?

Holger <private_lock>
Tue 12 Jul 2022 01:26:34 PM UTC, comment #1: 

So you're saying that there is input on the master side of the pty that bash reads when the child process exits? This is an indeterminate situation.

Flushing the input buffer defeats all typeahead. How is bash supposed to know whether the input it reads is intended or `left over' from a terminated child process? For instance, it's easy to type input while, say, `sleep 5' is running, and bash reads and executes the input when sleep exits. This is common and expected.

If someone wants this flush to happen, there should be a way to configure the terminal emulator to flush the input on the pty when a child process exits, but it should not be the default setting.

Chet Ramey <chet>
Group administrator
Mon 11 Jul 2022 07:57:58 PM UTC, original submission:  

Hi,

I reported this also against Konsole as:
https://bugs.kde.org/show_bug.cgi?id=456373

If I run a program like glxgears, it disables the "bracketed paste protocol" -> fine, either the program has the responsibility to handle the text - e.g. store it in a text file, if it is an editor, or it ignores the input completely.

But what happens, when the program exits and there is still input left in the buffers? Well it seems, bash takes over and happily executes, before any "bracketed paste protocol" can prevent execution.

Now, who is responsible for cleaning out the buffer to stop this? Is it bash (the shell) or Konsole (the terminal-GUI)?
Is there a valid use-case, where a program leaves some input in the buffers, to run AFTER it exits?

Holger <private_lock>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by chet (Posted a comment)
  • -email is unavailable- added by private_lock (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-08-03 chet StatusNeed Info Wont Do
    2022-07-12 chet StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code