bugGNU Screen - Bugs: bug #51890, screen randomly injects \b into...

 
 

bug #51890: screen randomly injects \b into UTF8 streams when processing combining characters

Submitted by:  Mike Frysinger <vapier>
Submitted on:  Tue 29 Aug 2017 09:50:37 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Open/Closed: OpenRelease: 4.5.0
Fixed Release: NonePlanned Release: None
Work Required: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed 20 Sep 2017 08:48:46 AM UTC, comment #3:

Is this system is "weird" in any way? Some unusual kernel/libc?

I don't recall anything that would cause inserting '\b' bytes into a stream...

but it sounds bit similar to https://savannah.gnu.org/bugs/?31336

Amadeusz Sławiński <amade>
Project Administrator
Sun 03 Sep 2017 11:29:25 PM UTC, comment #2:

i tried a few diff terminal emulators and observed the same behavior

on a completely different system i tried screen 4.6.1 and it did not insert \b into the stream. not sure if it's because of the upgrade or because of some other system aspect.

i guess if you know of a change that went in to change this behavior, it's fine to punt the bug.

Mike Frysinger <vapier>
Wed 30 Aug 2017 07:39:37 AM UTC, comment #1:

Hi Mike,
thanks for bugreport.

Do you use xterm?
Can you try to reproduce it in 4.6.1 please?

The output of your example of printf-string looks same for me in xterm and in screen.

Alexander Naumov <anaumov>
Project Administrator
Tue 29 Aug 2017 09:50:37 PM UTC, original submission:

simple example:
printf 'xA\U0000030Ax\n'

that will write out the UTF-8 byte stream (hexdump view):
78 41 cc 8a 78 0a |xA..x.|

when i'm not using screen, the terminal emulator sees that exactly. however, screen will read that and then mangle it, passing along:
xA\bA\U0000030Ax\n
78 41 08 41 cc 8a 78 0a |xA.A..x.|

a proper terminal emulator is able to deal with this. but the question still stands: why is it doing this ? i couldn't locate the logic in the screen source though.

poking it through strace shows the screen process doing the read() on the pty master (/dev/ptmx) and getting the correct UTF-8 stream, then doing a write on its slave pty with the mangled stream. so it doesn't seem like it's an external-to-screen mangling.

my locale is set to en_US.UTF8, screen was launched with -U, and .screenrc has:
defutf8 on
defencoding utf8
using screen 4.05.00

since the whole pipeline is UTF-8 aware, i can't explain why screen would need to interject these things. i might understand if it was dealing with some semi-broken systems where it tried to get slightly better output, but that doesn't apply here.

the odd thing is that when screen dumps lines from its history (e.g. when you attach or otherwise scrollback), it doesn't inject the \b logic. only for new content.

noticed originally with a bit more pathological line:
1.001.01a अ॒ग्निमी॑ळे पु॒रोहि॑तं य॒ज्ञस्य॑ दे॒वमृ॒त्विज॑म् ।
that inserts a number of \b (all around combining chars? didn't look super close).

Mike Frysinger <vapier>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by amade (Posted a comment)
  • -unavailable- added by anaumov (Posted a comment)
  • -unavailable- added by vapier (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup1