bugGNU nano - Bugs: bug #30397, The whitespace setting warns when...

 
 

bug #30397: The whitespace setting warns when two UTF-8 characters are used in a non-utf8 terminal.

Submitted by:  Dan Mahoney <gushi>
Submitted on:  Fri 09 Jul 2010 03:20:11 AM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Benno Schulenberg <bens>
Open/Closed: ClosedRelease: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 10 Jun 2014 08:14:18 PM UTC, comment #10:

Considering this fixed.

Benno Schulenberg <bens>
Project MemberIn charge of this item.
Wed 26 Mar 2014 12:54:25 PM UTC, comment #9:

Are you content with the locale-dependent whitespace-character seeting that is now in SVN? Will it do the job for you?

Benno Schulenberg <bens>
Project MemberIn charge of this item.
Wed 26 Feb 2014 07:12:12 PM UTC, comment #8:

Actually -- I have a better answer than locale-specific whitespace options, which might be easier for me to implement, which is this:

Add a config file option called safe-whitespace or something like that, that's basically tried if the error-condition hits, the idea being you can set your normal whitespace to whatever you like, but you really should set safe-whitespace to something pure 7-bit-ascii.

As a bonus, I don't have to change the way the current whitespace option is parsed.

Whether it's accepted or not as a patch, I think it'll be fun to try to make work.

Dan Mahoney <gushi>
Wed 26 Feb 2014 09:14:06 AM UTC, comment #7:

I don't know how hard it would be to implement -- I am not a C coder either. :)

Whether a patch for a locale-specific nanorc or whitespace option would be accepted, I don't know either -- I am not the main nano maintainer. Personally, I wouldn't accept it. It is not nano's business to deal with situations where people use multiple encodings. This whole idea of having different encodings is a nightmare from the past and we should leave it behind.

(By the way, the default example /etc/nanorc does not contain any UTF-8 characters, it only suggests to use them in the comments. The default example contains two spaces, which I think... is entirely pointless, so I'm going to change that.)

Benno Schulenberg <bens>
Project MemberIn charge of this item.
Tue 25 Feb 2014 07:58:44 PM UTC, comment #6:

Right -- my question was as to how hard this would be to implement the way I asked. I don't have to do magic in /etc/profile (or the other various config files for the other shells my users may use) for any other piece of software currently -- admittedly, no other programs have utr-8 values in their config files.

I don't tend to tweak binaries on my systems -- I tend to copy config files around. I really don't want to have to make a bespoke "nano" package to accomplish this.

I'm not a C-coder, but I'm attempting to read the source now, to see how hard this might be. The source looks really clean and readable, for whatever that's worth.

If I can come up with something clean, is there a chance it'll be accepted?

...where'd I put my copy of K&R...?

-Dan

Dan Mahoney <gushi>
Tue 25 Feb 2014 04:56:46 PM UTC, comment #5:

As a site admin, can't you do some magic in maybe /etc/profile and "redirect" things based on the locale of the user?

Failing that, how about tweaking the site nano in src/rcfile, around line 1265:

if LC_MESSAGES ends with ".utf8"
then use SYSCONFDIR "/nanorc.utf8"
else use SYSCONFDIR "/nanorc.ascii"

(About the "better" characters not being confusable with anything one can type: I can type « and », and use them regularly.)

Benno Schulenberg <bens>
Project MemberIn charge of this item.
Mon 24 Feb 2014 06:47:47 PM UTC, comment #4:

My workaround has been to put the pure-ascii characters as a global, and then override them locally, but as the site admin, I'd like it if others using pure UTF-8 (i.e. anyone on a mac or anyone on ubuntu's default setup) could see the "better" characters, as they're not confusable with something you can type.

My suggestion for the scope of this bug was to have the "set" command be conditional:

set whitespace UTF-8 "xy" <-- this one would apply in the given encoding

set whitespace "ab" <-- or would fall back to this one, which would make it backwards compatible with existing config files.

How hard would this be to make happen?

Dan Mahoney <gushi>
Mon 24 Feb 2014 12:53:46 PM UTC, comment #3:

Yes, well, if you use UTF-8 on a non-UTF terminal, you get misrepresentations, that is not nano's fault. If you wish whitespace representation to work across all encodings, you will have to choose plain ASCII characters -- maybe ":.".

With nano throwing up a warning you mean the "Error in ~/.nanorc on line x: Two single-column characters required" when you have two UTF chars for whitespace? Should that warning be made more clear? "Your current locale requires two single-column characters here"?

Benno Schulenberg <bens>
Project MemberIn charge of this item.
Sun 23 Feb 2014 09:10:26 PM UTC, comment #2:

Sorry, please ignore that last comment, it was for another bug.

-Dan

Dan Mahoney <gushi>
Sun 23 Feb 2014 09:09:06 PM UTC, comment #1:

Okay, so this is under FreeBSD, which is termcap, not terminfo based.

It's my understanding that screen does some magic to translate the actual terminal it's running on and present that to an application in the form of the TERMCAP environment variable.

All I know is that with term type "screen" set, mouse doesn't work, if I do a setenv TERM xterm, it does. If gnu screen's not doing some special magic to look at that name directly, then the answer pretty much has to be here:

It also sets TERM, which should cause nano to use whatever's in /etc/termcap.db

I can paste in what's in "env" and what's in the requisite files:

TERM=screen
TERMCAP=SC|screen|VT 100/ANSI X3.64 virtual terminal:\
:DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:bs:bt=\E[Z:\
:cd=\E[J:ce=\E[K:cl=\E[H\E[J:cm=\E[%i%d;%dH:ct=\E[3g:\
:do=^J:nd=\E[C:pt:rc=\E8:rs=\Ec:sc=\E7:st=\EH:up=\EM:\
:le=^H:bl=^G:cr=^M:it#8:ho=\E[H:nw=\EE:ta=^I:is=\E)0:\
:li#33:co#169:am:xn:xv:LP:sr=\EM:al=\E[L:AL=\E[%dL:\
:cs=\E[%i%d;%dr:dl=\E[M:DL=\E[%dM:dc=\E[P:DC=\E[%dP:\
:im=\E[4h:ei=\E[4l:mi:IC=\E[%d@:ks=\E[?1h\E=:\
:ke=\E[?1l\E>:vi=\E[?25l:ve=\E[34h\E[?25h:vs=\E[34l:\
:ti=\E[?1049h:te=\E[?1049l:us=\E[4m:ue=\E[24m:so=\E[3m:\
:se=\E[23m:md=\E[1m:mr=\E[7m:me=\E[m:ms:\
:Co#8:pa#64:AF=\E[3%dm:AB=\E[4%dm:op=\E[39;49m:AX:G0:\
:as=\E(0:ae=\E(B:\
:ac=\140\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\
:po=\E[5i:pf=\E[4i:k0=\E[10~:k1=\EOP:k2=\EOQ:k3=\EOR:\
:k4=\EOS:k5=\E[15~:k6=\E[17~:k7=\E[18~:k8=\E[19~:\
:k9=\E[20~:k;=\E[21~:F1=\E[23~:F2=\E[24~:kb=^H:kh=\E[1~:\
:@1=\E[1~:kH=\E[4~:@7=\E[4~:kN=\E[6~:kP=\E[5~:kI=\E[2~:\
:kD=\E[3~:ku=\EOA:kd=\EOB:kr=\EOC:kl=\EOD:km:

From /etc/termcap:

SC|screen|VT 100/ANSI X3.64 virtual terminal:\
:am:xn:ms:mi:G0:km:\
:DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:bs:bt=\E[Z:\
:cb=\E[1K:cd=\E[J:ce=\E[K:cl=\E[H\E[J:cm=\E[%i%d;%dH:ct=\E[3g:\
:do=^J:nd=\E[C:pt:rc=\E8:rs=\Ec:sc=\E7:st=\EH:up=\EM:\
:le=^H:bl=^G:cr=^M:it#8:ho=\E[H:nw=\EE:ta=^I:is=\E)0:\
:li#24:co#80:us=\E[4m:ue=\E[24m:so=\E[3m:se=\E[23m:\
:mb=\E[5m:md=\E[1m:mr=\E[7m:me=\E[m:sr=\EM:al=\E[L:\
:AL=\E[%dL:dl=\E[M:DL=\E[%dM:cs=\E[%i%d;%dr:dc=\E[P:\
:DC=\E[%dP:im=\E[4h:ei=\E[4l:IC=\E[%d@:\
:ks=\E[?1h\E=:ke=\E[?1l\E>:vb=\Eg:\
:ku=\EOA:kd=\EOB:kr=\EOC:kl=\EOD:kb=^H:\
:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:k5=\E[15~:k6=\E[17~:\
:k7=\E[18~:k8=\E[19~:k9=\E[20~:k;=\E[21~:F1=\E[23~:F2=\E[24~:\
:F3=\E[25~:F4=\E[26~:F5=\E[28~:F6=\E[29~:\
:F7=\E[31~:F8=\E[32~:F9=\E[33~:FA=\E[34~:\
:kh=\E[1~:kI=\E[2~:kD=\E[3~:@7=\E[4~:kP=\E[5~:\
:kN=\E[6~:eA=\E(B\E)0:as=^N:ae=^O:ti=\E[?1049h:te=\E[?1049l:\
:vi=\E[?25l:ve=\E[34h\E[?25h:vs=\E[34l:\
:Co#8:pa#64:AF=\E[3%dm:AB=\E[4%dm:op=\E[39;49m:AX:\
:ac=``aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:

...and the chain that builds the xterm here.

xterm|xterm-color|X11 terminal emulator:\
:ti@:te@:tc=xterm-xfree86:

xterm-xfree86|XFree86 xterm:\
:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:\
:k5=\E[15~:k6=\E[17~:k7=\E[18~:k8=\E[19~:\
:k9=\E[20~:k;=\E[21~:F1=\E[23~:F2=\E[24~:\
:@7=\EOF:@8=\EOM:kI=\E[2~:\
:kh=\EOH:kP=\E[5~:kN=\E[6~:\
:ku=\EOA:kd=\EOB:kr=\EOC:kl=\EOD:Km=\E[M:tc=xterm-basic:
#
# This chunk is used for building the VT220/Sun/PC keyboard variants.
xterm-basic|xterm common (XFree86):\
:li#24:co#80:am:kn#12:km:mi:ms:xn:AX:bl=^G:\
:is=\E[!p\E[?3;4l\E[4l\E>:rs=\E[!p\E[?3;4l\E[4l\E>:le=^H:\
:AL=\E[%dL:DL=\E[%dM:DC=\E[%dP:al=\E[L:dc=\E[P:dl=\E[M:\
:UP=\E[%dA:DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:\
:ho=\E[H:cd=\E[J:ce=\E[K:cl=\E[H\E[2J:cm=\E[%i%d;%dH:cs=\E[%i%d;%dr:\
:im=\E[4h:ei=\E[4l:ks=\E[?1h\E=:ke=\E[?1l\E>:kD=\E[3~:kb=^H:\
:sf=\n:sr=\EM:st=\EH:ct=\E[3g:sc=\E7:rc=\E8:\
:eA=\E(B\E)0:as=\E(0:ae=\E(B:ml=\El:mu=\Em:up=\E[A:nd=\E[C:\
:md=\E[1m:me=\E[m:mr=\E[7m:so=\E[7m:se=\E[27m:us=\E[4m:ue=\E[24m:\
:ti=\E[?1049h:te=\E[?1049l:vi=\E[?25l:ve=\E[?25h:\
:ut:Co#8:pa#64:op=\E[39;49m:AB=\E[4%dm:AF=\E[3%dm:

Best,

Dan

Dan Mahoney <gushi>
Fri 09 Jul 2010 03:20:11 AM UTC, original submission:

I have a UTF-8 terminal, and have put in my global nanorc:

set whitespace "»·", which were pasted in from doc samples.

Now, if you're on UTF8, you will see, essentially:

">>" and "." (the . would be at mid-line height). Both of these would be a single character wide.

If, however, you're on some OTHER terminal type, you see this:

http://www.gushi.org/nano_bug/nano_stuff.png

(as reported by a user)

However, nano throws up a warning that it can't parse that line because at that point it's 4 chars wide.

I might suggest that the "set whitespace" warning needs to be modified such that it is encoding matched.

set whitespace UTF-8 "xy"
set whitespace "ab"

Dan Mahoney <gushi>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by bens (Posted a comment)
  • -unavailable- added by gushi (Submitted the item)
  • -unavailable- added by gushi (Original Reporter)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 10 Jun 2014 08:14:18 PM UTCbensStatusNeed Info=>Fixed
      Open/ClosedOpen=>Closed
    Wed 26 Mar 2014 12:54:25 PM UTCbensStatusNone=>Need Info
      Assigned toNone=>bens
    Tue 25 Feb 2014 04:56:46 PM UTCbensSummaryNano\'s whitespace handling throws errors when two UTF-8 characters are used in a non-utf8 terminal.=>The whitespace setting warns when two UTF-8 characters are used in a non-utf8 terminal.
    Fri 09 Jul 2010 03:20:11 AM UTCgushiCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup