bugGNU nano - Bugs: bug #58130, [Wish] extra syntax highlighting...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #58130: [Wish] extra syntax highlighting files -- Haskell and Markdown

Submitter:  Ryan Westlund <yujiri>
Submitted:  Mon 06 Apr 2020 09:08:00 PM UTC
   
 
Severity:  1 - Wish Status:  Fixed
Assigned to:  bens Open/Closed:  Closed

Jump to the original submission

Wed 16 Sep 2020 05:41:26 PM UTC, comment #34: 

Thanks for the confirmation.  Now waiting for Hussam to repost.

Benno Schulenberg <bens>
Group administrator
Mon 14 Sep 2020 08:02:30 PM UTC, comment #33: 

I agree that the new syntax bring improvements as Hussam mentioned, including the highlighting of "error", the fixes to comments and character literals and the separation of the contextual keywords "as", "qualified", and "hiding" which are not reserved as identifiers. I did approve of it. I didn't notice the problem with using white because I tested it on a black terminal.

Ryan Westlund <yujiri>
Mon 14 Sep 2020 07:22:42 AM UTC, comment #32: 

Ryan?  Do you think the newly proposed Haskell syntax is better than the current one?

Benno Schulenberg <bens>
Group administrator
Thu 30 Jul 2020 07:13:13 AM UTC, comment #31: 

Released in nano-5.0.

Benno Schulenberg <bens>
Group administrator
Tue 26 May 2020 03:00:00 PM UTC, comment #30: 

(Issues get closed when the relevant fixes/changes have been released.  That will still take some weeks.)

Benno Schulenberg <bens>
Group administrator
Mon 25 May 2020 02:12:47 PM UTC, comment #29: 

I believe this can be marked as closed now, given the last comment.

Ryan Westlund <yujiri>
Thu 14 May 2020 01:23:58 PM UTC, comment #28: 

If there are any other syntaxes you would like to see added in nano's tarball, then please post a new issue.

Benno Schulenberg <bens>
Group administrator
Tue 21 Apr 2020 10:05:21 AM UTC, comment #27: 

Thanks.  Pushed to master in commit 36e22c99, with a few tweaks.

I've credited only Alex Taber, as the contributions of the four others (including yourself) are not substantial changes but just a little polishing.  I've included the URL of the original file in the commit message, so people can easily find the full history, when interested.

Benno Schulenberg <bens>
Group administrator
Tue 21 Apr 2020 01:08:11 AM UTC, comment #26: 

Ah, I assumed the file was written by Scopatz. Having received the go-ahead from Alex Taber, I'll attach a new version with the undefined thing fixed.

I'm a little unsure about the credit at the top of the file, since Scopatz asked "include credit to me / the other authors", but it looks like none of the commits on the Haskell file were his.

(file #48888)

Ryan Westlund <yujiri>
Mon 20 Apr 2020 04:49:24 PM UTC, comment #25: 

Thanks for the full patch.  However, when I look at the history of that haskell syntax file [1], it was authored by astronautlevel2 (Alex Taber).  So, we will have to ask him directly whether it is okay if we include his file into nano, under the GPL version 3.  Please send him an email (aft.pokemon (at) gmail.com) and CC me.

[1]https://github.com/scopatz/nanorc/commit/f0db3a7bbb7b26f16cb3f9f2fa230460d96acc64#diff-6470a046c397df010366b3eacdbcbec2

Further, I don't think we want to color the word "undefined" inside strings either, so the rule has to move up some more.

Benno Schulenberg <bens>
Group administrator
Mon 20 Apr 2020 02:26:06 PM UTC, comment #24: 

Ah, thanks. Attaching patch file.

(file #48880)

Ryan Westlund <yujiri>
Mon 20 Apr 2020 11:48:27 AM UTC, comment #23: 

Thanks.  I've changed also the regex for strikethrough, to avoid coloring the text between two stricken-through words.  Pushe to master in commit c859fd40.

For the haskell file, please provide a proper patch: the "replica" of a commit.  Produce it like this:


git checkout -b with-haskell
git commit -s syntax/extra/haskell.nanorc
git format-patch master


Benno Schulenberg <bens>
Group administrator
Sun 19 Apr 2020 01:51:57 PM UTC, comment #22: 

Ah. I should've tested with text.md.

Yeah, your version looks good. I'll attach a new patch.

(file #48870)

Ryan Westlund <yujiri>
Sun 19 Apr 2020 09:05:59 AM UTC, comment #21: 

Your description of what is needed is good: forbid whitespace immediately after the leading asterisk.  But your implementation forbids /any/ space between the asterisks.  Look at text.md and see that This text will be italic is not green.

This seems to work well:

color green "\*[^* \t][^*]*\*|_[^_ \t][^_]*_"

(where the \t is a literal tab).

Benno Schulenberg <bens>
Group administrator
Fri 17 Apr 2020 07:43:17 PM UTC, comment #20: 

Ah, indeed! Requiring not a space after the opening emphasis character for italics also solves the problem of `**bold** normal *bold*` coloring the text intended as normal as italics. I had assumed that was unsolvable, like the similar one with Python or Javascript strings, but this seems to fix it without causing additional issues. I'm going to attach a patch that has this change applied.

(file #48861)

Ryan Westlund <yujiri>
Fri 17 Apr 2020 07:31:12 PM UTC, comment #19: 

Ah, I'm surprised I didn't notice the issue with list items and italics. I wonder if changing the italics regex from `\*[^*]+\*|_[^_]+_` to `\*[^* ]+\*|_[^_]+_` would work, making it not match if there's a space immediately after the first asterisk.

Ryan Westlund <yujiri>
Fri 17 Apr 2020 05:42:43 PM UTC, comment #18: 

My criterium for adding syntaxes is roughly: files to which the syntax applies can be found on an average default install of a Unixy system or in an average software package.  Man files are there for sure, and some people edit them.  PO files are in many tarballs, and translators want to edit them.  On my system there are some awk and tcl files too.  But indeed, I would think that asm, fortran, and groff files are rare enough that those syntaxes can be pushed down to extra/.

I had added the ^ for symmetry: if a fenced block needs to end at EOL then it seemed logical that it must also start at BOL.

I've looked at some of the .md files on my system and found two problems.  I attach the two files.  In the first, the rule for italics sees the * of a bullet item erroneously as the start of italics.  In the second, the sloppy continuation line of an indented bullet item (after Method 1 and Method 2) is colored as if it were code.  This latter problem seems hard or impossible to fix -- the writer of the text should just indent the line a bit more, to line up with the text above it.  But for the first there ought to be a solution, but I think it will become ugly.

(file #48859, file #48860)

Benno Schulenberg <bens>
Group administrator
Fri 17 Apr 2020 02:25:23 PM UTC, comment #17: 

Patch files for Markdown and Haskell added. Good catch noticing my plusses were pointless. But it looks like you added a ^ for the starting regex for fenced code blocks in the last upload. I didn't include this in the patch because it meant in fenced code blocks that were also indented under a list item, content starting with > would be seen as a block quote (I found this because I'm using it for a Python REPL excerpt in a list) and colored as that.

As for syntaxes that could go in extra/: nftables? Maybe the tex ones? Honestly, I feel quite a lot of these are more obscure (or at least less commonly edited as text files) than Haskell, including man, groff, asm, awk, po, tcl, email, fortran, and cmake. But maybe I think less of those things than most Nano users do. I'm kind of biased because I don't use any of those things and do use Haskell.

(file #48855, file #48856)

Ryan Westlund <yujiri>
Fri 17 Apr 2020 10:21:41 AM UTC, comment #16: 

(When improving a file, please use the last version that I posted, otherwise it's extra work for me to figure out what you changed.)

The plusses for the fenced code don't bring anything, so I've removed those.  Apart from that, it looks good.  Please provide a signed-off patch against current git for the attached version.

Yes, you can have Haskell in the extra/ directory, if you provide the file as a signed-off patch against current git, after adding a leading comment line, plus a comment line that states the license, and normalize the other comment lines (one #), and sort the Data Classes alphabetically (not the Constructors).  Furthermore, should "undefined" be colored red also inside comments?  If not, it needs to move up.  It also needs a comment, "Special keyword"?

Of the existing syntaxes in nano, which ones do you think are candidates for pushing down into extra/?  (There won't be a release for several weeks, so there's no hurry.)

(file #48854)

Benno Schulenberg <bens>
Group administrator
Thu 16 Apr 2020 10:41:00 PM UTC, comment #15: 

Oh! I can't believe I forgot this: trailing whitespace should be colored because two spaces at the end of a line render as a <br>. I'll attach a new nanorc file with a line for that added.

I chose blue as a color because I thought it looked less attention-grabbing, which is nicer for when trailing whitespace is part of the syntax rather than a mistake.

(file #48853)

Ryan Westlund <yujiri>
Thu 16 Apr 2020 05:49:37 PM UTC, comment #14: 

I just came across an issue with the markdown one: indented code blocks require a blank line after them or else they color the paragraph after them as code. The regex for terminating an indented code block was only satisfied by a blank line, and not by an unindented line.

I'm uploading a new version of markdown.nanorc and a test case (test.md). I've made three changes:

1. Match each line of indented code blocks individually instead of using start= and end=. I'm not aware of why they had to be matched as a group. Let me know if this breaks anything.

2. Spaces after the indentation don't fail it. I noticed that a line starting with 5 spaces wasn't colored, but it should've been - it's code that starts with a space. This was tough to fix without messing up indented list items with extra spaces, but I think I got it.

3. Fenced code blocks allow more than three backquotes to enclose. Markdown allows this so it's possible to put a literal ``` inside a fenced code block if you enclose it with four of more. I don't think we can replicate that perfectly in Nano, but it should help to not recognize ``` as ending the code block if it's not at the end of a line, since the ``` that ends a code block can't have stuff after it on the line.

If this looks good, I hope it can get committed before next release.

What's your opinion of adding Haskell? I'll go ahead and attach that file. (For clarity, the original file was made by Anthony Scopatz, with improvements by me.)

(file #48849, file #48850, file #48851)

Ryan Westlund <yujiri>
Tue 14 Apr 2020 02:56:44 PM UTC, comment #13: 

Scopatz does have a systemd file, but I can't comment on it since I run FreeBSD. I would like to see the Haskell one added, even if it goes to extra/. (My PR to improve it got merged a few days ago.)

I'm going to go ahead and contact him. Even though his repo is GPL, I think I should invite him to the discussion.

Ryan Westlund <yujiri>
Tue 14 Apr 2020 02:19:21 PM UTC, comment #12: 

Thanks.  The markdown syntax has been pushed to master, commit fb53ad9c.

Any other syntax that needs adding?  One for systemd files?

Benno Schulenberg <bens>
Group administrator
Mon 13 Apr 2020 01:45:51 PM UTC, comment #11: 

Ah, I see. Okay. I found two more things:

  • Blockquotes were not highlighted if they were indented. I added `[ ]*` to after the ^ of the regex.


  • (After changing that) indented blockquotes were colored as code. I added > to the negated character class that must be found after the indentation.


I'm attaching a file with both changes made. This might be the final markdown.nanorc.

(file #48828)

Ryan Westlund <yujiri>
Sun 12 Apr 2020 05:31:38 PM UTC, comment #10: 

In email a tab may get converted to spaces, but if you download file #48801, it should contain two real tabs.

Hmmm...  The leading whitespace for list items is somewhat complicated.  From https://daringfireball.net/projects/markdown/syntax#list : """List markers typically start at the left margin, but may be indented by up to three spaces. List markers must be followed by one or more spaces or a tab."""

Only the markers of a list within a list will be indented by four or more spaces (or a tab).  See attached file for improved regexes for list markers and indented code.  Don't hesitate to point out any mistakes or other things.

(file #48824)

Benno Schulenberg <bens>
Group administrator
Sun 12 Apr 2020 01:27:45 PM UTC, comment #9: 

It looks like the tab characters got replaced with 8 spaces? I don't think this is the intended effect since each `[        ]` in regex is identical to `[ ]` as far as I know. The previous version had `[ ]` (matching either a space or a tab). Markdown normally allows either four spaces or a tab as indentation for list item levels or code blocks, so I think `^(    | )*` (either four spaces or a tab, any number of levels) is the proper regex for both. (I realize now that my original file allowed only one space, which it shouldn't. I've heard some implementations treat that as indentation but the standard says it has to be four or a tab.)

Ryan Westlund <yujiri>
Fri 10 Apr 2020 08:55:53 AM UTC, comment #8: 

Attached version colors blue anything between parentheses after a pair or square brackets.  This also means that anything between square brackets becomes magenta even when not followed by something between parentheses, but.. the other markdown syntaxes I've seen do that too, so...

This version also colors indented text as code.  This is not perfect because it has to avoid coloring list items as code, but I think in practice it will work well enough.

I've also reshuffled the order a bit.  Let me known if another order would be better -- I don't know, as I have no experience with Markdown.

If you wish, please create a signed-off patch from this.  Or shall I just add an "Original-patch-by:" tag to the commit?

(file #48801, file #48802)

Benno Schulenberg <bens>
Group administrator
Fri 10 Apr 2020 01:41:31 AM UTC, comment #7: 

Your syntax seems really good! I'm not sure if it's a good idea to have inline code after headings, and I'm a little put off by how link hrefs are only colored blue if they're a full URL, meaning for example that relative path links (which I use a lot) stay magenta. But there might not be a good way around that, and I see the usefulness of having URLs highlighted outside of `[]()` links since some Markdown implementations turn a standalone URL into a link automatically.

Ryan Westlund <yujiri>
Thu 09 Apr 2020 12:39:07 PM UTC, comment #6: 

Thanks.  Here is a further developed version, made while looking at https://en.wikipedia.org/wiki/Markdown#Example and using the attached text.md as a test file.

Your syntax did not distinguish between italic and bold, and did not recognize underscores for them, and used brightwhite (which will be invisible on a light background).  I like the brightyellow for headings, but for the rest I've modified the colors.

(file #48791, file #48792)

Benno Schulenberg <bens>
Group administrator
Wed 08 Apr 2020 06:04:54 PM UTC, comment #5: 

His markdown file might be better than mine; I left some features out of mine because the markdown processor I'm currently using (mistune) doesn't have them. (No strikethrough, etc.)

Mine doesn't support indented code blocks (only fenced ones), because I can't think of a way to color indented code blocks without coloring things like subsequent paragraphs in list items.

Ryan Westlund <yujiri>
Wed 08 Apr 2020 06:02:42 PM UTC, comment #4: 

Alright, I wasn't sure if it made sense to post the files here since this thread isn't a patch.


(file #48784)

Ryan Westlund <yujiri>
Wed 08 Apr 2020 05:33:08 PM UTC, comment #3: 

Please post your self-constructed markdown.nanorc.  We can built it up from there, and include it as a main syntax.

Haskell seems like a good candidate for the extra/ subdir.  The CSV file looks horrendous -- I have no idea what it's for.  And Nginx looks too niche.

(See also https://lists.gnu.org/archive/html/nano-devel/2020-04/msg00023.html.)

Benno Schulenberg <bens>
Group administrator
Wed 08 Apr 2020 01:00:42 PM UTC, comment #2: 

Nginx and CSV are things that seem like good includes (although on testing them out, I've noticed the Nginx one has an issue that makes comments invisible on a black terminal). I would personally benefit from the Fish one. Markdown also seems like an excellent idea (although I've already half-written my own for that which is very different).

As for licensing, the repo has custom-written license at https://github.com/scopatz/nanorc/blob/master/license, but it says the files are under GPLv3.

Oh, I forgot there's still no file for Haskell included by default! He has a Haskell one. That reminds me that I had copied his to my machine long ago (I must have found it somewhere else) and made a couple of changes I thought were improvements. I'll open an issue on his repo for that.

Let me know what you think of these file types, particularly Haskell and Markdown.

Ryan Westlund <yujiri>
Tue 07 Apr 2020 10:21:37 AM UTC, comment #1: 

Yes, those files have been there for a long time.  The author(s) never submitted any of them upstream.

Which of those files would you like nano to pack by default?  And what precise license covers each individual one of them?  In other words: is the license compatible with GPL v3?

Soon I will post an RFC to nano-devel, proposing to trim some of the existing syntax files (debian, gentoo, spec...), and to ask if nano is missing some syntaxes for widely used file types.

Benno Schulenberg <bens>
Group administrator
Mon 06 Apr 2020 09:08:00 PM UTC, original submission:  

I recently came across a whole bunch of syntax highlighting files at https://github.com/scopatz/nanorc, many for things Nano doesn't already come with a file for. How about pulling some of these?

I've also written one for Renpy (https://github.com/yujiri8/configs/tree/master/nano/renpy.nanorc). I don't know if that's considered too obscure to be included but if not I would submit it as a patch.

Ryan Westlund <yujiri>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #48870:  markdown-comment22.patch added by yujiri (1008B - text/x-patch)
file #48861:  markdown-comment20.patch added by yujiri (1018B - text/x-patch)
file #48859:  psmisc-README.md added by bens (2KiB - text/markdown)
file #48860:  simple-resume.md added by bens (1KiB - text/markdown)
file #48856:  haskell.patch added by yujiri (1KiB - text/x-patch)
file #48855:  markdown.patch added by yujiri (981B - text/x-patch)
file #48854:  markdown.nanorc added by bens (830B - application/octet-stream)
file #48853:  markdown.nanorc added by yujiri (828B - application/octet-stream)
file #48849:  haskell.nanorc added by yujiri (907B - application/octet-stream)
file #48850:  markdown.nanorc added by yujiri (794B - application/octet-stream)
file #48851:  test.md added by yujiri (610B - text/markdown)
file #48828:  markdown.nanorc added by yujiri (804B - application/octet-stream)
file #48824:  markdown.nanorc added by bens (798B - application/octet-stream)
file #48801:  markdown.nanorc added by bens (780B - application/octet-stream)
file #48802:  text.md added by bens (886B - text/markdown)
file #48791:  markdown.nanorc added by bens (649B - application/octet-stream)
file #48792:  text.md added by bens (771B - text/markdown)
file #48784:  markdown.nanorc added by yujiri (399B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bens (Posted a comment)
  • -email is unavailable- added by yujiri (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-09-16 bens Open/ClosedOpen Closed
    2020-09-14 bens Open/ClosedClosed Open
    2020-07-30 bens Open/ClosedOpen Closed
    2020-05-14 bens StatusIn Progress Fixed
        Summary[Wish] extra syntax highlighting files [Wish] extra syntax highlighting files -- Haskell and Markdown
    2020-04-21 yujiri Attached File- Added 0002-syntax-new-file-Haskell.patch, #48888
    2020-04-20 yujiri Attached File- Added 0001-syntax-new-file-Haskell.patch, #48880
    2020-04-20 bens StatusNone In Progress
    2020-04-19 yujiri Attached File- Added markdown-comment22.patch, #48870
    2020-04-17 yujiri Attached File- Added markdown-comment20.patch, #48861
    2020-04-17 bens Attached File- Added psmisc-README.md, #48859
        Attached File- Added simple-resume.md, #48860
    2020-04-17 yujiri Attached File- Added markdown.patch, #48855
        Attached File- Added haskell.patch, #48856
    2020-04-17 bens Attached File- Added markdown.nanorc, #48854
    2020-04-16 yujiri Attached File- Added markdown.nanorc, #48853
    2020-04-16 yujiri Attached File- Added haskell.nanorc, #48849
        Attached File- Added markdown.nanorc, #48850
        Attached File- Added test.md, #48851
    2020-04-13 yujiri Attached File- Added markdown.nanorc, #48828
    2020-04-12 bens Attached File- Added markdown.nanorc, #48824
    2020-04-10 bens Attached File- Added markdown.nanorc, #48801
        Attached File- Added text.md, #48802
    2020-04-09 bens Attached File- Added markdown.nanorc, #48791
        Attached File- Added text.md, #48792

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code