bugGNU nano - Bugs: bug #62056, screen-drawing at the end of a...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #62056: screen-drawing at the end of a large XML file is too slow

Submitter:  Benno Schulenberg <bens>
Submitted:  Tue 15 Feb 2022 04:12:37 PM UTC
   
 
Severity:  3 - Normal Status:  Fixed
Assigned to:  bens Open/Closed:  Closed

Thu 17 Mar 2022 03:32:13 PM UTC, comment #4: 

Fixed in git, commit 15a0a129, by making sure the multidata is properly precalculated when needed.

Benno Schulenberg <bens>
Group administrator
Sun 20 Feb 2022 01:34:41 PM UTC, comment #3: 

However... if we recalculate the multiline data every time a multiline-regex match has changed (or might have changed -- when pasting multiple lines we don't check all the pasted lines), then we can trust the multidata to be correct, but that also means that the drawing routines must not set or changed any of this multidata.  In theory the drawing routines would always want to set the multidata to what it already set to, but I'm not sure whether this actually is the case.

(Well, that's easy to check: precalculate the multidata for every keystroke, and instead of letting the coloring code set the multidata, let it check whether the multidata is set to what it wants it to set to.  Result: no.  When the coloring routine finds an end match on a line, it wants to set the data to ENDSHERE, but if there is another start match on that line (after the end match), then... the precalculated multidata is different, either WOULDBE or STARTSHERE or JUSTONTHIS, depending on whether and where there is a matching end match.  This also shows that if the precalculated multidata is anything other than WHOLELINE or NOTHING, then the coloring routine should check the entire line for end matches and start matches and end matches.)

Benno Schulenberg <bens>
Group administrator
Sun 20 Feb 2022 12:41:05 PM UTC, comment #2: 

(It is worth noting that the 324 ms refresh time at the end of the example XML file is caused entirely by the last multiline regex in the xml syntax ('color yellow start="<!--" end="-->").  The only two comments in the example file occur at the start, so when doing backtracking for this regex pair, nano has to recede all the way to the start of the file, to the second of these two comments.  Majorly inefficient.)

Benno Schulenberg <bens>
Group administrator
Tue 15 Feb 2022 04:51:17 PM UTC, comment #1: 

This backtracking for the first row is only needed when we made a jump in the file (with M-/, M-PgDn, M-0, Ctrl+↓, after a replacement, ...).  That is still for quite a few different movements, but not for all: not for simple scrolls of one row, and not for PageUps and PageDowns.  These latter movements are the most common (at least for me) and should be fast, and can be fast.

So... somehow the movements that cause the screen to scroll more than editwinrows (or could cause this), should set a 'jumped' flag.  And the line-drawing routine should then only do the backtracking for the first row when this 'jumped' flag is set.  At the end of drawing the first row, the flag should be reset.

Ah, no, that will not work: after making a change at the start of the file that should recolor the entire file, and then jumping to the end and paging back up, for each PageUp, the backtracking should be done...  :|

It's probably better to... remember whether a multiline-regex-affecting change was made, and when then any jump occurs, to recalculate all the multiline data.  For the example file the latter takes 668 ms, which is twice as long as a full backtracking, but... after that recalcualtion, all further movements (even jumps) can be drawn quick, because all the multiline data is valid (as long as no further multiline-affecting changes occur).  So... that would require two flags: 'major-change' and 'jumped'.  And edit_refresh() should first call precalc_multicolorinfo() when both are set, and then reset both.

Benno Schulenberg <bens>
Group administrator
Tue 15 Feb 2022 04:12:37 PM UTC, original submission:  

To reproduce, first download the Unicode data file [1], unzip it, and delete the zip file.

[1] https://www.unicode.org/Public/UCD/latest/ucdxml/ucd.all.grouped.zip

Then uncomment the "#define TIMEPRECALC" line in src/color.c and the "#define TIMEREFRESH" line in src/winio.c, and recompile.  Now run:

src/nano -xl -f syntax/xml.nanorc  ucd.all.grouped.xml

It takes several seconds just to load the file.  But note the time that precalculation takes (for me 668 ms).  Then note the refresh time at the top of the file (for me 11 ms).  Now type M-/ to jump to end-of-file.  See how long the refresh takes (for me 324 ms).  This latter value is too large -- it takes too long, it is too slow.

Now page up a few times.  Each screen draw takes around 325 ms.  Now make your terminal just three rows, and page up some more.  Each screen draw still takes 311 ms.  It's not the drawing that takes time, it's the backtracking for the first row that takes this 311 ms -- drawing the rest of the rows takes around 14 ms.

So, the problematic commit is 43d94692: "always do backtracking for the first row of the screen", which fixed bug #59948 and addressed bug #59945.

Benno Schulenberg <bens>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bens (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-04-29 bens Open/ClosedOpen Closed
    2022-03-17 bens StatusNone Fixed
    2022-02-20 bens Summaryscreen-drawing at the end of a large file is too slow screen-drawing at the end of a large XML file is too slow

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code