bugGNU nano - Bugs: bug #59028, [Wish] an option to switch off the...


You are not allowed to post comments on this tracker with your current authentication level.

bug #59028: [Wish] an option to switch off the use of libmagic

Submitter:  Benno Schulenberg <bens>
Submitted:  Fri 28 Aug 2020 08:11:27 AM UTC
Severity:  1 - Wish Status:  Fixed
Assigned to:  bens Open/Closed:  Closed

Thu 08 Oct 2020 08:55:04 AM UTC, comment #5: 

Option --magic and 'set magic' was released in version 5.3.

Benno Schulenberg <bens>
Project Administrator
Fri 25 Sep 2020 02:41:27 PM UTC, comment #4: 

Fixed in git, commit 58a3293e, by requiring --magic or 'set magic' to enable the use of libmagic.

It is better to not use libmagic by default.  On my machine, now doing 'src/nano NEWS' is "bam", instantaneous; doing 'src/nano NEWS -!' instead is "patam": there is a noticeable slight delay: the terminal screen goes blank during a fraction of a second.

Benno Schulenberg <bens>
Project Administrator
Mon 31 Aug 2020 02:34:31 PM UTC, comment #3: 

Here are some measurements on four different machines.  Apart from the laptop, all these machines have a real, spinning disk; the laptop has flash storage.  The "cold" columns list the average total startup time of nano upon its first run of the day (simulated with 'echo 3 >/proc/sys/vm/drop_caches'), and the "rerun" columns list the time of a warm start, when all needed files are already cached in RAM.  All times are in milliseconds.

           cold    cold   rerun  rerun
          magic    none   magic   none
netbook    1580    1420     100     60    Arch32
netbook    1410    1250      85     50    Lucid

old box    1005     860      50     29    Arch32
old box     990     835      60     20    Peppermint

laptop      210     140      55     25    Bionic (18.04)

desktop     880     750      20     11    Bionic (18.04)
desktop     900     750      15     13    Focal (20.04)
desktop     ---     ---      35     14    NomadBSD
desktop     ---     ---      35     31    Haiku

When doing a cold start, disabling the use of magic saves some 150 ms (when having a spinning disk).  Compared to the total startup time of 900 to 1500 milliseconds, this saving hardly seems worth the trouble.  But when having flash storage, disabling magic saves a full third of the startup time.  That is worth it.

For a warm start, when having a fast processor (like on the desktop), saving maybe ten milliseconds on startup time doesn't seem worth it.  But when having a poorer processor, it can make a significant difference, like on the old box with Peppermint: from 60 to 20 milliseconds.  That is the difference between quick and instantaneous.

(But even if the savings in time would normally not be noticeable, it will be nice to be able to avoid all the wasted CPU cycles when the user knows they won't need magic or even simply don't want magic.)

It has surprised how much the timings can change on the same machine just by using a different distro.  The differences aren't dramatic, but still: significant.  But most surprising was how much the startup times could vary on one and the same machine plus distro.  For example, the 1250 ms for a cold start without magic on the netbook with Lucid is an average of numbers between 761 and 1424 ms.  And the 750 ms on the desktop with Focal is an average of numbers between 640 and 2078 ms.  Yes, that is not a typo: the fastest machine would sometimes have the slowest startup time: a full two seconds.  Ridiculous.

Benno Schulenberg <bens>
Project Administrator
Sat 29 Aug 2020 11:22:09 AM UTC, comment #2: 

Well, it results that it doesn't make any significant difference whether a machine has a real disk or flash storage.  Below, the netbook, the old box, and the desktop all have a real disk, the laptop has flash storage.  The first three machines all have a low-end processor, and the desktop machine has an i5.

The "name" column lists the time it takes to fruitlessly run through all the filename and headerline regexes, the "magic" column lists the time to do the equivalent magic stuff, and "first" list the time for the latter for a cold start.  All times are in milliseconds.

          name  magic   first
netbook    3.8   43.5    87.5   Arch32
old box    1.1   20.0    23.7   Arch32
laptop     1.4   23.5    --.-   Bionic (18.04)

desktop    0.8    2.7     9.9   Focal (20.04)
desktop    1.2    4.1    20.4   Haiku

I'm surprised that on the relatively fast i5 machine the time to do the magic stuff is only roughly three to four times the time it takes to do the normal regexes, whereas on the slower machines the magic takes roughly fifteen times more time.  It must have something to do with the better processor also having more L1/L2/L3 cache, allowing it keep more stuff close to the processor and thus do the magic stuff so much faster.

Haiku and Focal run on the same machine, but although Haiku is slower, the nice thing about it is that its timings are stable: they vary less than ten percent.  But on Focal, the 0.8 is an average of a number that varies from 0.5 to 1.2, and 2.7 is an average from numbers between 1.4 and 3.9.

If all people had fast machines like the desktop, adding a 'set nomagic' option wouldn't be worth it: saving two milliseconds (or nine milliseconds on the first start) is not noticeable by anyone.  But... this only measures the times it takes to do the things in find_and_prime_applicable_syntax().  It would be better to measure the full time it takes for nano to fully start up, from pressing Enter on the command line to first showing the contents of the file, and compare that time for both a cold and a warm start, and both with and without using magic.

Benno Schulenberg <bens>
Project Administrator
Fri 28 Aug 2020 08:32:43 AM UTC, comment #1: 

On my machine, running fruitlessly through all the filename regexes and headerline regexes takes 1.4 milliseconds.  Doing a fruitless magic search takes 23.5 milliseconds.

Twenty three milliseconds in itself are not much, but this is on a machine with flash-memory storage.  On machines with a real disk, loading the magic stuff the first time could easily take more than a hundred milliseconds.  It is this needless delay (in the vast majority of cases) that I want to spare the user.

Benno Schulenberg <bens>
Project Administrator
Fri 28 Aug 2020 08:11:27 AM UTC, original submission:  

Almost all distros build a vanilla nano, without disabling anything during ./configure, which means that libmagic will be enabled.  But the average user uses nano mostly to edit some config files, and most of these config files are not covered by any specific syntax, which means that trying to determine the syntax with a call to magic_file() is just a waste of time.

So... it would be nice if the users of a vanilla nano would have a way to prevent this wasteful calling of magic_file() when they know that rarely they will be editing a file where such a call would help.

(Or maybe not calling magic_file() should be the default, and we should add an option to turn this call on: 'set magic', instead of 'set nomagic'.)

But first I will have to verify how much time precisely gets wasted by this calling of libmagic.

Benno Schulenberg <bens>
Project Administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by bens (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-10-08 bens Open/ClosedOpen Closed
    2020-09-25 bens StatusNone Fixed
    2020-09-03 bens Summary[Wish] an option to switch the use of libmagic off [Wish] an option to switch off the use of libmagic

    Back to the top

    Powered by Savane 3.9