bugGNU gettext - Bugs: bug #63641, Poor build parallelism in gettext

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #63641: Poor build parallelism in gettext

Submitter:  Sergei Trofimovich <slyfox>
Submitted:  Wed 11 Jan 2023 07:59:41 AM UTC
   
 
Category:  Build Severity:  2 - Minor
Item Group:  None Status:  Wont Fix
Privacy:  Public Assigned to:  haible
Open/Closed:  Closed

Sun 12 Feb 2023 09:39:16 AM UTC, comment #2: 

Not to challenge your decision but to add a bit of color on current state: `gettext` builds 2m11s on my machine.

As it's an early package to build during system bootstrap it's slow build delays many other packages (with better parallelism) that depend on it.

To put `gettext` slowness in context: the whole of `coreutils` and it's tests builds 1m13s on my machine. And even `ocaml` (a language compiler with it's own code generator) takes 1m45s.

I find it hard to believe gettext does as much useful work as these packages during the build be it time measurement or energy consumption. Or be it per-core measurement vs whole machine load.

Sergei Trofimovich <slyfox>
Sun 12 Feb 2023 08:55:20 AM UTC, comment #1: 

This improvement is too costly for too little benefit.

Explanation:

Makefiles (without assuming GNU make) are quite a constrained but at the same time also complicated environment, with the various types of rules and the need for it to work also in VPATH environments and across different 'make' implementations. Therefore this feature would cost several hours, up to possibly 3 days, of work.

The benefit is saving a couple of seconds (probably less than a minute on a fast machine), and only in wall clock time, not in energy consumption.

Instead, I plan to move out libtextstyle to a different tarball, which you can then build less often. This reduces the benefit of the proposed modification even more.

Bruno Haible <haible>
Group administrator
Wed 11 Jan 2023 07:59:41 AM UTC, original submission:  

I build gettext as part of fresh systems quite regularly (~daily). I noticed that gettext does not parallelize as good as it could. It is visibly slower than many other packages.

I think it's because `for` loops are used in a few places to build not that small libraries, like `libxml`, `glib`, `gnulib`:


      for f in gl_array_list.h gl_array_list.c basename-lgpl.c binary-io.h binary-io.c c-ctype.h c-ctype.c c-strcase.h c-strcasecmp.c c-strncasecmp.c cloexec.c concat-filename.c exitfail.c fatal-signal.h fatal-signal.c fd-hook.c fd-ostream.c file-ostream.c full-write.h full-write.c get_ppid_of.h get_ppid_of.c get_progname_of.h get_progname_of.c getprogname.h getprogname.c gettext.h html-ostream.c html-styled-ostream.c iconv-ostream.c gl_list.h gl_list.c glthread/lock.h glthread/lock.c malloca.c math.c mem-hash-map.h mem-hash-map.c memory-ostream.c minmax.h noop-styled-ostream.c ostream.c printf-frexp.c printf-frexpl.c safe-read.c safe-write.c sig-handler.c size_max.h sockets.h sockets.c stat-time.c styled-ostream.c sys_socket.c term-ostream.c term-style-control.c term-styled-ostream.c glthread/threadlib.c unistd.c unistr/u8-mbtouc.c unistr/u8-mbtouc-aux.c xalloc.h xmalloc.c xstrdup.c xconcat-filename.c xgethostname.h xgethostname.c gl_xlist.h gl_xlist.c xsize.h xsize.c xvasprintf.h xvasprintf.c xasprintf.c color.h color.c misc.h misc.c version.c  libcroco/cr-additional-sel.c libcroco/cr-attr-sel.c libcroco/cr-cascade.c libcroco/cr-declaration.c libcroco/cr-doc-handler.c libcroco/cr-enc-handler.c libcroco/cr-fonts.c libcroco/cr-input.c libcroco/cr-num.c libcroco/cr-om-parser.c libcroco/cr-parser.c libcroco/cr-parsing-location.c libcroco/cr-prop-list.c libcroco/cr-pseudo.c libcroco/cr-rgb.c libcroco/cr-sel-eng.c libcroco/cr-selector.c libcroco/cr-simple-sel.c libcroco/cr-statement.c libcroco/cr-string.c libcroco/cr-style.c libcroco/cr-stylesheet.c libcroco/cr-term.c libcroco/cr-tknzr.c libcroco/cr-token.c libcroco/cr-utils.c libxml/DOCBparser.c libxml/HTMLparser.c libxml/HTMLtree.c libxml/SAX.c libxml/SAX2.c libxml/buf.h libxml/buf.c libxml/c14n.c libxml/catalog.c libxml/chvalid.c libxml/debugXML.c libxml/dict.c libxml/enc.h libxml/encoding.c libxml/entities.c libxml/error.c libxml/globals.c libxml/hash.c libxml/legacy.c libxml/list.c libxml/nanoftp.c libxml/nanohttp.c libxml/parser.c libxml/parserInternals.c libxml/pattern.c libxml/relaxng.c libxml/save.h libxml/schematron.c libxml/threads.c libxml/timsort.h libxml/tree.c libxml/trionan.c libxml/uri.c libxml/valid.c libxml/xinclude.c libxml/xlink.c libxml/xmlIO.c libxml/xmlmemory.c libxml/xmlmodule.c libxml/xmlreader.c libxml/xmlregexp.c libxml/xmlsave.c libxml/xmlschemas.c libxml/xmlschemastypes.c libxml/xmlstring.c libxml/xmlunicode.c libxml/xmlwriter.c libxml/xpath.c libxml/xpointer.c glib/ghash.c glib/glist.c glib/gmessages.c glib/gprimes.c glib/gstrfuncs.c glib/gstring.c; do \

    gcc -DHAVE_CONFIG_H -I.  ...

...

    for f in ../src/str-list.c ../src/dir-list.c ../src/message.c ../src/msgl-ascii.c ../src/po-error.c ../src/po-xerror.c ../src/write-catalog.c ../src/write-po.c ../src/open-catalog.c ../src/po-charset.c ../src/po-lex.c ../src/po-gram-gen.c ../src/read-po.c ../src/read-catalog-abstract.c ../src/read-catalog.c ../src/plural-table.c ../src/format-c.c ../src/format-python.c ../src/format-python-brace.c ../src/format-java.c ../src/format-java-printf.c ../src/format-csharp.c ../src/format-javascript.c ../src/format-scheme.c ../src/format-lisp.c ../src/format-elisp.c ../src/format-librep.c ../src/format-ruby.c ../src/format-sh.c ../src/format-awk.c ../src/format-lua.c ../src/format-pascal.c ../src/format-smalltalk.c ../src/format-qt.c ../src/format-qt-plural.c ../src/format-kde.c ../src/format-kde-kuit.c ../src/format-boost.c ../src/format-tcl.c ../src/format-perl.c ../src/format-perl-brace.c ../src/format-php.c ../src/format-gcc-internal.c ../src/format-gfc-internal.c ../src/format.c ../src/plural-exp.c ../src/plural-eval.c ../src/msgl-check.c ../src/sentence.c basename-lgpl.c c-ctype.h c-ctype.c c-strcase.h c-strcasecmp.c c-strncasecmp.c c-strstr.h c-strstr.c cloexec.c concat-filename.c diffseq.h error-progname.h error-progname.c exitfail.c fd-hook.c fstrcmp.h fstrcmp.c fwriteerror.h fwriteerror.c gcd.h gcd.c getprogname.h getprogname.c gettext.h hard-locale.c gl_linked_list.h gl_linked_list.c gl_anylinked_list1.h gl_anylinked_list2.h gl_list.h gl_list.c localcharset.c glthread/lock.h glthread/lock.c malloca.c markup.h markup.c mbswidth.h mbswidth.c mem-hash-map.h mem-hash-map.c minmax.h setlocale_null.c size_max.h stat-time.c striconv.h striconv.c striconveh.h striconveh.c striconveha.h striconveha.c glthread/threadlib.c glthread/tls.h glthread/tls.c uniconv/u8-conv-from-enc.c unictype/ctype_alpha.c unictype/ctype_space.c unilbrk/lbrktables.c unilbrk/u8-possible-linebreaks.c unilbrk/u8-width-linebreaks.c unilbrk/ulc-common.c unilbrk/ulc-width-linebreaks.c unistd.c unistr/u16-mbtouc.c unistr/u16-mbtouc-aux.c unistr/u8-check.c unistr/u8-mblen.c unistr/u8-mbtouc.c unistr/u8-mbtouc-aux.c unistr/u8-mbtouc-unsafe.c unistr/u8-mbtouc-unsafe-aux.c unistr/u8-mbtoucr.c unistr/u8-next.c unistr/u8-prev.c unistr/u8-strmbtouc.c unistr/u8-uctomb.c unistr/u8-uctomb-aux.c uniwidth/width.c wctype-h.c xalloc.h xmalloc.c xstrdup.c xconcat-filename.c xerror.h xerror.c gl_xlist.h gl_xlist.c xmalloca.c xsize.h xsize.c xstriconv.h xstriconv.c xvasprintf.h xvasprintf.c xasprintf.c asnprintf.lo fcntl.lo mbrtowc.lo obstack.lo printf-args.lo printf-parse.lo vasnprintf.lo; do \


Would it make sense to use make-level parallelism to build those?

Thanks!

Sergei Trofimovich <slyfox>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Updated the item)
  • -email is unavailable- added by slyfox (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-02-12 haible StatusNone Wont Fix
        Assigned toNone haible
        Open/ClosedOpen Closed
    2023-01-11 haible CategoryNone Build
        Severity3 - Normal 2 - Minor

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code