bugGNU gettext - Bugs: bug #56109, many autoconf tests produce wrong...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #56109: many autoconf tests produce wrong results with CC="gcc -flto"

Submitted by:  Tomasz Kłoczko <kloczek>
Submitted on:  Tue 09 Apr 2019 12:33:34 PM UTC  
 
Category:  Build Severity:  2 - Minor
Item Group:  None Status:  Fixed
Privacy:  Public Assigned to:  Bruno Haible <haible>
Open/Closed:  Closed

( Jump to the original submission)

Sun 18 Aug 2019 04:12:48 PM UTC, comment #21: 

The fix for the latter issue is to use CC="gcc -flto -ffat-lto-objects", not CC="gcc -flto".

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Sun 18 Aug 2019 03:05:02 PM UTC, comment #20: 

Fixed in gnulib through this commit:
https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=bdf8b1d2b74e12648edafffcda43832bc35e3d1f

However, building gettext with CC="gcc -fto" does still not work, because of an issue in libtool: libtool's _LT_CMD_GLOBAL_SYMBOLS cannot determine lt_cv_sys_global_symbol_pipe in this situation, because 'nm' does not work on object files. Test case:

conftest.c:

char nm_test_var;
void nm_test_func(void);
void nm_test_func(void){}
int main(){nm_test_var='a';nm_test_func();return(0);}

$ gcc -flto -O2 -c conftest.c ; nm conftest.o
nm: conftest.o: plugin needed to handle lto object
0000000000000001 C __gnu_lto_slim
0000000000000001 C __gnu_lto_v1

Feel free to report this to the binutils people.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Fri 12 Apr 2019 08:36:57 PM UTC, comment #19: 

> I'm building everything with -Wl,--as-needed and using LTO optimisation.


The LTO optimization is the problem. And it's independent of the GCC version and of the distribution.

$ gcc -O2 test-pow.c
/tmp/ccAF8sCL.o:(.data+0x0): undefined reference to `pow'
collect2: error: ld returned 1 exit status
$ gcc -O2 -flto test-pow.c
<no error>

To be fixed in gnulib.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Fri 12 Apr 2019 11:29:52 AM UTC, comment #18: 

It was not possible to link it without -lm

$ gcc -O2 -o test-pow test-pow.c
/usr/bin/ld: /tmp/cczE8eXL.o:(.data+0x0): undefined reference to `pow'
collect2: error: ld returned 1 exit status
$ gcc -O2 -o test-pow test-pow.c -lm

(file #46758)

Tomasz Kłoczko <kloczek>
Fri 12 Apr 2019 11:26:25 AM UTC, comment #17: 

$ gcc -O2 -S test-pow.c && cat test-pow.s
        .file        "test-pow.c"
        .text
        .section        .text.startup,"ax",@progbits
        .p2align 4
        .globl        main
        .type        main, @function
main:
.LFB0:
        .cfi_startproc
        subq        $8, %rsp
        .cfi_def_cfa_offset 16
        movsd        .LC0(%rip), %xmm0
        movapd        %xmm0, %xmm1
        call        *funcptr(%rip)
        addsd        .LC1(%rip), %xmm0
        movl        $1, %eax
        movsd        .LC2(%rip), %xmm1
        comisd        %xmm0, %xmm1
        ja        .L1
        xorl        %eax, %eax
        comisd        .LC3(%rip), %xmm0
        seta        %al
.L1:
        addq        $8, %rsp
        .cfi_def_cfa_offset 8
        ret
        .cfi_endproc
.LFE0:
        .size        main, .-main
        .comm        l_ret,16,16
        .comm        d_ret,8,8
        .comm        f_ret,4,4
        .comm        i_ret,4,4
        .globl        funcptr
        .data
        .align 8
        .type        funcptr, @object
        .size        funcptr, 8
funcptr:
        .quad        pow
        .section        .rodata.cst8,"aM",@progbits,8
        .align 8
.LC0:
        .long        2610202341
        .long        1073341303
        .align 8
.LC1:
        .long        692754423
        .long        1073835387
        .align 8
.LC2:
        .long        858993459
        .long        1070805811
        .align 8
.LC3:
        .long        858993459
        .long        1073427251
        .ident        "GCC: (GNU) 9.0.1 20190328 (Red Hat 9.0.1-0.12)"
        .section        .note.GNU-stack,"",@progbits

Tomasz Kłoczko <kloczek>
Fri 12 Apr 2019 11:23:08 AM UTC, comment #16: 

$ gcc -O2 -S test-pow.c && cat test-pow.s
.file "test-pow.c"
.text
.section .text.startup,"ax",@progbits
.p2align 4
.globl main
.type main, @function
main:
.LFB0:
.cfi_startproc
subq $8, %rsp
.cfi_def_cfa_offset 16
movsd .LC0(%rip), %xmm0
movapd %xmm0, %xmm1
call *funcptr(%rip)
addsd .LC1(%rip), %xmm0
movl $1, %eax
movsd .LC2(%rip), %xmm1
comisd %xmm0, %xmm1
ja .L1
xorl %eax, %eax
comisd .LC3(%rip), %xmm0
seta %al
.L1:
addq $8, %rsp
.cfi_def_cfa_offset 8
ret
.cfi_endproc
.LFE0:
.size main, .-main
.comm l_ret,16,16
.comm d_ret,8,8
.comm f_ret,4,4
.comm i_ret,4,4
.globl funcptr
.data
.align 8
.type funcptr, @object
.size funcptr, 8
funcptr:
.quad pow
.section .rodata.cst8,"aM",@progbits,8
.align 8
.LC0:
.long 2610202341
.long 1073341303
.align 8
.LC1:
.long 692754423
.long 1073835387
.align 8
.LC2:
.long 858993459
.long 1070805811
.align 8
.LC3:
.long 858993459
.long 1073427251
.ident "GCC: (GNU) 9.0.1 20190328 (Red Hat 9.0.1-0.12)"
.section .note.GNU-stack,"",@progbits

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 08:26:50 PM UTC, comment #15: 

> gcc 9 from fedora rawhide.
> Additionally I'm building everything with -Wl,--as-needed and using LTO optimisation.


I tried to install Fedora Rawhide in a VM - in vain. The netinst installer does not produce a working system, and the Live CD does not reach a login prompt.

So, I need info from you:
1) The result of "gcc -O2 -S test-pow.c && cat test-pow.s".
2) The result of "gcc -O2 -o test-pow test-pow.c". Can you just attach the test-pow binary, so that I can see using objdump or gdb what happens?

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Tue 09 Apr 2019 07:45:33 PM UTC, comment #14: 

problem is that "install" target is in many cases used by people like me and not at all by end users.

You may be thinking that hanging all those html files installations across many Makefile.am file you are making gettext easier to consume. Reality is that it makes everything harder.

> Some distributors make choices for all their users. For example, OpenBSD turns off all internationalization because in their view, it's unnecessary and can only produce security issues.


What I've mention is not about i18n support generally but what gettext source code tree is doing during install target.

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 07:40:34 PM UTC, comment #13: 

> I'm working on distribution packges.
> second copy of all documentation in html is MEANINGLESS.


As someone who works on distribution packages, you ought to be aware of the main difference between the "make install" target and a distribution's structure. Namely, "make install" is supposed to install everything that is/might be useful to an average user (because there is only one install target!). Whereas in a distribution, one source package can correspond to multiple binary package. For example, in Debian, you have the packages 'octave', 'octave-doc' (doc in PDF format) and 'octave-info' (doc in info format).

Some distributors make choices for all their users. For example, OpenBSD turns off all internationalization because in their view, it's unnecessary and can only produce security issues.

Some distributors allow the end users to make choices for themselves, by offering distinct binary package. In the 'octave' example above, some users prefer PDF doc, some users prefer info doc, and some users don't need the doc at all, only the program.

I hope you are in the second camp, and will thus make libasprintf and the HTML documentation available in separate binary packages.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Tue 09 Apr 2019 05:18:05 PM UTC, comment #12: 

> You sound like your machine
> 1. has no web browser installed,
> 2. has a disk size of less than 5 GB.


You really do not understand that I'm working on distribution packges.
On builng such resources you are not using web browser and you care first about minimal size of installed resources.

If gettext command is used for example by grub fact that out of source tree is possible to build and install second copy of all documentation in html is MEANINGLESS.
Do you see my point?
I can underdsstand that as maintainer you may be interested to publish whole html documentation in come content of the project web server.
Most of the end uusers will be only using man and info commands to browse installed documentation because it is TYPICAL way of accessing to such documentation.
In other word unconditionally generating and installing all that stuff in install target is waste of time.
On preparing my rpm package with gettext I've already added

[tkloczko@domek SOURCES]$ ls -l gettext-do_not_install_some_doc.patch
-rw-rw-r-- 1 tkloczko tkloczko 15055 Apr  9 17:44 gettext-do_not_install_some_doc.patch

which unconditionally removes generating and installing all those resources as from point of view of end user OS distribution consumer all this stuff is USELESS!!!

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 05:06:39 PM UTC, comment #11: 

Yes I would.
Just please make new release :)

Current tar ball contains much more bugs than whatever would be possible to release out of current git repo.

Not releasing anything you are trapped in catch 22 case :)
You are not releasing because you are not sure about existing bus and not to many people are encouraged to test new code because you are not releasing it.

Just make new release and if it will be any critical bug make next minor release ASAP.
That is the only and proper way of testing original source tree .. really :)

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 04:52:41 PM UTC, comment #10: 

> Could you please tell do you have any plans to push out all
> already accumulated changes in git repo as new release?


Do you want to help with prerelease testing? If so, I can invite you to help testing the prerelease by private email, when time to do so has come.

> by default on linux gettext builds and installs libasprintf


Try './configure --help'.

> duplicates already installed documentation in form of roff
> and info pages.


You sound like your machine
1. has no web browser installed,
2. has a disk size of less than 5 GB.

Other users have a machine with a web browser and/or more disk space.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Tue 09 Apr 2019 04:26:24 PM UTC, comment #9: 

As long as it was not new releases in last few years anyone who will be trying trying to build gettext package will not be even able to follow those changes.

I don't remember in +10 years any project needlessly puffed by gnulib. Occasionally like in this case with pow() it breaks only other projects.

4th time .. PLEASE ..
Could you please tell do you have any plans to push out all already accumulated changes in git repo as new release?
Do I need to beg you to answer on very simple question?

PS. I just found that by default on linux gettext builds and installs libasprintf which is nowhere else used (even not in gettext). It would be good to at least disable build it.

Other thing is building and installing every piece of the gettext documentation in html form.

[tkloczko@domek gettext-0.19.8.1]$ ./configure --help | grep html
  --htmldir=DIR           html documentation [DOCDIR]

As you see current ac build suite does not offer to disable build and install this stuff.
almost 100% all those files duplicates already installed documentation in form of roff and info pages.

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 04:14:52 PM UTC, comment #8: 

> gnulib on Linux is CONSTANTLY broken.
> Because most of the systems are trying to compliant with most of the standards usability of the gnulib is only decreasing.


Recent glibcs have broken gnulib twice recently (removal of gets() and of <libio.h>). Both were fixed in gnulib very rapidly. The second one was over a year ago.

Are you sure you are not exaggerating when you say "constantly broken"?

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Tue 09 Apr 2019 02:17:52 PM UTC, comment #7: 

gnulib on Linux is CONSTANTLY broken.
Because most of the systems are trying to compliant with most of the standards usability of the gnulib is only decreasing.
In many cases it is possible to abandon use gnulib without changing main code of the project. In all projects is possible at least drop using few gnulib wrappers.
Result is that final code size is smaller.

In other words: executing gnulib test suite is pure waste of time.
QUED.

Could you* please* answer on the questions do you have any plans to make new gettext release soon?

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 01:47:39 PM UTC, comment #6: 

> I think that performing full gnulib test in gettext suite is waste of time.


I disagree. If the packages that use gnulib didn't enable the tests for the respective gnulib modules, hardly anyone would perform these tests, and as a result, gnulib would have lower quality.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Tue 09 Apr 2019 01:27:23 PM UTC, comment #5: 

Jusy found that SuSE has patch which completely disables gnulib test suite in gettext tests.

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 01:26:11 PM UTC, comment #4: 

gcc 9 from fedora rawhide.
Additionally I'm building everything with -Wl,--as-needed and using LTO optimisation.

Do you have any plans to make new release?
At least one CVE looks like is fixed in git and on top of this a lot of other changes and fixes. IMO project like gettext should make new release way often that every few years (monthly of ewveryt two months would be great).

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 01:21:12 PM UTC, comment #3: 

> If that bug already has been fixed in gnulib


No, there is no bug fix around pow() in gnulib in the last 4 years.

> checking whether pow can be used without linking with libm... yes


Which version of GCC and of glibc are you using?

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Tue 09 Apr 2019 12:51:00 PM UTC, comment #2: 

BTW: I think that performing full gnulib test in gettext suite is waste of time.
Looks like it is bug in gnulib.
If that bug already has been fixed in gnulib best way to fix it would be just make new gettext release.

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 12:37:43 PM UTC, comment #1: 

from configure output:

checking whether pow can be used without linking with libm... yes

Looks like bug in m4 macros.

Tomasz Kłoczko <kloczek>
Tue 09 Apr 2019 12:33:34 PM UTC, original submission:  

Fragment of the check target output:

make[3]: 'test-posix_spawn1' is up to date.
make[3]: 'test-posix_spawn2' is up to date.
/bin/sh ../libtool  --tag=CC --preserve-dup-deps  --mode=link gcc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -flto  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto -fuse-linker-plugin -o test-pow test-pow.o libtests.a ../gnulib-lib/libgettextlib.la libtests.a  
libtool: link: gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -flto -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -flto -fuse-linker-plugin -o .libs/test-pow test-pow.o  libtests.a ../gnulib-lib/.libs/libgettextlib.so -L/usr/lib64 -lacl -lunistring -lcroco-0.6 -lglib-2.0 -lxml2 -lncurses libtests.a -fopenmp
/usr/bin/ld: /tmp/test-pow.dBrO1K.ltrans0.ltrans.o: in function `main':
/home/tkloczko/rpmbuild/BUILD/gettext-0.19.8.1/gettext-tools/gnulib-tests/test-pow.c:38: undefined reference to `pow'
collect2: error: ld returned 1 exit status

PS. Do you have any plans to make new gettext release soon?

Tomasz Kłoczko <kloczek>

 

Attached Files
file #46758:  test-pow added by kloczek (22KiB - application/octet-stream)
file #46741:  test-pow.c added by haible (326B - text/x-csrc)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Posted a comment)
  • -email is unavailable- added by kloczek (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-08-18 haible StatusConfirmed => Fixed
        Open/ClosedOpen => Closed
    2019-04-12 haible Severity3 - Normal => 2 - Minor
        StatusNeed Info => Confirmed
        Summarypow() test fails => many autoconf tests produce wrong results with CC="gcc -flto"
    2019-04-12 kloczek Attached File- => Added test-pow, #46758
    2019-04-09 haible Attached File- => Added test-pow.c, #46741
        StatusNone => Need Info
        Assigned toNone => haible

    Back to the top


    Powered by Savane 3.5