bugGNU libunistring - Bugs: bug #54453, abort in u8_possible_linebreaks in...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #54453: abort in u8_possible_linebreaks in libunistring 0.9.10

Submitted by:  Natanael Copa <ncopa>
Submitted on:  Mon 06 Aug 2018 10:48:26 AM UTC  
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  Need Info
Privacy:  Public Assigned to:  None
Open/Closed:  Open

( Jump to the original submission)

Thu 09 Aug 2018 07:21:58 PM UTC, comment #7: 

Turns out to be a bug in gettext which bundles an old copy of libunistring. The shared, system, libunistring code is used but ends up use the data table from the bundled, older libunistring and things goes wrong.

(Thanks to Timo Teräs for helping me find that out).

Natanael Copa <ncopa>
Thu 09 Aug 2018 03:10:06 PM UTC, comment #6: 

The binaries comes from official Alpine Linux x86_64 builder:
http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/

I build them locally with debugging symbols to be able to provide backtraces.

Alpine Linux edge(rolling release) x86_64.

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/6.4.0/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-6.4.0/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --with-pkgversion='Alpine 6.4.0' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-cloog-backend --enable-languages=c,c++,objc,java,fortran,ada --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
Thread model: posix
gcc version 6.4.0 (Alpine 6.4.0)

musl libc 1.1.19 + some backported patches.

Same toolchain for both 0.9.9 and 0.9.10. It only happens with 0.9.10.

What is somewhat weird is that a couple of times when I have built libunistring with -g, the problem no longer manifests itself. When it does, the gettext have exactly 21 failing test in testsuite.

It is quite possible that this is a memory corruption or undefined behavior somewhere else, in either gettext, libunistring or gnulib, but I have no clue where the problem may be.

Natanael Copa <ncopa>
Thu 09 Aug 2018 02:45:42 PM UTC, comment #5: 

The gettext testsuite failures that you posted in https://lists.gnu.org/archive/html/bug-gettext/2018-08/msg00008.html also point to a broken u8_possible_linebreaks function.

However, the source code of u8_possible_linebreaks and related functions has not changed between libunistring 0.9.9 and 0.9.10. Therefore, compiling the two libunistring version with the same GCC and the same compiler options should produce the same result.

May I ask: Where did you get these two binary packages from? And with which GCC versions were they compiled, respectively?

Bruno Haible <haible>
Project Administrator
Thu 09 Aug 2018 09:01:36 AM UTC, comment #4: 

all tests in gettext-0.19.8.1 testsuite also passes with libunistring-0.9.9.

Natanael Copa <ncopa>
Thu 09 Aug 2018 08:52:28 AM UTC, comment #3: 

Interestingly, gettext-0.19.8.1's testsuite has 21 failing tests when its built with libunistring-0.9.10, which all passed when built with libunistring-0.9.7.

Natanael Copa <ncopa>
Thu 09 Aug 2018 08:39:18 AM UTC, comment #2: 

Another backtrace, with lt_TO.po:

Core was generated by `/usr/bin/msgmerge lt_LT.po.new filezilla.pot -o lt_LT.po.new~'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007ff7b9738f1b in __syscall4 (a4=<optimized out>, a3=<optimized out>,
    a2=<optimized out>, a1=<optimized out>, n=<optimized out>)
    at ./arch/x86_64/syscall_arch.h:38
38      ./arch/x86_64/syscall_arch.h: No such file or directory.
[Current thread is 1 (LWP 30519)]
(gdb) bt
#0  0x00007ff7b9738f1b in __syscall4 (a4=<optimized out>, a3=<optimized out>,
    a2=<optimized out>, a1=<optimized out>, n=<optimized out>)
    at ./arch/x86_64/syscall_arch.h:38
#1  __restore_sigs (set=set@entry=0x7ffcda4e3530) at src/signal/block.c:43
#2  0x00007ff7b9739035 in raise (sig=sig@entry=6) at src/signal/raise.c:13
#3  0x00007ff7b9710fb4 in abort () at src/exit/abort.c:9
#4  0x00007ff7b879b294 in u8_possible_linebreaks (
    s=0x55ff98668083 "ymel_s\271\367\177",
    s@entry=0x55ff98668080 "_ymel_s\271\367\177", n=n@entry=11,
    encoding=encoding@entry=0x7ff7b94e3c17 "UTF-8",
    p=0x55ff986231a3 "\001\001\001\001\001\001\001\001\271\367\177",
    p@entry=0x55ff986231a0 '\001' <repeats 11 times>, "\271\367\177")
    at unilbrk/u8-possible-linebreaks.c:161
#5  0x00007ff7b879b320 in u8_width_linebreaks (
    s=s@entry=0x55ff98668080 "_ymel_s\271\367\177", n=n@entry=11,
    width=width@entry=77, start_column=start_column@entry=7,
    at_end_columns=at_end_columns@entry=0, o=o@entry=0x55ff98643640 "",
    encoding=0x7ff7b94e3c17 "UTF-8",
    p=0x55ff986231a0 '\001' <repeats 11 times>, "\271\367\177")
    at unilbrk/u8-width-linebreaks.c:46
#6  0x00007ff7b879b83e in ulc_width_linebreaks (
    s=0x55ff98668080 "_ymel_s\271\367\177", n=11, width=77,

Natanael Copa <ncopa>
Mon 06 Aug 2018 12:15:45 PM UTC, comment #1: 

Can you please attach the files he_IL.po.new and filezilla.pot that are mentioned in the command-line? Without these files, I have little chance of reproducing the issue.

Bruno Haible <haible>
Project Administrator
Mon 06 Aug 2018 10:48:26 AM UTC, original submission:  

Hi,

After upgrading libunistring 0.9.10 on Alpine Linux, build of filezilla ends with an abort() when calling /usr/bin/msgmerge (from gettext).

coredump indicate that it happens in libunistring. Downgrading libunistring solves it.

Backtrace from core dump:

````
Core was generated by `/usr/bin/msgmerge he_IL.po.new filezilla.pot -o he_IL.po.new~'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f13cefb4f1b in __syscall4 (a4=<optimized out>, a3=<optimized out>, a2=<optimized out>,
    a1=<optimized out>, n=<optimized out>) at ./arch/x86_64/syscall_arch.h:38
38      ./arch/x86_64/syscall_arch.h: No such file or directory.
[Current thread is 1 (LWP 12586)]
(gdb) bt
#0  0x00007f13cefb4f1b in __syscall4 (a4=<optimized out>, a3=<optimized out>, a2=<optimized out>,
    a1=<optimized out>, n=<optimized out>) at ./arch/x86_64/syscall_arch.h:38
#1  __restore_sigs (set=set@entry=0x7ffecef34020) at src/signal/block.c:43
#2  0x00007f13cefb5035 in raise (sig=sig@entry=6) at src/signal/raise.c:13
#3  0x00007f13cef8cfb4 in abort () at src/exit/abort.c:9
#4  0x00007f13ce017294 in u8_possible_linebreaks (s=0x55bf9c4f8f2b "___\001",
    s@entry=0x55bf9c4f8f20 "%H:%M:%S __\001", n=n@entry=17, encoding=encoding@entry=0x7f13ced5fc17 "UTF-8",
    p=0x55bf9c5313eb "\001\001\001\001\001\001src/interface/QueueView.cpp",
    p@entry=0x55bf9c5313e0 "\001\002\001\001\002\001\001\002\003\001\001\001\001\001\001\001\001src/interface/QueueView.cpp") at unilbrk/u8-possible-linebreaks.c:161
#5  0x00007f13ce017320 in u8_width_linebreaks (s=s@entry=0x55bf9c4f8f20 "%H:%M:%S __\001", n=n@entry=17,
    width=width@entry=77, start_column=start_column@entry=7, at_end_columns=at_end_columns@entry=0,
    o=o@entry=0x55bf9c539f20 "", encoding=0x7f13ced5fc17 "UTF-8",
    p=0x55bf9c5313e0 "\001\002\001\001\002\001\001\002\003\001\001\001\001\001\001\001\001src/interface/QueueView.cpp") at unilbrk/u8-width-linebreaks.c:46
#6  0x00007f13ce01783e in ulc_width_linebreaks (s=0x55bf9c4f8f20 "%H:%M:%S __\001", n=17, width=77,
    start_column=7, at_end_columns=0, o=0x55bf9c539f20 "", encoding=0x7f13ced5fc17 "UTF-8",
    p=0x55bf9c5313e0 "\001\002\001\001\002\001\001\002\003\001\001\001\001\001\001\001\001src/interface/QueueView.cpp") at unilbrk/ulc-width-linebreaks.c:120
#7  0x00007f13ced42cdb in ?? () from /usr/lib/libgettextsrc-0.19.8.1.so
#8  0x00007f13ced44330 in ?? () from /usr/lib/libgettextsrc-0.19.8.1.so
#9  0x00007f13ced41584 in msgdomain_list_print () from /usr/lib/libgettextsrc-0.19.8.1.so
#10 0x000055bf9b265b7e in ?? ()
#11 0x00007f13cef8cad6 in __libc_start_main (main=0x55bf9b265288, argc=5, argv=0x7ffecef34668)
    at src/env/__libc_start_main.c:74
#12 0x000055bf9b265bd3 in ?? ()
#13 0x0000000000000000 in ?? ()
````

Natanael Copa <ncopa>

 

Attached Files
file #44755:  test-suite.log added by ncopa (11KiB - text/x-log - gettext-tools 0.19.8.1: tests/test-suite.log)
file #44754:  libunistring-0.9.10-r0.log added by ncopa (500KiB - text/x-log - build log of libunistring)
file #44753:  lt_LT.po added by ncopa (326KiB - text/x-gettext-translation)
file #44751:  he_IL.po added by ncopa (272KiB - text/x-gettext-translation)
file #44752:  filezilla.pot added by ncopa (221KiB - application/vnd.ms-powerpoint)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haible (Posted a comment)
  • -email is unavailable- added by ncopa (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-08-09 haible StatusNone => Need Info
    2018-08-09 ncopa Attached File- => Added test-suite.log, #44755
    2018-08-09 ncopa Attached File- => Added libunistring-0.9.10-r0.log, #44754
    2018-08-09 ncopa Attached File- => Added lt_LT.po, #44753
    2018-08-09 ncopa Attached File- => Added he_IL.po, #44751
        Attached File- => Added filezilla.pot, #44752

    Back to the top


    Powered by Savane 3.5