GNU Core Utilities - Bugs: bug #11004, Results of "sort" fail...
You are not allowed to post comments on this tracker with your current authentication level.
bug #11004: Results of "sort" fail "sort -c" check if LANG is set and memory is low
Submitter: | James Youngman <jay> | ||
Submitted: | Mon 15 Nov 2004 02:27:54 AM UTC | ||
Category: | None | Severity: | 3 - Normal |
Item Group: | None | Status: | Wont Fix |
Privacy: | Public | Assigned to: | None |
Open/Closed: | Open |
( Jump to the original submission )
Tue 15 May 2007 04:39:51 PM UTC, comment #11: |
Paul Eggert <eggert>![]() |
Tue 15 May 2007 02:35:59 PM UTC, comment #10: glibc Bugzilla entry: http://sourceware.org/bugzilla/show_bug.cgi?id=4416
Debian glibc bug tracker entry: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=281313%3A |
James Youngman <jay> |
Tue 15 May 2007 01:03:59 PM UTC, comment #9: How was it fixed in glibc? Do you have a pointer to the fix? Are any changes needed to GNU "sort"? |
Paul Eggert <eggert>![]() |
Tue 15 May 2007 08:41:47 AM UTC, comment #8: This has now been fixed (as a bug in glibc). |
James Youngman <jay> |
Thu 05 Oct 2006 01:26:55 PM UTC, comment #7: Please note the the two invocations of "sort" are always with the same value of $LANG. The only aspect of the environment which is different between the initial and the "-c" invocation of sort is the virtual memory limit (ulimit -v).
If the two invocations of sort were with different values of LC_ALL or LANG, I would strongly agree with you.
As it is, I would still assert that if LANG or LC_ALL is set, and sort produces output which is not correctly sorted according to the rules for the selected environment, it should return a nonzero status in order to indicate to the caller that it has failed.
I think then that your suggestion regarding the value returned by setlocale() and errno is probably the right answer.
|
James Youngman <jay> |
Wed 04 Oct 2006 05:28:28 PM UTC, comment #6: I looked at the bug again, and now I think I understand it
You suggested 'if sort is unable to produce an output that
One possible fix would be for "sort" to report an error
A better fix might be to modify glibc setlocale to set errno
|
Paul Eggert <eggert>![]() |
Wed 04 Oct 2006 09:47:25 AM UTC, comment #5: The bug still exists with glibc 2.3.6 and coreutils 5.97.
|
James Youngman <jay> |
Tue 19 Sep 2006 10:21:08 PM UTC, comment #4: No activity for a while on this one, so I assume the bug got fixed in glibc or something. |
Paul Eggert <eggert>![]() |
Tue 21 Dec 2004 09:01:42 PM UTC, comment #3: As suggested, I have raised this witht he Debian project as a possible glibc problem. You can visit the bug report at
|
James Youngman <jay> |
Tue 21 Dec 2004 07:15:54 AM UTC, comment #2: I suspect this is a problem with your GNU C library setup, not with "sort" per se. glibc attempts to mmap the locale-archive file as a performance speedup, and falls back on the locale.alias file if the locale-archive can't be mmapped (which in this case it can't, because you're low on memory). Perhaps these two sources of locale information are inconsistent on your host. Or possibly you have found a glibc bug. Most likely you can work around the bug by setting LC_ALL="C" in your environment. |
Paul Eggert <eggert>![]() |
Mon 15 Nov 2004 02:30:23 AM UTC, comment #1: I forgot to mention that this is with coreutils 5.2.1. |
James Youngman <jay> |
Mon 15 Nov 2004 02:27:54 AM UTC, original submission:
I normally run with $LANG set. If I artificially restrict the amount of VM that sort can use, it will exit with return status 0, and yet the output file will fail the check performed by sort -c with the same value for $LANG.
If I unset $LANG or I don't restrict the amount of VM available, everything works fine. If sort is unable to produce an output that would pass a "sort -c" check, I think it should exit with a nonzero status (probably 2).
The problem can be reproduced with a three line test file, as shown below. In the session below, I demonstrate the problem, unset LANG to show that the problem goes away, and then set it again and show that the problem comes back.
Specifying "-S8000" seems to alter the set of system calls made (no reference to /proc/meminfo) but doesn't change the behaviour. I attach an "strace" output file of the initial sort, both a working example (without the VM limit) and a non-woring example. Comparison of the two files shows that the significant difference in behaviour occurs where the program attempts to mmap() the file /usr/share/locale/locale.alias.
james@orbital:~$ cat -vet testfile
|
James Youngman <jay> |
Depends on the following items: None found
Items that depend on this one: None found
There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.
Follow 5 latest changes.
Date | Changed by | Updated Field | Previous Value | => | Replaced by |
---|---|---|---|---|---|
2006-10-04 | eggert | Open/Closed | Closed | ![]() |
Open |
2006-09-19 | eggert | Status | None | ![]() |
Wont Fix |
Open/Closed | Open | ![]() |
Closed | ||
2004-11-15 | jay | Attached File | - | ![]() |
Added TRACE.working.nopid.gz, #1871 |
2004-11-15 | jay | Attached File | - | ![]() |
Added TRACE.failing.nopid.gz, #1870 |
I'm afraid the glibc fix doesn't mean that we have an obvious fix for coreutils. I guess the idea is that we should go through the coreutils source and replace all instances of "setlocale (LC_ALL, "");" with "if (! setlocale (LC_ALL, "") && errno == ENOMEM) error (exit_failure, ENOMEM, "memory exhausted");". But we have no way of telling whether setlocale reliably sets errno. The Debian glibc bug tracker entry doesn't confirm that setlocale sets errno reliably in glibc 2.5. And even if it did, I don't offhand see any way at run-time, or even at configure-time, to tell whether the errno value is reliable. So I'm afraid coreutils is still stuck.