glibc Bugzilla entry: http://sourceware.org/bugzilla/show_bug.cgi?id=4416

Debian glibc bug tracker entry: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=281313%3A

This has now been fixed (as a bug in glibc).

I looked at the bug again, and now I think I understand it
better.  You're calling "sort" twice, in two different
environments.  In one environment "sort' has an artificially
restricted amount of memory, so that the call to "setlocale"
fails, and "sort" behaves as if LC_ALL is equal to C.  In
the other, there is no such restriction and "sort" behaves
as if LC_ALL is equal to en_GB.ISO-8859-15.  The two locales
are not consistent, so "sort" sorts differently under them.

You suggested 'if sort is unable to produce an output that
would pass a "sort -c" check, I think it should exit with a
nonzero status (probably 2).'  But if you invoke "sort -c"
in the same environment as "sort", it will pass the "sort
-c" check.  So I don't think this is a serious bug.

One possible fix would be for "sort" to report an error
message if setlocale fails.  However, this would have
negative consequences.  I've been on systems that behave
that way, and whenever you set the locale to something
"wrong", you start getting diagnostics from nearly every
program that you run.  The normal Unix utility tradition is
therefore to ignore setlocale failures, and silently fall
back on the POSIX locale.

A better fix might be to modify glibc setlocale to set errno
reliably (currently it doesn't), and to report an error if
we are using glibc and if the errno value indicates a memory
problem.  This will require a change to glibc.  Perhaps you
could tackle that task?

No activity for a while on this one, so I assume the bug got fixed in glibc or something.

I suspect this is a problem with your GNU C library setup, not with "sort" per se.  glibc attempts to mmap the locale-archive file as a performance speedup, and falls back on the locale.alias file if the locale-archive can't be mmapped (which in this case it can't, because you're low on memory).  Perhaps these two sources of locale information are inconsistent on your host.  Or possibly you have found a glibc bug. Most likely you can work around the bug by setting LC_ALL="C" in your environment.

I normally run with $LANG set.   If I artificially restrict the amount of VM that sort can use, it will exit with return status 0, and yet the output file will fail the check performed by sort -c with the same value for $LANG. 

If I unset $LANG or I don't restrict the amount of VM available, everything works fine.  If sort is unable to produce an output that would pass a "sort -c" check, I think it should exit with a nonzero status (probably 2).

The problem can be reproduced with a three line test file, as shown below.    In the session below, I demonstrate the problem, unset LANG to show that the problem goes away, and then set it again and show that the problem comes back.

Specifying "-S8000" seems to alter the set of system calls made (no reference to /proc/meminfo) but doesn't change the behaviour.  I attach an "strace" output file of the initial sort, both a working example (without the VM limit) and a non-woring example.  Comparison of the two files shows that the significant difference in behaviour occurs where the program attempts to mmap() the file /usr/share/locale/locale.alias.



james@orbital:~$ cat -vet  testfile
/var/cache/apt/archives/libpopt-dev_1.7-5_i386.deb$
/var/cache/apt/archives/libpopt0_1.7-5_i386.deb$
/var/cache/apt/archives/libwrap0_7.6.dbs-6_i386.deb$
james@orbital:~$ ( ulimit  -v 1800 >/dev/null ; sort < testfile ) | sort -c
sort: -:2: disorder: /var/cache/apt/archives/libpopt0_1.7-5_i386.deb
james@orbital:~$ echo $LANG
en_GB.ISO-8859-15
james@orbital:~$ uname -a
Linux orbital 2.6.8-1-k7-smp #1 SMP Thu Oct 7 03:43:05 EDT 2004 i686 GNU/Linux
james@orbital:~$ unset LANG
james@orbital:~$ ( ulimit  -v 1800 >/dev/null ; sort < testfile ) | sort -c
james@orbital:~$ LANG=en_GB.ISO-8859-15; export LANG
james@orbital:~$ ( ulimit  -v 1800 >/dev/null ; sort < testfile ) | sort -c
sort: -:2: disorder: /var/cache/apt/archives/libpopt0_1.7-5_i386.deb
james@orbital:~$ ( ulimit  -v 1800 >/dev/null ; sort < testfile ) | cat -vet
/var/cache/apt/archives/libpopt-dev_1.7-5_i386.deb$
/var/cache/apt/archives/libpopt0_1.7-5_i386.deb$
/var/cache/apt/archives/libwrap0_7.6.dbs-6_i386.deb$