bugGNU Octave - Bugs: bug #40436, median no longer passes make check

 
 

bug #40436: median no longer passes make check

Submitted by:  Michael Godfrey <godfrey>
Submitted on:  Thu 31 Oct 2013 08:23:07 PM UTC  
 
Category:  Interpreter Severity:  5 - Blocker
Priority:  5 - Normal Item Group:  Regression
Status:  Fixed Assigned to:  John W. Eaton <jwe>
Originator Name:  Godfrey Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Thu 07 Nov 2013 09:47:40 PM UTC, comment #58: 

  PASS     11503
  FAIL         1
  XFAIL        6

and:
octave-gui:1> test median
PASSES 12 out of 12 tests
octave-gui:2>
+ more of the same.

So, it works for me!  Great!

Michael Godfrey <godfrey>
Project Member
Thu 07 Nov 2013 09:10:51 PM UTC, comment #57: 

I looked at the stl_algo.h file in 4.8.0 and 4.8.1 and it did not seem to have the same problem.  The change that introduced the bug is here:

http://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=12fc14041555b50c2fbe25175e9047d33e4baae4

It seems that it was a performance issue that prompted the change.
The bug report for that is here:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58437

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Thu 07 Nov 2013 08:59:43 PM UTC, comment #56: 

I am compiling this now.

The issue with other levels may bit be too severe.
It is claimed that 4.8.3 is fixed and 4.8.1 worked.
But, you never know...  Surely, not the last gcc bug.

Michael Godfrey <godfrey>
Project Member
Thu 07 Nov 2013 08:04:58 PM UTC, comment #55: 

I checked in the following changesets to work around the GCC bug:

http://hg.savannah.gnu.org/hgweb/octave/rev/540d1c122b08
http://hg.savannah.gnu.org/hgweb/octave/rev/0d660785098e
http://hg.savannah.gnu.org/hgweb/octave/rev/8b7bbe021d0f
http://hg.savannah.gnu.org/hgweb/octave/rev/51a3b3fa981f

It seems to work for me.

Note that it will warn but refuse to fix the problem if it is detected for any compiler other than g++ 4.8.2.  I don't know what else to do since the only thing we know how to fix is the problem with the 4.8.2 header file.  I also tried to ensure that a bits/stl_algo.h file that has been created in the build tree by a previous build is removed if it is no longer needed.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Thu 07 Nov 2013 03:36:04 AM UTC, comment #54: 

This problem is certainly a nuisance.  Since my first
suggestion seems not to work how about keeping a
corrected copy of the 4.8.1 bits/stl_algo.h and reference it
if the version of gcc is 4.8.1.

Or, at least, provide a script to be run as root
which applies the patch to the system stl_algo.h file?
Users who can su to root can just run it, others can
try to get their sysadmin to do it (in many cases the
sysadmin will, of course, refuse...).

If any procedure is chosen such that users could build
a system containing this bug, the build MUST fail unconditionally.

By the way, is there really no way to undefine a template
and then redefine it?

Michael Godfrey <godfrey>
Project Member
Thu 07 Nov 2013 01:40:48 AM UTC, comment #53: 

The tradeoff is between inconvenience for users who are stuck with this broken version of gcc and the maintenance burden on Octave to carry this fix for some period of time and any unforeseen problems that may arise.

I'm with Rik for declaring this to be a gcc bug that needs to be fixed by gcc/distributions/sysadmins, and maybe we make configure abort if it detects this version of gcc, but even provide an option to override the check if the user is certain that their gcc has been patched. For a similar example, see the configure.ac in this project I also work with:

http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/configure.ac#l421

This will affect me personally, it's not only Fedora, since my Debian dev system is also using gcc 4.8.2 as the default compiler now, so I will have to work around this myself for the time being. I consider that an acceptable cost for riding the bleeding edge, which is what Fedora always seems to be doing.

Mike Miller <mtmiller>
Project Administrator
Wed 06 Nov 2013 11:36:03 PM UTC, comment #52: 

Unfortunately, I don't think it is possible to redefine a template simply by doing something like

#include <some_header_that_defines_template_foo>

template <class T> foo (...) { }

The only way I know to do this is to get the compiler to see our replacement version of <some_header_that_defines_template_foo> instead of the system version.  It is possible to do that, but it seems somehow unpleasant to me because now Octave is distributing a header file from a specific version of GCC.  And we have to continue distributing that file as long as someone somewhere might be compiling with GCC 4.8.1.

OTOH, since the template is defined in the bits/stl_algo.h header file, it is possible for users (sysadmins) to fix this problem simply by editing the affected header file and making two trivial changes (in two locations, they need to change a '2' to a '1').  There is no need to recompile GCC or install different packages.  Now, I suppose some sysadmins will object to doing this since it alters some system files.  But that just seems stupid to me.  This is not some random fix that we picked up from some blog somewhere that we are suggesting.  It is a change that is already in the GCC source tree.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Wed 06 Nov 2013 11:03:20 PM UTC, comment #51: 

Rik,

I wish that your "six months" was all it will take.
But, Fedora 19 already fails, Fedora 20 will have the
bug for some time, and many people skip fedora releases
since each update is extra work. And, overriding
the standard packages in Fedora is more work. About
20 packages are required to make this gcc change.
Not many users will be willing or able to do this.

Is it true that the patch can go in data.cc overriding
the bad code in bits/dtl_algo.h defines? If this can
be done it does not seem like a lot of trouble or
"baggage."

We could try to move the problem to the various packagers.
The Fedora packagers already know about the problem, and
they have more leverage with the gcc folks...

However, if the patch is pretty simple I still would
like to see it go in.  This is not just because I am
one of the users who will have to fix it myself...

Michael Godfrey <godfrey>
Project Member
Wed 06 Nov 2013 10:37:08 PM UTC, comment #50: 

My vote earlier was to consider this a gcc bug, rather than an Octave bug.  It seems that the problem exists only for 4.8.2, and neither the previous 4.8.1 nor the following 4.8.3 are affected.  As such, it is a burden to maintain support for this very transitory bug.  I agree that it would be good to inform users and abort compiling Octave, but fixing it ourselves seems like too much work for a situation that is going to go away in just a few months.  Fedora 20 would be affected, but they release every 6 months and they may backport a working gcc if there are enough other bug reports about it.

Rik <rik5>
Project Administrator
Wed 06 Nov 2013 10:11:51 PM UTC, comment #49: 

John,

A problem with replacing the entire bits/stl_algo.h file is
that there seem to be frequent changes to this (large)
header.  I diff'ed the 8.4.1 and 8.4.2 files and lots of
changes were made. So, just replacing what is needed seems
better.  I prefer this choice.  But, it should also
provide a pretty loud message saying a bug in gcc had to
be corrected in the build.  Remember that gcc has a very
large number of bugs, so the developers cannot consider any
one of them to be of great importance.  (Maybe some priority
is given if the bug prevents compiling gcc or the kernel...)
Preventing users from being able to compile Octave without
extra trouble will tend to discourage users.

The baggage is a nuisance, but gcc updates are quite frequent
so it may not last all that long.

Obviously, do what you think is best.

Michael Godfrey <godfrey>
Project Member
Wed 06 Nov 2013 09:02:09 PM UTC, comment #48: 

Looking at bits/stl_algo.h, it seems that the affected functions, two variations of __unguarded_partition_pivot, are ultimately used in both std::nth_element and std::sort.  We use both in Octave.

Would it be better to replace the bits/stl_algo.h file when building Octave (can easily be done by putting a modified file in the Octave source tree and finding it with a -I compiler flag) or should we just provide replacements for std::sort and std::nth_element?  Or should we just fail to build Octave if this bug is found and tell people they need to install a version of GCC that does not have this problem?

I hesitate to install a fix in Octave (either replacing the header file or the functions) because we'll likely have to carry this baggage around for years to come.  If we just refuse to build, we'll be pushing people in the direction of fixing the problem.  And the fix is trivial and doesn't even require rebuilding the compiler.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Sun 03 Nov 2013 06:52:22 PM UTC, comment #47: 

I thought I also modified the state of this bug report and assigned it to myself, but when I checked just now the changes I thought I made weren't done, so I'm trying again.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Sun 03 Nov 2013 06:46:41 PM UTC, comment #46: 

Actually, John gave his fix in comment #40

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 06:19:10 PM UTC, comment #45: 

This is what John said he is doing now.
As he said on the bug_tracker list (but not here)

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 06:10:58 PM UTC, comment #44: 

If the upstream bug report I found is the cause of this error, then it should be possible to patch the 4.8.2 bits/stl_algo.h header file with the following change from svn:

http://gcc.gnu.org/viewcvs/gcc/branches/gcc-4_8-branch/libstdc%2B%2B-v3/include/bits/stl_algo.h?r1=203873&r2=203872&pathrev=203873

and rebuild with just that one change to verify.

If this header could somehow be included with Octave, but only used when gcc 4.8.2 is being used for compilation, that could work around this mess. We probably can't use this header file uncondtionally as it will likely be incompatible with other earlier or later gcc versions.

Mike Miller <mtmiller>
Project Administrator
Sun 03 Nov 2013 06:01:52 PM UTC, comment #43: 

Rik,

Does John's analysis convince you that fixing
bits/stl_algo.h corrects the problem?

One concern is that there are numerious other changes
in stl_algo.h between 8.4.1 and 8.4.2.  The Fedora
packaging folks might be able to raise this to a
level that it gets fixed sooner.

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 05:47:36 PM UTC, comment #42: 

configure.ac extracts the version of gcc.  It would be possible to put in a test to abort the compilation if 4.8.2 is detected.  That at least prevents incorrect answers from being promulgated.

There is still a question abouth whether the problem is with libstdc++-4.8.2 or gcc-4.8.2.  Usually it's not worth deciphering because people upgrade both packages at the same time.  However, I suppose there is the possibility of someone using a different compiler that links against stdc++.  This might expose the bug again if the real problem is the library rather than the compiler.

Finally, has someone verified that 4.8.3 fixes this?  I don't feel bad about blacklisting a single version, but if this is something that gcc isn't going to fix for a while then we might need a different strategy.

Rik <rik5>
Project Administrator
Sun 03 Nov 2013 05:40:25 PM UTC, comment #41: 

I also agree that it would also be good to try to get people who package GCC to fix the bug in their distributions.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Sun 03 Nov 2013 05:39:38 PM UTC, comment #40: 

As noted in comment 30, it looks like a bug in std_algo.h in GCC 4.8.2.  Debian testing has that version now, though my system still has 4.7.3 as the default compiler version.

In any case, I think we should test for this bug and work around it.

One way to do that would be to replace nth_element with our own, but that doesn't really fix the bug.  How do we know that the buggy utility function isn't used somewhere else?

Another way would be to provide a replacement of the buggy utility function.  But since it is a template, they only way I can see to do that is to replace the bits/stl_algo.h file with a fixed version.

I modified my copy of stl_algo.h with the fix given here:

http://gcc.gnu.org/viewcvs/gcc?view=revision&revision=203874

and then all the tests pass for median.m.  I'll work on a configure test and patch.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Sun 03 Nov 2013 05:30:32 PM UTC, comment #39: 

libstdc++-4.8.2-1. in Fedora appears to be where the
error is (in nth_element).  This is in final testing
for Fedora 20.

If someone knows how to reach the Fedora Octave
packagers, it would be very good to inform them of
this error.

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 04:59:19 PM UTC, comment #38: 

Although I'm listed as the author of median.m and probably did write the original version, it looks like Jarslav rewrote it almost completely:

http://hg.savannah.gnu.org/hgweb/octave/annotate/2ff47214de72/scripts/statistics/base/median.m

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Sun 03 Nov 2013 04:34:26 PM UTC, comment #37: 

Well:

1. Blacklisting 4.8.2 may be the best choice, but
   it appears that it will mean blacklisting fedora 20
   for users who do not want to build the whole set
   of 4.8.1 or maybe 4.8.3 stuff from source. The set
   that has to be removed and recompiled is about 20
   large RPMs.

2. If you try "help nth_element" it gives a sort description
   of the tradeoff between it and sort. Changing nth_element
   to actually use sort(x)(n) is pretty simple. If there
   is a simple way to test for 4.8.2 within nth_element
   the switch to sort could be made only if the lib
   is level 4.8.2.

3. The fact that the problem is in 4.8.2 does not mean we
   should just say "not our fault."  An Octave user who
   gets wrong ansers for median or nth_element will be
   upset and will view the error as an Octave error.
   We should not knowingly produce wrong answers regardless
   of whose fault it is.

Finally, often the wrong data are obvious garbage,
but when I was working on this I noticed that fairly often
the incorrect elements were reasonable looking floating
point, sometimes only off from the correct value by a factor
of 2 or so.  This means that leaving things as they are would
mean that there was a good chance that some users would get
wrong answers without any obvious indication.  While wrong
answers do happen, it is unacceptable to knowingly allow them.

I will not be party to such a choice.  If this goes ahead
I will have to blacklist Octave 3.8.  And, of course, I would
not use it myself without a patch to nth_element.

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 04:02:46 PM UTC, comment #36: 

We don't want to fix what is an upstream error by using a massive workaround.  We call std::nth_element to do the work and if core routines in libstdc++ are buggy for 4.8.2 I would just say that we blacklist 4.8.2.

Rik <rik5>
Project Administrator
Sun 03 Nov 2013 03:48:52 PM UTC, comment #35: 

Yes I think it is a good idea to release 3.8 with this known error.

There is no bug to be fixed in Octave.

We are fairly certain it is a bug in gcc 4.8.2, since only you and I are seeing it with that version, and not with 4.8.1 or any earlier version. Someone could build the current gcc 4.8 branch from svn to see if the error goes away.

How would you change the algorithm? Surely not by doing a complete sort of an array when only a partial sort is required, just to fix this for one buggy version of gcc?

Mike Miller <mtmiller>
Project Administrator
Sun 03 Nov 2013 02:48:23 PM UTC, comment #34: 

Is it a good idea to consider release of 3.8
with this bug?

I do not think that waiting for all systems which
are on 4.8.2 to move to 4.8.3 is viable.  It is not yet
verified that the problem is fixed in 4.8.3 in
any case.

Is there anything wrong with changing the algorithm in
nth_element in data.cc?  This would permit going
ahead with the release plan.

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 01:28:53 PM UTC, comment #33: 

Sounds good to me.

Also note that mxe-octave is currently on gcc 4.8.1. If it is updated to 4.8.2 this bug will start appearing in the cross-built Windows binaries.

Mike Miller <mtmiller>
Project Administrator
Sun 03 Nov 2013 05:43:23 AM UTC, comment #32: 

So is the consensus that this is not an Octave bug, but a gcc one?  If so, it's not really our responsibility to fix it before the release.

Rik <rik5>
Project Administrator
Sun 03 Nov 2013 03:57:16 AM UTC, comment #31: 

I forgot to paste some intermediate results I found.

I generated a random vector of length 100 and call nth_element with argument 98:100.

Before calling std::nth_element at http://hg.savannah.gnu.org/hgweb/octave/file/efbed19b8e04/liboctave/util/oct-sort.cc#l1916

data[97] = 0.162404
data[98] = 0.154987
data[99] = 0.964765

Afterwards:

data[97] = -4.11178e-50
data[98] = -6.48408e-256
data[99] = 0.993401

Mike Miller <mtmiller>
Project Administrator
Sun 03 Nov 2013 03:53:40 AM UTC, comment #30: 

It is perhaps this gcc bug:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58800

That bug indicates a segfault, which we don't see here, but in our case the random values we are seeing could be from a pointer indexing out-of-range which is what this bug seems to have been addressing.

Fixed in gcc 4.8.3, wait for distros to get that out and see if this goes away.

Mike Miller <mtmiller>
Project Administrator
Sun 03 Nov 2013 02:55:57 AM UTC, comment #29: 

Looking a bit more at the documentation, would
it make sense to just use the sort(x)(n) algorithm?
The nth_element algorithm is claimed to be faster
only for samll length (N)/size (X, DIM).
Would this change have a significant performance
penalty?

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 02:48:12 AM UTC, comment #28: 

But, it will still take some work to
isolate what has gone wrong with liboctave
Array::nth_element. And, how to make
it work with whatever has changed in gnu
nth_element code, or wherever the problem
really lies.

Michael Godfrey <godfrey>
Project Member
Sun 03 Nov 2013 12:08:01 AM UTC, comment #27: 

Very good!  I was just looking at nth_element.
Any ideas what can be done about this?  A bug
report to the GNU folks is probably called for.
At least they do put out frequent updates. In
fact there is already a newer set beyond 4.8.2-1.

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 11:15:16 PM UTC, comment #26: 

I've traced this down to liboctave Array::nth_element, here's an example using the call to nth_element that the failing median test case makes. It does look like uninitialized memory, because it's random which element of this result is wrong, and it doesn't happen for me every time.

octave-cli:1> b = rand (3, 4, 6, 5);
octave-cli:2> tmp = nth_element (b, 3:4, 3)
tmp =

ans(:,:,1,1) =

   0.72585   0.35493   0.89880   0.37920
   0.75690   0.62664   0.54848   0.12803
   0.38214   0.57314   0.23438   0.86337

ans(:,:,2,1) =

   0.90676   0.63536   0.92257   0.53978
   0.79579   0.73971   0.73077   0.68045
   0.67474   0.64598   0.59156   0.95027

ans(:,:,1,2) =

  -1.4411e+296    2.5586e-01    4.0583e-01    3.5271e-01
    4.8441e-01    3.8306e-01    4.3900e-01    2.4635e-01
    3.7203e-01    3.5613e-01    6.8720e-01  -7.4608e+232

ans(:,:,2,2) =

  -3.2079e+176    2.5827e-01    5.8781e-01    6.4392e-01
    6.6424e-01    5.7252e-01    5.2495e-01    3.6968e-01
    5.1522e-01    4.7913e-01    8.9407e-01  -1.3622e-138

ans(:,:,1,3) =

   0.46609   0.48197   0.47079   0.35431
   0.36301   0.31437   0.54804   0.62638
   0.40946   0.48275   0.61334   0.19517

ans(:,:,2,3) =

   0.70109   0.56286   0.69651   0.44498
   0.54481   0.39687   0.65335   0.75202
   0.83503   0.61924   0.90139   0.72982

ans(:,:,1,4) =

   0.51566   0.17927   0.60041   0.44344
   0.56499   0.00000   0.50164   0.21559
   0.21606   0.26750   0.39789   0.49834

ans(:,:,2,4) =

   0.55074   0.18725   0.61739   0.61690
   0.56823   0.00000   0.86750   0.34780
   0.34628   0.39065   0.41223   0.54986

ans(:,:,1,5) =

   0.22879   0.11877   0.34887   0.64783
   0.50557   0.43964   0.00000   0.44355
   0.59332   0.67293   0.55430   0.36468

ans(:,:,2,5) =

   0.56224   0.54101   0.51298   0.68682
   0.57307   0.51974   0.21241   0.61531
   0.59603   0.74638   0.82469   0.50562

Mike Miller <mtmiller>
Project Administrator
Sat 02 Nov 2013 10:46:39 PM UTC, comment #25: 

Right!  Puzzling over median.m now.
John wrote it.  Maybe he could take a
look, too.!!!

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 10:39:50 PM UTC, comment #24: 

It must be wrong.  After all, you started with a call to rand() which only has values in the range [0,1] and the last value in column 1 is -2e+219.  That sure looks like a read of unitialized memory.

Rik <rik5>
Project Administrator
Sat 02 Nov 2013 10:25:51 PM UTC, comment #23: 

To be a bit more specific,
b = rand (3,4,6,5);
median(b,3);
Sometimes produces valid data except:
ans(:,:,1,2) =

    6.8587e-01    4.2765e-01    2.4749e-01    1.7470e-01
    2.9131e-01    3.9197e-01    6.1856e-01    6.2355e-01
  -2.4950e+219    6.3926e-01    2.2892e-01    3.7294e-01

assert correctly says that the last entry in col 1 is
wrong.

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 10:19:01 PM UTC, comment #22: 

I was just doing that.  And, I have discovered that it
is the median(b,3) call that fails.

assert appears to run correctly.

Time to work on median.m  Simpler, I hope.

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 09:50:48 PM UTC, comment #21: 

When stepping, check what the value af A_null and B_null are.  They seem to have some crazy values in them.

Rik <rik5>
Project Administrator
Sat 02 Nov 2013 08:52:33 PM UTC, comment #20: 

I tried gcc -g -O3. No luck.

Next, some more stepping through assert.m

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 01:37:07 PM UTC, comment #19: 

I tend to blame gcc.  4.8.2 is described as a "bug fix"
update.  But, with 70 bugs fixed it is pretty likely
that new ones have been introduced.  This has happened
plenty of times in the past.

Trying to run different levels of gcc and the lib elements
is not simple and tends to cause other problems. To do this
using the distributed RPMs means breaking the dependency
rules.

One question: is there a reason why the tests for assert.m
are not at the bottom of the file?  There are 2 internal
functions which are after the test block.  I moved the tests to
the end of the file and it did not appear to make any difference.

I have a few more ideas, but I am not hopeful of finding
a work around.

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 06:15:27 AM UTC, comment #18: 

This does start to sound like there is an issue with gcc (it does happen) since the exact same source code and same options works under 4.8.1 and fails under 4.8.2.

If you want to be thorough you could try and see if there is a problem with the compiler or with the libc library.  Try upgrading just the library components such as libstdc++.  Now try running the octave compiled under 4.8.1.  If it has problems then there has actually been a regression in one of the libraries (this is rarer than a regression in gcc).

Rik <rik5>
Project Administrator
Sat 02 Nov 2013 05:17:03 AM UTC, comment #17: 

I tried a number of things in assert.m, but none
had any real effect. A change would appear to work
and then fail...

Something is not being initialized.

Tomorrow I may try other gcc options.

Running under gcc 4.8.1 always works.

Michael Godfrey <godfrey>
Project Member
Sat 02 Nov 2013 02:08:48 AM UTC, comment #16: 

I get this error also now, gcc 4.8.2 on Debian. It happens intermittently when run as

$ ./run-octave --eval "test median"

Seems to fail all the time when run interactively, maybe 4/5 times when run from the shell as above.

Mike Miller <mtmiller>
Project Administrator
Sat 02 Nov 2013 01:20:00 AM UTC, comment #15: 

I tried compiling with -g -O0, but
this did no good.

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 09:16:55 PM UTC, comment #14: 

OK. Backing out gcc and its dependencies
(see attached file: fc19_downgrade) makes
median.m pass its tests, both in make check
and test median.

This means that the error is not simply in
assert, but in some compiled code used by
assert who's behavior has changed.

This could affect other code, of course.

So, I am re-upgrading my main system, and
will try to find the difference in behavior of
assert on it and on my backup.

Any suggestions would be welcome.

(file #29525)

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 06:56:44 PM UTC, comment #13: 

I will work on this soon, following your suggestions.
This is fairly important since this level of Fedora or
above will be the current release by the time 3.8 is
released.

It is very odd that this problem only affects. the median
tests.

Also, do you think that A_null in assert should be complex?
Seems very odd to me.  What does the median of real numbers
have to do with complex variables?

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 06:18:02 PM UTC, comment #12: 

The difference in octave prompt between octave-cli and octave-gui was addressed by jwe on the maintainer's list.  It should be cosmetic and not a source of the problem.

I do see that you updated not only your gcc compiler but also libc.  That is a pretty big change.  It's hard to say though who is right.  It may be that the new gcc/libc is not initializing some value in which case it is gcc's problem.  Or it may be that Octave was not initializing something and relying on some default behavior, in which case the fix should be in Octave.  You have already determined that it wasn't LLVM which caused the problem.  You might try updating all the other packages except those that involve gcc and libc and see what happens.

Rik <rik5>
Project Administrator
Fri 01 Nov 2013 05:31:58 PM UTC, comment #11: 

OK. Here is some "good" news, or what must
pass for good news about this problem:

Building on my backup system which was last
updated using yum on 27 Oct. the median test
in make check passes. And, test median also
works correctly. The backup and my main
system are very nearly identical with respect
to Fedora configuration.  The backup is
only updated after updates seem OK on main system.
This procedure seems to have paid off in this case.

So, something in the set of updates since 27 Oct. has
caused this problem. I do not have an efficient
strategy for closing down on which RPM has caused
this problem.  Any suggestions?  Some RPMs are
easy to back out, others are not.

On the backup, I still get the prompt:
octave-gui:1>
when I type: ./run-octave --no-gui

Another point about the test failures. For each new
invocation of Octave some random number turns up in
the assert fail display. This number stays the same
within that execution, but a new fail value appears
for each new invocation of Octave.  Somehow, assert
gets garbage and then remembers it!

Also, the prompt: octave-gui for ./run-octave --no-gui
seems to be a feature.  It happens on the backup
system which runs test median correctly.  And, with
this version of Octave installed, then: octave --no-gui
gives the correct prompt: octave:1>

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 04:51:52 PM UTC, comment #10: 

I should also have said that when I quit Octave
after ./run-octave --no-gui

>>then run ther median tests, and dbquit

and quit
I get:
octave-gui:12> quit

warning: reloading /run/media/godfrey/GoFlex08/qss/d/src/octave/hg/octave/libinterp/dldfcn/__init_fltk__.oct clears the following functions:
warning:   _have_fltk_
warning:   _init_fltk_
warning: library /run/media/godfrey/GoFlex08/qss/d/src/octave/hg/octave/libinterp/dldfcn/__init_fltk__.oct not reloaded due to existing references
===============================
It appears that something is not quite right in the new
gui/cli setup.

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 04:47:18 PM UTC, comment #9: 

One more item to report: After
./run-octave --no-gui I get:
octave-gui:1>

Is this expected?

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 04:19:18 PM UTC, comment #8: 

System compiled with --disable-jit also
fails in essentially the same way.
One good thing!

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 03:47:12 PM UTC, comment #7: 

Since you are suspicious of LLVM (and it was just updated
on Fedora) I am building with it disabled.
More when the build completes.

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 03:44:03 PM UTC, comment #6: 

Here is what happens consistently on my system
(fedora 19 x86_64):
I have the lines below in a script.
If the script is run assert always fails.
If the last line of the script is then run it passes.
The failure with debug_on_error(true) is at line 375,
which is obvious. But, at that point
erridx = [](0x1)
and A_null=

ans(:,:,1,1) =

   0.66855   0.54710   0.53196   0.29479
   0.66464   0.51617   0.55531   0.69112
   0.32747   0.56386   0.36310   0.65944

ans(:,:,1,2) =

    6.8587e-01    4.2765e-01    2.4749e-01    1.7470e-01
    2.9131e-01    3.9197e-01    6.1856e-01    6.2355e-01
  -5.6666e+171    6.3926e-01    2.2892e-01    3.7294e-01

ans(:,:,1,3) =

   0.32483   0.63250   0.34819   0.45336
   0.49242   0.59707   0.57103   0.33424
   0.62111   0.37522   0.44119   0.63128

ans(:,:,1,4) =

   0.26956   0.61410   0.43620   0.36742
   0.46206   0.64779   0.62195   0.39042
   0.57359   0.70165   0.38829   0.19057

ans(:,:,1,5) =

   0.56813   0.61866   0.61677   0.29744
   0.23062   0.66617   0.27415   0.63008
   0.74869   0.34095   0.51082   0.23169

debug>
ans(:,:,1,2) is wrong.

One more fact: I recently ran yum update and
a lot of relevant packages were updated:

Oct 30 16:44:24 localhost yum[14603]: Updated: libgcc-4.8.2-1.fc19.x86_64
Oct 30 16:44:25 localhost yum[14603]: Updated: libstdc++-4.8.2-1.fc19.x86_64
Oct 30 16:44:25 localhost yum[14603]: Updated: libsss_idmap-1.11.1-4.fc19.x86_64
Oct 30 16:44:26 localhost yum[14603]: Updated: gnutls-3.1.15-1.fc19.x86_64
Oct 30 16:44:29 localhost yum[14603]: Updated: libstdc++-devel-4.8.2-1.fc19.x86_64
Oct 30 16:44:30 localhost yum[14603]: Updated: xorg-x11-server-common-1.14.3-2.fc19.x86_64
Oct 30 16:44:30 localhost yum[14603]: Updated: libquadmath-4.8.2-1.fc19.x86_64
Oct 30 16:44:30 localhost yum[14603]: Updated: gnutls-dane-3.1.15-1.fc19.x86_64
Oct 30 16:44:32 localhost yum[14603]: Updated: llvm-libs-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:44:33 localhost yum[14603]: Updated: llvm-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:44:34 localhost yum[14603]: Updated: libgfortran-4.8.2-1.fc19.x86_64
Oct 30 16:44:34 localhost yum[14603]: Updated: gnutls-c++-3.1.15-1.fc19.x86_64
Oct 30 16:44:36 localhost yum[14603]: Updated: gnutls-devel-3.1.15-1.fc19.x86_64
Oct 30 16:44:36 localhost yum[14603]: Updated: qpdf-libs-5.0.1-1.fc19.x86_64
Oct 30 16:44:37 localhost yum[14603]: Updated: libgomp-4.8.2-1.fc19.x86_64
Oct 30 16:44:39 localhost yum[14603]: Updated: cpp-4.8.2-1.fc19.x86_64
Oct 30 16:44:41 localhost yum[14603]: Updated: gcc-4.8.2-1.fc19.x86_64
Oct 30 16:44:45 localhost yum[14603]: Updated: clang-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:44:45 localhost yum[14603]: Updated: libquadmath-devel-4.8.2-1.fc19.x86_64
Oct 30 16:44:45 localhost yum[14603]: Updated: python-sssdconfig-1.11.1-4.fc19.noarch
Oct 30 16:44:46 localhost yum[14603]: Updated: libsoup-2.42.2-5.fc19.x86_64
Oct 30 16:44:47 localhost yum[14603]: Updated: sssd-client-1.11.1-4.fc19.x86_64
Oct 30 16:44:48 localhost yum[14603]: Updated: sssd-common-1.11.1-4.fc19.x86_64
Oct 30 16:44:49 localhost yum[14603]: Updated: sssd-krb5-common-1.11.1-4.fc19.x86_64
Oct 30 16:44:49 localhost yum[14603]: Updated: sssd-common-pac-1.11.1-4.fc19.x86_64
Oct 30 16:44:50 localhost yum[14603]: Updated: sssd-ad-1.11.1-4.fc19.x86_64
Oct 30 16:44:50 localhost yum[14603]: Updated: sssd-krb5-1.11.1-4.fc19.x86_64
Oct 30 16:44:51 localhost yum[14603]: Updated: sssd-ldap-1.11.1-4.fc19.x86_64
Oct 30 16:44:51 localhost yum[14603]: Updated: sssd-proxy-1.11.1-4.fc19.x86_64
Oct 30 16:44:53 localhost yum[14603]: Updated: gcc-base-debuginfo-4.8.2-1.fc19.x86_64
Oct 30 16:44:53 localhost yum[14603]: Installed: perl-Future-0.18-1.fc19.noarch
Oct 30 16:44:54 localhost yum[14603]: Updated: libipa_hbac-1.11.1-4.fc19.x86_64
Oct 30 16:44:54 localhost yum[14603]: Updated: sssd-ipa-1.11.1-4.fc19.x86_64
Oct 30 16:44:55 localhost yum[14603]: Updated: cups-filters-libs-1.0.40-4.fc19.x86_64
Oct 30 16:44:55 localhost yum[14603]: Installed: perl-Try-Tiny-0.12-2.fc19.noarch
Oct 30 16:44:56 localhost yum[14603]: Installed: perl-Test-Fatal-0.010-5.fc19.noarch
Oct 30 16:44:56 localhost yum[14603]: Updated: perl-IO-Async-0.60-1.fc19.noarch
Oct 30 16:44:59 localhost yum[14603]: Updated: cups-filters-1.0.40-4.fc19.x86_64
Oct 30 16:45:01 localhost yum[14603]: Updated: sssd-1.11.1-4.fc19.x86_64
Oct 30 16:45:59 localhost yum[14603]: Updated: gcc-debuginfo-4.8.2-1.fc19.x86_64
Oct 30 16:46:00 localhost yum[14603]: Updated: libsoup-devel-2.42.2-5.fc19.x86_64
Oct 30 16:46:01 localhost yum[14603]: Updated: gcc-gfortran-4.8.2-1.fc19.x86_64
Oct 30 16:46:02 localhost yum[14603]: Updated: clang-analyzer-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:46:04 localhost yum[14603]: Updated: clang-devel-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:46:05 localhost yum[14603]: Updated: gcc-c++-4.8.2-1.fc19.x86_64
Oct 30 16:46:06 localhost yum[14603]: Updated: libtool-2.4.2-21.fc19.x86_64
Oct 30 16:46:07 localhost yum[14603]: Updated: clang-doc-3.3-0.10.rc3.fc19.noarch
Oct 30 16:46:08 localhost yum[14603]: Installed: llvm-doc-3.3-0.10.rc3.fc19.noarch
Oct 30 16:46:10 localhost yum[14603]: Updated: llvm-devel-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:46:10 localhost yum[14603]: Updated: gnutls-utils-3.1.15-1.fc19.x86_64
Oct 30 16:46:11 localhost yum[14603]: Updated: xorg-x11-server-Xephyr-1.14.3-2.fc19.x86_64
Oct 30 16:46:12 localhost yum[14603]: Updated: xorg-x11-server-Xorg-1.14.3-2.fc19.x86_64
Oct 30 16:46:13 localhost yum[14603]: Updated: xorg-x11-server-Xvfb-1.14.3-2.fc19.x86_64
Oct 30 16:46:14 localhost yum[14603]: Updated: perl-HTML-Parser-3.71-1.fc19.x86_64
Oct 30 16:46:14 localhost yum[14603]: Updated: libtool-ltdl-2.4.2-21.fc19.x86_64
Oct 30 16:46:15 localhost yum[14603]: Updated: diffutils-3.3-4.fc19.x86_64
Oct 30 16:46:16 localhost yum[14603]: Updated: python-markupsafe-0.18-1.fc19.x86_64
Oct 30 16:46:16 localhost yum[14603]: Updated: createrepo-0.10-1.fc19.noarch
Oct 30 16:46:17 localhost yum[14603]: Updated: lirc-libs-0.9.0-17.fc19.x86_64
Oct 30 16:46:45 localhost yum[14603]: Updated: llvm-debuginfo-3.3-0.10.rc3.fc19.x86_64
Oct 30 16:46:45 localhost yum[14603]: Updated: ebook-tools-libs-0.2.2-1.fc19.x86_64
Oct 30 16:46:46 localhost yum[14603]: Updated: libgcc-4.8.2-1.fc19.i686
Oct 30 16:46:46 localhost yum[14603]: Updated: libstdc++-4.8.2-1.fc19.i686
Oct 30 16:46:47 localhost yum[14603]: Updated: libgomp-4.8.2-1.fc19.i686
Oct 30 16:46:48 localhost yum[14603]: Updated: gnutls-3.1.15-1.fc19.i686
Oct 30 16:46:48 localhost yum[14603]: Updated: libtool-ltdl-2.4.2-21.fc19.i686
Oct 31 08:46:10 localhost yum[29754]: Updated: 1:openssl-libs-1.0.1e-30.fc19.x86_64
Oct 31 08:46:11 localhost yum[29754]: Updated: gpaste-libs-3.2.2-1.fc19.x86_64
Oct 31 08:46:12 localhost yum[29754]: Updated: gpaste-3.2.2-1.fc19.x86_64
Oct 31 08:46:14 localhost yum[29754]: Updated: 1:openssl-devel-1.0.1e-30.fc19.x86_64
Oct 31 08:46:15 localhost yum[29754]: Updated: 1:openssl-1.0.1e-30.fc19.x86_64
Oct 31 08:46:16 localhost yum[29754]: Updated: 2:tar-1.26-27.fc19.x86_64
Oct 31 08:46:18 localhost yum[29754]: Updated: poppler-data-0.4.6-4.fc19.noarch
Oct 31 08:46:22 localhost yum[29754]: Updated: cmake-2.8.12-2.fc19.x86_64
Oct 31 08:46:23 localhost yum[29754]: Updated: 1:openssl-libs-1.0.1e-30.fc19.i686
[root@pbdsl3 log]#

And, I just noticed that in assert A_null is complex. Is
this expected??
stopped in /d/src/octave/hg/octave/scripts/testfun/assert.m
debug> whos
Variables in the current scope:

   Attr Name             Size                     Bytes  Class
   ==== ====             ====                     =====  =====
        A                3x4x1x5                    480  double
   c    A_null           3x4x1x5                    960  double
        A_null_imag      3x4x1x5                    480  double
        A_null_real      3x4x1x5                    480  double
        B                3x4x1x5                    480  double
   c    B_null           3x4x1x5                    960  double
        B_null_imag      3x4x1x5                    480  double
        B_null_real      3x4x1x5                    480  double
        ans              1x1                          8  double
        argin            1x53                        53  char
  a     argn             2x37                        74  char
      p call_depth       1x1                          8  double
    f   cond             3x4x1x5                    480  double
        err              1x1                         51  struct
        erridx           0x1                          0  double
 ==================================

Something has seriously confused asset().  erridx
should not be empty...

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 03:09:44 PM UTC, comment #5: 

I can't reproduce this.

From the build directory, I did

./run-octave --no-gui
rand ("seed", 2);
a = rand (2,3,4,5);
b = rand (3,4,6,5);
x = sort (a, 4);
y = sort (b, 3);
assert (median (b, 3), (y(:, :, 3, :) + y(:, :, 4, :))/2);

And it passed without problem.  I tried various combinations with the gui, with '--no-gui-libs', with '-f' and without, and they all pass.  So I think it is local to your system.  Maybe you could try setting "debug_on_error (true)" before running the assert and you could find out why your getting a value that is 1e+88.

Rik <rik5>
Project Administrator
Fri 01 Nov 2013 02:14:49 PM UTC, comment #4: 

No cruft.  I have done 2 full installs from clone
since this problem appeared.  Note:
1. It only happens once per Octave invocation.
2. In make check it fist happens after other calls to
   assert.
3. If running test median from command line it only
   seems to happen on the sequence shown below.
4. I have also tried octave --non-init-file, but this
   has no effect.
5. test median always works for me too.  Try the sequence
   below. 

This is weird.

Michael Godfrey <godfrey>
Project Member
Fri 01 Nov 2013 02:08:28 PM UTC, comment #3: 

I wonder if you don't have cruft in your system.  Could you try an 'hg clone' to a brand new directory, and then bootstrap, configure, make, 'make test'?

I can run 'test median' and everything passes just fine.  But, I have also been running 'make maintainer-clean' and the whole bootstrap cycle because I have been fiddling with the Autotools stuff.

Rik <rik5>
Project Administrator
Thu 31 Oct 2013 08:53:18 PM UTC, comment #2: 

Additional information:
One the current dev system the failure
only occurs on the first use of assert:

octave-gui:1> rand ("seed", 2);
octave-gui:2> a = rand (2,3,4,5);
octave-gui:3> b = rand (3,4,6,5);
octave-gui:4> x = sort (a, 4);
octave-gui:5> y = sort (b, 3);
octave-gui:6> assert (median (b, 3), (y(:, :, 3, :) + y(:, :, 4, :))/2);
error: ASSERT errors for:  assert (median (b, 3),(y (:, :, 3, :) + y (:, :, 4, :)) / 2)

  Location  |  Observed  |  Expected  |  Reason
 (3,1,1,2)   -1.058402066116045e+88         0.6608      Abs err 1.0584e+88 exceeds tol 0
 (3,2,1,4)   0.3542919456958771        0.8        Abs err 0.44571 exceeds tol 0
octave-gui:6> assert (median (b, 3), (y(:, :, 3, :) + y(:, :, 4, :))/2);
octave-gui:7>

Further uses of assert work.

Michael Godfrey <godfrey>
Project Member
Thu 31 Oct 2013 08:34:38 PM UTC, comment #1: 

This appears to be a problem in assert since
median(b,3) == (y(:, :, 3, :) + y(:, :, 4, :))/2

produces the same result in both systems.
The results of the above is:
ans =

ans(:,:,1,1) =

   1   1   1   1
   1   1   1   1
   1   1   1   1

ans(:,:,1,2) =

   1   1   1   1
   1   1   1   1
   1   1   1   1

ans(:,:,1,3) =

   1   1   1   1
   1   1   1   1
   1   1   1   1

ans(:,:,1,4) =

   1   1   1   1
   1   1   1   1
   1   1   1   1

ans(:,:,1,5) =

   1   1   1   1
   1   1   1   1
   1   1   1   1

Michael Godfrey <godfrey>
Project Member
Thu 31 Oct 2013 08:23:07 PM UTC, original submission:  

I just checked:

1. Current dev system (31 Oct) fails.
2. System built on 26 Oct passes.

The file median.m has not been changed between these
2 systems.

Code that fails (from the median.m tests) is:

rand ("seed", 2);
a = rand (2,3,4,5);
b = rand (3,4,6,5);
x = sort (a, 4);
y = sort (b, 3);
assert (median (b, 3), (y(:, :, 3, :) + y(:, :, 4, :))/2);

error: ASSERT errors for:  assert (median (b, 3),(y (:, :, 3, :) + y (:, :, 4, :)) / 2)

  Location  |  Observed  |  Expected  |  Reason
 (3,1,1,2)   -6.060400129782798e+78         0.6608      Abs err 6.0604e+78 exceeds tol 0
 (3,2,1,4)   0.3134371936321259        0.8        Abs err 0.48657 exceeds tol 0

This was run from the command line using the 2 dev versions.
The Oct 26 version passes.

The same error
is written to fntests.log on current dev system.

Michael Godfrey <godfrey>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #29525:  fc19_downgrade added by godfrey (3KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by godfrey (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 19 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-11-07 rik5 StatusReady For Test => Fixed
        Open/ClosedOpen => Closed
    2013-11-07 jwe StatusIn Progress => Ready For Test
    2013-11-03 jwe Severity3 - Normal => 5 - Blocker
        StatusInvalid => In Progress
        Assigned toNone => jwe
        Open/ClosedClosed => Open
    2013-11-03 rik5 Severity5 - Blocker => 3 - Normal
        StatusIn Progress => Invalid
        Assigned tojwe => None
        Open/ClosedOpen => Closed
    2013-11-03 jwe Severity3 - Normal => 5 - Blocker
        StatusInvalid => In Progress
        Assigned toNone => jwe
        Open/ClosedClosed => Open
    2013-11-03 rik5 StatusNeed Info => Invalid
        Open/ClosedOpen => Closed
    2013-11-01 godfrey Attached File- => Added fc19_downgrade, #29525
    2013-11-01 rik5 StatusNone => Need Info

    Back to the top


    Powered by Savane 3.5