bugGNU Octave - Bugs: bug #48312, MXE Octave: test fails

 
 

bug #48312: MXE Octave: test fails

Submitter:  Avinoam Kalma <avinoam>
Submitted:  Sat 25 Jun 2016 09:15:42 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Avinoam Open/Closed:  * Closed
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 16 Nov 2016 08:49:31 PM UTC, comment #24: 

Most of the failures have been fixed, or changed to XFAIL.
The remaining failures (only in 64-bit Octave) have individual reports.

Closing as fixed.

Avinoam Kalma <avinoam>
Group Member
Fri 16 Sep 2016 06:33:21 AM UTC, comment #23: 

See also bug #49091

Philip Nienhuis <philipnienhuis>
Group Member
Thu 15 Sep 2016 09:06:43 PM UTC, comment #22: 

Instead of %!xtest, should the bugs use %!test <48312>?

Rik <rik5>
Group administrator
Mon 12 Sep 2016 06:29:32 PM UTC, comment #21: 

The 2 mappers.cc acosh errors from this report are not specific to windows. They also occur on RHEL 6.8, gcc 4.9.3, HG-ID 9d4cb0cf9cd2

Anonymous
Fri 01 Jul 2016 12:17:51 PM UTC, comment #20: 

After http://hg.savannah.gnu.org/hgweb/octave/rev/18af3ea1d387 there are only 7 FAILs o Windows.
Though I think we see convergence rather than divergence (= confusion) :-) I'll open separate bug reports for the remaining things.

Below I concentrate only on acos-related FAILs:
There are still two acosh-related errors left in mappers-cc.tst, looking like simple tolerance issues:

:
>>>>> processing L:\philip\devel\octdev\mxe\mxe_64b_20160623\dist\octave-2016-07-01-12-48\share\octave\4.1.0+\etc\tests\libinterp\corefcn\mappers.cc-tst
***** test
 re = 2.99822295029797;
 im = pi/2;
 assert (acosh (-10i), re - i*im);
!!!!! test failed
ASSERT errors for:  assert (acosh (-10i),re - i * im)

  Location  |  Observed  |  Expected  |  Reason
     ()      2.9982-1.5708i 2.9982-1.5708i   Abs err 6.2172e-015 exceeds tol 0
shared variables
  scalar structure containing the fields:

    rt2 =  1.4142
    rt3 =  1.7321
***** test
 re = single (2.99822295029797);
 im = single (pi/2);
 assert (acosh (single (10i)), re + i*im, 5*eps ("single"));
 assert (acosh (single (-10i)), re - i*im, 5*eps ("single"));
!!!!! test failed
ASSERT errors for:  assert (acosh (single (-10i)),re - i * im,5 * eps ("single"))

  Location  |  Observed  |  Expected  |  Reason
     ()      2.9982-1.5708i 2.9982-1.5708i   Abs err 7.1526e-006 exceeds tol 5.9605e-007
shared variables
  scalar structure containing the fields:

    rt2 =  1.4142
    rt3 =  1.7321
:


($ hg summary
parent: 22019:a49ed7b7b66f tip
 build: delete HG-ID on 'make distclean' and 'make maintainer-clean')

Philip Nienhuis <philipnienhuis>
Group Member
Thu 30 Jun 2016 09:23:16 PM UTC, comment #19: 

I have "fixed" four failures in mappers.cc by ignoring them for systems with buggy math libraries:

http://hg.savannah.gnu.org/hgweb/octave/rev/18af3ea1d387

I'd recommend opening discrete bugs for the remaining test failures (printd and tmpfile already have their own tracker items), since several have been fixed here, and this kind of bug report only gets more confusing the longer it stays open.

Mike Miller <mtmiller>
Group Member
Wed 29 Jun 2016 04:08:02 PM UTC, comment #18: 

The four FAILs that you are seeing with acos, acosh, asin, and asinh where the value is off by either pi or pi/2 are all new tests that I added recently (http://hg.savannah.gnu.org/hgweb/octave/rev/e9765b62d4e8). I may just change these into xtests since they fail on a variety of systems that don't have the newest libraries yet.

Mike Miller <mtmiller>
Group Member
Tue 28 Jun 2016 08:57:29 PM UTC, comment #17: 

@comment #16:
Done, see bug #48337

Well, 11 FAILs have been identified now. That's 11 more to go :-)
Maybe the resurfaced NaN/Inf mix-ups may also count as identified.

The acos() errors look more serious. The test results have been attached to comment #6.
Later this week I can investigate them, if no one else beats me to it, but still I can't fix them.

The assert.m and axis.m FAILs "have always been there" on my Windows systems so they don't bother me. Invoked separately (outside tests) they've always worked fine.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 28 Jun 2016 07:27:22 PM UTC, comment #16: 

Yes, tmpfile failing seems like it might be worth a bug report in its own right. The library call was changed recently from gnulib::tmpfile to std::tmpfile, so that may have exposed a bug on Windows that gnulib was solving for us.

Mike Miller <mtmiller>
Group Member
Tue 28 Jun 2016 07:18:32 PM UTC, comment #15: 

As to fwrite, the culprit is the call to tmpfile in L.605 of io-cc.tst:

>> id = tmpfile ()

id = -1


and later on fwrite tries to write to id.
Further down in io-cc.tst are 5 more of these constructs.

Shall I enter a separate bug report for tmpfile() ?

I'll look further into the other more isolated tests, so please leave this bug report open.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 28 Jun 2016 04:15:36 PM UTC, comment #14: 

@Mike comment #12
fwrite - I'll have a look tonight. Those tests in io-cc.tst look quite complicated when I peeked at them last weekend, so I may need a little time.

Thanks for the get*id bug fix.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 28 Jun 2016 04:58:49 AM UTC, comment #13: 

@Mike: thans for fixing the "test system.tst" failures.

I have opened a new bug report for the test printd, which was confirmed (bug #48326)

Avinoam Kalma <avinoam>
Group Member
Mon 27 Jun 2016 11:30:27 PM UTC, comment #12: 

Any idea where the fwrite errors are in io.tst?

I think it's expected that geteuid, getuid, getegid, and getgid fail on non-POSIX systems (because these are just thin wrappers around the system calls of the same names). I pushed an easy fix for that, they should show up as 4 skipped tests now:

http://hg.savannah.gnu.org/hgweb/octave/rev/e2e0e353c3ef

Mike Miller <mtmiller>
Group Member
Mon 27 Jun 2016 05:50:22 AM UTC, comment #11: 

Thanks.


[a,b,c,d] = test ("io.tst", "quiet", stdout)

yields 6 times "fwrite: invalid stream number = -1"


[a,b,c,d] = test ("system.tst", "quiet", stdout)
***** assert (geteuid () >= 0)
!!!!! test failed
assert (geteuid () >= 0) failed
***** assert (getuid () >= 0)
!!!!! test failed
assert (getuid () >= 0) failed
***** assert (getegid () >= 0)
!!!!! test failed
assert (getegid () >= 0) failed
***** assert (getgid () >= 0)
!!!!! test failed
assert (getgid () >= 0) failed


I'll look into the rest later (work week started)

Philip Nienhuis <philipnienhuis>
Group Member
Sun 26 Jun 2016 09:45:09 PM UTC, comment #10: 

Re fntests.log: maybe, no idea.

Don't use length, use size. I get r2 is a character matrix of size 7×30, r4 is a row vector of size 1×217. Then after the line r2=disp(r2), r2 also becomes a row vector of size 1×217 (the array is interpolated into a single string with newline characters). Since you have r4 of length 224, which is 217+7, I would bet money on r4 containing CR-NL sequences and r2 not, resulting in the test failing.

To run all tests in a file, use the function form of test() and request output arguments. This lets all the tests run through and it reports which ones passed or failed. If you just run "test FILE" it will bail out at the first failure.

Try for example


>> more off
>> [a,b,c,d] = test ("io.tst", "quiet", stdout)


This should print to your command window exactly the same output that you would normally see in the fntests.log file, and return the numbers of passes, total tests, fails, and disabled tests.


Mike Miller <mtmiller>
Group Member
Sun 26 Jun 2016 09:31:27 PM UTC, comment #9: 

AFAIK Nan/Inf mix-ups have been a MinGW issue for several years.
IIRC JWE has "fixed" these (or xfailed them) in the tests several months if not a year ago or so.

printd.m - I based my conclusions on fntests.log (which is attached to comment #6), I just noted similar output. I didn't look further :-)
But running the test by hand it shows strings r2 and r4 differ (leght of r4 = 224, r2 = 30).

I have no details on the fixed tests; I do not know how to run them.
"test io.tst" bails out with "fwrite: invalid stream number = -1"  and  "test system.tst" gives:

***** assert (geteuid () >= 0)
!!!!! test failed
assert (geteuid () >= 0) failed


both "manual" tests stop right there.

It is really unfortunate that fntests.log gets clipped after statistics\tests\z_test_2.m.  Maybe a case of improper file buffer flushing?
I've been a bit sloppy in rigorously running __run_test_suite__ in cross-builds during the last months. I just noted that it also happens in a w64 cross-build from early May.
I have quite a few older cross-builds archived back to Oct 2015, so I could try a few more. Should I enter a separate bug report for this?

Philip Nienhuis <philipnienhuis>
Group Member
Sun 26 Jun 2016 08:58:55 PM UTC, comment #8: 

The data.cc and mappers.cc failures are probably all the new tests that I added recently, possibly failing because of either mingw runtime libraries or the version of gcc used to cross-build Octave. See bug #45507, those tests may be converted to xtest until the improved support libraries are a little more widespread.

printd.m - Are you sure that the strings match, not off by some whitespace or non-printable character? Newline endings?

io.tst / system.tst - Any details? Or are these "normal" failures, or reported elsewhere?

Mike Miller <mtmiller>
Group Member
Sun 26 Jun 2016 08:45:18 PM UTC, comment #7: 

A little more results from a w64 build for Windows:
+verbatim-
  libinterp\corefcn\data.cc-tst ..... PASS   1003/1005
                                        FAIL    2  (NaN / Inf)
  libinterp\corefcn\mappers.cc-tst .. PASS    390/396
                                        FAIL    6  (serious acos errors, 1/2 pi off)
  plot\appearance\axis.m ............ PASS      2/3
                                        FAIL    1  (tolerance > 0)
  plot\util\axes.m .................. PASS      0/1
                                        FAIL    1  (fails due to missing legend.m)
  plot\util\printd.m ................ PASS      0/1
                                        FAIL    1  ("Strings don't match" - values do match however)
  testfun\assert.m .................. PASS     94/95
                                        FAIL    1  (AFAIK a usual FAIL on Windows)
  io.tst ............................ PASS    129/135
                                        FAIL    6  (???)
  system.tst ........................ PASS     88/92
                                        FAIL    4  (???)
-verbatim-

Ctrl-C / Ctrl-V don't work, but right-clicking and selectingfrom the menu does work.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 26 Jun 2016 08:21:48 PM UTC, comment #6: 

Avinaom,
I think Octave had no stray functions in its path; it was started from the installer in an empty subdir.

In a "clean" enable-windows-64 bit cross-build (da218a61ce4a, June 21) _run_test_suite_ gives immediately after installation (Octave started from setup screen:)

PASS       14021
FAIL          22
XFAIL         31
SKIPPED       54

A second attempt "hangs" at legend.m. In subsequent attempts Octave hangs earlier, on copyobj.m. I have to kill Octave through Task Manager then.
After moving legend.m and copyobj.m out of the way _run_test_suite_ can run to the end.


An fntests_64.log is attached to this comment; I've only kept the FAILs . It only goes until statistics\tests\z_test_2.m, for unknown reasons that is all that Octave has been writing to disk.
In the attached fntests_64.log NaN/Inf issues in data.cc and  acos() issues in mappers.cc can be seen.


(file #37587)

Philip Nienhuis <philipnienhuis>
Group Member
Sun 26 Jun 2016 05:11:25 PM UTC, comment #5: 

@Philip, could you please check some of the failures?
May you have in the path some wrong function(s),
that responsible to this?

Could please check for example why "test help" fails?


>> strfind (help ("ls"), "List directory contents")
ans =  95
>> strfind (help ("."), "||")
ans =  87
>> help (42)
error: help: invalid input
>> help ("abc", "def")
error: help: invalid input
>>  help ("_! UNLIKELY_FCN! _")
error: help: '_! UNLIKELY_FCN! _' not found


Avinoam Kalma <avinoam>
Group Member
Sun 26 Jun 2016 11:19:31 AM UTC, comment #4: 

<A quick report>

Running _run_test_suite_ on a "clean" (= without the patches I usually include) 32-bit cross build on Win7 64b:

  • crashes the first time (after installation and checking "Run Octave") soon after entering mappers.cc


  • second time finishes, but with


PASS    13900
FAIL      138
XFAIL      31
SKIPPED    55


(I had type this over as copy/paste didn't work in the terminal. IOW, another FAIL).

The fails are (AFAICS)

4  in data.cc-tst
1  in error.cc-tst
10 in mappers.cc-tst
2  in fftw.cc-tst
1  in _unimplemented_.m
1  in get_first_help_sentence.m
1  in help.m
1  in printd.m
2  in deblank.m
3  in regexptranslate
2  in strjust
2  in strtrim
6  in testfun/test.m
35 in io.tst
6  in struct.tst
58 in system.tst

  • After a 3rd start, _run_test_suite_ hangs on legend.m


If I get time I'll try to analyse the log.



Philip Nienhuis <philipnienhuis>
Group Member
Sat 25 Jun 2016 09:17:01 PM UTC, comment #3: 

64 bit.
But I have a recent 32b build lying around, I'll install it and report back later

(slightly OT:
On Linux I always get 2 osmesa FAILs. As no one else reported that I think that must be related to the distro-supplied stuff (Mageia-5 64b). I haven't run into problems with it yet so I don't bother.)

Philip Nienhuis <philipnienhuis>
Group Member
Sat 25 Jun 2016 12:10:42 PM UTC, comment #2: 

@Philip: I test with 32 bit. Did you run with 32 bit or 64 bit?

All these tests passed on linux (Ubuntu).

attached fntest.log


(file #37572)

Avinoam Kalma <avinoam>
Group Member
Sat 25 Jun 2016 10:11:30 AM UTC, comment #1: 

Here, with Octave-dev at 21961:307a2d7ff29b, Octave does not even get that far:

>> __run_test_suite__

Integrated test scripts:

  liboctave\array\Array.cc-tst ................................median: X cannot be an
empty matrix
>>


Philip Nienhuis <philipnienhuis>
Group Member
Sat 25 Jun 2016 09:15:42 AM UTC, original submission:  


Running __run_test_suite__ with
changeset http://hg.savannah.gnu.org/hgweb/octave/rev/d31e0f704d0b
on Windows-7, I got 13 failures:


  libinterp\corefcn\data.cc-tst ............................... PASS   1003/1005
                                                                  FAIL    2
  libinterp\corefcn\mappers.cc-tst ............................ PASS    390/396
                                                                  FAIL    6
 plot\util\printd.m .......................................... PASS      0/1
                                                                  FAIL    1
  system.tst .................................................. PASS     88/92
                                                                  FAIL    4
                                                                  SKIP   11


Can someone confirm these failures?

Avinoam Kalma <avinoam>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #37587:  fntests_64.log added by philipnienhuis (8KiB - text/x-log)
file #37572:  fntests.log added by avinoam (109KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by avinoam (Submitted the item)
  • -email is unavailable- added by avinoam
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-11-16 avinoam StatusNeed Info Fixed
        Open/ClosedOpen Closed
    2016-06-26 philipnienhuis Attached File- Added fntests_64.log, #37587
    2016-06-25 avinoam Attached File- Added fntests.log, #37572
    2016-06-25 avinoam Carbon-Copy- Added lostbard

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code