Wed 15 Mar 2017 11:19:59 PM UTC, comment #12:
Per comment #10, all of the Java tests could have a run time check. For example, the cross-build of MXE Octave for Windows includes Java functionality (available at compile time). But the PC where Octave is installed might not have a JRE.
As an example, see javachk.m
A number of the functions like java_set or java_get have no BIST tests because there was no good way make tests conditional on the presence of a JVM.
|
Wed 15 Mar 2017 08:07:26 PM UTC, comment #11:
I checked in the following change to allow run-time conditions to determine whether to skip a test.
http://hg.savannah.gnu.org/hgweb/octave/rev/67d741321e21
The changeset also updates the 10 cases the Rik identified. Are there others?
|
Fri 23 Sep 2016 04:06:55 PM UTC, comment #10:
I prefaced the 10 tests that depend at runtime on a working window environment with
I'm not going to tackle the larger question of how to get these tests reported as XFAILs. That question needs to be answered for the Java tests as well, and it makes sense to address both cases at the same time.
|
Thu 22 Sep 2016 07:53:44 AM UTC, comment #9:
I cannot think of a way of marking a test as XFAIL at runtime
but what we could do is to add
to all FLTK tests so at least the reason for the failure is clear.
As Rik noted the tests that require this are few enough that we can do it manually.
|
Wed 21 Sep 2016 04:46:45 PM UTC, comment #8:
Agree, we do need a way to differentiate between a test that is disabled because we know it would fail (such as --without-foo-feature), and a test that fails because of some run time condition that we may not have expected.
The latter are more similar to XFAILS, they are expected failures, but not because of a bug, because of some external condition like not having a display, or not having a home directory, or not having internet access.
We do want to see that the test failed, and why it failed, which is what is shown now, but it would also be nice to be able to more easily see that they are not failures due to a real problem in Octave or in some external library.
|
Wed 21 Sep 2016 04:37:37 PM UTC, comment #7:
There are 3 testif HAVE_FLTK in the libinterp directory and 7 in the scripts directory. The number is not large so they could be retrofitted with a run-time test, perhaps using have_window_system, to decide whether they should be executed.
But that still feels like the wrong strategy. If 'make check' is being run to test for regressions on several different platforms then we would like it to run the graphical tests, not just skip them.
|
Wed 21 Sep 2016 04:35:12 PM UTC, comment #6:
Mike,
you are right, if I ssh with no remote display
and
skipping the tests would make sense to me and it is probably
as simple as checking if have_window_system is true ...
c.
|
Wed 21 Sep 2016 04:28:09 PM UTC, comment #5:
So that seems consistent with what I am thinking is happening.
The available_graphics_toolkits will not list fltk if have_window_system returns false, and I think that's what happens in the buildbot environment.
When you remote in to test it manually, it probably works because there is a display available, either local or remotely. To get these tests to pass would need a VNC or other headless display to dump figures to.
I guess disabling the tests would be an option, similar to what we discussed about Java tests that have "testif HAVE_JAVA", but fail when the JVM is not actually available at run time.
|
Wed 21 Sep 2016 08:41:22 AM UTC, comment #4:
Hi, thanks for all your replies ...
@Rik
available_graphics_toolkits
does list fltk whether I add the library location to
LD_LIBRARY_PATH or not (which is actually called DYLD_LIBRARY_PATH on OSX, btw).
graphics_toolkit fltk
plot (randn (5))
does produce a plot (although extremely slow, but that may be due to remote connection?)
@Mike
The issue here is that FLTK test are NOT suppressed,
all tests prepended with testif HAVE_FLTK are run but fail,
if FLTK cannot access a display maybe they should be skipped?
Xvfb cannot be used as we we link to the aqua version (not x11)
of FLTK and QT
I think a headless display can be produced with VNC as well but
I don't remeber how, I'll look into this.
|
Tue 20 Sep 2016 04:21:21 PM UTC, comment #3:
On the Debian builds, we install the "xvfb" package and run make check with DISPLAY set to an Xvfb running in the background. If there is a xvfb package available in homebrew we should set that up also.
|
Tue 20 Sep 2016 04:20:13 PM UTC, comment #2:
Carlo - if you ssh in to the mac mini and try to test fltk with a VNC or with X protocol forwarding over ssh, does it work? I think the tests you are seeing skipped are simply because the build environment has no DISPLAY variable.
I get the same results on Debian when I run
These messages that Octave normally produces
are suppressed when running make check.
|
Tue 20 Sep 2016 04:09:42 PM UTC, comment #1:
What does this return?
Is fltk one of the listed options? Can you do a simple plot?
Since your FLTK libraries are in a non-standard location, what if you add that location to LD_LIBRARY_PATH so they can be found at runtime?
In csh syntax
|
Tue 20 Sep 2016 01:13:12 PM UTC, original submission:
Hi,
This appears again to be an inconsistency between
configure and make check on OSX:
When running configure FLTK and OpenGL appear
to be correctly detected [1]:
but, when running 'make check' all tests related to the
fltk toolkit fail [2]:
the message about missing fltk toolkit seems inconsistent
with HAVE_FLTK evaluating to true ...
c.
[1] http://buildbot.octave.org:8010/builders/clang-osx/builds/23/steps/configure/logs/stdio
[2] http://www1.mate.polimi.it/~carlo/fntests.log
|