bugGNU Octave - Bugs: bug #59081, [MXE Octave] Octave binary cannot...

 
 

bug #59081: [MXE Octave] Octave binary cannot be found by of-image

Submitter:  Kai Torben Ohlhus <siko1056>
Submitted:  Tue 08 Sep 2020 10:25:46 AM UTC
   
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Installation Failure
Status:  Wont Fix Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 6.0.90 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 08 Sep 2020 03:37:25 PM UTC, comment #15: 

@John and @Markus: Thank you for the clarifications and valuable comments.  I think I will have to go the way of comment #13 and make a real minimal Octave installation to the Docker Buildbot Worker building the mxe-octave later.

I modify the scripts tomorrow and give them a try.  Maybe the Docker Buildbot system will get to work before the Octave 6 release ^^

This report can be closed to avoid being stale for another 4 years =D  Reopen if you get other ideas about solving this issue.

Kai Torben Ohlhus <siko1056>
Group Member
Tue 08 Sep 2020 03:33:36 PM UTC, comment #14: 

There is a open bug report for the fact that we need octave in some cases

https://savannah.gnu.org/bugs/?49503


John Donoghue <lostbard>
Group Member
Tue 08 Sep 2020 03:28:57 PM UTC, comment #13: 

Looks like of-image needs Octave for these tests:

## Some functions have been moving between Octave core and the Image
## package so make sure we are not shadowing anything on this
## installation.
##
##  im2double - moved into core for core version 4.2
##  ntsc2rgb - moved into the image package for core version 4.4
##  rgb2ntsc - moved into the image package for core version 4.4

M_FILES=""
m4_foreach([FNAME], [im2double, ntsc2rgb, rgb2gray, rgb2ntsc], [
  AC_MSG_CHECKING([Octave core for function FNAME])
  HAS_FUNCTION=`$OCTAVE -qfH --eval "printf ('%i', exist ('FNAME'))"`
  if test "$HAS_FUNCTION" == 0; then
    AC_MSG_RESULT([no])
    M_FILES="${M_FILES} FNAME.m"
  else
    AC_MSG_RESULT([yes])
  fi
])


So exactly what John wrote in comment #11.

Building a minimal Octave should take only a few minutes if using ccache is an option.
E.g. jwe's no-extras-debian worker took about 3 minutes for the configure and compile steps: http://buildbot.octave.org:8010/#/builders/7/builds/1786
Additionally, you'd need to call "make install". But I don't think that will take very long.

Markus Mützel <mmuetzel>
Group administrator
Tue 08 Sep 2020 03:22:35 PM UTC, comment #12: 

In the caseof image, it uses octave to check whether the functions im2double, ntsc2rgb, rgb2gray, rgb2ntsc are available.

statistics does a similar thing.


John Donoghue <lostbard>
Group Member
Tue 08 Sep 2020 03:19:44 PM UTC, comment #11: 

octave is only required if the pkg requires octave for detection of settings or functions - most do not need it

in mxe, the package install is done by a python script

John Donoghue <lostbard>
Group Member
Tue 08 Sep 2020 03:01:02 PM UTC, comment #10: 

I also often wondered why a native Octave is required. Is it used to run e.g., "pkg install" into the ./usr/x86_64-w64-mingw32/ directory tree?

Philip Nienhuis <philipnienhuis>
Group Member
Tue 08 Sep 2020 02:45:07 PM UTC, comment #9: 

@Markus comment #7 and comment #8: I also thought about something similar.  I would not like to make mxe more complicated by another target.  Also I try to avoid the extra +15 minutes of build time (without docs, checks etc.) just to satisfy (maybe unnecessary) OF package "needs" 😇

First, I thought about a patch removing those lines from of-image configure.ac.

@John comment #6: If there is no easy way to teach the of-image package about the octave binary from mxe-side, I can close the bug report and will make a small "fake" installation to satisfy the "needs" of-image (and maybe other of-packages). [Maybe a fake script file even will do to trick of-image 🤔]

Kai Torben Ohlhus <siko1056>
Group Member
Tue 08 Sep 2020 02:37:01 PM UTC, comment #8: 

Sorry. Please, disregard my last comment. The mkoctfile cross-compiler is built already.

Not sure why packages need to run Octave during their build process. So I can't tell if a version or API mismatch might be a problem or not.

Markus Mützel <mmuetzel>
Group administrator
Tue 08 Sep 2020 02:25:58 PM UTC, comment #7: 

Maybe we could add a "native-octave" target that builds and installs a native Octave version from a provided tarball (any of default, stable, release selectable if that is possible).

Using a fixed version of Octave to build the package binaries might lead to .oct files that are incompatible to the cross-built Octave version.

Markus Mützel <mmuetzel>
Group administrator
Tue 08 Sep 2020 02:17:48 PM UTC, comment #6: 

For the octave detection, AC_PATH_PROG appears to ignore the env var if its not a full path, but if is a fullpath does use the env var.

Which is why the mkoctfile part worked in the initaial comment.


mkoctfile is also already sent to the pkg install script in the build process, as it needed to point to the cross built tools.

the octave variable currently isn't being set though or being passed through. We could probably change that easily enough.

It does check whether the correct octave version can be found and prints a warning on the start of the make process if it couldn't find the correct octave version


John Donoghue <lostbard>
Group Member
Tue 08 Sep 2020 02:00:08 PM UTC, comment #5: 

To give a better answer to comment #2:

So far I didn't use


make JOBS=8 hg-octave-dist hg-octave-branch=default


I build the respective "octave-6.0.91.tar.xz" tarball in a separate Buildbot process.  Note that this is not the "offical" Buildbot server run by jwe.

https://buildbot.octave.space/#/builders/1/builds/1

The build results get published automatically

https://www.octave.space/data/stable/a927fe30358d/

to facilitate and shorten release cycles of Octave's stable branch (especially before a major release more frequent releases of stable without uploading them to alpha.gnu.org might be helpful).

In the long term this should become a "lightweight" release.  Ensuring that all mxe (32,64,64-64) are build from the same release tarball.

Kai Torben Ohlhus <siko1056>
Group Member
Tue 08 Sep 2020 01:50:00 PM UTC, comment #4: 

Thank you John for looking into this 🙂

Would of-image be compiled to the mxe built "octave" and "mkoctfile", if the block from

https://sourceforge.net/p/octave/image/ci/default/tree/src/configure.ac

was removed and the variables OCTAVE and MKOCTFILE are properly set by mxe-octave?


AC_PATH_PROG([OCTAVE], [octave])
if test -z "$OCTAVE"; then
  AC_MSG_ERROR([*** 'octave' not found.])
fi

AC_PATH_PROG([MKOCTFILE], [mkoctfile])
if test -z "$MKOCTFILE"; then
  AC_MSG_ERROR([*** 'mkoctfile' not found.])
fi


Kai Torben Ohlhus <siko1056>
Group Member
Tue 08 Sep 2020 01:31:46 PM UTC, comment #3: 

For image, even explicity setting the env variable seems to be ignored for some reason:

in image-2.12.0/src:

OCTAVE=fail MKOCTFILE=error ./configure

checking for octave... /usr/local/bin/octave
checking for mkoctfile... /usr/local/bin/mkoctfile




John Donoghue <lostbard>
Group Member
Tue 08 Sep 2020 01:23:04 PM UTC, comment #2: 

On mine:

checking for octave... /usr/local/bin/octave

On the buildbots are/were? building the octave repo as part of the process and running

make JOBS=8 hg-octave-dist hg-octave-branch=default

Before building the rest of octave

John Donoghue <lostbard>
Group Member
Tue 08 Sep 2020 10:29:45 AM UTC, comment #1: 

P.S. link to the failing Buildbot

https://buildbot.octave.space/#/builders/3/builds/5

https://github.com/gnu-octave/octave-buildbot

I think when this is solved, I can more frequently provide public accessible mxe-builds for Octave 🙂

Kai Torben Ohlhus <siko1056>
Group Member
Tue 08 Sep 2020 10:25:46 AM UTC, original submission:  

When building Octave packages, the distribution built Octave binary seems not to be exposed to the package configure script.

I suspect, that most mxe build systems have some Octave installed, that is detected by of-image.  Can someone check his mxe "log" directory for "of-image" and confirm the detected value for "OCTAVE=''"?



In general I do not like to install an Octave on my system, as I run the mxe-builds inside Docker containers, that shall be as "clean" as possible.

When looking at the logs, is there a way to set an OCTAVE variable in the same way it is set for MKOCTFILE and copy the octave binary, for example?

Excerpt from "2020-09-08_of-image_log.txt":


LDFLAGS='-L/buildbot/octave-mxe-stable-w64/build/usr/x86_64-w64-mingw32/lib'
LIBOBJS=''
LIBS=''
LTLIBOBJS=''
MKOCTFILE='/buildbot/octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-mkoctfile --verbose'
M_FILES=''
OBJEXT=''
OCTAVE=''


Code where binaries are installed in mxe-octave:

https://hg.octave.org/mxe-octave/file/tip/src/stable-octave.mk#l205


ls -al /buildbot/octave-mxe-stable-w64/build/usr/bin/*oct*
-rwxr-xr-x. 1 root root 949344 Sep  8 17:04 octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-mkoctfile
-rwxr-xr-x. 1 root root 305168 Sep  8 17:04 octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-octave-config




Excerpt from "2020-09-08_of-image_log.txt"


[build]    of-image

Failed to build package of-image!
------------------------------------------------------------
image-2.12.0/COPYING
checking for a sed that does not truncate output... /buildbot/octave-mxe-stable-w64/build/usr/bin/sed
checking for octave... no
configure: error: *** 'octave' not found.
operation= install
mkoctfile= /buildbot/octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-mkoctfile --verbose
arch= x86_64-w64-mingw32-api-v55
prefix= /buildbot/octave-mxe-stable-w64/build/usr/x86_64-w64-mingw32
files= ['image-2.12.0.tar.gz']
verbose= True
Installing /buildbot/octave-mxe-stable-w64/build/tmp-of-image/image-2.12.0.tar.gz
running ./configure --host='x86_64-w64-mingw32' --build='x86_64-pc-linux-gnu'
cleaning up


Kai Torben Ohlhus <siko1056>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by siko1056 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-09-08 siko1056 StatusNone Wont Fix
        Open/ClosedOpen Closed
    2020-09-08 siko1056 Attached File- Added 2020-09-08_of-image_log.txt, #49764
        Attached File- Added 2020-09-08_of-image_log.txt, #49765
        Attached File- Added 2020-09-08_mxe_build_of-image_log.txt, #49766

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code