bugGNU Octave - Bugs: bug #59081, [MXE Octave] Octave binary cannot...

 
 

bug #59081: [MXE Octave] Octave binary cannot be found by of-image

Submitted by:  Kai Torben Ohlhus <siko1056>
Submitted on:  Tue 08 Sep 2020 10:25:46 AM UTC  
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Installation Failure
Status:  Wont Fix Assigned to:  None
Originator Name:  Open/Closed:  Closed
Release:  6.0.90 Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 08 Sep 2020 03:37:25 PM UTC, comment #15: 

@John and @Markus: Thank you for the clarifications and valuable comments.  I think I will have to go the way of comment #13 and make a real minimal Octave installation to the Docker Buildbot Worker building the mxe-octave later.

I modify the scripts tomorrow and give them a try.  Maybe the Docker Buildbot system will get to work before the Octave 6 release ^^

This report can be closed to avoid being stale for another 4 years =D  Reopen if you get other ideas about solving this issue.

Kai Torben Ohlhus <siko1056>
Project Member
Tue 08 Sep 2020 03:33:36 PM UTC, comment #14: 

There is a open bug report for the fact that we need octave in some cases

https://savannah.gnu.org/bugs/?49503

John Donoghue <lostbard>
Project Member
Tue 08 Sep 2020 03:28:57 PM UTC, comment #13: 

Looks like of-image needs Octave for these tests:

## Some functions have been moving between Octave core and the Image
## package so make sure we are not shadowing anything on this
## installation.
##
##  im2double - moved into core for core version 4.2
##  ntsc2rgb - moved into the image package for core version 4.4
##  rgb2ntsc - moved into the image package for core version 4.4

M_FILES=""
m4_foreach([FNAME], [im2double, ntsc2rgb, rgb2gray, rgb2ntsc], [
  AC_MSG_CHECKING([Octave core for function FNAME])
  HAS_FUNCTION=`$OCTAVE -qfH --eval "printf ('%i', exist ('FNAME'))"`
  if test "$HAS_FUNCTION" == 0; then
    AC_MSG_RESULT([no])
    M_FILES="${M_FILES} FNAME.m"
  else
    AC_MSG_RESULT([yes])
  fi
])

So exactly what John wrote in comment #11.

Building a minimal Octave should take only a few minutes if using ccache is an option.
E.g. jwe's no-extras-debian worker took about 3 minutes for the configure and compile steps: http://buildbot.octave.org:8010/#/builders/7/builds/1786
Additionally, you'd need to call "make install". But I don't think that will take very long.

Markus Mützel <mmuetzel>
Project Member
Tue 08 Sep 2020 03:22:35 PM UTC, comment #12: 

In the caseof image, it uses octave to check whether the functions im2double, ntsc2rgb, rgb2gray, rgb2ntsc are available.

statistics does a similar thing.

John Donoghue <lostbard>
Project Member
Tue 08 Sep 2020 03:19:44 PM UTC, comment #11: 

octave is only required if the pkg requires octave for detection of settings or functions - most do not need it

in mxe, the package install is done by a python script

John Donoghue <lostbard>
Project Member
Tue 08 Sep 2020 03:01:02 PM UTC, comment #10: 

I also often wondered why a native Octave is required. Is it used to run e.g., "pkg install" into the ./usr/x86_64-w64-mingw32/ directory tree?

Philip Nienhuis <philipnienhuis>
Project Member
Tue 08 Sep 2020 02:45:07 PM UTC, comment #9: 

@Markus comment #7 and comment #8: I also thought about something similar.  I would not like to make mxe more complicated by another target.  Also I try to avoid the extra +15 minutes of build time (without docs, checks etc.) just to satisfy (maybe unnecessary) OF package "needs" 😇

First, I thought about a patch removing those lines from of-image configure.ac.

@John comment #6: If there is no easy way to teach the of-image package about the octave binary from mxe-side, I can close the bug report and will make a small "fake" installation to satisfy the "needs" of-image (and maybe other of-packages). [Maybe a fake script file even will do to trick of-image 🤔]

Kai Torben Ohlhus <siko1056>
Project Member
Tue 08 Sep 2020 02:37:01 PM UTC, comment #8: 

Sorry. Please, disregard my last comment. The mkoctfile cross-compiler is built already.

Not sure why packages need to run Octave during their build process. So I can't tell if a version or API mismatch might be a problem or not.

Markus Mützel <mmuetzel>
Project Member
Tue 08 Sep 2020 02:25:58 PM UTC, comment #7: 

Maybe we could add a "native-octave" target that builds and installs a native Octave version from a provided tarball (any of default, stable, release selectable if that is possible).

Using a fixed version of Octave to build the package binaries might lead to .oct files that are incompatible to the cross-built Octave version.

Markus Mützel <mmuetzel>
Project Member
Tue 08 Sep 2020 02:17:48 PM UTC, comment #6: 

For the octave detection, AC_PATH_PROG appears to ignore the env var if its not a full path, but if is a fullpath does use the env var.

Which is why the mkoctfile part worked in the initaial comment.

mkoctfile is also already sent to the pkg install script in the build process, as it needed to point to the cross built tools.

the octave variable currently isn't being set though or being passed through. We could probably change that easily enough.

It does check whether the correct octave version can be found and prints a warning on the start of the make process if it couldn't find the correct octave version

John Donoghue <lostbard>
Project Member
Tue 08 Sep 2020 02:00:08 PM UTC, comment #5: 

To give a better answer to comment #2:

So far I didn't use

make JOBS=8 hg-octave-dist hg-octave-branch=default

I build the respective "octave-6.0.91.tar.xz" tarball in a separate Buildbot process.  Note that this is not the "offical" Buildbot server run by jwe.

https://buildbot.octave.space/#/builders/1/builds/1

The build results get published automatically

https://www.octave.space/data/stable/a927fe30358d/

to facilitate and shorten release cycles of Octave's stable branch (especially before a major release more frequent releases of stable without uploading them to alpha.gnu.org might be helpful).

In the long term this should become a "lightweight" release.  Ensuring that all mxe (32,64,64-64) are build from the same release tarball.

Kai Torben Ohlhus <siko1056>
Project Member
Tue 08 Sep 2020 01:50:00 PM UTC, comment #4: 

Thank you John for looking into this 🙂

Would of-image be compiled to the mxe built "octave" and "mkoctfile", if the block from

https://sourceforge.net/p/octave/image/ci/default/tree/src/configure.ac

was removed and the variables OCTAVE and MKOCTFILE are properly set by mxe-octave?

AC_PATH_PROG([OCTAVE], [octave])
if test -z "$OCTAVE"; then
  AC_MSG_ERROR([*** 'octave' not found.])
fi

AC_PATH_PROG([MKOCTFILE], [mkoctfile])
if test -z "$MKOCTFILE"; then
  AC_MSG_ERROR([*** 'mkoctfile' not found.])
fi

Kai Torben Ohlhus <siko1056>
Project Member
Tue 08 Sep 2020 01:31:46 PM UTC, comment #3: 

For image, even explicity setting the env variable seems to be ignored for some reason:

in image-2.12.0/src:

OCTAVE=fail MKOCTFILE=error ./configure

checking for octave... /usr/local/bin/octave
checking for mkoctfile... /usr/local/bin/mkoctfile

John Donoghue <lostbard>
Project Member
Tue 08 Sep 2020 01:23:04 PM UTC, comment #2: 

On mine:

checking for octave... /usr/local/bin/octave

On the buildbots are/were? building the octave repo as part of the process and running

make JOBS=8 hg-octave-dist hg-octave-branch=default

Before building the rest of octave

John Donoghue <lostbard>
Project Member
Tue 08 Sep 2020 10:29:45 AM UTC, comment #1: 

P.S. link to the failing Buildbot

https://buildbot.octave.space/#/builders/3/builds/5

https://github.com/gnu-octave/octave-buildbot

I think when this is solved, I can more frequently provide public accessible mxe-builds for Octave 🙂

Kai Torben Ohlhus <siko1056>
Project Member
Tue 08 Sep 2020 10:25:46 AM UTC, original submission:  

When building Octave packages, the distribution built Octave binary seems not to be exposed to the package configure script.

I suspect, that most mxe build systems have some Octave installed, that is detected by of-image.  Can someone check his mxe "log" directory for "of-image" and confirm the detected value for "OCTAVE=''"?

In general I do not like to install an Octave on my system, as I run the mxe-builds inside Docker containers, that shall be as "clean" as possible.

When looking at the logs, is there a way to set an OCTAVE variable in the same way it is set for MKOCTFILE and copy the octave binary, for example?

Excerpt from "2020-09-08_of-image_log.txt":

LDFLAGS='-L/buildbot/octave-mxe-stable-w64/build/usr/x86_64-w64-mingw32/lib'
LIBOBJS=''
LIBS=''
LTLIBOBJS=''
MKOCTFILE='/buildbot/octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-mkoctfile --verbose'
M_FILES=''
OBJEXT=''
OCTAVE=''

Code where binaries are installed in mxe-octave:

https://hg.octave.org/mxe-octave/file/tip/src/stable-octave.mk#l205

ls -al /buildbot/octave-mxe-stable-w64/build/usr/bin/*oct*
-rwxr-xr-x. 1 root root 949344 Sep  8 17:04 octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-mkoctfile
-rwxr-xr-x. 1 root root 305168 Sep  8 17:04 octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-octave-config

Excerpt from "2020-09-08_of-image_log.txt"

[build]    of-image

Failed to build package of-image!
------------------------------------------------------------
image-2.12.0/COPYING
checking for a sed that does not truncate output... /buildbot/octave-mxe-stable-w64/build/usr/bin/sed
checking for octave... no
configure: error: *** 'octave' not found.
operation= install
mkoctfile= /buildbot/octave-mxe-stable-w64/build/usr/bin/x86_64-w64-mingw32-mkoctfile --verbose
arch= x86_64-w64-mingw32-api-v55
prefix= /buildbot/octave-mxe-stable-w64/build/usr/x86_64-w64-mingw32
files= ['image-2.12.0.tar.gz']
verbose= True
Installing /buildbot/octave-mxe-stable-w64/build/tmp-of-image/image-2.12.0.tar.gz
running ./configure --host='x86_64-w64-mingw32' --build='x86_64-pc-linux-gnu'
cleaning up

Kai Torben Ohlhus <siko1056>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by siko1056 (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-09-08 siko1056 StatusNone => Wont Fix
        Open/ClosedOpen => Closed
    2020-09-08 siko1056 Attached File- => Added 2020-09-08_of-image_log.txt, #49764
        Attached File- => Added 2020-09-08_of-image_log.txt, #49765
        Attached File- => Added 2020-09-08_mxe_build_of-image_log.txt, #49766

    Back to the top


    Powered by Savane 3.5