bugGNU Octave - Bugs: bug #50339, configure detection for BLAS does...

 
 

bug #50339: configure detection for BLAS does not use -fdefault-integer-8

Submitted by:  Kai Torben Ohlhus <siko1056>
Submitted on:  Fri 17 Feb 2017 04:11:13 PM UTC  
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Regression
Status:  Confirmed Assigned to:  John W. Eaton <jwe>
Originator Name:  Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Mon 18 Sep 2017 07:50:08 AM UTC, comment #12: 

Agree to Mike (comment #11).  `enable-64` is now default and obsolete.  The users need to know which options should be passed to Octave's configure to get their custom build Fortran libraries to work with Octave.

Kai Torben Ohlhus <siko1056>
Project Member
Sat 09 Sep 2017 02:08:42 AM UTC, comment #11: 

That's what I thought.

I am now able to build Octave with all of the Fortran libraries built for 64-bit integers and all tests pass. But I have to build with the following:

./configure \
  CPPFLAGS='-DLONGBLAS=long -DBLAS64' \
  F77_INTEGER_8_FLAG=-fdefault-integer-8 \
  ...

The -D options might not be necessary, they are only for the SuiteSparse include files. I'll see if I can simplify that part.

But the F77_INTEGER_8_FLAG is necessary to pass into Octave's configure so that it applies it when testing the various libraries. Otherwise configure just errors out like I showed in comment #9.

Alternatively I think I can set FFLAGS="-g -O2 -fdefault-integer-8" and configure will just think that the Fortran compiler does everything right by default.

This is still an improvement over the previous --enable-64 behavior, we now have a consistent octave_idx_type on 64-bit systems regardless of how the Fortran libraries were built. I don't know that I'd call this a regression. Before the user had to pass --enable-64, now they have to pass something else, everything works both ways assuming the right libraries and the right flags are given.

Mike Miller <mtmiller>
Project Administrator
Sat 09 Sep 2017 12:41:13 AM UTC, comment #10: 

What we currently want to support is the following:

Building with 32-bit or 64-bit indexing (octave_idx_type).

If using a 32-bit integer for octave_idx_type, we need to ensure that the BLAS and other Fortran libraries also use 32-bit indexing.

If 64-bit octave_idx_type, we can use either 32-bit or 64-bit libraries, we just need to know which one we have and whether special flags are needed so that the integer size for the Fortran code we compile matches the BLAS and other Fortran libraries that we find on the system.

I think it's sufficient to check the BLAS library on the system and assume that the other system Fortran libraries have been compiled in the same way, but I'm 100% sure someone will screw that up with self-compiled and installed libraries and blame us for our lousy configure script when it fails.  :-)

Currently, I think the tests are a bit jumbled up, so getting the configure script to do what I've described above may take some significant refactoring.

I don't even know whether it is possible to do this automatically with a reasonable amount of effort, even if we only care about one compiler (gfortran), or if it is just best to expect the user to supply the compiler option to use, and then we determine whether it is functional (options works and sizes match expectations).

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Fri 08 Sep 2017 12:07:44 AM UTC, comment #9: 

Alternatively I can also build by passing F77_INTEGER_8_FLAG=-fdefault-integer-8 as Kai said in comment #4. If that variable is filled in at the beginning of configure, then everything else seems to work together. Without that option specific up front, I get

configure: defining FFLAGS to be -g -O2
checking for sgemm_ in -lblas... yes
checking whether LSAME is called correctly from Fortran... yes
checking whether ISAMAX is called correctly from Fortran... no
checking whether SDOT is called correctly from Fortran... yes
checking whether DDOT is called correctly from Fortran... yes
checking whether CDOTU is called correctly from Fortran... no
checking whether ZDOTU is called correctly from Fortran... yes
checking BLAS library integer size... 8
checking for sgemm_ in -lblas... yes
checking whether LSAME is called correctly from Fortran... yes
checking whether ISAMAX is called correctly from Fortran... no
checking whether SDOT is called correctly from Fortran... no
checking whether DDOT is called correctly from Fortran... yes
checking whether CDOTU is called correctly from Fortran... no
checking whether ZDOTU is called correctly from Fortran... no
checking BLAS library integer size... 8
configure: error: BLAS and LAPACK libraries are required

Some of these tests are segfaulting because the integer size is obviously incorrect at this point, so it thinks that the BLAS library simply doesn't work and configure fails.

I'm not sure if the intent here was to test BLAS with the default options first, let it fail, and then try again with the option to use 64-bit integers?

Mike Miller <mtmiller>
Project Administrator
Thu 07 Sep 2017 09:21:25 PM UTC, comment #8: 

No. I can confirm this bug is still present.

The way it affects me is

./configure

does not work, but

./configure FFLAGS='-g -O2 -fdefault-integer-8'

does work (at least configure passes).

My understanding of the rework of configure was that it was supposed to detect the BLAS integer size automatically and add the appropriate option to FFLAGS automatically. The --enable-64 option is now redundant/default.

The error I get seems to be related to configure does check for the default integer size, twice, but it never actually gets added to FFLAGS. I'll see if I can figure out why that is.

Mike Miller <mtmiller>
Project Administrator
Mon 04 Sep 2017 02:29:56 PM UTC, comment #7: 

This report is about 5 months old.  Has building Octave with 64-bit Fortran libraries been fixed now?

Rik <rik5>
Project Administrator
Fri 24 Feb 2017 10:59:37 PM UTC, comment #6: 

I should also say, my goal is for this to work now without needing --enable-64 since that is now the default.

The idea is that Octave will use 64-bit indexing internally, but will check the BLAS to determine whether 64-bit indexing should be used for Fortran (and other) libraries that depend on the BLAS.  If the BLAS is determined to use 64-bit indexing, then we should attempt to use the -fdefault-integer-8 (or equivalent) option to compile the Fortran bits of Octave.  Since I only know what the right option is for gfortran, users of other compilers will have to supply this option.

Anyway, probably I just have the logic mixed up in the configure script.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Fri 24 Feb 2017 10:52:05 PM UTC, comment #5: 

OK, thanks.

My goal is to have this case detected automatically.  I'll see what I can do.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Fri 24 Feb 2017 10:42:44 PM UTC, comment #4: 

Thank you jwe. I have a custom built OpenBLAS, SuiteSparse, QRUPDATE, and ARPACK-NG, each using 8 byte FORTRAN integers and changed SONAME to avoid any conflict with system wide installed libraries. This has been working for Octave 4.2.0.

My Makefile is available from

https://github.com/siko1056/GNU-Octave-enable-64/blob/dev/Makefile (from line 212 on) and would expand to something like (usually without the highlighted line, this flag, if not passed, will not be used while detecting the custom BLAS library):

./configure \
  CPPFLAGS='-I$(INSTALL_DIR)/include' \
  LDFLAGS='-L$(INSTALL_DIR)/lib' \
  \
  \
  F77_INTEGER_8_FLAG='-fdefault-integer-8' \
  \
  \
  LD_LIBRARY_PATH='$(INSTALL_DIR)/lib' \
  --prefix=$(INSTALL_DIR) \
  --libdir='$(INSTALL_DIR)/lib' \
  --enable-64 \
  --with-blas='-lopenblas_Octave64' \
  --with-amd='-lamd_Octave64 \
              -lsuitesparseconfig_Octave64' \
  --with-camd='-lcamd_Octave64 \
               -lsuitesparseconfig_Octave64' \
  --with-colamd='-lcolamd_Octave64 \
                 -lsuitesparseconfig_Octave64' \
  --with-ccolamd='-lccolamd_Octave64 \
                  -lsuitesparseconfig_Octave64' \
  --with-cxsparse='-lcxsparse_Octave64 \
                   -lsuitesparseconfig_Octave64' \
  --with-cholmod=-lamd_Octave64 \
   -lcamd_Octave64 \
   -lcolamd_Octave64 \
   -lccolamd_Octave64 \
   -lcxsparse_Octave64 \
   -lumfpack_Octave64 \
   -lcholmod_Octave64 \
   -lsuitesparseconfig_Octave64 \
  --with-umfpack=-lamd_Octave64 \
   -lcamd_Octave64 \
   -lcolamd_Octave64 \
   -lccolamd_Octave64 \
   -lcxsparse_Octave64 \
   -lumfpack_Octave64 \
   -lcholmod_Octave64 \
   -lsuitesparseconfig_Octave64 \
  --with-qrupdate='-lqrupdate_Octave64' \
  --with-arpack='-larpack_Octave64'

Kai Torben Ohlhus <siko1056>
Project Member
Fri 24 Feb 2017 09:31:10 PM UTC, comment #3: 

I will take a look a this problem but I need some additional info to understand exactly what you are saying doesn't work properly.

Exactly what options were you passing to configure when it failed to work as you expect?

Do you have BLAS and other libraries built to use 64-bit indexing?

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Fri 24 Feb 2017 09:23:08 PM UTC, comment #2: 

Regarding comment #1, I tried both FFLAGS and F77_INTEGER_8_FLAG, but only

F77_INTEGER_8_FLAG='-fdefault-integer-8'

as `configure` parameter worked for me.

Kai Torben Ohlhus <siko1056>
Project Member
Fri 24 Feb 2017 05:33:09 PM UTC, comment #1: 

Adding jwe to the CC list for this bug since his cset 22986:21707d6a02c7 seems to be at the root of the new behavior.

@Kai: Does it work if you add "-fdefault-integer-8" to FFLAGS at the shell before running configure?

Rik <rik5>
Project Administrator
Fri 17 Feb 2017 04:11:13 PM UTC, original submission:  

After reordering configure to use --enable-64 by default the check for the BLAS library happens by compiling FORTRAN code without using the -fdefault-integer-8 flag and fails in segmentation faults. In http://hg.savannah.gnu.org/hgweb/octave/file/5f79bb3fdd28/configure.ac in lines 963 and 977 the variable F77_INTEGER_8_FLAG is used, but empty. It gets filled by the necessary flag in about line 1050.

I assume the order was swapped in CSET http://hg.savannah.gnu.org/hgweb/octave/rev/21707d6a02c7 but never tested. My last test of this detection feature happened about 3 months ago.

Kai Torben Ohlhus <siko1056>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by rik5
  • -email is unavailable- added by siko1056 (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-09-18 siko1056 Open/ClosedOpen => Closed
    2017-09-07 mtmiller Dependencies- => bugs #44359 is dependent
    2017-09-07 mtmiller StatusNeed Info => Confirmed
    2017-02-24 jwe Assigned toNone => jwe
    2017-02-24 rik5 StatusNone => Need Info
        Carbon-Copy- => Added jwe

    Back to the top


    Powered by Savane 3.7