bugGNU Octave - Bugs: bug #42176, Octave built without OpenMP...

 
 

bug #42176: Octave built without OpenMP crashes when unloading OpenMP-enabled oct-files

Submitted by:  Mike Miller <mtmiller>
Submitted on:  Wed 23 Apr 2014 12:15:04 PM UTC  
 
Category:  Configuration and Build System Severity:  2 - Minor
Priority:  3 - Low Item Group:  Segfault, Bus Error, etc.
Status:  Fixed Assigned to:  Mike Miller <mtmiller>
Originator Name:  Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 22 Feb 2015 04:02:03 PM UTC, comment #12: 

Oh, our comments passed in the tubes.  I'll leave ax_openmp.m4 alone after your change.  But the requirement of 2.59 is kind of irrelevant since Octave's configure.ac appears to require 2.63 anyway.

John W. Eaton <jwe>
Project Administrator
Sun 22 Feb 2015 03:59:55 PM UTC, comment #11: 

I checked the sources of autoconf 2.58 and even it has _AC_LANG_PREFIX.  Then I tried a bootstrap for the Octave sources with autoconf 2.62 and automake 1.11.  2.62, the largest numbers I could find in the sources.  That failed with the following messages:

running: AUTOPOINT=true LIBTOOLIZE=true autoreconf --verbose --install --force -I m4  --no-recursive
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
main::scan_file() called too early to check prototype at /tmp/ac/bin/aclocal line 617.
autoreconf: configure.ac: tracing
autoreconf: running: true --copy --force
main::scan_file() called too early to check prototype at /tmp/ac/bin/aclocal line 617.
autoreconf: running: /tmp/ac/bin/autoconf --include=m4 --force
configure.ac:1638: error: possibly undefined macro: AC_MSG_WARN
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure:7074: error: possibly undefined macro: m4_set_add
configure:122937: error: possibly undefined macro: m4_set_foreach
autoreconf: /tmp/ac/bin/autoconf failed with exit status: 1
./bootstrap: autoreconf failed

The m4_set_add, m4_set_foreach macros were added in autoconf 2.63.  Using that version, bootstrap succeeded and the resulting configure script seemed to work OK, so I bumped the required version in Octave's configure.ac and removed the AC_PREREQ from the ax_openmp.m4 file.

OK?

John W. Eaton <jwe>
Project Administrator
Sun 22 Feb 2015 03:57:46 PM UTC, comment #10: 

For now, I pushed a cset to change our local copy of the macro in the m4 directory to use version 2.59 (http://hg.savannah.gnu.org/hgweb/octave/rev/ced2313b8d56).

Rik <rik5>
Project Administrator
Sun 22 Feb 2015 03:34:37 PM UTC, comment #9: 

I'll test it with Autoconf 2.59 just to be sure, but I like the typo theory.

Reopening to remind myself to test.

Mike Miller <mtmiller>
Project AdministratorIn charge of this item.
Sun 22 Feb 2015 03:18:34 PM UTC, comment #8: 

Looking at the git repository, the change happened very recently in 10/2014.

http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=history;f=m4/ax_openmp.m4

The AC_PREREQ line was changed to add m4 quoting around the version number and at the same time the version number was changed.  Might be coincidence, but I'm still thinking typo.

Rik <rik5>
Project Administrator
Sun 22 Feb 2015 03:10:07 PM UTC, comment #7: 

This is kind of silly, but it looks to me like the AC_PREREQ line in ax-openmp.m4 may be a typo.  It says

AC_PREREQ([2.69]) dnl for _AC_LANG_PREFIX

I searched the web for _AC_LANG_PREFIX and in lots of other m4 files I find the same line with a different version number.

AC_PREREQ([2.59]) dnl for _AC_LANG_PREFIX

In fact, even ax_openmp.m4 used 2.59 in 2008.  Maybe they upped the version number for some other reason and forgot to remove the comment?

I do know that I changed the AC_PREREQ down to 2.68 which is what I have and Autoconf runs successfully.

Rik <rik5>
Project Administrator
Sun 22 Feb 2015 03:04:51 PM UTC, comment #6: 

That requirement is only if you need to run autoconf.  It doesn't affect the generated configure script, which doesn't require autoconf.  So I don't see a problem.  People who want to do development for Octave and build from the mercurial sources should also be capable of building or installing a newer version of autoconf if needed.  The mxe-octave build system (what I use for building on old systems like RHEL 5 that lack the necessary packages) already builds a newer version of autotools as part of its build process.

John W. Eaton <jwe>
Project Administrator
Sun 22 Feb 2015 02:48:48 PM UTC, comment #5: 

One wrinkle is that the ax_openmp.m4 file requires Autoconf > 2.69.  I can no longer build on Ubuntu 12.04 LTS (admittedly 3 years old this April).  It seems like users on really long term stable platforms like Red Hat Enterprise Linux will have the same issue.

Maybe we should still enable OpenMP but continue to use our own compile test?

Alternatively, there's hacking the ax_openmp.m4 file.  They have an AC_PREREQ([2.69]) because they are using the _AC_LANG_PREFIX macro.  Perhaps there is a way to simulate that macro through a longer sequence of commands in earlier versions.

If we do just decide to go with it then we should raise the required minimum version of Autoconf in Octave's own configure.ac file and make a note that the version required for building has changed.

Rik <rik5>
Project Administrator
Sun 22 Feb 2015 01:00:08 AM UTC, comment #4: 

Ok, I took another look and fixed this one. OpenMP is now enabled by default:

http://hg.savannah.gnu.org/hgweb/octave/rev/81078b0e39e8

I've been building with OpenMP for a while now anyway and have had no problems, all tests pass with or without it. Octave still does not use OpenMP in its libraries anywhere yet, but all oct-files are built such that they can use OpenMP with no extra handling necessary.

Mike Miller <mtmiller>
Project AdministratorIn charge of this item.
Sat 21 Feb 2015 08:40:34 PM UTC, comment #3: 

Depends how busy you are, but I think we could fit this in to the 4.0 release.  We try not to break ABI compatibility between minor versions which means the next time this might get in would be 4.2 if I understand the OpenMP stuff correctly.

Rik <rik5>
Project Administrator
Sat 21 Feb 2015 08:02:33 PM UTC, comment #2: 

No, I think I'll delay this until after the release.

I want to fix this by enabling OpenMP by default, and replace our custom OpenMP configuration with http://www.gnu.org/software/autoconf-archive/ax_openmp.html. We've been building with OpenMP enabled in Debian for a long time with no problems. But since the impact of this is pretty low and we're close to the release now, I'll make that change later.

Mike Miller <mtmiller>
Project AdministratorIn charge of this item.
Sat 21 Feb 2015 07:01:36 PM UTC, comment #1: 

@Mike: Is this issue something we want to worry about for the 4.0 release?  The Severity is marked as minor, so maybe not.

Rik <rik5>
Project Administrator
Wed 23 Apr 2014 12:15:04 PM UTC, original submission:  

When Octave unloads an oct-file that uses OpenMP, either directly or indirectly via a third-party library, Octave can crash, either immediately or after receiving a signal of some kind, such as a SIGCHLD or SIGCONT. The cause of the crash appears seems to be because OpenMP creates a thread pool that runs in the background until the process exits. When the OpenMP library is unloaded from the process, one of the OpenMP threads will segfault when scheduled to run.

This is not just a problem with some theoretical oct-files that users may create with OpenMP. The GraphicsMagick library uses OpenMP, and the _magick_read_ oct-file that acts as the interface to the library for Octave image functions suffers from this problem right now.

Example:

octave:1> im = imread ("large_file.tif");
warning: your version of GraphicsMagick limits images to 8 bits per pixel
octave:2> clear -f
octave:3> system ("pwd");
/home/mike
panic: Segmentation fault -- stopping myself...
attempting to save variables to 'octave-workspace'...
save to 'octave-workspace' complete
octave:4> Segmentation fault

The same occurs for me when backgrounding Octave with Ctrl-Z and resuming after the clear -f. When running Octave in gdb, the segfault happens immediately after the clear -f.

If OMP_NUM_THREADS is set to 1, no segfault occurs because OpenMP does not create any additional threads.

If Octave is compiled without OpenMP itself, then it has no reason to keep the OpenMP library (libgomp in the case of gcc) in process memory, so it is unloaded when the oct-file that loaded it is removed. If Octave is built with -fopenmp, then all Octave libraries and executables will keep the libgomp library in memory for the entire execution lifetime, so unloading an oct-file will not unload libgomp.

If there are any other suggestions for how to solve the problem of unloading an oct-file that may have done some OpenMP work without unloading libgomp from the process, please comment here. The only reasonable solutions I've seen when researching this are to either use LD_PRELOAD when the program is started, or to build the entire program with -fopenmp. If I don't hear any better solutions or strong arguments not to do it, I will work on a change to enable OpenMP in Octave by default from now on.

Mike Miller <mtmiller>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mtmiller (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2015-02-22 jwe StatusIn Progress => Fixed
        Open/ClosedOpen => Closed
    2015-02-22 mtmiller StatusFixed => In Progress
        Open/ClosedClosed => Open
    2015-02-22 mtmiller StatusPostponed => Fixed
        Open/ClosedOpen => Closed
    2015-02-21 mtmiller Priority5 - Normal => 3 - Low
        StatusNone => Postponed

    Back to the top


    Powered by Savane 3.5