bugGNU Octave - Bugs: bug #37713, HDF5 error message on exit

 
 

bug #37713: HDF5 error message on exit

Submitter:  Torsten Lilge <ttl>
Submitted:  Sun 11 Nov 2012 03:28:02 PM UTC
   
 
Category:  GUI Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Invalid / Not an Octave Bug Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 25 Oct 2013 04:44:16 AM UTC, comment #14: 

Yes, no ore message on my system, too.

Torsten Lilge <ttl>
Group Member
Thu 24 Oct 2013 09:22:36 PM UTC, comment #13: 

I'm not sure why this report was closed.  It still seems to be a problem.  The following changeset seems to avoid the problem for me:

http://hg.savannah.gnu.org/hgweb/octave/rev/9c03b071fd7b


John W. Eaton <jwe>
Group administrator
Fri 18 Oct 2013 03:51:55 AM UTC, comment #12: 

Should we submit a bug report to HDF5 developers?  Do we have enough specific to create a useful bug report?

http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2010-November/003928.html

Dan Sebald <sebald>
Thu 17 Oct 2013 12:42:56 PM UTC, comment #11: 

Dan, if you read my last two comments you'd see that I already narrowed this down to a specific usage of the HDF5 library. Namely, calling the H5Fis_hdf5 function and no other HDF5 functions from a thread that is not the main thread in a program. If you link with the ordinary HDF5 library, this error is printed when the program shuts down. If you link with one of the MPI-enabled HDF5 libraries, no error.

To reproduce this you can either

  • use the example program in comment #9
  • run the Octave GUI with one or more packages installed and quit without doing anything
  • run the Octave GUI without any packages (octave -f) and do something like this:



>> x = 0;
>> save foo.txt x
>> load foo.txt
>> quit


Other than noise, I don't see any harmful behavior. For the moment, I consider this a bug in HDF5 and not a bug in Octave.

Mike Miller <mtmiller>
Group Member
Thu 17 Oct 2013 05:28:31 AM UTC, comment #10: 

Don't close this one just yet.  Although I followed the links and see some very similar bug reports for HD5, the code sequence is different from the pattern listed here.

Here is the hunk of HD5 code (from H5.c) generating this sequence:


    /*
     * Terminate each interface. The termination functions return a positive
     * value if they do something that might affect some other interface in a
     * way that would necessitate some cleanup work in the other interface.
     */
#define DOWN(F)                                                                      \
    (((n = H5##F##_term_interface()) && (at + 8) < sizeof loop)?              \
     (sprintf(loop + at, "%s%s", (at ? "," : ""), #F),                              \
      at += HDstrlen(loop + at),                                              \
      n):                                                                     \
     ((n > 0 && (at + 5) < sizeof loop) ?                                      \
     (sprintf(loop + at, "..."),                                              \
      at += HDstrlen(loop + at),                                              \
     n) : n))

    do {
        pending = 0;
        /* Try to organize these so the "higher" level components get shut
         * down before "lower" level components that they might rely on. -QAK
         */
        pending += DOWN(R);
        pending += DOWN(D);
        pending += DOWN(L);
        pending += DOWN(G);
        pending += DOWN(A);
        pending += DOWN(S);
        pending += DOWN(T);
        /* Don't shut down the file code until objects in files are shut down */
        if(pending == 0)
            pending += DOWN(F);

        /* Don't shut down "low-level" components until "high-level" components
         * have successfully shut down.  This prevents property lists and IDs
         * from being closed "out from underneath" of the high-level objects
         * that depend on them. -QAK
         */
        if(pending == 0) {
            pending += DOWN(AC);
            pending += DOWN(Z);
            pending += DOWN(FD);
            pending += DOWN(P);
            pending += DOWN(PL);
            /* Don't shut down the error code until other APIs which use it are shut down */
            if(pending == 0)
                pending += DOWN(E);
            /* Don't shut down the ID code until other APIs which use them are shut down */
            if(pending == 0)
                pending += DOWN(I);
            /* Don't shut down the skip list code until everything that uses it is down */
            if(pending == 0)
                pending += DOWN(SL);
            /* Don't shut down the free list code until _everything_ else is down */
            if(pending == 0)
                pending += DOWN(FL);
        }
    } while(pending && ntries++ < 100);


Basically, this loop attempts to shutdown higher level interfaces, then midlevel interfaces, then low level interfaces.  I assume the low level interfaces are used by the higher level interface.  Therefore, those can't be shut down until the higher level interfaces have successfully disassociated all APIs.

It looks as though the library is getting hung up on trying to shutdown the Error Code support.  Here is the H5E_term_interface routine:


int
H5E_term_interface(void)
{
    int        n = 0;

    FUNC_ENTER_NOAPI_NOINIT_NOERR

    if(H5_interface_initialize_g) {
        int ncls, nmsg, nstk;

        /* Check if there are any open error stacks, classes or messages */
        ncls = H5I_nmembers(H5I_ERROR_CLASS);
        nmsg = H5I_nmembers(H5I_ERROR_MSG);
        nstk = H5I_nmembers(H5I_ERROR_STACK);

        n = ncls + nmsg + nstk;
        if(n > 0) {
            /* Clear any outstanding error stacks */
            if(nstk > 0)
                H5I_clear_type(H5I_ERROR_STACK, FALSE, FALSE);

            /* Clear all the error classes */
            if(ncls > 0) {
                H5I_clear_type(H5I_ERROR_CLASS, FALSE, FALSE);

                /* Reset the HDF5 error class, if its been closed */
                if(H5I_nmembers(H5I_ERROR_CLASS) == 0)
                    H5E_ERR_CLS_g = -1;
            } /* end if */

            /* Clear all the error messages */
            if(nmsg > 0) {
                H5I_clear_type(H5I_ERROR_MSG, FALSE, FALSE);

                /* Reset the HDF5 error messages, if they've been closed */
                if(H5I_nmembers(H5I_ERROR_MSG) == 0) {
                    /* Include the automatically generated error code termination */
                    #include "H5Eterm.h"
                } /* end if */
            } /* end if */
        } /* end if */
        else {
            /* Destroy the error class, message, and stack id groups */
            H5I_dec_type_ref(H5I_ERROR_STACK);
            H5I_dec_type_ref(H5I_ERROR_CLASS);
            H5I_dec_type_ref(H5I_ERROR_MSG);

            /* Mark closed */
            H5_interface_initialize_g = 0;
            n = 1; /*H5I*/
        } /* end else */
    } /* end if */

    FUNC_LEAVE_NOAPI(n)
} /* end H5E_term_interface() */


Well, there are three elements to error handling that could be coming back always positive, i.e., n = ncls + nmsg + nstk > 0.

I'll make a wild guess: imagine that something in this process fails to happen correctly.  Then, stack is cleared, messages are cleared, classes are cleared...all fine and dandy.  Then it has to go through the loop again and tries again n = ncls + nmsg + nstk.  If an error happens the the error stack (or one of the other two) is now non-empty.  Basically, I'm wondering if HD5 can get past this stage at all if it keeps generating an error for some reason.

Anyway, can anyone think of something in the packages that is creating or not creating an HD5 error message, different from what might have been done in the past?

Dan Sebald <sebald>
Sat 12 Oct 2013 10:48:46 PM UTC, comment #9: 

It appears that the call to H5Fis_hdf5 in load-save.cc (get_file_format) is indeed what's setting up Octave to generate this error on exit, because it's being called in a secondary thread when run in GUI mode.

The following quick-n-dirty standalone program demonstrates the same error on exit for me:


#include <hdf5.h>
#include <pthread.h>

void *
func (void *p)
{
  char *arg = (char*)p;
  int x = H5Fis_hdf5 (arg);
  return 0;
}

int main(int argc, char *argv[])
{
  pthread_t thread;
  pthread_create (&thread, 0, func, argv[1]);
  pthread_join (thread, 0);
  return 0;
}


Mike Miller <mtmiller>
Group Member
Sat 12 Oct 2013 07:43:20 PM UTC, comment #8: 

Yes, that could be it. I deleted ~/.octave_packages and the error isn't printed now.

Mike Miller <mtmiller>
Group Member
Sat 12 Oct 2013 07:37:22 PM UTC, comment #7: 

Do you have any packages installed?  I see the message if Octave loads the octave_packages file.  If that file is not present, then I don't see the message.

I think this happens because Octave calls some HDF5 functions while it is trying to determine the type of the file that it is loading.

John W. Eaton <jwe>
Group administrator
Sat 12 Oct 2013 07:33:06 PM UTC, comment #6: 

I am also seeing the error on a Debian system with 1.8.11.

John W. Eaton <jwe>
Group administrator
Sat 12 Oct 2013 07:28:08 PM UTC, comment #5: 

I know we've marked this as something Octave can't do anything about. But for reference, I'm still seeing this error message on exit with HDF5 version 1.8.11 on Debian (non-MPI version).

Mike Miller <mtmiller>
Group Member
Thu 29 Aug 2013 05:26:01 PM UTC, comment #4: 

I don't think this is an Octave bug.  See here for a report on the HDF5 team having trouble with their own library (https://elist.ornl.gov/pipermail/visit-developers/2010-January/005486.html).  The fix definitely went in after version 1.8.4.  I just downloaded the latest released version, 1.8.11, and no error message is generated.  So, I suggest upgrading to a newer version of the HDF5 library.  Closing report.

Rik <rik5>
Group administrator
Thu 29 Aug 2013 04:20:10 PM UTC, comment #3: 

I can confirm this with a recent tip (hg id 91691d74845d) and I'm also using hdf5 version 1.8.4.

Interestingly, if I do any sort of load/save with an HDF5 file during the GUI session then the error message is not produced.

This code produces an error:


run-octave
exit


This code does not:


run-octave
x = 1;
save -hdf5 junk.hdf5 x
exit


So, it would seem that we are instantiating something, like an instance of the HDF5 class, which doesn't get initialized until an operation is done on it.

Rik <rik5>
Group administrator
Fri 12 Apr 2013 08:24:46 AM UTC, comment #2: 

OK, now I'm able to reproduce it by starting Octave, loading a MAT file (one produced by Matlab that someone sent) and then exiting Octave.

John W. Eaton <jwe>
Group administrator
Thu 11 Apr 2013 06:07:58 AM UTC, comment #1: 

I'm not seeing this error.  I'm using hdf5 1.8.8 on a Debian 64-bit system.

Do you have to do anything with hdf5, or just start and exit octave?

John W. Eaton <jwe>
Group administrator
Sun 11 Nov 2012 03:28:02 PM UTC, original submission:  

When closing the gui the following error message shows up (only visible if stderr is not redirected):


HDF5: infinite loop closing library
D,T,F,FD,P,FD,P,FD,P,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E


ubuntu 12.04 (32bit) and hdf5 version 1.8.4

Torsten Lilge <ttl>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by sebald (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by ttl (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-08-29 rik5 StatusNone Invalid / Not an Octave Bug
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code