Thu 23 Jun 2016 08:08:16 AM UTC, comment #40:
Hi, I attach an exercise I made in order to detect LAPACK version (but not if it is from MKL or whatever).
(file #37557)
|
Wed 13 Apr 2016 08:26:26 PM UTC, comment #39:
Yeah, I saw ATL_buildinfo, but it prints to the standard output which I deemed useless to us. I don't think we want to open a pipe and fork just so we can get this string from the ATLAS library.
If we can figure out the best way to do this dlsym function detection, and the equivalent for Windows, then maybe we should just detect the presence of each of them until we get a way to actually get a version string from all of them.
|
Wed 13 Apr 2016 07:58:36 PM UTC, comment #38:
For ATLAS, there is ATL_buildinfo [1]. That function prints the version info. Maybe we can parse its output...
[1] http://math-atlas.sourceforge.net/faq.html#probeV
|
Wed 13 Apr 2016 07:30:41 PM UTC, comment #37:
I also don't like that we are giving special treatment to proprietary software here.
How about asking the ATLAS and OpenBLAS maintainers to provide functions that will return version information so that we can use those as well?
|
Wed 13 Apr 2016 07:27:50 PM UTC, comment #36:
In Octave, we prefer C++ style casts instead of C-style casts.
I'd probably use a typedef to define a type for the function and then an assignment with a reinterpret_cast<>:
The blas version function definitely doesn't belong in the oct-shlib file. Maybe it would be better to add a function to the oct-shlib class to either open the current executable as a shared library, then we could just use the resulting octave_shlib object as we do any other to search for symbols.
But I'm not sure what the best organization is for this yet. I'll have to thing about this a bit more.
|
Wed 13 Apr 2016 05:42:49 PM UTC, comment #35:
As suggested by Mike, the attached patch checks for functions specific to OpenBLAS or ATLAS and prints that (but not the version numbers). If MKL is loaded, it returns the string returned by mkl_get_version_string.
I put the new function "blas_version" into file "oct-shlib.cc" because it already had some logic for including the correct headers for dlsym. If it should be moved somewhere else please let me know (or feel free to move it).
When compiling with this patch, I get the following warning:
Is this important? I do not know what the current-style cast would be in this case. "dlsym" does not seem to be standard conformant anyway judging from what I could find. Is there any other better way?
Neither can I test whether this works because I do not have access to the MKL.
The output for me with this patch is:
Would a similar approach for LAPACK be OK? Check for ATLAS, give MKL version string, default answer "unknown or reference LAPACK"? Are there any other LAPACK implementations that should be checked? Are these supported at all? I could not find any in ax-lapack.m4.
Another minor change in this patch: Matlab supports any mix of upper and lowercase letters in the options to version. This fixes this difference.
(file #36918)
|
Tue 12 Apr 2016 07:29:03 PM UTC, comment #34:
Just to clarify, both ATLAS and OpenBLAS do have ways to get each of their versions, but only if you include and compile against their header files. Their versions are not encoded in the compiled libraries anywhere, so that is not useful to us the way Octave currently uses those libraries.
|
Tue 12 Apr 2016 07:21:14 PM UTC, comment #33:
Correct, there is no standard way that I know of to get version information, nor is there even a non-standard way to get a version string out of any of the BLAS libraries that I can see, other than Intel's MKL (mkl_get_version_string).
I'd say the best we could possibly do would be to detect that either OpenBLAS or ATLAS are loaded and print that (but not the version numbers), or detect that MKL is loaded and print its version string, or failing any of those shrug and say "unknown or reference BLAS".
|
Mon 11 Apr 2016 08:09:15 PM UTC, comment #32:
I don't see where flexiblas extracts version info from the blas or lapack libraries. Can you point directly to that info?
In any case, I don't see how that can be done for an arbitrary blas or lapack library given that there is no standard way in the blas and lapack specs to determine version info.
|
Mon 11 Apr 2016 07:36:18 PM UTC, comment #31:
The FlexiBLAS project ( http://www.mpi-magdeburg.mpg.de/projects/flexiblas ) is an abstraction layer to the BLAS library to select a BLAS library during runtime. FlexiBLAS can display vendor string and version info for the currently selected BLAS backend, so may be their solution can also be used here.
Matlab, at least under Linux, can handle changing the BLAS and LAPACK backends via LD_LIBRARY_PATH and the version function returns the correct version strings.
|
Mon 11 Apr 2016 06:04:43 PM UTC, comment #30:
As for the blas and lapack versions, there is no simple platform-independent way to check for that information.
The info at build time isn't useful because you can build against the reference blas but use OpenBLAS, ATLAS, or some other vendor blas at run time.
Additionaly, there is no standard way for the blas or lapack library to report version info. I don't think that information is even stored for the reference versions of the lapack or blas libraries.
Matlab can provide this information because they distribute proprietary binaries that are linked with speciic blas and lapack libraries. As I understand it replacing the Matlab blas or lapack libraries is explicitly unsupported, so if you swap them out yourself and something breaks, it's your fault.
|
Mon 11 Apr 2016 05:54:37 PM UTC, comment #29:
I checked in an additional change to store the id in libgui and the main cli and gui executables and check for consistency at startup.
|
Mon 11 Apr 2016 04:33:03 PM UTC, comment #28:
I checked in some changes to store the hg id separately in liboctave and liboctinterp so we can easily detect mismatches.
A more common problem might be picking up incorrect versions of those libraries from the main octave-gui or octave-cli programs due to some bad LD_LIBRARY_PATH setting or other installation problem. So maybe we should also store another copy of the hg id in the src directory and check for consistency when Octave starts. If we do this, should a mismatch be fatal or just produce a warning?
If we require consistency for Octave to start, then I guess we could just pick one and go back to having only "hg_id" as a single field in the _octave_config_info_ struct.
|
Mon 11 Apr 2016 03:43:25 PM UTC, comment #27:
+1 for reproducible builds also. If I clone the same source and build twice, I hope to get the same output. I'd prefer to not have the time that I happen to be building affect the product of that build.
|
Mon 11 Apr 2016 03:31:46 PM UTC, comment #26:
Sorry about the duplicate text in my last comment, I did some cut and paste and failed to remove the copied text.
For Matlab compatibility we just need the release date, correct? In that case, I think we should not put the build date and time in a source file. I don't think that matters to users. Developers who want to know the build time can look at timestamps in their build tree and compare checksums of installed files with the ones in the build tree to know whether they are the same.
I'll make that change.
|
Mon 11 Apr 2016 03:21:17 PM UTC, comment #25:
I checked in your changes to add fftwf_version to _octave_config_info_. Thanks.
It's not the HG-ID target being tagged with .PHONY that is causing the trouble. There are move-if-change rules to prevent further action if there are no changes to the build-info.cc source file.
The problem is that we are now putting the build time into the source file. So that will always change and I see no simple way to fix that.
If you use _TIME_ and _DATE_ as you did before, those could be signifcantly different from the actual last build time if your hg id wasn't changing. For example, if you were working on some changes and your hg id was always SOMETHING+ (you aren't checking in any changes) but you are repeatedly building. Then the "build time" would not change because the source file with the _TIME_ and _DATE_ would not change.
If you change %OCTAVE_BUILD_TIME% to 0 in libinterp/build-info.in.cc, then run make a few times you should see what I mean. HG-ID and build-info.cc will be generated, but build-info.cc shouldn't change so no compiling or linking should happen either.
If we use _TIME_ and _DATE_ as you did before, then the timestamps recorded in the object file could be signifcantly different from the actual last build time if your hg id wasn't changing. For example, if you were working on some changes and your hg id was always SOMETHING+ (you aren't checking in any changes) but you are repeatedly building. Then the "build time" would not change because the build-info.cc source file with the _TIME_ and _DATE_ macros would not change, so would not be recompiled.
Do we really want build time and date embedded in a source file? Is this really a Matlab compatibility issue? Who cares? The more important info is the HG ID.
Maybe just record the build time in a text file somewhere and have Octave read it at startup? Maybe just edit into the version.m file directly?
But I still think that it would be useful to have the hg id stored in liboctave, libinterp and possibly libgui so that we can ensure that the libraries are consistent and the ones that are expected.
|
Mon 11 Apr 2016 09:19:51 AM UTC, comment #24:
The attached patch adds "fftwf_version" to octave_config_info. Under normal circumstances, this should always be the same as "fftw_version".
I did not know about ctime. Maybe it could be added to the "see also" of datestr?
A comment on the make rules for HG-ID and libinterp/build-info.cc: As it stands right now, libinterp/build-info.cc implicitly is a phony target because it depends on the phony target HG-ID. Thus, build-info.cc is created on every run of make. The changed file is then re-compiled and linked in every time. This might lead to an unnecessarily increased compile (and linker!) time when only changes on code for unrelated libraries were done.
My first approach was making only the target that created HG-ID phony and making libinterp/build-info.cc depend on the file instead of on the target.
This however meant that the build time (in the same file) was only updated (automatically) when the hg-id changed. Maybe a small trade-off for local wip builds.
Any of the two approaches has their advantages, imho. Just wanted to re-phrase what I tried to explain in my (not very clear) comment #12.
(file #36891)
|
Sun 10 Apr 2016 07:58:28 PM UTC, comment #23:
I checked in the following changeset:
http://hg.savannah.gnu.org/hgweb/octave/rev/184b85b31e03
It drops the local date and time. Worse than there being some small difference between the time the build-info.cc file is generated and compiled is that the _TIME_ and _DATE_ depend on timezone.
You can use the Unix-style time functions that Octave already provides to convert the time to some other form. See for example, ctime or localtime + asctime or strftime, etc.
Since the fftw version function was defined in oct-fftw.cc, the declaration belongs in oct-fftw.h, not build-info.h. Should we care about the fftwf version separately from the fftw version? I suppose they could be different, most likely if there is some installation problem. Having both could help in diagnosing that but it is probably not likely to see that problem.
|
Sun 10 Apr 2016 09:37:34 AM UTC, comment #22:
The last patch did not work because I forgot to include my local changes to the build-info.h header. This version should compile.
It also adds the new field "build_unixtime" to the __octave_config_info___ structure.
I left the build_time and build_date fields in because they are in a more user readable form. However, they will be a little bit off depending on how fast the system is for compilation. I am OK with removing them if you deem it better.
To get a more user readable date string from the unix time, you can use the following line of code:
Please, observe that this is UTC+0 independent on your local time zone. The same holds for build_date and build_time (at least when compiling with gcc).
Maybe it would make sence to add small helper functions to convert between unix time and Matlab time (if not already there) when removing build_date and build_time. These could maybe also contain optional conversion to local timezone. But maybe better keep this out of this bug...
(file #36887)
|
Sun 10 Apr 2016 01:30:47 AM UTC, comment #21:
Another small change to the makefile rule for the HG-ID target: The presence of the .hg directory does not necessarily mean that hg is available. It could have been deinstalled or it could be a copy to another PC...
(file #36885)
|
Sun 10 Apr 2016 01:10:23 AM UTC, comment #20:
It seems I was a little late with the last patch. Attached please find the patch rebased to the current tip. Basicly just the part with the fftw version and a change to reflect the new field names in __octave_config_info__.
(file #36884)
|
Sun 10 Apr 2016 12:26:37 AM UTC, comment #19:
With the attached patch the makefile directly inserts the hg id into the template for build-info.cc. Some tests should make sure that hg is present and that we are building from an hg repo. The hg id would be 0 otherwise.
I added FFTW to the version function. Are there any alternative libraries that are supported by Octave that might have their version info available with a different call?
The respective output for me is:
Regarding BLAS and LAPACK: Does the build system "know" which versions or variants it is using? If yes, can this info be stored somehow?
(file #36883)
|
Sat 09 Apr 2016 11:46:11 PM UTC, comment #18:
Markus: I checked in your changeset then another one to make the changes I was suggesting in comment #17:
http://hg.savannah.gnu.org/hgweb/octave/rev/fe1447ae68cf
http://hg.savannah.gnu.org/hgweb/octave/rev/cf552443c104
I tested it by doing "make dist" and then building from the tar.gz file. Both were out-of-source-tree VPATH builds.
Now I'm wondering whether it would be better to store a universal time stamp (Unix time?) instead of the local time and date.
|
Fri 08 Apr 2016 09:54:18 PM UTC, comment #17:
Markus:
Since the shell script in build-aux just runs a single shell command, I don't think we need it. The other scripts that do substitutions are there because they substitute configuration variables and it is better to put those variables in a shell script that embed them in the Makefile. That's also why those shell scripts are generated at configure time.
What happens with this hg id substitution if you are building from a tar file distribution? In that case "hg id" won't give us a version number and we can't even assume that hg will be available.
|
Fri 08 Apr 2016 07:47:35 PM UTC, comment #16:
I don't know for sure, but I think the BLAS and LAPACK versions may be the ugliest part of this.
It may involve something like checking for specific functions that are only in MKL or ATLAS or OpenBLAS, and then querying each for their particular version strings. I have no idea if netlib BLAS and LAPACK provide any way to query a version.
The undocumentedmatlab link in comment #1 also shows that the version of FFTW can be queried in Matlab.
|
Fri 08 Apr 2016 07:41:53 PM UTC, comment #15:
In this version of the patch, the release date and the build date and time are part of the __octave_config_info__ struct.
Now build-info.cc is the file that is generated by the script and the header is static like you suggested. I agree that this is a much better approach.
Regarding the version functions defined in each library: Would these be versions that only change once in a while (hence not the hg id)? Would these be the same as OCTAVE_VERSION?
With this patch version.m supports all documented flags.
Is it OK to use the try catch statement for the Java functions? Is it reliable to check _octave_config_info_.build_features.JAVA instead?
I don't know how to get the info requested in comment #0. Does anyone have an idea how to check the vendor and version of the linked in BLAS and LAPACK libraries?
The current output of version.m and the new fields of __octave_config_info__ with this patch on my system are:
(file #36869)
|
Thu 07 Apr 2016 04:13:38 PM UTC, comment #14:
Just a heads up, but I'm also looking at something where I need version and config info in liboctave, so some of these things may eventually be moved there. My changes are not urgent though, so they can wait until the issues in this bug report are resolved.
|
Thu 07 Apr 2016 04:06:00 PM UTC, comment #13:
Thanks for working on this.
Instead of adding a new function to return the release date, wouldn't it be sufficient to just add the release date to the octave_config_info structure?
I'd rather include header files to provide function declarations instead of inserting them by hand (your extern decl for oct_hg_id) in toplev.cc. So instead of generating the header file, why not make that static and generate the source file instead? Then toplev.cc could include build-info.h (which won't change with each change to the hg id) and the hg ID can be substituted into the build-info.cc file.
After thinking about it a little more, and also encountering numerous times when bugs are reported and we aren't sure whether the correct versions of libraries are being used, I think it might be useful to have version functions defined in each library (liboctave_version, libinterp_version, and libgui_version). Then the octave_config_info could contain fields for each of these. If things are working properly, each of these should return the same result. But if there is some build problem, we could detect that there is a library version mismatch and at least warn about it.
Also, a small comment about commit messages. Instead of writing "Add this or that ...", please write things like
|
Thu 07 Apr 2016 08:22:22 AM UTC, comment #12:
I removed the build date and time from version.m.
The output of version for the attached patch is now:
The date is returned as stored in OCTAVE_RELEASE_DATE. Thus, it is formatted differently from what Matlab returns ("yyyy-mm-dd" instead of "mmmm d, yyyy"). Should I reformat with datestr for maximum compatibility?
The __octave_config_info__ struct now has a new field "hgid" holding the hg id.
The only file that has to be rebuild when the hg id changes is the new build-info.cc with its generated header.
I left the functions exposing the build date and time and the hg id directly to Octave intact for whom it might interest. My reasoning was that they can reside in the same file that has to be re-build anyway. However, date and time are only updated when build-info.cc is build (e.g. when it has to be re-build to update the hg id).
If it is agreed upon that these are not needed, I am OK with removing them.
Instead of making the header file holding the hg id directly a phony target, I took the following approach to keep the build time as efficient as possible:
- store the current hg id in a text file
- generate the header file that stores the hg id
- keep the function exposing the hg id in a seperate file
On subsequent calls:
- use a phony target that compares the current hg id with the one stored in the file. Only if the hg id has changed:
- overwrite the text file with the current hg id
- generate the header file that stores the hg id
Thus, on subsequent builds of the same revision (only local changes), nothing hg id related has to be rebuild. On updates of the hg id, only one additional object has to be rebuild (and linked).
Are you OK with these changes?
(file #36853)
|
Wed 06 Apr 2016 04:47:02 AM UTC, comment #11:
I think it would be fine to have the hg revision (if available) stored somewhere in the binary. Can it be stored separately from the version info? Possibly as a separate field in the _octave_config_info_ struct? To make that fairly efficient for the build, there could be a single file that contains a function that returns the hg version. Then the only thing that would have to happen at build time is to compile that file (if it has changed) and relink.ee
I don't know whether we want to include the build time and date in the binary. That would mean recompiling at least one file and relinking every time make runs. That seems like a bit much to me. But if others want this feature and someone wants to implement it, it's OK with me as long as it is optional.
|
Wed 06 Apr 2016 04:14:37 AM UTC, comment #10:
Thanks for starting to work on this.
Do we want the build date and time to be updated every time Octave is compiled? Or do we want to just use the OCTAVE_RELEASE_DATE string that's already defined?
Do we want to append the hg revision to the version string like this? Or do we only want it to be queryable with a different option?
I wouldn't be surprised if some people depend on the version return value to be a single word with the dotted version string, appending the hg revision might break that expectation.
And if we do append the hg revision to the version, do we want to suppress that when building an officially tagged release?
|
Tue 05 Apr 2016 04:35:53 PM UTC, comment #9:
Thank you all for your help and support.
The attached patch shows a work in progress without much comments and tests. Please, let me know if this goes in the correct direction.
As already pointed out, the added functions should reside in files which have as few dependencies as possible to cause as little rebuilding as necessary. Thus, I reasoned it would be best to add new files for them right next to where version.h is.
At the moment, it is necessary to touch build_timestamp.in.h for the hg ID, build date and time to update. Maybe it is possible to change the makefiles such that the corresponding files are build each time. They are quite lightweight and it should not cost too much to build them each time. However, I do not know what changes would be necessary.
I did not yet research the -java, -blas and -lapack options. But I am looking forward too hearing feedback on the parts already done.
Here is what Matlab R2012a returns for version:
Here is what Octave returns for me with the attached patch:
The plus after the hg id indicates local changes.
Arguments are still ignored.
(file #36841)
|
Tue 05 Apr 2016 03:29:13 PM UTC, comment #8:
See
https://bitbucket.org/snippets/mtmiller/5rb7o/autoconf-version-suffix-from-mercurial
for a script I worked up to generate a mercurial revision string. Note that my goal was for it to be an empty string if building from a revision that is exactly a tagged release (so the version string would not have a suffix in that case). That may not be desired for this depending on how it is used.
|
Tue 05 Apr 2016 03:23:46 PM UTC, comment #7:
Yeah, I took a look at adding the mercurial id to the Octave version but ran into a couple of problems (if this bug is indeed growing to adopt that feature).
First, I tried adding it to configure.ac, but that frankly causes way too much rebuilding.
Second, adding it to the build without needing to reconfigure / rebuild a bunch of things needs a build-aux shell script to produce the mercurial ID on stdout (easy) that can be $(shell) evaluated into a make variable. This means that make would have to check whether version.h needs to be updated every time make is run. Doable, but unfortunate.
That part aside, I think that adding it to version.h may work. I think there may still be too many cc files including version.h that don't actually need it, if we could reduce that list it would help keep rebuilds fast every time a hg commit is made.
|
Tue 05 Apr 2016 01:29:28 PM UTC, comment #6:
Sorry, it wasn't a bug report, it was a short thread on the maintainers mailing list:
http://lists.gnu.org/archive/html/octave-maintainers/2015-12/msg00229.html
|
Tue 05 Apr 2016 01:18:02 PM UTC, comment #5:
More useful would be the hg id (if building from the mercurial sources). We already had a discussion about that in the comments for another bug report.
For releases, I don't think there's much we can do that is better than using the version number. It would be difficult to know whether the build is from modified sources. Well, I suppose that's true for builds from the mercurial sources as well...
|
Tue 05 Apr 2016 11:34:00 AM UTC, comment #4:
I thought about exposing OCTAVE_RELEASE_DATE for this function. That would be "2015-05-26" for the foreseeable time.
If desired, I could look into how to insert the current date and time at compilation into the code...
|
Tue 05 Apr 2016 10:25:48 AM UTC, comment #3:
I think that this
[v, d] = version
v = 4.1.0+
d = 2015-05-26
would be very helpful.
For me the date and maybe the time, would be a way of telling which compile I am running. The version number stays the same for months at at time, but the time will be different each compile.
|
Tue 05 Apr 2016 07:14:33 AM UTC, comment #2:
I would be happy to work on this one if no one else wants to take it first...
|
Mon 04 Apr 2016 06:24:02 PM UTC, comment #1:
Thanks for filing this bug report.
According to
https://www.mathworks.com/help/matlab/ref/version.html
the following should work:
And according to this bug report, the versions of BLAS and LAPACK libraries should work with the -blas and -lapack options. These options are not documented at all on Matlab's site, but are demonstrated here:
http://undocumentedmatlab.com/blog/math-libraries-version-info-upgrade
This might be a good small project for a new contributor to work on.
|
Thu 30 Jul 2015 06:12:57 PM UTC, original submission:
Matlab returns the BLAS and LAPACK "vendor string" and version by the version command:
and
|