bugGNU Octave - Bugs: bug #67140, Make built octave.jar reproducible

 
 

bug #67140: Make built octave.jar reproducible

Submitter:  Atri Bhattacharya <badshah400>
Submitted:  Tue 20 May 2025 01:20:10 PM UTC
   
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Feature Request
Status:  In Progress Assigned to:  None
Originator Name:  Atri Open/Closed:  * Open
Release:  * 10.1.0 Release: 
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 30 May 2025 06:02:11 PM UTC, comment #10: 

Because the test for JAR_HAVE_DATE_OPTION is somewhat slow (a call to the shell to invoke an executable), we should do the test once and cache the result.  The macro to use is AC_CACHE_CHECK (https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Caching-Results.html).  The cache variable naming scheme is documented at https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Cache-Variable-Names.html#Cache-Variable-Names.  If I may, I suggest 'octave_cv_jar_has_date_option'.

For a template, I would modify this test for jni.h


  AC_MSG_CHECKING([for include file <jni.h>])
  AC_CACHE_VAL([octave_cv_java_have_jni],[
    JNI_PATH=`echo $JAVA_CPPFLAGS | $SED -e 's/-I//g'`
    octave_cv_java_have_jni=no
    for dir in $JNI_PATH; do
      if test -f "${dir}/jni.h"; then
        octave_cv_java_have_jni=yes
        octave_cv_java_jni_h_path=$dir
        break
      fi
    done
  ])
  if test $octave_cv_java_have_jni = yes; then
    AC_MSG_RESULT([$octave_cv_java_jni_h_path])
  else
    AC_MSG_RESULT([not found])
    warn_java="Include file <jni.h> not found.  Octave will not be able to call Java methods."
    break
  fi


  Also, I think I agree with Markus that there probably isn't a need to do AC_SUBST since we don't substitute this into any output files using @VAR_NAME@ markers.


Rik <rik5>
Group administrator
Fri 30 May 2025 05:35:16 PM UTC, comment #9: 

I would initialize JAR_HAVE_DATE_OPTION to 'no'.  This is the sensible starting value, and it makes the resulting shell test sequence easier to understand.


JAR_HAVE_DATE_OPTION=no


followed by


        if test $JAR_HAVE_DATE_OPTION = yes; then \




Rik <rik5>
Group administrator
Thu 29 May 2025 11:07:19 AM UTC, comment #8: 

Pasting patch here rather that attaching, so it is easier to quote and review. Will attach the version we agree looks final.

I went with your suggestion, only I renamed to AC variable to 'JAR_HAVE_DATE_OPTION' instead of 'JAR_HAVE_DATE_FLAG'.

I agree with you that it may be redundant to check for `date`.


---
 configure.ac           |   11 +++++++++++
 scripts/java/module.mk |   11 +++++++++--
 2 files changed, 20 insertions(+), 2 deletions(-)

Index: octave-10.1.0/scripts/java/module.mk
===================================================================
--- octave-10.1.0.orig/scripts/java/module.mk
+++ octave-10.1.0/scripts/java/module.mk
@@ -52,11 +52,18 @@ $(%canon_reldir%_JAVA_CLASSES) : %.class
                      -d $(abs_top_builddir)/scripts/java \
                      $(org_octave_dir)/$(<F) )

+JAR_DATE = $(shell date -u -d"@$(SOURCE_MTIME)" -I'seconds')
+
 if AMCOND_HAVE_JAVA
 %reldir%/octave.jar: $(%canon_reldir%_JAVA_CLASSES)
         $(OCT_V_JAR)rm -f $@-t $@ && \
-        ( cd scripts/java; \
-          "$(JAR)" cf octave.jar-t $(JAVA_CLASSES) ) && \
+        if test "x$(JAR_HAVE_DATE_OPTION)" = "xyes"; then \
+          ( cd scripts/java; \
+            "$(JAR)" -c --date="$(JAR_DATE)" -f octave.jar-t $(JAVA_CLASSES) ) \
+        else \
+          ( cd scripts/java; \
+            "$(JAR)" -c -f octave.jar-t $(JAVA_CLASSES) ) \
+        fi && \
         mv $@-t $@
 endif

Index: octave-10.1.0/configure.ac
===================================================================
--- octave-10.1.0.orig/configure.ac
+++ octave-10.1.0/configure.ac
@@ -2479,6 +2479,7 @@ fi
 JAVA=
 JAVAC=
 JAR=
+JAR_HAVE_DATE_OPTION=
 JAVA_LIBS=

 dnl Fake loop so that "break" can be used to skip code blocks.
@@ -2697,6 +2698,15 @@ do
     break
   fi

+  ## Test if jar supports --date for reproducible builds
+  AC_MSG_CHECKING([whether jar supports '--date' option])
+  if "$JAR" --help | grep -- '--date=' > /dev/null 2>&1; then
+      JAR_HAVE_DATE_OPTION=yes
+      AC_MSG_RESULT($JAR_HAVE_DATE_OPTION)
+  else
+      AC_MSG_RESULT(no)
+  fi
+
   ## Passed all configuration tests.  A workable Java installation was found.
   build_java=yes
   AC_DEFINE(HAVE_JAVA, 1,
@@ -2711,6 +2721,7 @@ AM_CONDITIONAL([AMCOND_HAVE_JAVA], [test
 AC_SUBST(JAVA)
 AC_SUBST(JAVAC)
 AC_SUBST(JAR)
+AC_SUBST(JAR_HAVE_DATE_OPTION)
 AC_SUBST(JAVA_CPPFLAGS)
 AC_SUBST(JAVA_LIBS)
 AC_DEFINE_UNQUOTED([JAVA_HOME], ["$JAVA_HOME"], [Java home (top-level installation dir)])


Atri Bhattacharya <badshah400>
Thu 29 May 2025 09:44:23 AM UTC, comment #7: 

Another point that we might need to consider:
This might be the first place where we rely on the `date` command to work correctly.
Afaict, that command is part of the GNU coreutils. So, expecting that it exists and does the the right thing might be fine.

Markus Mützel <mmuetzel>
Group administrator
Thu 29 May 2025 09:38:07 AM UTC, comment #6: 

Thank you for the quick update.

Another point that I missed in the first round (sorry for that):
That's no hard rule. But we usually try to not conditionally AC_SUBST Makefile variables.

Could you please take a look at how, e.g., `JAR` is handled around there in configure.ac and handle `JAR_SUPPORT_DATE` similarly?
I'm not sure how the naming convention would be for this case. But maybe, something like `JAR_HAVE_DATE_FLAG` would be better?

Markus Mützel <mmuetzel>
Group administrator
Thu 29 May 2025 09:15:17 AM UTC, comment #5: 

comment #3:

>
> at least change the expression to be safer for the case that $JAR is empty.
> E.g., something like that for the if-expression:


>   if "$JAR" --help 2>&1 | grep -- '--date=' > /dev/null 2>&1; then


>
> Other than that, this looks good to me.
>


Thanks for the review. That is indeed useful. Updated patch attached.

(file #57251)

Atri Bhattacharya <badshah400>
Thu 29 May 2025 09:00:08 AM UTC, comment #4: 

Oops. Misread the condition for the check for presence of "javac" and "jar". Both must be found to be able to continue with Java.

It might still be worth adding at least the double-quotes there (in case of white-spaces).

Markus Mützel <mmuetzel>
Group administrator
Thu 29 May 2025 08:50:41 AM UTC, comment #3: 

Thank you very much for the proposed patch.

Afaict, Octave currently doesn't require that "$JAR" is actually defined. (It continues to configure if any of "javac" or "jar" was found.)
Would it be sensible to include a test for that? Or at least change the expression to be safer for the case that $JAR is empty.
E.g., something like that for the if-expression:

  if "$JAR" --help 2>&1 | grep -- '--date=' > /dev/null 2>&1; then


Other than that, this looks good to me.

Markus Mützel <mmuetzel>
Group administrator
Tue 20 May 2025 08:54:39 PM UTC, comment #2: 

I have attached the (hopefully) final version of my patch here. This should work with older versions of jar that do not support the '--date' option as well. In such a case, it will simply not use the option when jar-ing and lose out on reproducibility (cannot be helped).

Sorry for the multiple messages.

Atri Bhattacharya <badshah400>
Tue 20 May 2025 03:07:48 PM UTC, comment #1: 

I should note that the --date option is supported for jar from openjdk versions 19 (released 2022) and above. This will not work with older openjdk versions.

Atri Bhattacharya <badshah400>
Tue 20 May 2025 01:20:10 PM UTC, original submission:  

At present, when 'octave.jar' is built from sources, it embeds the host date-time affecting consistent reproducibility of the built archive. This is a problem for distros such as openSUSE where we are striving to reduce reproducibility problems like this. The following patch passes the --date option to jar to set the jar date to the SOURCE_MTIME to avoid jar picking up the host build time/date instead. I would be happy to revise the patch if you have any suggestions.

Thanks in advance.


---
 scripts/java/module.mk |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Index: octave-10.1.0/scripts/java/module.mk
===================================================================
--- octave-10.1.0.orig/scripts/java/module.mk
+++ octave-10.1.0/scripts/java/module.mk
@@ -52,11 +52,13 @@ $(%canon_reldir%_JAVA_CLASSES) : %.class
                      -d $(abs_top_builddir)/scripts/java \
                      $(org_octave_dir)/$(<F) )

+JAR_DATE = $(shell date -u -d"@$(SOURCE_MTIME)" -I'seconds')
+
 if AMCOND_HAVE_JAVA
 %reldir%/octave.jar: $(%canon_reldir%_JAVA_CLASSES)
         $(OCT_V_JAR)rm -f $@-t $@ && \
         ( cd scripts/java; \
-          "$(JAR)" cf octave.jar-t $(JAVA_CLASSES) ) && \
+          "$(JAR)" -c --date="$(JAR_DATE)" -f octave.jar-t $(JAVA_CLASSES) ) && \
         mv $@-t $@
 endif


Atri Bhattacharya <badshah400>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #57251:  octave-reproducible-jar.patch added by badshah400 (2KiB - text/x-patch - Updated patch)
file #57232:  octave-reproducible-jar.patch added by badshah400 (2KiB - text/x-patch - patch: Check if jar supports --date and use it to make jar builds deterministic/reproducible)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by badshah400 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2025-05-29 badshah400 Attached File- Added octave-reproducible-jar.patch, #57251
    2025-05-29 mmuetzel StatusNone In Progress
    2025-05-20 badshah400 Attached File- Added octave-reproducible-jar.patch, #57232

    Back to the top

    Powered by Savane 3.15-26b0.
    Corresponding source code