bugGNU Octave - Bugs: bug #62135, `make dist` fails with bash error

 
 

bug #62135: `make dist` fails with bash error

Submitter:  Markus Mützel <mmuetzel>
Submitted:  Thu 03 Mar 2022 08:16:12 AM UTC
   
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 04 Mar 2022 05:18:05 PM UTC, comment #11: 

Thanks.

`make all`, `make check`, and `make dist` work for me with hg id 1385d0c8fa8c. The buildbots seem to be ok, too.

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Thu 03 Mar 2022 09:20:35 PM UTC, comment #10: 

I think it was working for me because I had a test/mex directory from a previous build.  I pushed two more changesets that I hope will fix the remaining problems.


John W. Eaton <jwe>
Group administrator
Thu 03 Mar 2022 08:30:43 PM UTC, comment #9: 

That seems to break "make check":



make[4]: Entering directory '/scratch/buildbot/workers/jwe-debian-x86_64-5/gcc-debian/build/test'
DL_LDFLAGS="-shared" ../src/mkoctfile --mex -I.. -I../../src/libinterp/corefcn -I../libinterp/corefcn -L../libinterp/.libs -L../liboctave/.libs -v ../../src/test/mex/bug_54096.c -o mex/bug_54096.mex || rm -f mex/bug_54096.mex
gcc -c  -fPIC -I/usr/local/include/octave-8.0.0/octave/.. -I/usr/local/include/octave-8.0.0/octave -I/usr/local/include  -pthread -fopenmp -g -O2   -I. -I.. -I../../src/libinterp/corefcn -I../libinterp/corefcn  -DMEX_DEBUG ../../src/test/mex/bug_54096.c -o /tmp/oct-7tnkMg.o
g++ -I/usr/local/include/octave-8.0.0/octave/.. -I/usr/local/include/octave-8.0.0/octave -I/usr/local/include  -pthread -fopenmp -g -O2  -o mex/bug_54096.mex  /tmp/oct-7tnkMg.o   -L../libinterp/.libs -L../liboctave/.libs -shared  -L/usr/local/lib -shared  -L/usr/local/lib -shared
/usr/bin/ld: cannot open output file mex/bug_54096.mex: No such file or directory
collect2: error: ld returned 1 exit status
DL_LDFLAGS="-shared" ../src/mkoctfile --mex -I.. -I../../src/libinterp/corefcn -I../libinterp/corefcn -L../libinterp/.libs -L../liboctave/.libs -v ../../src/test/mex/bug_51725.c -o mex/bug_51725.mex || rm -f mex/bug_51725.mex
gcc -c  -fPIC -I/usr/local/include/octave-8.0.0/octave/.. -I/usr/local/include/octave-8.0.0/octave -I/usr/local/include  -pthread -fopenmp -g -O2   -I. -I.. -I../../src/libinterp/corefcn -I../libinterp/corefcn  -DMEX_DEBUG ../../src/test/mex/bug_51725.c -o /tmp/oct-2lB7po.o
g++ -I/usr/local/include/octave-8.0.0/octave/.. -I/usr/local/include/octave-8.0.0/octave -I/usr/local/include  -pthread -fopenmp -g -O2  -o mex/bug_51725.mex  /tmp/oct-2lB7po.o   -L../libinterp/.libs -L../liboctave/.libs -shared  -L/usr/local/lib -shared  -L/usr/local/lib -shared
/usr/bin/ld: cannot open output file mex/bug_51725.mex: No such file or directory
collect2: error: ld returned 1 exit status
DL_LDFLAGS="-shared" ../src/mkoctfile --mex -I.. -I../../src/libinterp/corefcn -I../libinterp/corefcn -L../libinterp/.libs -L../liboctave/.libs -v ../../src/test/mex/mexnumtst.c -o mex/mexnumtst.mex || rm -f mex/mexnumtst.mex
gcc -c  -fPIC -I/usr/local/include/octave-8.0.0/octave/.. -I/usr/local/include/octave-8.0.0/octave -I/usr/local/include  -pthread -fopenmp -g -O2   -I. -I.. -I../../src/libinterp/corefcn -I../libinterp/corefcn  -DMEX_DEBUG ../../src/test/mex/mexnumtst.c -o /tmp/oct-gsxBhn.o
g++ -I/usr/local/include/octave-8.0.0/octave/.. -I/usr/local/include/octave-8.0.0/octave -I/usr/local/include  -pthread -fopenmp -g -O2  -o mex/mexnumtst.mex  /tmp/oct-gsxBhn.o   -L../libinterp/.libs -L../liboctave/.libs -shared  -L/usr/local/lib -shared  -L/usr/local/lib -shared
/usr/bin/ld: cannot open output file mex/mexnumtst.mex: No such file or directory
collect2: error: ld returned 1 exit status
( /bin/bash ../run-octave   --no-init-file --silent --no-history -p /scratch/buildbot/workers/jwe-debian-x86_64-5/gcc-debian/build/test/mex /scratch/buildbot/workers/jwe-debian-x86_64-5/gcc-debian/build/../src/test/fntests.m /scratch/buildbot/workers/jwe-debian-x86_64-5/gcc-debian/build/../src/test ) && if :; then echo ""; if [ -f fntests.log ]; then echo "Contents of test/fntests.log:"; echo ""; gawk -f ../../src/test/show-failures.awk fntests.log; else echo "fntests.log is missing!"; fi; fi
Integrated test scripts:
  liboctave/array/Array.cc-tst ................................... pass   21/21


Dmitri A. Sergatskov <dasergatskov>
Thu 03 Mar 2022 08:05:08 PM UTC, comment #8: 

Oops, somehow I messed up renaming test/module.mk to test/Makefile.am.  I pushed another change to fix that.

John W. Eaton <jwe>
Group administrator
Thu 03 Mar 2022 07:32:19 PM UTC, comment #7: 

Also all buyildbots are failing

Dmitri A. Sergatskov <dasergatskov>
Thu 03 Mar 2022 07:27:32 PM UTC, comment #6: 

Don't know if it is related, but I can't build right now: during configure, I get the following error


...
config.status: creating doc/doxyhtml/Doxyfile
config.status: creating libgnu/Makefile
config.status: error: cannot find input file: `test/Makefile.in'
 cd ../octaveclone && /bin/bash /media/pantxo/datadisk/dev/octaveclone/build-aux/missing automake-1.16 --gnu
configure.ac:3099: error: required file 'test/Makefile.in' not found
make: *** [Makefile:11507 : ../octaveclone/Makefile.in] Erreur 1


I ran bootstrap in the source directory and ran configure from a clean directory.

Pantxo Diribarne <pantxo>
Group Member
Thu 03 Mar 2022 06:41:43 PM UTC, comment #5: 

I pushed the following change:

http://hg.savannah.gnu.org/hgweb/octave/rev/103efb7e1600

It's not perfect, but it avoids the 'make dist' error for me.  I also see the bad side effect of having the summary lines from 'make check' followed by output from some other commands.  I think that's happening because the recursive Make invocation in the test directory happens before commands in the top-level Makefile are executed.  I'm not sure of an easy fix for that.

Marking as ready for test.

John W. Eaton <jwe>
Group administrator
Thu 03 Mar 2022 02:35:31 PM UTC, comment #4: 

I've made a quick attempt at implementing what I suggested in comment #3 and it seems possible without too much effort.  I'll try to check in a change later today.

John W. Eaton <jwe>
Group administrator
Thu 03 Mar 2022 01:59:36 PM UTC, comment #3: 

Without having to fix GNU Make or Automake, maybe we could use a sub-make for the test directory instead of using recursive rules for that entire directory tree in the main Makefile.am?  That will be easiest if it is possible for to have the structure


Makefile.am -- use recursive rules for all but test and gnulib dirs
test/Makefile.am -- use recursive rules for subdirs of test dir
gnulib/Makefile.am -- does not use recursive rules because gnulib doesn't support that properly yet


John W. Eaton <jwe>
Group administrator
Thu 03 Mar 2022 01:24:48 PM UTC, comment #2: 
John W. Eaton <jwe>
Group administrator
Thu 03 Mar 2022 08:38:20 AM UTC, comment #1: 

Oops, wrong file.
The command it is complaining about is probably this one here:

        @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
        topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
        list='$(DISTFILES)'; \
          dist_files=`for file in $$list; do echo $$file; done | \
          sed -e "s|^$$srcdirstrip/||;t" \
              -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \
        case $$dist_files in \
          */*) $(MKDIR_P) `echo "$$dist_files" | \
                           sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \
                           sort -u` ;; \
        esac; \
        for file in $$dist_files; do \
          if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \
          if test -d $$d/$$file; then \
            dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \
            if test -d "$(distdir)/$$file"; then \
              find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \
            fi; \
            if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \
              cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \
              find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \
            fi; \
            cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \
          else \
            test -f "$(distdir)/$$file" \
            || cp -p $$d/$$file "$(distdir)/$$file" \
            || exit 1; \
          fi; \
        done


That is pretty long and contains $dist_files twice. Probably that variable grew too long...

Markus Mützel <mmuetzel>
Group administrator
Thu 03 Mar 2022 08:16:12 AM UTC, original submission:  

It looks like we are hitting a bash limit when building the `dist` target on the default branch. E.g. on the buildbot:
http://buildbot.octave.org:8010/#/builders/21/builds/1281/steps/7/logs/stdio

if test -d "octave-8.0.0"; then find "octave-8.0.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "octave-8.0.0" || { sleep 5 && rm -rf "octave-8.0.0"; }; else :; fi
test -d "octave-8.0.0" || mkdir "octave-8.0.0"
make[3]: /bin/bash: Argument list too long
make[3]: *** [Makefile:28343: distdir-am] Error 127
make[3]: Leaving directory '/scratch/buildbot/workers/jwe-debian-x86_64-5/w64-32-on-debian/src/octave-hg-repo/.build'
make[2]: *** [Makefile:28338: distdir] Error 2
make[2]: Leaving directory '/scratch/buildbot/workers/jwe-debian-x86_64-5/w64-32-on-debian/src/octave-hg-repo/.build'
make[1]: *** [Makefile:28445: dist] Error 2
make[1]: Leaving directory '/scratch/buildbot/workers/jwe-debian-x86_64-5/w64-32-on-debian/src/octave-hg-repo/.build'
make: *** [/scratch/buildbot/workers/jwe-debian-x86_64-5/w64-32-on-debian/src/hg-octave-dist.mk:34: hg-octave-dist] Error 2
program finished with exit code 2


The corresponding lines in the Makefile read the following when I reproduce here:

        @(echo "$(distdir) archives ready for distribution: "; \
          list='$(DIST_ARCHIVES)'; for i in $$list; do echo $$i; done) | \
          sed -e 1h -e 1s/./=/g -e 1p -e 1x -e '$$p' -e '$$x'


Markus Mützel <mmuetzel>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by pantxo (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mmuetzel (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-03-04 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-03-03 jwe StatusNone Ready For Test

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code