bugGNU Octave - Bugs: bug #49126, build-env-features.cc missing all...

 
 

bug #49126: build-env-features.cc missing all defines (e.g. HAVE_CHOLMOD) due to dependency on gawk features

Submitted by:  Carlo de Falco <cdf>
Submitted on:  Mon 19 Sep 2016 10:39:56 AM UTC  
 
Category:  Configuration and Build System Severity:  5 - Blocker
Priority:  5 - Normal Item Group:  Installation Failure
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  Closed
Release:  4.2.0-rc1 Operating System:  Mac OS

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 20 Sep 2016 09:08:50 AM UTC, comment #16: 

Re: comment #14

jwe, I have installed GNU awk into /usr/local/bin
after discovering this issue that's why you were
not able to reproduce it, previously there was only the
one in /usr/bin

c.

Carlo de Falco <cdf>
Project Member
Mon 19 Sep 2016 11:45:21 PM UTC, comment #15: 

Done

http://hg.savannah.gnu.org/hgweb/octave/rev/b3268dbf9781

So the build should no longer need gawk (but will use it over other awks if it is found in PATH).

The sed vs gsed issue I mentioned on the mailing list remains.

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 11:40:22 PM UTC, comment #14: 

It seems to fix the problem for me on the mac mini system that we are using for buildbot.  I was initially confused because I wasn't seeing a problem even before applying the patch, but that was apparently because my PATH had /usr/local/bin first so I was getting awk == gawk.  I was able to duplicate the problem by putting /usr/bin first, and then verify that the patch fixes the problem.

So, yes, please push this change.

Thanks.

John W. Eaton <jwe>
Project Administrator
Mon 19 Sep 2016 11:11:24 PM UTC, comment #13: 

I think it's a safe change, but I would like some confirmation from Carlo or another macOS user first that this change actually fixes the problem, since I don't have the exact same awk on hand. There are so many possible variations that it could be this change or there could be something else going on.

I think I remember right that Debian used to carry a "nawk" program, which I think was the same as the BSD awk that is still used on all *BSD and macOS systems, but I can't find it in the Debian repositories anymore, so I'd like independent confirmation on macOS.

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 11:05:00 PM UTC, comment #12: 

I already wrote to the maintainers-list my preference for sed/awk/perl etc.

But I certainly think you can safely make the change proposed in comment #11.  As written, the regular expression is hard to understand by humans, let alone awk.  Your patch makes it clearer what is supposed to be happening.

Rik <rik5>
Project Administrator
Mon 19 Sep 2016 08:39:12 PM UTC, comment #11: 

As far as I can tell the only GNU-specific feature in this script is the regular expression syntax

/(OCTAVE_|)HAVE_/

This is not an error with gawk --posix, but is with mawk on my system.

Can you test whether the attached patch fixes this when building without gawk installed but with gsed installed (the script uses both)?

This is a pretty trivial change, if that's all there is to it (of course there are sed issues discussed elsewhere). The biggest problem is that it's not easy to catch these kinds of things, and they tend to creep in again because most of us just use the GNU tools and have no easy way to test with all possible variants.

(file #38556)

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 03:44:27 PM UTC, comment #10: 

Yes, I definitely get build failures with the --posix options on both, so this is reproducible on any system. I think that it should be possible to make the scripts portable, but I'm not sure about pattern alternation (a\|b) as I mentioned on the mailing list. I can certainly help work on this to some degree.

As far as the scripts that use sed instead of $SED, I think the intent is that they are portable so it doesn't matter what $SED is. I think that's the approach that both gnulib and automake try to take, to only write sed commands that will work with any sed so it doesn't matter as long as some sed can be called.

As I said I lean towards portability, but other of our co-maintainers may say it's not worth the effort, what's the harm in requiring GNU awk and sed?

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 03:31:06 PM UTC, comment #9: 

Mike,

That's a good point.

We should also consider that many third party scripts
which are part of the build system (e.g. from gnulib and automake) use sed, so we acnnot give up sed alltogether ...

Also note that not all those third party scripts allow
choosing gsed instead of sed, for example

  build-aux/ylwrap

uses just 'sed' instead of $SED ...

c.

Carlo de Falco <cdf>
Project Member
Mon 19 Sep 2016 03:27:51 PM UTC, comment #8: 

Thank you (smh), now testing with AWK="gawk --posix" SED="gsed --posix".

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 03:23:37 PM UTC, comment #7: 

Re: needing both sed and awk, I haven't look too closely at it, but they definitely do have different features. I think simple text replacement is much more easily expressed with sed, and awk is much better at text processing that requires tracking state or printing large blocks of text in addition to text substitution or generating new text.

In short, I think everything could be done with awk, but it may make the things that are currently done with sed a little more awkward. Just my impression from years of using both tools and knowing their strengths and weaknesses, I would use each in different contexts.

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 03:22:29 PM UTC, comment #6: 

if you want to test portability of SED scripts
you could invoke gsed with the '--posix' option

Carlo de Falco <cdf>
Project Member
Mon 19 Sep 2016 03:18:03 PM UTC, comment #5: 

I get the same file if I configure with AWK=mawk on Debian.

I think I lean towards portability, jwe and others can weight in on what they think. Now that I have an awk that I can test with that makes things easier. I don't have a sed program that I can test with to make sure our usage of sed is portable.

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 03:10:31 PM UTC, comment #4: 

Indeed it seems to be an issue related
to awk/gawk the attached libinterp/build-env-features.cc
produced after setting AWK=gawk looks much better.

So again, do we really need both tools and do we really
need GNU version of both tools?

Thanks,
c.

(file #38553)

Carlo de Falco <cdf>
Project Member
Mon 19 Sep 2016 03:08:13 PM UTC, comment #3: 

Thanks, that confirms that macOS awk cannot produce the right output file, the way the script is written now (see libinterp/build-env-features.sh).

See also my post to the list about sed vs gsed (http://lists.gnu.org/archive/html/octave-maintainers/2016-09/msg00193.html), same applies for awk vs gawk, we need to decide whether we value portability of these scripts to any version of sed/awk, or features for easier maintenance and testing. I can't say, I don't write most of these scripts.

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 03:01:07 PM UTC, comment #2: 

Hi Mike,

Here you are, I'll try to rebuild using

  AWK=gawk

and report back.
But do we really need to use both awk AND sed?

c.


$ cat libinterp/build-env-features.cc
// DO NOT EDIT!  Generated automatically from config.h by Make."

#if defined (HAVE_CONFIG_H)
#  include "config.h"
#endif

#include "build-env.h"
#include "ov.h"

namespace octave
{
  namespace build_env
  {
    octave_scalar_map
    features (void)
    {
      static bool initialized = false;

      static octave_scalar_map m;

      if (! initialized)
        {
          static octave_value ov_true = (true);
          static octave_value ov_false = (false);


          initialized = true;
        }

      return m;
    }
  };
};
mathicsepc32:octave_default_build carlo$


Carlo de Falco <cdf>
Project Member
Mon 19 Sep 2016 02:50:27 PM UTC, comment #1: 

Good testing, thank you. I see that you passed SED=gsed, but not AWK=gawk. I also see that configure tests for gawk before awk, and didn't find it.

The script that puts the "features" into the Octave executable is a awk/gawk script, and maybe the awk features that we are using are not portable. Can you also attach or provide a link to the file libinterp/build-env-features.cc?

Mike Miller <mtmiller>
Project Administrator
Mon 19 Sep 2016 10:39:56 AM UTC, original submission:  

Octave 4.2 built on OSX El Capitan with the following options:

brew_prefix=/usr/local
../octave/configure \
    SED=gsed \
    CC=clang CXX=clang++ \
    F77=gfortran \
    LDFLAGS="-L${brew_prefix}/lib" \
    --prefix=/opt/octave/4.2.0 \
    --enable-dependency-tracking \
    --enable-link-all-dependencies \
    --with-x=no \
    --without-OSMesa \
    --disable-java \
    --with-blas="openblas"

Here [1] and [2] you can find the files config.log and fntests.log.

configure does not complain about
suitesparse components (e.g. cholmod or amd) not being found, and even
reports the flags used to link to them, yet the tests for Octave functions
depending on those libraries are skipped by 'make check'.

[1] http://www1.mate.polimi.it/~carlo/config.log
[2] http://www1.mate.polimi.it/~carlo/fntests.log

Carlo de Falco <cdf>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #38556:  awk.diff added by mtmiller (678B - text/x-diff)
file #38553:  build-env-features.cc added by cdf (37KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by cdf (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 11 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-09-19 mtmiller StatusConfirmed => Fixed
        Open/ClosedOpen => Closed
    2016-09-19 mtmiller Attached File- => Added awk.diff, #38556
    2016-09-19 mtmiller Summarybuild-env-featuers.cc missing all defines (e.g. HAVE_CHOLMOD) due to dependency on gawk features => build-env-features.cc missing all defines (e.g. HAVE_CHOLMOD) due to dependency on gawk features
    2016-09-19 mtmiller StatusNone => Confirmed
        Summaryconfigure finds suitesparse components but flags HAVE_CHOLMOD, HAVE_AMD, etc. are => build-env-featuers.cc missing all defines (e.g. HAVE_CHOLMOD) due to dependency on gawk features
    2016-09-19 cdf Attached File- => Added build-env-features.cc, #38553
        StatusConfirmed => None
        Summaryconfigure finds suitesparse components but build-env-features HAVE_CHOLMOD, HAVE_AMD, etc. wrong due to awk vs gawk => configure finds suitesparse components but flags HAVE_CHOLMOD, HAVE_AMD, etc. are
    2016-09-19 mtmiller StatusNone => Confirmed
        Summaryconfigure finds suitesparse components but flags HAVE_CHOLMOD, HAVE_AMD, etc. are => configure finds suitesparse components but build-env-features HAVE_CHOLMOD, HAVE_AMD, etc. wrong due to awk vs gawk

    Back to the top


    Powered by Savane 3.5