Tue 20 Sep 2016 09:08:50 AM UTC, comment #16:
Re: comment #14
jwe, I have installed GNU awk into /usr/local/bin
after discovering this issue that's why you were
not able to reproduce it, previously there was only the
one in /usr/bin
c.
|
Mon 19 Sep 2016 11:45:21 PM UTC, comment #15:
Done
http://hg.savannah.gnu.org/hgweb/octave/rev/b3268dbf9781
So the build should no longer need gawk (but will use it over other awks if it is found in PATH).
The sed vs gsed issue I mentioned on the mailing list remains.
|
Mon 19 Sep 2016 11:40:22 PM UTC, comment #14:
It seems to fix the problem for me on the mac mini system that we are using for buildbot. I was initially confused because I wasn't seeing a problem even before applying the patch, but that was apparently because my PATH had /usr/local/bin first so I was getting awk == gawk. I was able to duplicate the problem by putting /usr/bin first, and then verify that the patch fixes the problem.
So, yes, please push this change.
Thanks.
|
Mon 19 Sep 2016 11:11:24 PM UTC, comment #13:
I think it's a safe change, but I would like some confirmation from Carlo or another macOS user first that this change actually fixes the problem, since I don't have the exact same awk on hand. There are so many possible variations that it could be this change or there could be something else going on.
I think I remember right that Debian used to carry a "nawk" program, which I think was the same as the BSD awk that is still used on all *BSD and macOS systems, but I can't find it in the Debian repositories anymore, so I'd like independent confirmation on macOS.
|
Mon 19 Sep 2016 11:05:00 PM UTC, comment #12:
I already wrote to the maintainers-list my preference for sed/awk/perl etc.
But I certainly think you can safely make the change proposed in comment #11. As written, the regular expression is hard to understand by humans, let alone awk. Your patch makes it clearer what is supposed to be happening.
|
Mon 19 Sep 2016 08:39:12 PM UTC, comment #11:
As far as I can tell the only GNU-specific feature in this script is the regular expression syntax
This is not an error with gawk --posix, but is with mawk on my system.
Can you test whether the attached patch fixes this when building without gawk installed but with gsed installed (the script uses both)?
This is a pretty trivial change, if that's all there is to it (of course there are sed issues discussed elsewhere). The biggest problem is that it's not easy to catch these kinds of things, and they tend to creep in again because most of us just use the GNU tools and have no easy way to test with all possible variants.
(file #38556)
|
Mon 19 Sep 2016 03:44:27 PM UTC, comment #10:
Yes, I definitely get build failures with the --posix options on both, so this is reproducible on any system. I think that it should be possible to make the scripts portable, but I'm not sure about pattern alternation (a\|b) as I mentioned on the mailing list. I can certainly help work on this to some degree.
As far as the scripts that use sed instead of $SED, I think the intent is that they are portable so it doesn't matter what $SED is. I think that's the approach that both gnulib and automake try to take, to only write sed commands that will work with any sed so it doesn't matter as long as some sed can be called.
As I said I lean towards portability, but other of our co-maintainers may say it's not worth the effort, what's the harm in requiring GNU awk and sed?
|
Mon 19 Sep 2016 03:31:06 PM UTC, comment #9:
Mike,
That's a good point.
We should also consider that many third party scripts
which are part of the build system (e.g. from gnulib and automake) use sed, so we acnnot give up sed alltogether ...
Also note that not all those third party scripts allow
choosing gsed instead of sed, for example
build-aux/ylwrap
uses just 'sed' instead of $SED ...
c.
|
Mon 19 Sep 2016 03:27:51 PM UTC, comment #8:
Thank you (smh), now testing with AWK="gawk --posix" SED="gsed --posix".
|
Mon 19 Sep 2016 03:23:37 PM UTC, comment #7:
Re: needing both sed and awk, I haven't look too closely at it, but they definitely do have different features. I think simple text replacement is much more easily expressed with sed, and awk is much better at text processing that requires tracking state or printing large blocks of text in addition to text substitution or generating new text.
In short, I think everything could be done with awk, but it may make the things that are currently done with sed a little more awkward. Just my impression from years of using both tools and knowing their strengths and weaknesses, I would use each in different contexts.
|
Mon 19 Sep 2016 03:22:29 PM UTC, comment #6:
if you want to test portability of SED scripts
you could invoke gsed with the '--posix' option
|
Mon 19 Sep 2016 03:18:03 PM UTC, comment #5:
I get the same file if I configure with AWK=mawk on Debian.
I think I lean towards portability, jwe and others can weight in on what they think. Now that I have an awk that I can test with that makes things easier. I don't have a sed program that I can test with to make sure our usage of sed is portable.
|
Mon 19 Sep 2016 03:10:31 PM UTC, comment #4:
Indeed it seems to be an issue related
to awk/gawk the attached libinterp/build-env-features.cc
produced after setting AWK=gawk looks much better.
So again, do we really need both tools and do we really
need GNU version of both tools?
Thanks,
c.
(file #38553)
|
Mon 19 Sep 2016 03:08:13 PM UTC, comment #3:
Thanks, that confirms that macOS awk cannot produce the right output file, the way the script is written now (see libinterp/build-env-features.sh).
See also my post to the list about sed vs gsed (http://lists.gnu.org/archive/html/octave-maintainers/2016-09/msg00193.html), same applies for awk vs gawk, we need to decide whether we value portability of these scripts to any version of sed/awk, or features for easier maintenance and testing. I can't say, I don't write most of these scripts.
|
Mon 19 Sep 2016 03:01:07 PM UTC, comment #2:
Hi Mike,
Here you are, I'll try to rebuild using
AWK=gawk
and report back.
But do we really need to use both awk AND sed?
c.
|
Mon 19 Sep 2016 02:50:27 PM UTC, comment #1:
Good testing, thank you. I see that you passed SED=gsed, but not AWK=gawk. I also see that configure tests for gawk before awk, and didn't find it.
The script that puts the "features" into the Octave executable is a awk/gawk script, and maybe the awk features that we are using are not portable. Can you also attach or provide a link to the file libinterp/build-env-features.cc?
|
Mon 19 Sep 2016 10:39:56 AM UTC, original submission:
Octave 4.2 built on OSX El Capitan with the following options:
brew_prefix=/usr/local
../octave/configure \
SED=gsed \
CC=clang CXX=clang++ \
F77=gfortran \
LDFLAGS="-L${brew_prefix}/lib" \
--prefix=/opt/octave/4.2.0 \
--enable-dependency-tracking \
--enable-link-all-dependencies \
--with-x=no \
--without-OSMesa \
--disable-java \
--with-blas="openblas"
Here [1] and [2] you can find the files config.log and fntests.log.
configure does not complain about
suitesparse components (e.g. cholmod or amd) not being found, and even
reports the flags used to link to them, yet the tests for Octave functions
depending on those libraries are skipped by 'make check'.
[1] http://www1.mate.polimi.it/~carlo/config.log
[2] http://www1.mate.polimi.it/~carlo/fntests.log
|