Tue 07 Feb 2017 02:04:45 PM UTC, comment #21:
I going to make a 1.19.1 release in a few days... still waiting for more reports ;-)
|
Tue 07 Feb 2017 12:34:03 AM UTC, comment #20:
Mind publishing a wget 1.19.1 or 1.20 release tarball?
|
Mon 06 Feb 2017 10:02:55 PM UTC, comment #19:
As you say, Test-iri-disabled and 'invalid multibyte sequence' do not belong here.
I pushed the changes in configure.ac and tests/WgetTests.pm.
Thanks everybody for info & helping.
|
Sun 05 Feb 2017 11:43:32 PM UTC, comment #18:
I just ran make check with Wget 1.18 and a whopping 9 tests failed on macOS 10.12.3, including Test-iri-disabled, so this test failure isn't a regression and doesn't really belong to this thread.
|
Sun 05 Feb 2017 11:30:00 PM UTC, comment #17:
I also added another log where wget is run with --debug, in case it is more useful. Also, I've seen this
error message with IRIs with older versions of Wget before (compiled with IRI support).
|
Sun 05 Feb 2017 10:28:15 PM UTC, comment #16:
Thanks, removing the test "X$iri" = "Xno" branch does fix the problem.
Now that compilation succeeds, I ran make check to make sure everything's working, and insterestingly, whether I configure with --disable-iri or not, there's one single test failure: Test-iri-disabled.px. I'm attaching the log for this test. Looks like wget insists on percent encoding the filenames, despite the "blah blah saved" showing the raw unencoded filenames.
By the way, I also noticed a small problem in tests/WgetTests.pm's error message. Here's a patch to fix it
As you can see, without the patch (which puts parentheses around the join), @unexpected_downloads is treated in a scalar context and join ', ' does nothing, resulting in messages like
instead of the presumably intended
|
Sun 05 Feb 2017 07:06:24 PM UTC, comment #15:
I guess I found the prolem in configure.ac:
if test "X$iri" = "Xno"; then
# we don't need libunistring - clear settings from gnulib module
LIBUNISTRING=""
LTLIBUNISTRING=""
fi
That is old code and it hits you when you don't build with IRI support (your gist shows IRI: no).
Please test again without that code (remove it from configure.ac and autoreconf && ./configure && make).
If that works for you, I'll make a commit without it.
|
Sun 05 Feb 2017 06:41:56 PM UTC, comment #14:
$ ls -la m4/libunistring*.m4
-rw-r--r-- 1 tim tim 6320 Feb 4 15:22 m4/libunistring-base.m4
-rw-r--r-- 1 tim tim 6348 Feb 4 15:22 m4/libunistring.m4
-rw-r--r-- 1 tim tim 961 Feb 4 15:22 m4/libunistring-optional.m4
What you could do to debug the emptiness of (LT)LIBUNISTRING:
Edit m4/libunistring.m4, put some 'echo' lines printing $LIBUNISTRING. The *.m4 files are translated into shell code (the ./configure script). You will see your echoes in the output of ./configure. That is what I would do (and did a few times when developing m4 macros).
There are not so many places where (LT)LIBUNISTRING is set:
```
m4/libunistring.m4: HAVE_LIBUNISTRING=no
m4/libunistring.m4: LIBUNISTRING=
m4/libunistring.m4: LTLIBUNISTRING=
m4/libunistring.m4: LIBUNISTRING=
m4/libunistring.m4: LTLIBUNISTRING=
m4/libunistring.m4: LIBUNISTRING="$LIBUNISTRING $LIBICONV"
m4/libunistring.m4: LTLIBUNISTRING="$LTLIBUNISTRING $LTLIBICONV"
```
|
Sun 05 Feb 2017 12:35:14 AM UTC, comment #13:
I expanded the set of modules to all of those included in bootstrap.conf, but couldn't reproduce the empty LIBUNISTRING and LTLIBUNISTRING. In case someone wants to take a look, here are the files for my dummy project: https://gist.github.com/642a4acc94da8f487d7e3cacd2b12f98.
Wget's current gnulib submodule is 16 commits behind master; fast forwarding to master doesn't seem to change anything.
|
Sat 04 Feb 2017 09:43:08 PM UTC, comment #12:
Thanks for putting so work into this issue !
It might be a gnulib issue, but we need a work-around anyways.
Could you run your dummy project with all the gnulib modules from wget (bootstrap.conf) ? Just to see if the problem occurs or not.
If the problem occurs, maybe you could track it down to a few modules (libunistring-optional and unistr/u8-strlen, unicase/u8-tolower might interfere in some ways).
Too bad that I can't reproduce it...
BTW, in the wget directory, you could test with the latest gnulib: 'cd gnulib; git checkout master; git pull; cd ..; git commit -m "gnulib: update" gnulib; ./bootstrap; ./configure; make clean; make'
|
Sat 04 Feb 2017 08:45:17 PM UTC, comment #11:
The commit to add wcwidth to bootstrap.conf did not help me, but upon further investigation, I realized that it was because I have libunistring installed (which, if linked, provides uc_width); if I move libunistring out of the way, I have no problem with compilation. Therefore, this may not be a problem for most users.
That said, the libunistring problem is a weird one. configure does check for libunistring:
but it doesn't set the proper ld flags:
(the above should instead be:
)
Interestingly, I created a dummy project where I ran
then with boilerplate configure.ac and automake.am, the generated configure would set LIBUNISTRING and LTLIBUNISTRING to the correct values. Not sure where the problem is.
|
Sat 04 Feb 2017 06:34:17 PM UTC, comment #10:
Ah, OK.
'wcwidth' gnulib module was missing in bootstrap.conf.
Please try again the latest git master.
|
Sat 04 Feb 2017 05:43:13 PM UTC, comment #9:
Issues other than uc_width have been fixed by master.
Sorry for not being specific enough about uc_width. It is used by rpl_wcwidth:
wcwidth replacement is used because
Logs can be found at https://gist.github.com/51e9245975f3081bba345184c5be6b08.
|
Sat 04 Feb 2017 05:13:01 PM UTC, comment #8:
Could anyone test latest 'master' from git repo ? It needs a ./bootstrap && ./configure && make clean && make && make check
I tried to fix all three issues mentioned here, but I am unsure about uc_width (likely used by u8_strlen()).
@zmwang There is the gnu-info mailing list for announcements. But only non-alpha release should be announced there. Maybe there is a way to change the policy...
|
Fri 03 Feb 2017 11:42:49 PM UTC, comment #7:
By the way, may I suggest that an announcement mailing list (say, wget-announce), be created to notify people about releases, including alphas? I assume many of us would be willing to test alpha releases but may not want to subscribe to bug-wget.
(I just put together a cron job to monitor https://alpha.gnu.org/gnu/wget/ so I will be notified of future alphas, but subscribing to an announcement list would be so much easier.)
|
Fri 03 Feb 2017 11:00:44 PM UTC, comment #6:
Including unistd.h before wget.h (provided that "config.h" is included before <unistd.h>) does fix the issue. However, I uncovered two more bugs while testing this fix.
First, in src/xattr.h, the fsetxattr macro is declared with a space between the macro name and the argument list parenthesis, which is apparently wrong. Trivial to fix.
Secondly, for some reason, lib/uniwidth/width.c isn't added to the source list, leading to undefined symbol error for uc_width. Since autotools make my head spin... I just uncommented the "if LIBUNISTRING_COMPILE_UNIWIDTH_WIDTH" conditional guarding "libgnu_a_SOURCES += uniwidth/width.c" in lib/Makefile.am, and the build proceeded to a success. Apparently this is not a proper fix, but at least it tells us how many bugs there are in total...
I have attached my "sketchy patch", if you just want to get your hands on a copy of wget 1.19 on macOS as soon as possible and don't care for proper fixes.
|
Fri 03 Feb 2017 10:54:42 PM UTC, comment #5:
And adding
first before
leads back to the originally reported error.
|
Fri 03 Feb 2017 10:47:11 PM UTC, comment #4:
Making the suggested change leads to:
|
Fri 03 Feb 2017 09:48:12 PM UTC, comment #3:
Thanks for the additional info.
It looks like unlink() doesn't work as expected on MacOSX, so that gnulib add wrapper code, doing it in <unistd.h> with
#define unlink rpl_unlink
Now, in ftp.c we include "wget.h" before <unistd.h>.
"wget.h" includes "options.h" which declares 'struct options' with a member called 'unlink'. Thereafter <unistd.h> makes the above define and thus all references to member 'unlink' are rewritten to 'rpl_unlink' by the preprocessor => syntax error.
Could anyone test to include unistd.h before wget.h in ftp.c ?
Maybe other sources are affected similarly... but it would be good to know if ftp.c compiles with such a change.
I guess the best fix would be to rename member 'unlink' to somethink else, e.g. 'do_unlink'.
|
Fri 03 Feb 2017 09:30:56 PM UTC, comment #2:
I have attached config.log. Other log files can be found in https://gist.github.com/14bb04872e454f7769c661c4e0ff680f.
The problem is not limited to macOS 10.12.3. We see the same error on 10.10 and 10.11.
Note that macOS does have unlink(2), declared in unistd.h. The difference between wget 1.18 and wget 1.19 is that configure performs additional checks on unlink and unlinkat which fail:
|
Fri 03 Feb 2017 04:40:48 PM UTC, comment #1:
That's why we had an alpha release... so everyone could test and report before the release... ;-)
We could easily solve that problem by renaming the field...
But is there really no unlink() function available on MacOS 10.12.3 ? If there is, the gnulib detection of unlink() doesn't seem to work correctly and we have to talk with the gnulib people.
Could you upload config.log here ?
|
Fri 03 Feb 2017 04:14:45 PM UTC, original submission:
I never had any problems with wget 1.18, but trying to build 1.19 with the same ./configure flags as before will fail to build. My flags are:
--with-ssl=openssl
--with-libssl-prefix=/usr/local/ssl
--diasble-pcre
--disable-iri
The `make` step fails with this:
------------------8<------------------------------------
/Applications/Xcode.app/Contents/Developer/usr/bin/make all-am
CC connect.o
CC convert.o
CC cookies.o
CC ftp.o
ftp.c:1466:19: error: no member named 'rpl_unlink' in 'struct options'
if (opt.unlink && file_exists_p (con->target))
~~~ ^
../lib/unistd.h:1851:19: note: expanded from macro 'unlink'
# define unlink rpl_unlink
^
1 error generated.
make[3]: *** [ftp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
----------------------------8<-----------------------------
|