bugGNU Wget - Bugs: bug #61229, "unable to resolve host...


bug #61229: "unable to resolve host address" for URI's when from a long list

Submitter:  None
Submitted:  Sun 26 Sep 2021 10:48:58 PM UTC
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  None
Operating System:  Mac OS Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup

Sun 26 Sep 2021 10:48:58 PM UTC, original submission:  


When I run wget with a long list of URIs from a file, after a couple of hundered entries the URI's stop to resolve (the "wget: unable to resolve host address <...>" lines in the log), although they still do if supplied in shorter lists. It's not an issue of the timeout (it is set in the example to 10 seconds just not to wait 15 minutes for each single URI), nor the URI's themselves, which are valid.

wget \
--debug \
--spider \
--no-directories \
--timeout=10 \
--tries=2 \
--input-file="/Users/comp/wget/url-list.txt" \

The input and output files are attached.

I'm on MacOS 11.5.2, M1

wget version:
GNU Wget 1.21.2 built on darwin20.6.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
+ntlm +opie +psl +ssl/openssl

    /opt/homebrew/etc/wgetrc (system)
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/opt/homebrew/etc/wgetrc"
    -DLOCALEDIR="/opt/homebrew/Cellar/wget/1.21.2/share/locale" -I.
    -I../lib -I../lib -I/opt/homebrew/opt/openssl@1.1/include
    -I/opt/homebrew/Cellar/libpsl/0.21.1_3/include -DNDEBUG -g -O2
    clang -I/opt/homebrew/Cellar/pcre/8.45/include
    -I/opt/homebrew/Cellar/libpsl/0.21.1_3/include -DNDEBUG -g -O2
    -L/opt/homebrew/Cellar/pcre/8.45/lib -lpcre -lidn2
    -L/opt/homebrew/opt/openssl@1.1/lib -lssl -lcrypto -ldl
    -L/opt/homebrew/Cellar/zlib/1.2.11/lib -lz
    -L/opt/homebrew/Cellar/libpsl/0.21.1_3/lib -lpsl ftp-opie.o
    openssl.o http-ntlm.o ../lib/libgnu.a -liconv -lintl -Wl,-framework
    -Wl,CoreFoundation -lunistring



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #51974:  test-output.txt added by None (384KiB - text/plain)
file #51975:  url-list.txt added by None (13KiB - text/plain)


Depends on the following items: None found

Items that depend on this one: None found


CC list is empty


Follow 2 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2021-09-26 None Attached File- Added test-output.txt, #51974
    Attached File- Added url-list.txt, #51975

Back to the top

Powered by Savane 3.13-c54d.
Corresponding source code