bugGNU Wget - Bugs: bug #61229, "unable to resolve host...

 
 

bug #61229: "unable to resolve host address" for URI's when from a long list

Submitter:  None
Submitted:  Sun 26 Sep 2021 10:48:58 PM UTC
   
 
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  None
Operating System:  Mac OS Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 26 Sep 2021 10:48:58 PM UTC, original submission:  

Hello,

When I run wget with a long list of URIs from a file, after a couple of hundered entries the URI's stop to resolve (the "wget: unable to resolve host address <...>" lines in the log), although they still do if supplied in shorter lists. It's not an issue of the timeout (it is set in the example to 10 seconds just not to wait 15 minutes for each single URI), nor the URI's themselves, which are valid.


wget \
--debug \
--spider \
--no-directories \
--timeout=10 \
--tries=2 \
--input-file="/Users/comp/wget/url-list.txt" \
--output-file="/Users/comp/wget/test-output.txt"


The input and output files are attached.

I'm on MacOS 11.5.2, M1

wget version:
GNU Wget 1.21.2 built on darwin20.6.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
+ntlm +opie +psl +ssl/openssl

Wgetrc:
    /opt/homebrew/etc/wgetrc (system)
Locale:
    /opt/homebrew/Cellar/wget/1.21.2/share/locale
Compile:
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/opt/homebrew/etc/wgetrc"
    -DLOCALEDIR="/opt/homebrew/Cellar/wget/1.21.2/share/locale" -I.
    -I../lib -I../lib -I/opt/homebrew/opt/openssl@1.1/include
    -I/opt/homebrew/Cellar/pcre/8.45/include
    -I/opt/homebrew/Cellar/zlib/1.2.11/include
    -I/opt/homebrew/Cellar/icu4c/69.1/include
    -I/opt/homebrew/Cellar/libpsl/0.21.1_3/include -DNDEBUG -g -O2
Link:
    clang -I/opt/homebrew/Cellar/pcre/8.45/include
    -I/opt/homebrew/Cellar/zlib/1.2.11/include
    -I/opt/homebrew/Cellar/icu4c/69.1/include
    -I/opt/homebrew/Cellar/libpsl/0.21.1_3/include -DNDEBUG -g -O2
    -L/opt/homebrew/Cellar/pcre/8.45/lib -lpcre -lidn2
    -L/opt/homebrew/opt/openssl@1.1/lib -lssl -lcrypto -ldl
    -L/opt/homebrew/Cellar/zlib/1.2.11/lib -lz
    -L/opt/homebrew/Cellar/libpsl/0.21.1_3/lib -lpsl ftp-opie.o
    openssl.o http-ntlm.o ../lib/libgnu.a -liconv -lintl -Wl,-framework
    -Wl,CoreFoundation -lunistring

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #51974:  test-output.txt added by None (384KiB - text/plain)
file #51975:  url-list.txt added by None (13KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

Follow 2 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2021-09-26 None Attached File- Added test-output.txt, #51974
    Attached File- Added url-list.txt, #51975

Back to the top

Powered by Savane 3.14-f13d.
Corresponding source code