bugGNU Wget - Bugs: bug #61038, Windows post fails to post files...


bug #61038: Windows post fails to post files over 65536 bytes,

Submitter:  None
Submitted:  Fri 13 Aug 2021 08:54:25 PM UTC
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Paul Breed Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  1.20
Operating System:  Microsoft Windows Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  None
* Mandatory Fields

Add a New Comment (Rich Markup)

Comment Type & Canned Response:


( Jump to the original submission )

Mon 31 Jan 2022 09:04:47 PM UTC, comment #8: 


I successfully reproduce the problem with wget 1.21.2 on Windows which is build with gcc/mingw 9.3

I wrote a simple server app that accepts post data and return HTTP 200 if expected file size equals to actual one and return 400 if somethings goes wrong.

I tested the app on two different Window 10 desktop PC and works without error even for 1GB file uploads.

Then I published the app to a GitHub and test the app on GitHub Actions and tests failed after 16KB. wget uploads 16KB successfully but failed after 20th try for 256KB file.

Please see the logs from this url: https://github.com/webfolderio/wget-file-upload-test/runs/5012162406?check_suite_focus=true

I the tested server app on a public VPS and executed the tests and faced with the same results similar to GitHub actions test.

Similar to GitHub actions, wget fails to upload 256KB file after lots of try.

I attached the Wireshark capture data for the VPS test.
Hope that it helps to find the bug.

reference wget executable for tests: https://github.com/webfolderio/wget-windows

test scenarios:
localhost (wget) - localhost (server) - DESKTOP - OK
localhost (wget) - localhost (server) - GitHub Actions - 256KB FAILED
localhost (wget) - (server)   - VPS Server - 256KB FAILED

(file #52783)

Ozhan Duz <ozhanduz>
Fri 21 Jan 2022 12:29:28 PM UTC, comment #7: 

On the first glimpse, https://github.com/rockdaboot/wget2/issues/256 is unrelated.

But then it seems that the poll/select function on Windows sometimes reports "ready to read" but the next read returns 0 bytes read. (Wget2 used non-blocking I/O).

Wget uses blocking I/O - but that doesn't matter. I assume that a read also sometimes reports 0 bytes.

Now this issue is about POST, so mainly write() functionality. I assume we might see the same or similar thing here as well, that write()/send() returns 0 and wget stops.

I never saw this behavior on native Linux nor with Wine emulation.
Also, testing on my wife's Win11 laptop never showed this behavior.

The aforementioned GH issue 256 contains two links to a statically wget2 binary. Please feel free to test with these binaries in order to reproduce the issue and report on that bug if possible.
More information may lead to a fix in gnulib which also affects wget1.x (just needs a recompilation then).

Tim Ruehsen <rockdaboot>
Project Administrator
Mon 06 Dec 2021 03:33:57 PM UTC, comment #6: 



I tested many binaries from there and all are broken, most notably 1.19.4 and 1.21.1 , and IIRC also some older 1.17.xx and 1.15.xx .

Wed 01 Dec 2021 10:01:38 PM UTC, comment #5: 

Unfortunately, I am unable to reproduce this on Linux.

This seems to be a Windows only issue. Could you please mention where you got your Windows binaries from? Maybe raise an issue with them about it as well.

Darshit Shah <darnir>
Project Administrator
Tue 05 Oct 2021 04:05:08 PM UTC, comment #4: 

1.21.2 is out ... and 2.0.0 out ... and reported to be broken. :-(

To   : -email is unavailable-
Subj : BUG in recent versions of WGET with POSTing more than 8 KiO data

> You are welcome to submit bug reports via the GNU Wget bug
> tracker (see <https://savannah.gnu.org/bugs/?func=additem&group=wget>)
> or to our mailing list <bug-wget@gnu.org>.
> Visit <https://lists.gnu.org/mailman/listinfo/bug-wget> to get
> more info (how to subscribe, list archives, ...).

There is (most likely) a severe bug in recent versions of WGET,
including 1.19.4 and 1.21.1, but not in good old 1.11.4 from
year 2008.

Expected behaviour: POSTing will work irrespective size of the postdata

Observed behaviour: POSTing fails with exit code 4 if size of the
                    postdata exceeds ca 8 KiO

With WGET 1.11.4 I could POST data irrespective size, only
percent-encoding tested at that time.

With CURL I can POST data irrespective size, both percent-encoded
and multipart-encoded.

With WGET 1.19.4 and 1.21.1 I can post data of limited size, both
percent-encoded and multipart-encoded.

With WGET 1.19.4 and 1.21.1 posting reliably fails if size of the
postdata exceeds ca 8 KiO.

I do the encoding myself and run WGET and CURL in a pretty raw way,
not caring about the fact that CURL can multipart-encode whereas
WGET can't.

Using the Win32 versions of WGET, Linux not installed.

Result from "--debug" success (exitcode is 0 and data arrives
at the server):

> ---request end---
> [writing BODY file YPOST.TMP ... done]
> HTTP request sent, awaiting response... seconds 60.00, Winsock error: 0
> seconds 60.00, Winsock error: 0
> seconds 60.00, Winsock error: 0

> ---response begin---
> HTTP/1.1 200 OK

Result from "--debug" failure (exitcode is 4 after 3 attempts
and data does not arrive at the server):

> ---request end---
> [writing BODY file YPOST.TMP ... Closed 4/SSL 0x009fb810
> Giving up.

> Saving cookies to YCOOK.TMP.
> Done saving cookies.

The report is not very verbose, "Closed" is the only hint about the
problem. The file YPOST.TMP contains the postdata and is good in
both cases, but bigger in the lower one.

Of course I can use CURL instead of WGET, but it's good to have
options, and bad to have bugs, given that this probably is a bug
of WGET. I am aware that CURL is more intended for posting than WGET,
but this used to work with old versions (1.11.4) before they got
unusable due to 3rd party hard cryptographic deprecation.

Tue 17 Aug 2021 12:56:45 PM UTC, comment #3: 
Sun 15 Aug 2021 01:07:05 PM UTC, comment #2: 

Since this is a Windows issue, could you please report this issue to where you got the binary from ?

Please feel free to update this issue with any information you get from there.

Tim Ruehsen <rockdaboot>
Project Administrator
Fri 13 Aug 2021 09:05:54 PM UTC, comment #1: 

Confirmed this bug does not exist on a mac with version 1.20 It is a windows only issue.

Fri 13 Aug 2021 08:54:25 PM UTC, original submission:  

If you post a file > 65K the system sends (Size % 65K) of the file then closes the socket.

Version 1.11 works correctly. I've tried 1.19,1.20 and 1.21 and all three fail.

I've attached  --debug logs of working 1.11 and failing 1.20

Exact command line:

wget --post-file=obj\Release\SimpleHtml.bin

I created a simple post target that doe snothing on the target.
And tried sending files of various sizes and any file < 16bits works and any file > 16 bits fails.

Have not yet tested oin Linux and Mac, suspect the issue is probably in the windows networking adapter layer.

Can also send pcaps of these failures if you want them.



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #52783:  wget-file-upload-test-capture.pcapng.gz added by ozhanduz (17KiB - application/x-gzip - wget windows file upload wireshark capture)
file #51773:  Works_Log_1P11.txt added by None (1001B - text/plain)
file #51774:  Fails_Log_1P20.txt added by None (3KiB - text/plain)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by ozhanduz (Updated the item)
  • -email is unavailable- added by darnir (Posted a comment)
  • -email is unavailable- added by rockdaboot (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-01-31 ozhanduz Attached File- Added wget-file-upload-test-capture.pcapng.gz, #52783
    2021-08-13 None Attached File- Added Works_Log_1P11.txt, #51773
        Attached File- Added Fails_Log_1P20.txt, #51774

    Back to the top

    Powered by Savane 3.9