bugGNU Wget - Bugs: bug #48232, Sometimes wget restarts download...

 
 

bug #48232: Sometimes wget restarts download from the beginning, even if the server supports resumed downloads

Submitted by:  Evgeny Kapun <abacabadabacaba>
Submitted on:  Wed 15 Jun 2016 03:06:05 PM UTC  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: 1.18Operating System: GNU/Linux
Reproducibility: Every TimeFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 16 Jun 2016 02:38:59 PM UTC, comment #1:

See the attached patch. It fixes the issue on my side.

I'm specially afraid of having introduced an off-by-one error, so please remind me to write a test case for this next week. Just no time now.

(file #37503)

Ander Juaristi <ajuaristi>
Wed 15 Jun 2016 03:06:05 PM UTC, original submission:

If the connection is interrupted during download, wget normally tries to continue the download from the same place where it left off. This only works if the server supports resumed downloads, otherwise, download restarts from the beginning. However, sometimes wget would restart the download even when the server does support resumption. Testing shows that this happens if a network error occurs before wget receives HTTP response from the server, a situation which is quite common on poor networks.

For testing, I created a web server which would behave as follows:

  • On the first request, it will send a response with `Content-Length: 1000` and 500 bytes of data, then wait.
  • On all other requests, it will just wait without sending any data.

Using wget to download from such server produces this:

As you may see, only the second request includes `Range` header. Starting from the third request, `Range` header is not included, so the download would not be resumed at this point. In practice, this means that a big download would suddenly restart because the network was down for some time, which is undesirable.

I attached a test program to reproduce the issue. It listens on [::1]:8888 and acts as a web server. You need to restart it before every test.

Related bugs:

  • #31653: a fix for that bug is probably what introduced this bug. Read the discussion there.
  • #48123: a bug similar to this one, but there it is not clear that the server supports resumed downloads.
Evgeny Kapun <abacabadabacaba>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #37488:  wget-test added by abacabadabacaba (597B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by ajuaristi (Updated the item)
  • -unavailable- added by abacabadabacaba (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 16 Jun 2016 02:38:59 PM UTCajuaristiAttached File-=>Added 0001-Do-not-overwrite-restval-if-len-is-smaller.patch, #37503
    Wed 15 Jun 2016 03:06:05 PM UTCabacabadabacabaAttached File-=>Added wget-test, #37488

    Back to the top


    Powered by Savane 3.1-cleanup1