bugGNU Wget - Bugs: bug #24642, wget 1.11.4 redownloads entire...

 
 

bug #24642: wget 1.11.4 redownloads entire file after completion (when transfer is interrupted)

Submitter:  None
Submitted:  Fri 24 Oct 2008 10:20:36 AM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  Duplicate
Privacy:  Public Assigned to:  None
Originator Name:  Aleph0 Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  trunk
Operating System:  Microsoft Windows Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  Yes Work Required:  None
Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 27 Oct 2008 07:39:14 PM UTC, comment #1: 

Actually, the server indicated conflicting content lengths after the reconnect:

...
Content-Length: 6963752
Content-Range: 4299196-11262948/11262948
...

While the content-length header correctly indicates that it is going to send 6963752 bytes, the content-range actually indicates that it is going to send 6963753 bytes (because byte 4299196 must also be counted)). It is nonsensically claiming that it is going to send byte 11262948, which is the 11262949th byte out of 11262948 total bytes. Wget 1.11.x now uses the content-range to determine length when appropriate, as some servers have also been known to issue incorrect content-lengths, or sometimes doesn't specify it at all, when serving partial content).

So that's the server's issue, not Wget's, and you should contact rapidshare.com about that (I will try to contact them as well).

However, Wget's behavior when receiving a 200 response with a content-length that matches the local file's size is currently to download the full content, where it should probably just close the connection. This behavior is also present in 1.10, was reported recently to the mailing list, and appears to be a regression over 1.9.1 or thereabouts. It is being tracked at https://savannah.gnu.org/bugs/index.php?24662

Micah Cowan <micahcowan>
Fri 24 Oct 2008 10:20:36 AM UTC, original submission:  

Tested on Windows XP 32-bit Italian and English language. wget version 1.11.4 w/SSL support.

When the connection drops during a transfer, wget requests the remaining range and completes the download, but when the server closes the connection at the end of the transfer wget interprets it as another drop.

Thus it requests a "Range: [Last_byte]-" and the server resends the entire file, so wget downloads that file twice (or potentially an infinite number of times on especially flaky connections).

I've attached a console screenshot and a HTTP sniff (look at the GET # 7257). I've run it through a local proxy (in bypass mode) so that I could manually drop the connection and look at the HTTP messages.

The previous version I tested was 1.10.1, and it didn't show this behaviour.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #16727:  command.png added by None (48KiB - image/png)
file #16728:  http_dump.png added by None (50KiB - image/png)
file #16729:  wgetrc added by None (359B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by micahcowan (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2008-10-27 micahcowan StatusNone Duplicate
        Open/ClosedOpen Closed
    2008-10-24 None Attached File- Added command.png, #16727
        Attached File- Added http_dump.png, #16728
        Attached File- Added wgetrc, #16729

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code