bugGNU Wget - Bugs: bug #50579, wget --continue: Download fails...

 
 

bug #50579: wget --continue: Download fails when server does not support HEAD request

Submitted by:  Noël Köthe <nok>
Submitted on:  Sat 18 Mar 2017 06:07:36 PM UTC  
 
Category: Feature RequestSeverity: 3 - Normal
Priority: 5 - NormalStatus: Wont Fix
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Closed
Release: NoneOperating System: None
Reproducibility: NoneFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: No

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 24 Mar 2017 08:19:17 AM UTC, comment #3:

As Dale mentions, what Wget does is exactly according to the RFC. In fact it is the server that breaks the spec. And a workaround for this server may cause breakages for other servers that actually follow the specifications. As a result, I vote that we close this as a non-issue

Darshit Shah <darnir>
Project Administrator
Fri 24 Mar 2017 01:34:05 AM UTC, comment #2:

RFC 2616 section 9.4 says that an HTTP server should return the same headers for a HEAD request as it does for a GET of the same URL. So I see nothing incorrect with wget relying on that behavior.

Dale Worley <worley>
Sat 18 Mar 2017 06:40:57 PM UTC, comment #1:

As you already stated, the server should have answered the HEAD request properly. If a server refuses to do so, --continue currently doesn't work.

Especially with --content-disposition, Wget doesn't know about the filename to look/save at, so it simply gives up here.

If now Wget tries to be 'clever' and (silently) falls back to the filename derived from the URL, it could be that the wrong file becomes saved, clobbered, destroyed (depending on additional options).

So IMO, we better leave it up to the user to exactly say what (s)he wants.

The title is misleading, the problem is the same wether with or without trailing slash.

I leave this bug open for discussion and mark it feature request.

Tim Ruehsen <rockdaboot>
Project Administrator
Sat 18 Mar 2017 06:07:36 PM UTC, original submission:

Hello,

a forwarded bug report from a Debian user https://bugs.debian.org/843620

--8<--
It would be nice if the behavior of --continue would also handle
corner cases, such as the following:

$ wget --content-disposition --continue
http://www.mckesson.com/documents/providers/mckesson-radiology-12-dicom-conformance-statement/
--2016-11-08 12:48:07--
http://www.mckesson.com/documents/providers/mckesson-radiology-12-dicom-conformance-statement/
Resolving http://www.mckesson.com (www.mckesson.com)... 2.20.250.32, 2.20.250.41
Connecting to http://www.mckesson.com (www.mckesson.com)|2.20.250.32|:80... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2016-11-08 12:48:08 ERROR 503: Service Unavailable.

On the other hand this one works OK:

$ wget --content-disposition --continue
http://www3.gehealthcare.com/~/media/documents/us-global/products/interoperability/dicom/nuclear-medicine/gehc-dicom-conformance_ventri_5179057dhf_rev1.pdf
--2016-11-08 12:49:25--
http://www3.gehealthcare.com/~/media/documents/us-global/products/interoperability/dicom/nuclear-medicine/gehc-dicom-conformance_ventri_5179057dhf_rev1.pdf
Resolving www3.gehealthcare.com (www3.gehealthcare.com)...
2.20.250.19, 2.20.250.10
Connecting to www3.gehealthcare.com
(www3.gehealthcare.com)|2.20.250.19|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 709762 (693K) [application/pdf]
--2016-11-08 12:49:25--
http://www3.gehealthcare.com/~/media/documents/us-global/products/interoperability/dicom/nuclear-medicine/gehc-dicom-conformance_ventri_5179057dhf_rev1.pdf
Reusing existing connection to www3.gehealthcare.com:80.
HTTP request sent, awaiting response... 206 Partial Content
Length: 709762 (693K), 0 remaining [application/pdf]
Saving to: ‘GEHC-DICOM-Conformance_Ventri_5179057DHF_Rev1.pdf’

GEHC-DICOM-Conformance_Ventri_5179057DHF_Re
100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++]
693.13K --.-KB/s in 0s

2016-11-08 12:49:26 (0.00 B/s) -
‘GEHC-DICOM-Conformance_Ventri_5179057DHF_Rev1.pdf’ saved
[709762/709762]

For some reason HEAD would not get the Content-Disposition tag, but
looking at curl output I can see it:

  • Hostname was NOT found in DNS cache

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:--
--:--:-- 0* Trying 2.20.250.32...

> GET /documents/providers/mckesson-radiology-12-dicom-conformance-statement/ HTTP/1.1
> User-Agent: curl/7.38.0
> Host: http://www.mckesson.com
> Accept: /
>

0 0 0 0 0 0 0 0 --:--:-- --:--:--
--:--:-- 0< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Length: 3292374
< Content-Type: application/pdf

  • Server Microsoft-IIS/7.5 is not blacklisted

< Server: Microsoft-IIS/7.5
< Content-Disposition: inline; filename="McKesson Radiology 12.1.1
DICOM Conformance Statement rev1.0.pdf"
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< X-UA-Compatible: IE=edge,chrome=1
< Date: Tue, 08 Nov 2016 11:45:47 GMT
< Connection: keep-alive
< Set-Cookie: EktGUID=3f3d7804-163d-4a59-bbf4-676abf1a1dab;
expires=Wed, 08-Nov-2017 11:45:46 GMT; path=/
< Set-Cookie: EkAnalytics=0; expires=Wed, 08-Nov-2017 11:45:46 GMT; path=/
< Set-Cookie: ASP.NET_SessionId=nprzkvb3mnsu41ahuslx1auy; path=/; HttpOnly
< Set-Cookie: ecm=user_id=0&isMembershipUser=0&site_id=&username=&new_site=/&unique_id=0&site_preview=0&langvalue=0&DefaultLanguage=1033&NavLanguage=1033&LastValidLanguageID=1033&DefaultCurrency=840&SiteCurrency=840&ContType=&UserCulture=1033&dm=www.mckesson.com&SiteLanguage=1033;
path=/
< Set-Cookie: BIGipServerMCKESSON.COM_80=801417408.20480.0000;
expires=Tue, 08-Nov-2016 12:05:46 GMT; path=/
<
{ [data not shown]

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by darnir (Posted a comment)
  • -unavailable- added by worley (Posted a comment)
  • -unavailable- added by rockdaboot (Posted a comment)
  • -unavailable- added by nok (Submitted the item)
  • -unavailable- added by nok
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 24 Mar 2017 07:29:32 PM UTCrockdabootStatusNeeds Discussion=>Wont Fix
      Open/ClosedOpen=>Closed
    Sat 18 Mar 2017 06:40:57 PM UTCrockdabootCategoryProgram Logic=>Feature Request
      StatusNone=>Needs Discussion
      Summarywget --continue: URL with trailing slash '/' (but Content-Disposition)=>wget --continue: Download fails when server does not support HEAD request
    Sat 18 Mar 2017 06:07:36 PM UTCnokCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup1