bugGNU Wget - Bugs: bug #40819, wget does not send Accept-Encoding...

 
 

bug #40819: wget does not send Accept-Encoding and fails to handle Content-Encoding

Submitter:  Noël Köthe <nok>
Submitted:  Thu 05 Dec 2013 09:28:45 PM UTC
   
 
Category:  Protocol Issue Severity:  3 - Normal
Priority:  5 - Normal Status:  Fixed
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Closed
Release:  trunk Operating System:  GNU/Linux
Reproducibility:  None Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  No
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 04 Aug 2015 01:13:11 PM UTC, comment #1: 

just for completness:

commit 176c2b17fc9fd5e5e9db654e599fbfd2ae313bef
Author: Tim Rühsen <tim.ruehsen@gmx.de>
Date:   Tue Nov 11 16:01:58 2014 +0100

    Add 'Accept-Encoding: identity' to request header
   
    Fixes bug #40819
    Reported-by: Noël Köthe <noel@debian.org>

Noël Köthe <nok>
Thu 05 Dec 2013 09:28:45 PM UTC, original submission:  

Hello,

a forwarded bug from a Debian user: http://bugs.debian.org/731110

--8<--
I observed that wget downloads "wrong" content from a squid cache. Both tools operate within the bounds of RFC2616, but the resulting combination is wrong. Consider the following interaction:

A http client that supports Accept-Encoding and Content-Encoding
downloads an object via a squid3 proxy. An encoding that is not the identity encoding is chosen and the response is considered cacheable by squid3. At a later time wget is instructed to retrieve the same object via the same proxy. Unlike the other client wget does not send an Accept-Encoding header. The squid3 proxy exercises an assumption explicitly granted by RFC2616 14.3:

| If no Accept-Encoding field is present in a request, the server MAY
| assume that the client will accept any content coding.

It thus sends back the cached object including the Content-Encoding header, but wget does not handle the header and stores the object in compressed form.

Note that RFC2616 goes on to say:

| In this case, if "identity" is one of the available content-codings,
| then the server SHOULD use the "identity" content-coding, unless it has
| additional information that a different content-coding is meaningful to
| the client.

So arguably squid3 is violating a SHOULD clause and should be fixed.

A simple mechanism to resolve the situation is to make wget send an empty Accept-Encoding header by default or setting its value to "identity".

Helmut
--8<--

Thank you.

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by darnir (Updated the item)
  • -email is unavailable- added by darnir
  • -email is unavailable- added by nok (Submitted the item)
  • -email is unavailable- added by nok
  •  

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2015-03-07 darnir StatusNone Fixed
        Open/ClosedOpen Closed
    2014-01-17 darnir Carbon-Copy- Added -email is unavailable-
    2013-12-05 nok Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code