bugGNU Wget - Bugs: bug #53968, Decompressed data is written to...

 
 

bug #53968: Decompressed data is written to WARC file when using --compression=gzip

Submitter:  None
Submitted:  Thu 24 May 2018 12:50:50 PM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  Fixed
Privacy:  Public Assigned to:  rockdaboot
Originator Name:  William Prescott Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  1.19.5
Operating System:  GNU/Linux Reproducibility:  Every Time
Fixed Release:  trunk Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 29 May 2018 10:19:24 AM UTC, comment #2: 

Thanks for fixing it. I applied it as a patch to the tar release of 1.19.5 and it appears to work correctly.

Anonymous
Tue 29 May 2018 08:56:00 AM UTC, comment #1: 

Thanks for reporting.

I made a potential fix (latest master branch). Could you test it or tell us how to easily reproduce, please.

If the fix doesn't work out, we reopen the issue.

Tim Ruehsen <rockdaboot>
Group administrator
Thu 24 May 2018 12:50:50 PM UTC, original submission:  

When using the "--compression=gzip" option, Wget will write decompressed data into "response" WARC records instead of recording the response exactly as it was received.
This also prevents one from correctly decoding data in the WARC file that had been transferred using chunked encoding.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44208:  example.warc.gz added by None (5KiB - application/gzip - WARC and terminal output from running "wget -d --compression=gzip --warc-file=example 'http://lists.gnu.org/'")
file #44209:  wgetOutput.txt added by None (2KiB - text/plain - WARC and terminal output from running "wget -d --compression=gzip --warc-file=example 'http://lists.gnu.org/'")

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rockdaboot (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-05-29 rockdaboot StatusNone Fixed
        Assigned toNone rockdaboot
        Open/ClosedOpen Closed
        Fixed ReleaseNone trunk
    2018-05-24 None Attached File- Added example.warc.gz, #44208
        Attached File- Added wgetOutput.txt, #44209

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code