bugGNU Wget - Bugs: bug #66338, fails to fetch files that redirect...

 
 

bug #66338: fails to fetch files that redirect from a directory (workaround: use aria2c)

Submitter:  Eo Koochu <eokoochu>
Submitted:  Wed 16 Oct 2024 10:10:53 AM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  None Operating System:  GNU/Linux
Reproducibility:  Every Time Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  No
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 16 Oct 2024 10:10:53 AM UTC, original submission:  

When an URL refers to a directory that redirects to a file, wget gives up instantly without checking the redirect header. This is a sample broken session:

===8<------------------------------
$ wget 'https://consumer.georgia.gov/document/document/consumer-complaint-form-english/download'
--[timestamp]--  https://consumer.georgia.gov/document/document/consumer-complaint-form-english/download
Resolving consumer.georgia.gov (consumer.georgia.gov)... 104.18.74.48
Connecting to consumer.georgia.gov (consumer.georgia.gov)|104.18.74.48|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 183944 (180K) [application/pdf]
download: Is a directory
===8<------------------------------

This happened in version 1.21.3. It may be a similar bug to this one (not sure):

https://savannah.gnu.org/bugs/?func=detailitem&item_id=45992

I did not experiment with the --trust-server-names option. But if that’s a factor then wget still needs to more accurately and thoroughly inform the user, in which case it’s arguably a defect nonetheless. E.g. it should say in the error msg something like “not following redirect for security reasons”, if that’s the case.

*workaround*

The workaround is basically to use anything other than wget. Firefox works but FF suggests the wrong filename. Aria2c works flawlessly and sets the correct filename. This is what the aria2c session looks like:

===8<------------------------------
$ aria2c 'https://consumer.georgia.gov/document/document/consumer-complaint-form-english/download'
[timestamp] [NOTICE] Downloading 1 item(s)
[#6234b1 0B/0B CN:1 DL:0B]
[timestamp] [NOTICE] Download complete: $PWD/Consumer Complaint Form - 2024-09-30.pdf

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
6234b1|OK  |   459KiB/s|$PWD/Consumer Complaint Form - 2024-09-30.pdf

Status Legend:
(OK):download completed.
===8<------------------------------


Eo Koochu <eokoochu>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by eokoochu (Submitted the item)
  •  

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.14-f13d.
    Corresponding source code