bugGNU Wget - Bugs: bug #61050, Early abort should not write 0...

 
 

bug #61050: Early abort should not write 0 Byte output file

Submitter:  None
Submitted:  Tue 17 Aug 2021 02:20:37 PM UTC
   
 
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  Invalid
Privacy:  Public Assigned to:  None
Originator Name:  Matthias Hörmann Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  trunk
Operating System:  GNU/Linux Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  No
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 08 Sep 2021 04:03:22 PM UTC, comment #1: 

Sorry, this is documented behaviour and will not be changed in Wget. The manual explicitly states that the -O option is equivalent to the shell redirection operator ">". Thus, creating / truncating an existing file to 0 bytes immediately is the correct behaviour.

I completely understand where you are coming from. I don't like this either. But this has been the documented behaviour for many  many years now and multiple scripts rely on it to be so. As a result, the default will not be changed in the interests of backwards compatibility.

GNU Wget2, the next major version of Wget changes the meaning of -O and functions exactly as you described / expected it to. It was the first breaking change I introduced in there :)

Darshit Shah <darnir>
Group administrator
Tue 17 Aug 2021 02:20:37 PM UTC, original submission:  

Using Debian wget 1.21 or Gentoo 1.21.1 the following behaviour can be reproduced:

Calling

/usr/bin/wget -O 'GeoLiteCity.dat.gz' 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz'

the download fails because the domain no longer exists with

--2021-08-17 16:11:53--  http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
Resolving geolite.maxmind.com (geolite.maxmind.com)... failed: Name or service not known.
wget: unable to resolve host address ‘geolite.maxmind.com’

and an exit code of 4.

It also creates the output file as a zero byte file.

Clearly a download had not even started here so there is no reason to create the file already (and especially to not clean it up on failure if it needs to be created early for technical reasons).

This became an issue when our Puppet manifests then proceeded to skip the download of the file that was there and tried to unpack the 0 byte file in the next Puppet run (it correctly displayed the failed download command as failed in the original run).

It just makes it harder to determine whether the download already happened for a significant portion of failure cases and I also do not see a situation where it would be useful to have an empty file that has literally nothing to do with the file on the server (no metadata, content,...).

I understand that detecting a download by existence of the file can also be a problem on partial downloads but for reasonably small files that happens much more rarely than all the various errors that can occur before a download even starts.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by darnir (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-09-08 darnir StatusNone Invalid
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-bb6a.
    Corresponding source code