Fri 02 Nov 2012 09:07:14 PM UTC, original submission:
Combining --page-requisites and an output file that does not meaningfully support reading (such as /dev/null or stdout) fails.
It fails because the logic for --page-requisites goes back to the output file and reads it in order to build the list of URLs to download. This is not an intuitive behavior for most users since many would assume that an output file would not need to support reading.
To reproduce:
[output appended]
$ wget -O /dev/null --page-requisites http://www.gnu.org # Stops after initial file
$ wget -O - --page-requisites http://www.gnu.org # Hangs
Would it be possible to address this either by documenting the behavior within the man page or by parsing the document in memory?
WGET VERSION: 1.12 and 1.14
BEHAVIOR EXPECTED: to fetch an html file and all its prerequisites, throwing the actual data away
BEHAVIOR SEEN: wget either hangs (in the case of using -O -) or does not fetch the requisites (in the case of -O /dev/null)
WHY WOULD ANYONE DO THIS: In order to prime a cache or ensure all of a page's requisites are accessible.
Thank you,
Jason
Output:
$ wget -O /dev/null --page-requisites http://www.gnu.org
WARNING: combining -O with -r or -p will mean that all downloaded content
will be placed in the single file you specified.
--2012-11-02 13:42:28-- http://www.gnu.org/
Resolving http://www.gnu.org... 208.118.235.148, 2001:4830:134:3::a
Connecting to http://www.gnu.org|208.118.235.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `/dev/null'
[ <=> ] 19,514 --.-K/s in 0.1s
2012-11-02 13:42:28 (139 KB/s) - `/dev/null' saved [19514]
FINISHED --2012-11-02 13:42:28--
$ wget -O - --page-requisites http://www.gnu.org
WARNING: combining -O with -r or -p will mean that all downloaded content
will be placed in the single file you specified.
--2012-11-02 13:43:01-- http://www.gnu.org/
Resolving http://www.gnu.org... 208.118.235.148, 2001:4830:134:3::a
Connecting to http://www.gnu.org|208.118.235.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `STDOUT'
[<=> ] 0 --.-K/s <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
[...]
2012-11-02 13:43:28 (139 KB/s) - written to stdout [19514]
[hangs]
|