bugGNU Wget - Bugs: bug #60956, stylesheet and icon <link>...


bug #60956: stylesheet and icon <link> elements not properly classified as page requisites

Submitter:  None
Submitted:  Wed 21 Jul 2021 08:25:49 PM UTC
Category:  Program Logic Severity:  3 - Normal
Priority:  5 - Normal Status:  Confirmed
Privacy:  Public Assigned to:  rockdaboot
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  None
Operating System:  None Reproducibility:  Every Time
Fixed Release:  None Planned Release:  None
Regression:  None Work Required:  None
Patch Included:  No
* Mandatory Fields

Add a New Comment Rich Markup

Wed 15 Sep 2021 09:32:04 AM UTC, comment #3: 

OP here.  Thanks to all for working on this.

I confirm that version 1.21.2 built from https://ftp.gnu.org/gnu/wget/wget-1.21.2.tar.gz considers link elements with rel="icon" or rel="alternate stylesheet" to be page requisites, resolving the issue for the two test cases given in the original report.

Looking at commit ebb96761, it seems that a link element with rel="stylesheet alternate" still won't be considered a page requisite though.  Of course with this approach the same is true for all sorts of things like rel="random nonstandard keywords stylesheet", but if a line is going to be drawn somewhere, I think it would make sense to add "stylesheet alternate" to the list.  Unfortunately I can't provide a test case for this at the moment.  Thoughts?

I see that the same commit also causes link elements with rel="manifest" to be considered page requisites as well, but this is not mentioned in NEWS.  I point this out in case the omission was unintentional.

Wed 08 Sep 2021 04:04:16 PM UTC, comment #2: 

I believe this has been fixed in 1.21.2 now?
Can we close it?

Darshit Shah <darnir>
Group administrator
Sat 07 Aug 2021 09:39:28 AM UTC, comment #1: 

Thank you for the detailed bug report !

Tim Ruehsen <rockdaboot>
Group administrator
Wed 21 Jul 2021 08:25:49 PM UTC, original submission:  

stylesheet and icon <link> elements not properly classified as page requisites

The version of GNU Wget I was using:
1.20.3 from https://eternallybored.org/misc/wget/releases/wget-1.20.3-win64.zip
(None of the changes listed in NEWS for newer versions seem relevant to this issue.)

How I invoked wget:

wget --debug --page-requisites https://mdn.github.io/css-examples/alt-style-sheets/

What I expected wget to do:
fetch https://mdn.github.io/css-examples/alt-style-sheets/ and save as index.html
fetch https://mdn.github.io/css-examples/alt-style-sheets/default.css and save as default.css
fetch https://mdn.github.io/css-examples/alt-style-sheets/simple.css and save as simple.css
fetch https://mdn.github.io/css-examples/alt-style-sheets/fancy.css and save as fancy.css

What wget did:
fetch https://mdn.github.io/css-examples/alt-style-sheets/ and save as index.html
fetch https://mdn.github.io/css-examples/alt-style-sheets/default.css and save as default.css

Output messages (Hopefully the relevant portion; the rest can be provided if needed):

[IRI Enqueuing 'https://mdn.github.io/css-examples/alt-style-sheets/default.css' with 'utf-8'
Not following due to 'link inline' flag: https://mdn.github.io/css-examples/alt-style-sheets/simple.css
Not following due to 'link inline' flag: https://mdn.github.io/css-examples/alt-style-sheets/fancy.css
Not following due to 'link inline' flag: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/link


The page used in the example above references three stylesheets:

<link href="default.css" rel="stylesheet" type="text/css" title="Default Style">
<link href="simple.css" rel="alternate stylesheet" type="text/css" title="Simple">
<link href="fancy.css" rel="alternate stylesheet" type="text/css" title="Fancy">

Wget only considers the first one (with rel="stylesheet") to be a page requisite as far as --page-requisites is concerned.  This seems incorrect because the value of the rel attribute is specified by the HTML Standard section 4.2.4 (The link element) to be an unordered set of unique space-separated tokens.  All three in this example include "stylesheet", so they should all be considered page requisites.

A similar problem occurs with favicons.  Wget seems to consider those referenced with rel="shortcut icon" to be page requisites, but not those referenced with rel="icon".  For an example of the latter, try the following:

wget --page-requisites https://savannah.gnu.org/

The relevant portion of the HTML is:

<link rel="icon" type="image/png" href="/images/Savannah.theme/icon.png" />

It seems that rel="shortcut icon" is actually deprecated but still allowed for historical reasons.  See HTML Standard section (Link type "icon"), especially the note at the end of the section.

Posssible cause and solution:

I haven't investigated thoroughly and I'm not very familiar with C, but I'm guessing that the function tag_handle_link in the file src/html-url.c is responsible for this issue and is just checking the rel attribute of the <link> element for case-insensitive equality to "stylesheet" or "shortcut icon".

A better approach might be to split the rel attribute on ASCII whitespace as described by the HTML Standard section 4.6.6 (Link types) and then check each of the resulting tokens for case-insensitive equality to "stylesheet" and "icon".

An alternative which would probably be less correct and less reliable but may be easier to implement and still useful would be to check the rel attribute for case-insensitive equality to "stylesheet", "alternate stylesheet", "stylesheet alternate", "shortcut icon", and "icon".

See also:
HTML Standard section (Link type "alternate")
HTML Standard section (Link type "stylesheet")
<link>: The External Resource Link element (MDN)
Link types (MDN)



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by darnir (Posted a comment)
  • -email is unavailable- added by rockdaboot (Posted a comment)
  • -email is unavailable- added by atechgeek (Bug Updates)

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-08-07 rockdaboot StatusNone Confirmed
        Assigned toNone rockdaboot
    2021-08-02 atechgeek Carbon-Copy- Added atechgeek

    Back to the top

    Powered by Savane 3.13-a854.
    Corresponding source code