Tue 30 Jun 2009 06:57:13 PM UTC, original submission:
HTML 5 is approaching, for better or worse, which means a new slew of tags/attributes to parse for URIs.
OTOH, this also highlights a bit of future-proofing that Wget lacks. Users of existing Wget releases will need to upgrade just to get support for new tags. Also, if a page uses "custom" tags that a user might want to parse, there's no real way to specify that, currently.
Would be nice to extend the --folow-tags/--ignore-tags feature to accept lists of arbitrary tag/attr-list specifications, and not just tags that Wget already knows about. For instance, if a new "foo" element takes src uref attributes, both of which may contain URIs, then we could specify --follow-tags="<default>,foo:src|uref", where "<default>" means the ones Wget already knows about.
There might be some value in using an XPath-compatible syntax to specify tags/attributes. This feature would also potentially open Wget up to parse links from non-HTML XML files as well (RSS and the like?), if combined with a feature that told Wget to parse non-html content-types.
|