GNU Wget - Bugs: bug #59086, --page-requisites not always...

Group

Main
- Main
- View members
- Search
Homepage
Download
Docs
Mailing lists
Source code
- Use CVS
- Browse Web Pages Repository
- Use Git
- Browse Git repositories
- GNU Wget
- wget/wget2.git
Bugs
- Submit new
- Browse
- Reset to open
- Digest
- Dependencies
- ~~Export~~
- Get statistics
- Search
News
- Browse
- Atom feed
- ~~Submit~~
- ~~Manage~~

bug #59086: --page-requisites not always working when creating a warc file

Submitter:	Thomas Egense <thomasegense>
Submitted:	Wed 09 Sep 2020 08:52:02 AM UTC


Category:	Program Logic	Severity:	3 - Normal
Priority:	5 - Normal	Status:	None
Privacy:	Public	Assigned to:	None
Originator Name:		Open/Closed:	Open
Release:	None	Operating System:	GNU/Linux
Reproducibility:	None	Fixed Release:	None
Planned Release:	None	Regression:	None
Work Required:	None	Patch Included:	None

* Mandatory Fields

Add a New Comment Rich Markup

Wed 09 Sep 2020 08:59:37 AM UTC, comment #1:

minor correction:
and links are also NOT followed

Thomas Egense <thomasegense>

Wed 09 Sep 2020 08:52:02 AM UTC, original submission:

Url example: https://jyllands-posten.dk/

How to reproduce:
echo "https://jyllands-posten.dk/" >> url_list.txt

wget --level=1 --recursive --warc-cdx --page-requisites --warc-file=jp --warc-max-size=1G -i url_list.txt

The source code for the page is downloaded in the warc (last record). But none of the images are downloaded and links are also followed (--recursive parameter).

It is probably due to some HTTPS redirection, but since the
source code is downloaded correct, it should still be possible to follow links and download page requisites.

Thomas Egense <thomasegense>

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:


Comment:

No files currently attached

Depends on the following items: None found

Items that depend on this one: None found

Carbon-Copy List

-email is unavailable- added by thomasegense (Submitted the item)

Please enter the title of George Orwell's famous dystopian book (it's a date):

No changes have been made to this item

GNU Wget - Bugs: bug #59086, --page-requisites not always...

bug #59086: --page-requisites not always working when creating a warc file

<a href="#postcomment">Post a Comment</a>

<a href="#discussion">Discussion</a>

<a href="#attached">Attached Files</a>

<a href="#dependencies">Dependencies</a>

<a href="#cc">Mail Notification Carbon-Copy List</a>

<a href="#history">History</a>