bugGNU Wget - Bugs: bug #50320, Bad link conversion with mixed...

 
 

bug #50320: Bad link conversion with mixed HTTP/HTTPS content plus --mirror --adjust-extension

Submitted by:  None
Submitted on:  Wed 15 Feb 2017 06:08:54 PM UTC  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: Thomas ClaveiroleOriginator Email: -unavailable-
Open/Closed: OpenRelease: trunk
Operating System: GNU/LinuxReproducibility: Every Time
Fixed Release: NonePlanned Release: None
Regression: NoneWork Required: None
Patch Included: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Wed 15 Feb 2017 06:08:54 PM UTC, original submission:

Hello,

When I setup a local web server to provide :

<!DOCTYPE html>
<html>
<head>
<title>Wget test</title>
</head>
<body>
<script src="http://localhost/wget-test/script.js?foo=bar"></script>
<script src="https://localhost/wget-test/script.js?foo=bar"></script>
</body>
</html>

when requesting /wget-test/, either as HTTP or HTTPS, as well as a /wget-test/script.js resource (regardless of the scheme and query string; the content of this file is irrelevant).

Then,

wget --mirror --adjust-extension --convert-links http://localhost/wget-test/
rewrites the script links as follows:

<!DOCTYPE html>
<html>
<head>
<title>Wget test</title>
</head>
<body>
<script src="script.js%3Ffoo=bar"></script>
<script src="script.js%3Ffoo=bar.html"></script>
</body>
</html>

Note that the second link has an incorrect .html suffix appended. On the filesystem, the downloaded file does not have this suffix, so the link is broken. I guess the correct behavior should be not to append the .html suffix, but I am unsure whether two URLs that differ only in scheme (http:// vs. https://) should be considered the same resource and rewritten to point to the same location.

(This test case was derived from trying to mirror a much bigger site and it took me some time to pinpoint the issue. The bug also arises when multiple pages from the website link to the same resource using mixed http and https schemes -- which is a more realistic scenario.)

Looking at the bug tracker, I get the feeling that this bug might be related to #50173 and #25340, but this is unclear to me.

Find attached a debug log for :
wget -o wget.log --debug --no-check-certificate --mirror --adjust-extension --convert-links http://localhost/wget-test/
with my setup.

Regards,
Thomas Claveirole

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #39762:  wget.log added by None (9KiB - text/x-log)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by None (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 15 Feb 2017 06:08:54 PM UTCNoneAttached File-=>Added wget.log, #39762

    Back to the top


    Powered by Savane 3.1-cleanup1