bugGNU Wget - Bugs: bug #37212, When logging output to files, the...

 
 

bug #37212: When logging output to files, the URL is corrupted ( escaped : character sequence malformed)

Submitted by:  marius <mariush>
Submitted on:  Tue 28 Aug 2012 07:05:52 PM UTC  
 
Category: Program LogicSeverity: 3 - Normal
Priority: 5 - NormalStatus: None
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: NoneOperating System: Microsoft Windows
Reproducibility: NoneFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 28 Aug 2012 08:42:42 PM UTC, comment #1:

Did some further tests, after seeing that aria2c has the same issue.

Using --restrict-file-names=nocontrol solves the issue:

wget "http://www.example.org/gameswiki/index.php?search=102+Dalmatians%3A+P
uppies+To+The+Rescue&amp;go=Go&amp;title=Special%3ASearch" -o .\log\d0\200 -O .\
out\d0\200.out --restrict-file-names=nocontrol

./log/200.out:

--2012-08-28 23:30:36-- http://www.example.org/gameswiki/index.php?search=102+Dalmatians%3A+Puppies+To+The+Rescue&amp;go=Go&amp;title=Special%3ASearch

So it seems wget makes some changes to the url in an attempt to make the file save-able on disk.

BUT this isn't right, if I specify manually the name of the output file I don't see a reason it should mess about with the url.

There's something wrong happening somewhere

wget receives url in command line,
- when log to file and output to file enabled, wget changes in url %3A changes to A.
- when not logging and not output to file / output to console then url does not change, correct page is retrieved

--restrict-file-names=nocontrol makes wget not change the url

marius <mariush>
Tue 28 Aug 2012 07:05:52 PM UTC, original submission:

Version : wget 1.11.4 on Windows 7 64 bit (Home Premium)

Issue : When adding log parameters (-o and -O) and an url with %3A (:) is used, the url is corrupted, logs show the "%3A" changed to "A", wget tries to retrieve url using the corrupt url.

Apologies if this is due to my stupidity or if it's something very obvious, I don't have much experience with this application and various OS quirks.

Trying to automatize some tasks on a personal site based on Mediawiki.

The URL I have problems with is in the following form (domain and IP anonymized in the outputs below):

http://www.example.org/gameswiki/index.php?search=102+Dalmatians%3A+Puppies+To+The+Rescue&amp;go=Go&amp;title=Special%3ASearch

When running from the command line, everything works correctly:

wget "http://www.example.org/gameswiki/index.php?search=102+Dalmatians%3A+Puppies+To+The+Rescue&amp;go=Go&amp;title=Special%3ASearch"
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = c:/progra~1/wget/etc/wgetrc
--2012-08-28 21:40:16-- http://www.example.org/gameswiki/index.php?search=102+Dalmatians%3A+Puppies+To+The+Rescue&amp;go=Go&amp;title=Special%3ASearch
Resolving http://www.example.org... #.#.#.#
Connecting to http://www.example.org|#.#.#.#|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://www.example.org/games/102_Dalmatians:_Puppies_To_The_Rescue [following]
--2012-08-28 21:40:17-- http://www.example.org/games/102_Dalmatians:_Puppies_To_The_Rescue
Reusing existing connection to http://www.example.org:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `102_Dalmatians%3A_Puppies_To_The_Rescue.2'

So as you can see wget detects the redirect and retrieves the page from the right URL.

However, when adding the log output and log to file commands the URL gets messed up: "%3A" gets converted to "A" in the URL

command:

wget "http://www.example.org/gameswiki/index.php?search=102+Dalmatians%3A+Puppies+To+The+Rescue&amp;go=Go&amp;title=Special%3ASearch" -o .\log\d0\2 -O .\out\d0\2.out

and here's the result:

2.out:

--2012-08-28 21:31:07-- http://www.example.org/gameswiki/index.php?search=102+DalmatiansA+Puppies+To+The+Rescue&amp;go=Go&amp;title=SpecialASearch
Resolving http://www.example.org... #.#.#.#
Connecting to http://www.example.org|#.#.#.#|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `./out/d0/2.out'

0K .......... . 127M=0s

2012-08-28 21:31:08 (127 MB/s) - `./out/d0/2.out' saved [11285]

The saved output is also the incorrect one... page says "search results for 102 DalmationsA Puppies..."

--

Is there some special parameters to be used in Windows that I'm not aware of so that the URLs won't get corrupted when activating the logging?

Thank you for your help, it's very appreciated.

marius <mariush>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mariush (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup