bugGNU Wget - Bugs: bug #21392, --post-data should only POST to...


bug #21392: --post-data should only POST to the first URI, not recursive downloads

Submitter:  Micah Cowan <micahcowan>
Submitted:  Mon 22 Oct 2007 06:42:40 PM UTC
Category:  Program Logic Severity:  2 - Minor
Priority:  3 - Low Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  None Operating System:  None
Reproducibility:  None Fixed Release:  None
Planned Release:  1.14 Regression:  None
Work Required:  0 - Hours Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup

Thu 22 Apr 2010 07:05:43 PM UTC, comment #1: 

Stuart and others - here's a dirty hack that's working for what I need...

/usr/bin/wget --post-data  "Username=$USERNAME&Password=$PASSWORD" -O - "$SOURCE" |  /usr/bin/wget --base="$SOURCEBASE" --force-html -r -l 2 -nd -nH  -i -

Mon 22 Oct 2007 06:42:40 PM UTC, original submission:  

(From the mailing list:)


I've been using wget to recursively download the output of a CGI
script on a server, together with any documents linked to by the
output of that CGI script - and then to use -k to create a locally
linked version.

Due to the length of data sent to the CGI script, wget needs to be
invoked with --post-file

It seems that wget sends this post data to all the URLs it recursively
downloads - not just the base URL. Unfortunately this stings me
somewhat, as one of the webservers I need to download the linked files
from refuses post requests. (Even if I can get this changed, it seems
wrong to be sending post requests designed for one page to another

Is there any way to get wget to only use the post data for the first
file downloaded? I couldn't find any in the documentation - in fact
there seems to be nothing in the documentation regarding the
interaction of recursive downloading with post data. It would be great
to see the current behaviour documented somewhere.

Alternatively, if anyone can suggest any workarounds, that'd be much
appreciated. I need to convert the links, so just downloading the
first file using post, and then using that as wget's input (using -i)
won't work.

Stuart Moore

Micah Cowan <micahcowan>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by micahcowan (Submitted the item)

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2008-08-21 micahcowan Planned Release1.15 1.14
    2008-01-27 micahcowan Planned Release1.12 1.15
    2007-10-22 micahcowan Priority5 - Normal 3 - Low

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code