bugGNU Wget - Bugs: bug #45689, wget: opens a new connection for...

 
 

bug #45689: wget: opens a new connection for each ftp document

Submitter:  Noël Köthe <nok>
Submitted:  Tue 04 Aug 2015 12:18:38 PM UTC
   
 
Category:  Feature Request Severity:  3 - Normal
Priority:  5 - Normal Status:  Needs Discussion
Privacy:  Public Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  trunk Operating System:  GNU/Linux
Reproducibility:  Every Time Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 15 Aug 2015 01:01:24 PM UTC, comment #2: 

Hi Darshit,

This is not the same concept as the persistent connections in HTTP (those with Connect: Keep-Alive).

What happens here is that if you download two documents from the same FTP server (ie. `wget ftp://site.com/file1.txt ftp://site.com/file2.txt`) Wget will open a connection, download the file, and close the connection again for each of the files. This is inefficient if we take into account that both files are stored in the same server (ie, site.com). It'd be better to open the connection to site.com once, and send two RETRs over the same connection, and then close the connection. It's a simple concept, in reality.

This happens because the traversal is done in recur.c, whereas the actual connection-download-close process is done in ftp.c, and there's no relationship between both.

This issue was among my favorites, but I won't work on it until at least one month, so feel free to fix this, if you want :D

Ander Juaristi <ajuaristi>
Sat 15 Aug 2015 07:18:04 AM UTC, comment #1: 

I'm not very well versed with the FTP Protocol, so if I'm wrong here, someone please correct me.

Is re-using a FTP connection even possible? Because I haven't been able to find anything in the RFCs or through online searches to supplement this. I thought this was actually one of the largest drawbacks of the protocol that we need to open a lot of extra connections.

Can someone please point me to a resource explaining how this would work? If possible, I'd implement it.

Darshit Shah <darnir>
Group administrator
Tue 04 Aug 2015 12:18:38 PM UTC, original submission:  

Hello,

reporting an old bug/report from the mailing list:

if I use 'wget ftp://site.com/file1.txt ftp://site.com/file2.txt', wget will no reuse the ftp connection, but will open one for each document downloaded from the same site...

https://www.mail-archive.com/wget%40sunsite.dk/msg03316.html
https://bugs.debian.org/32353

thx and regards

Noël

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ajuaristi (Posted a comment)
  • -email is unavailable- added by darnir (Posted a comment)
  • -email is unavailable- added by nok (Submitted the item)
  • -email is unavailable- added by nok
  •  

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2015-08-15 darnir StatusNone Needs Discussion
    2015-08-04 nok Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code