bugGNU Wget - Bugs: bug #45689, wget: opens a new connection for...

 
 

bug #45689: wget: opens a new connection for each ftp document

Submitted by:  Noël Köthe <nok>
Submitted on:  Tue 04 Aug 2015 12:18:38 PM UTC  
 
Category: Feature RequestSeverity: 3 - Normal
Priority: 5 - NormalStatus: Needs Discussion
Privacy: PublicAssigned to: None
Originator Name: Open/Closed: Open
Release: trunkOperating System: GNU/Linux
Reproducibility: Every TimeFixed Release: None
Planned Release: NoneRegression: None
Work Required: NonePatch Included: None

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 15 Aug 2015 01:01:24 PM UTC, comment #2:

Hi Darshit,

This is not the same concept as the persistent connections in HTTP (those with Connect: Keep-Alive).

What happens here is that if you download two documents from the same FTP server (ie. `wget ftp://site.com/file1.txt ftp://site.com/file2.txt`) Wget will open a connection, download the file, and close the connection again for each of the files. This is inefficient if we take into account that both files are stored in the same server (ie, site.com). It'd be better to open the connection to site.com once, and send two RETRs over the same connection, and then close the connection. It's a simple concept, in reality.

This happens because the traversal is done in recur.c, whereas the actual connection-download-close process is done in ftp.c, and there's no relationship between both.

This issue was among my favorites, but I won't work on it until at least one month, so feel free to fix this, if you want :D

Ander Juaristi <ajuaristi>
Sat 15 Aug 2015 07:18:04 AM UTC, comment #1:

I'm not very well versed with the FTP Protocol, so if I'm wrong here, someone please correct me.

Is re-using a FTP connection even possible? Because I haven't been able to find anything in the RFCs or through online searches to supplement this. I thought this was actually one of the largest drawbacks of the protocol that we need to open a lot of extra connections.

Can someone please point me to a resource explaining how this would work? If possible, I'd implement it.

Darshit Shah <darnir>
Project Administrator
Tue 04 Aug 2015 12:18:38 PM UTC, original submission:

Hello,

reporting an old bug/report from the mailing list:

if I use 'wget ftp://site.com/file1.txt ftp://site.com/file2.txt', wget will no reuse the ftp connection, but will open one for each document downloaded from the same site...

https://www.mail-archive.com/wget%40sunsite.dk/msg03316.html
https://bugs.debian.org/32353

thx and regards

Noël

Noël Köthe <nok>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by ajuaristi (Posted a comment)
  • -unavailable- added by darnir (Posted a comment)
  • -unavailable- added by nok (Submitted the item)
  • -unavailable- added by nok
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 15 Aug 2015 07:18:04 AM UTCdarnirStatusNone=>Needs Discussion
    Tue 04 Aug 2015 12:18:38 PM UTCnokCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup1