bugGNU Wget - Bugs: bug #49458, Please search txt files named...

 
 

bug #49458: Please search txt files named ".ram" for links

Submitted by:  None
Submitted on:  Thu 27 Oct 2016 04:11:13 PM UTC  
 
Category: Feature RequestSeverity: 3 - Normal
Priority: 5 - NormalStatus: Wont Fix
Privacy: PublicAssigned to: None
Originator Name: powerKittenOriginator Email: -unavailable-
Open/Closed: ClosedRelease: 1.18
Operating System: NoneReproducibility: Every Time
Fixed Release: NonePlanned Release: None
Regression: NoWork Required: None
Patch Included: No

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 28 Oct 2016 07:36:44 PM UTC, comment #4:

Really, this opens up the can of worms of making which files are scanned for URLs configurable.

Ideally, the user could supply a list of extensions/MIME types and provide corresponding programs that would extract the URLs from the files.

But you would also want to implement --convert-links for these files, and that would be considerably more complicated.

Dale Worley <worley>
Fri 28 Oct 2016 07:53:53 AM UTC, comment #3:

This URL(s) within the .ram/.ra file likely points to some audio stream - it might be a radio station with changing (and endless !) content.
'Downloading' the contents could easily lead to a 'hanging' wget. That is perhaps not what you want.
--spider could at least 'check' that URL.
And the ICY-Metadata HTTP header makes a difference since the remote side likely is a shoutcast server.

BTW, there are lot's of other file formats that contain URLs.
E.g. Wget2 reads URLs from sitemaps and sitemap catalogs (compressed or uncompressed), from Atom and RSS feeds.
There is also examples/getstream.c that 'downloads' URLs (streams) from .ram, .m3u, .wax, .asx, .pls, .xspf (audio/x-mpegurl, audio/x-pn-realaudio, application/pls+xml, audio/x-scpls, application/xspf+xml).

Tim Ruehsen <rockdaboot>
Project Administrator
Thu 27 Oct 2016 05:03:10 PM UTC, comment #2:

Sorry, I didn't really word it that well. The idea was for it to scan .ram text files for resources when mirroring a site using the -m flag, as right now I have to make two scans when mirroring a site that uses .ram assets to a WARC.

Patrick Alan Marshall <powerkitten>
Thu 27 Oct 2016 04:55:34 PM UTC, comment #1:

I don't exactly know what you're asking, but I don't think it would be a good fit for wget given that you could easily automate it with a shell script.

The closest enhancement I can come across is to make -i accept multiple files, which, to the best of my knowledge, does not yet.

E.g. wget -i file1.ram,file2.ram,foo.txt,bar.html

What do you all think?

Ander Juaristi <ajuaristi>
Thu 27 Oct 2016 04:11:13 PM UTC, original submission:

The .ram "format" is simply an ANSI encoded text file. It consists of links to RealAudio resources. I think that it would benefit from this feature, as currently I have to go into each .ram file, copy the links into a list of urls, and then fetch them using the -i flag. I've attached a sample .ram file which can also be found at the (archived) address: http://web.archive.org/web/20161027160920/http://www.theorderoftime.com/art/music/ra-popular/theimportance.ram
or the original address:
http://www.theorderoftime.com/art/music/ra-popular/theimportance.ram

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #38823:  theimportance.ram added by None (64B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by worley (Posted a comment)
  • -unavailable- added by rockdaboot (Posted a comment)
  • -unavailable- added by powerkitten (Posted a comment)
  • -unavailable- added by ajuaristi (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 23 Nov 2016 02:59:14 PM UTCrockdabootStatusNone=>Wont Fix
      Open/ClosedOpen=>Closed
    Fri 28 Oct 2016 07:53:53 AM UTCrockdabootRelease1.17.1=>1.18
      Operating SystemMicrosoft Windows=>None
    Thu 27 Oct 2016 04:11:13 PM UTCNoneAttached File-=>Added theimportance.ram, #38823

    Back to the top


    Powered by Savane 3.1-cleanup1