bugGNU Wget - Bugs: bug #41741, page-requisites not working...

 
 

bug #41741: page-requisites not working another site

Submitted by:  None
Submitted on:  Sat 01 Mar 2014 11:39:17 AM UTC  
 
Category: Code ArchitectureSeverity: 3 - Normal
Priority: 5 - NormalStatus: Invalid
Privacy: PublicAssigned to: None
Originator Name: page-requisites not working another siteOriginator Email: -unavailable-
Open/Closed: ClosedRelease: trunk
Operating System: GNU/LinuxReproducibility: None
Fixed Release: NonePlanned Release: None
Regression: NoneWork Required: None
Patch Included: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 08 Jun 2014 02:37:00 PM UTC, comment #1:

The issue occurs because the site's robots.txt explicitly forbids access to all the directories that contain the page requisites.

If you still want to download them, you can use the `-e robots=off` command to force Wget to not respect the robots.txt file. Such behaviour is however discouraged.

Darshit Shah <darnir>
Project Member
Sat 01 Mar 2014 11:39:17 AM UTC, original submission:

--page-requisites not working another site

myrubapa@myrubapa:/var/www$ wget -p http://startandroid.ru/ 2>&1 |grep saved
2014-03-01 16:35:16 (56.5 KB/s) - ‘startandroid.ru/index.html’ saved [64735]
2014-03-01 16:35:16 (4.78 MB/s) - ‘startandroid.ru/robots.txt’ saved [865/865]
myrubapa@myrubapa:/var/www$

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by darnir (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 08 Jun 2014 02:37:00 PM UTCdarnirStatusNone=>Invalid
      Open/ClosedOpen=>Closed
    Sat 01 Mar 2014 11:39:17 AM UTCNoneAttached File-=>Added Учебник по Андроид. Уроки для начинающих.html, #30743

    Back to the top


    Powered by Savane 3.1-cleanup