bugGNU FM - Bugs: bug #32936, lastscrape: BeautifulSoup fails to...

 
 

bug #32936: lastscrape: BeautifulSoup fails to parse last.fm pages

Submitted by:  Petr Viktorin <encukou>
Submitted on:  Mon 28 Mar 2011 01:16:45 PM UTC  
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 09 Oct 2011 08:39:07 AM UTC, comment #2: 

  I was getting the following error using the patch:
"UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 238: ordinal not in range(128)"
  So I replaced the following line:
    contents = contents.replace(u'<!–[if IE]><![endif]–>', '')
With:
    contents = contents.replace('<!–[if IE]><![endif]–>', '')

Stephen Niedzielski <niedzielski>
Mon 28 Mar 2011 01:57:26 PM UTC, comment #1: 

The problem is caused by a weird comment: <!–[if IE]><![endif]–> (with en-dashes instead of two hyphen-minuses) on line 9 of every last.fm page. Removing this no-op comment makes everything work again.

I believe the attached patch is trivial enough so you can accept it without a copyright assignment. (And if not, it's trivial to write from scratch.)

I filed a bug in BeautifulSoup at https://bugs.launchpad.net/beautifulsoup/+bug/744278 – if that gets fixed no workarounds are necessary.

Also, BeautifulSoup 3.2 works. Lastscrape's README should be updated accordingly.

(file #23036)

Petr Viktorin <encukou>
Mon 28 Mar 2011 01:16:45 PM UTC, original submission:  

Apparently, last.fm made a subtle change to their pages that prevents BeautifulSoup from parsing the scrobbles correctly.

librefm/trunk/lastscrape$ ./lastscrape.py encukou lastfm_dump.txt
Traceback (most recent call last):
  File "./lastscrape.py", line 98, in <module>
    sys.exit(main(*sys.argv))
  File "./lastscrape.py", line 89, in main
    for artist, track, timestamp in fetch_tracks(args[1]):
  File "./lastscrape.py", line 75, in fetch_tracks
    for artist, track, timestamp in tracks:
  File "./lastscrape.py", line 34, in parse_page
    for row in soup.find('table', 'candyStriped tracklist').findAll('tr'):
AttributeError: 'NoneType' object has no attribute 'findAll'

(on a side note, BeautifulSoup 3.2 has been out for some while; it might make sense to tell users to use that instead of the 3.0.7a if it's made to work)

Petr Viktorin <encukou>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #23036:  lastscrape.diff added by encukou (1KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by niedzielski (Posted a comment)
  • -email is unavailable- added by encukou (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-03-28 encukou Attached File- => Added lastscrape.diff, #23036

    Back to the top


    Powered by Savane 3.7