I need to remove all the html tags from a given webpage data. I tried this using regular expressions:
import urllib2
import re
page = urllib2.urlopen("http://www.frugalrules.com")
from bs4 import BeautifulSoup, NavigableString, Comment
soup = BeautifulSoup(page)
link = soup.find('link', type='application/rss+xml')
print link['href']
rss = urllib2.urlopen(link['href']).read()
souprss = BeautifulSoup(rss)
description_tag = souprss.find_all('description')
content_tag = souprss.find_all('content:encoded')
print re.sub('<[^>]*>', '', content_tag)
But the syntax of the re.sub is:
re.sub(pattern, repl, string, count=0)
So, I modified the code as (instead of the print statement above):
for row in content_tag:
print re.sub(ur"<[^>]*>",'',row,re.UNICODE
But it gives the following error:
Traceback (most recent call last):
File "C:\beautifulsoup4-4.3.2\collocation.py", line 20, in <module>
print re.sub(ur"<[^>]*>",'',row,re.UNICODE)
File "C:\Python27\lib\re.py", line 151, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer
What am I doing wrong?
