Our database is filled with articles retrieved from RSS feeds. I was unsure
of what data I would be getting, and how much filtering was already setup (WP-O-Matic Wordpress plugin using the SimplePie library). This plugin does some basic encoding before insertion using Wordpress's built in post insert function which also does some filtering. I've figured out most
of the filters before insertion, but now I have whacko data that I need to remove.
This is an example
of whacko data that I have data in one field which the content I want in the front, but this part removed which is at the end:
<img src="http://feeds.feedburner.com/~ff/SoundOnTheSound?i=xFxEpT2Add0:xFbIkwGc-fk:V_sGLiPBpWU" border="0"></img>
<img src="http://feeds.feedburner.com/~ff/SoundOnTheSound?d=qj6IDK7rITs" border="0"></img>
<img src="http://feeds.feedburner.com/~ff/SoundOnTheSound?i=xFxEpT2Add0:xFbIkwGc-fk:D7DqB2pKExk"
Notice how some
of the images are escape and some aren't. I believe this has to do with the last part being cut off so as to be unrecognizable as an html tag, which then caused it to be html endcoded.
Another field has only this which is now filtered before insertion, but I have to get rid
of the others:
<img src="http://farm3.static.flickr.com/2183/2289902369_1d95bcdb85.jpg" alt="post_img" width="80"
(all examples are on one line, but broken up for readability)
Question: What is the
best way to work with the above escaped html (or portion
of an html tag)?
I can do it in Perl, PHP, SQL, Ruby, and even Python. I believe Perl to be the
best at text parsing, so that's why I used the Perl tag. And PHP times out on large database operations, so that's pretty much out unless I wanted to do batch processing and what not.
PS One
of the nice things about using Wordpress's insert post function, is that if you use php's strip_tags function to strip out all html, insert post function will insert <p> at the paragraph points.
Let me know if there's anything more that I can answer.
Some article that didn't quite answer my questions.
(http://stackoverflow.com/questions/2016751/remove-text-from-within-a-database-text-field)
(http://stackoverflow.com/questions/462831/regular-expression-to-escape-html-ampersands-while-respecting-cdata)
