How to detect if 2 news articles have the same topic? (Python language-comparison)

Posted by resopollution on Stack Overflow See other posts from Stack Overflow or by resopollution
Published on 2010-04-05T18:52:38Z Indexed on 2010/04/05 18:53 UTC
Read the original article Hit count: 305

Filed under:

python

|

comparison

|

language-comparisons

I'm looking for ideas on recommended approach.

I'm trying to scrape some headlines and body text from articles for a few specific sites, similar to what Google does with Google News.

The problem is across different sites, they may have articles on the same exact subject, worded slightly differently.

Can anyone point to me what I need to know in order to write a comparison algorithm to auto-detect similar articles?

Thanks very much in advance.

I use Python.

© Stack Overflow or respective owner

Related posts about python

unmet dependencies in Ubuntu 12.04

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I tried today to install a dvb-card on my Ubuntu 12.04 (Linux blauhai-linux 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux ). The installation failed with an error. After that, i tried to install python (it was already installed but i got this error): linux:~$… >>> More
How can I get sikuli-ide to work?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I installed sikuli-ide with sudo apt-get install sikuli-ide Everything was fine until I tried to start it from the terminal. I typed sikuli-ide But the only response I got was [info] locale: en_US The application was not started, furthermore there is no desktop file and sikuli-ide does not… >>> More
Getting PATH right for python after MacPorts install

as seen on Super User - Search for 'Super User'
I can't import some python libraries (PIL, psycopg2) that I just installed with MacPorts. I looked through these forums, and tried to adjust my PATH variable in $HOME/.bash_profile in order to fix this but it did not work. I added the location of PIL and psycopg2 to PATH. I know that Terminal is… >>> More
call python with system() in R to run a python script emulating the python console

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to pass a chunk of Python code to Python in R with something like system('python ...'), and I'm wondering if there is an easy way to emulate the python console in this case. For example, suppose the code is "print 'hello world'", how can I get the output like this in R? >>> print… >>> More
Python - Calling a non python program from python?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am currently struggling to call a non python program from a python script. I have a ~1000 files that when passed through this C++ program will generate ~1000 outputs. Each output file must have a distinct name. The command I wish to run is of the form: program_name -input -output -o1 -o2… >>> More

Related posts about comparison

Store comparison in variable (or execute comparison when it's given as an string)

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello everyone. I'd like to know if the super-powerful python allows to store a comparison in a variable or, if not, if it's possible calling/executing a comparison when given as an string ("==" or "!=") I want to allow the users of my program the chance of giving a comparison in an string. For… >>> More
On StringComparison Values

as seen on Geeks with Blogs - Search for 'Geeks with Blogs'
When you use the .NET Framework’s String.Equals and String.Compare methods do you use an overloStringComparison enumeration value? If not, you should be because the value provided for that StringComparison argument can have a big impact on the results of your string comparison. The StringComparison… >>> More
Use Comparison Operators in SQL to Remove/Return Columns with Null Values

as seen on Internet.com - Search for 'Internet.com'
When you use the comparison operators in SQL with the Where clause, you can remove the columns with null values. >>> More
Use Comparison Operators in SQL to Remove/Return Columns with Null Values

as seen on Internet.com - Search for 'Internet.com'
When you use the comparison operators in SQL with the Where clause, you can remove the columns with null values. >>> More
Building a Web Form in ASP.NET and PHP: a Comparison

as seen on ASP Free - Search for 'ASP Free'
While there are important differences between PHP and ASP.NET both are used to build websites. Because of this both need to enable developers to build web forms among other tasks. This article compares building a web form in PHP with building the same form in ASP.NET to help those familiar with… >>> More