Simple NLP: How to use ngram to do word similarity?

Posted by sadawd on Stack Overflow See other posts from Stack Overflow or by sadawd
Published on 2010-03-15T03:39:08Z Indexed on 2010/03/15 3:49 UTC
Read the original article Hit count: 325

Filed under:

nlp

Dear Everyone, I Hear that google uses up to 7-grams for their own data. I am interested in finding words that are similar in context (i.e. cat and dog) and I was wondering how do I compute the similarity of two words on a n-gram model given that n > 2.

Given a sample set like this forexample: (I, love cats), (cats, loves, dogs), (dogs, hate, human)

What is a good way to compare the similarity of this pair (I, cats)?

Also does anyone know of anyway to do levels for NLP? like: Army->Military->Solider ?

Related posts about nlp

stanford pos tagger runs out of memory?

as seen on Stack Overflow - Search for 'Stack Overflow'
my stanford tagger ran out of memory. Is it because the text has to be properly formatted? This is because i use it to tag html contents, with the tags stripped, but there may have quite a excessive amount of newlines. here is the error: BlockquoWARNING: Untokenizable: ? (char in decimal: 9829) … >>> More
NLP with greatly contrained input and abilities

as seen on Stack Overflow - Search for 'Stack Overflow'
Hat in hand here. I'm a seasoned developer and I would be grateful for a bit of help. I don't have time to read or digest long intricate discussions on theoretical concepts around NLP (or go get my PHD). That said, I have read a few and it's a damn interesting field. The problem is I need real world… >>> More
NLP - Word Alignment

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi..:) I am looking for word alignment tools and algorithms, I am dealing with bilingual English - Hindi text, Currently I am working on DTW(Dynamic Time Warping) algorithm, CLA(Competitive Linking Algorithm) , NATool, Giza++. Could you please suggest me any other alogrithm/tool which is language… >>> More
AGFL npx grammar nlp techniques dependency parsing

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi I am trying to obtain a dependency parse tree using AGFL. Unfortunately I cannot understand how to derive this. I am trying to generate the npx grammar but I am still lost can someone help me please? Thanks :) L >>> More
Starting out NLP - Python + large data set

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I've been wanting to learn python and do some NLP, so have finally gotten round to starting. Downloaded the english wikipedia mirror for a nice chunky dataset to start on, and have been playing around a bit, at this stage just getting some of it into a sqlite db (havent worked with dbs in the… >>> More

Developer IT

Simple NLP: How to use ngram to do word similarity? - Developer IT

Simple NLP: How to use ngram to do word similarity?

nlp

Related posts about nlp

stanford pos tagger runs out of memory?

NLP with greatly contrained input and abilities

NLP - Word Alignment

AGFL npx grammar nlp techniques dependency parsing

Starting out NLP - Python + large data set

Categories cloud