How to optimize an database suggestion engine

Posted by Dimitar Vouldjeff on Stack Overflow See other posts from Stack Overflow or by Dimitar Vouldjeff
Published on 2010-04-16T21:32:20Z Indexed on 2010/04/16 21:33 UTC
Read the original article Hit count: 258

Filed under:

collaborative-filtering

|

suggestions

|

correlation

|

optimization

Hi, I`m making an online engine for item-to-item recommending movies. I have made some researches and I think that the best way to implement that is using pearson correlation and make a table with item1, item2 and correlation fields, but the problem is that after each rate of item I have to regenerate the correlation for in the worst case N records (where N is the number of items).

Another think that I read is the following article, but I haven`t thought a way to implement it.

So what is your suggestion to optimize this process? Or any other suggestions? Thanks.

© Stack Overflow or respective owner

Related posts about collaborative-filtering

collaborative filtering in rails

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm looking for a solution for collaborative filtering in rails or even possible examples. So far I have only found acts_as_recommendable which looks useful but I noticed it hasn't had any updates in the last 2 years. Does anyone know of any other solutions and/or examples? >>> More
Collaborative Filtering Program: What to do for a Pearson Score When There Isn't Enough Data

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm building a recommendation engine using collaborative filtering. For similarity scores, I use a Pearson correlation. This is great most of the time, but sometimes I have users that only share a 1 or 2 fields. For example: User 1{ a: 4 b: 2 } User 2{ a: 4 b: 3 } Since this is only 2 data points… >>> More
Converting python collaborative filtering code to use Map Reduce

as seen on Stack Overflow - Search for 'Stack Overflow'
Using Python, I'm computing cosine similarity across items. given event data that represents a purchase (user,item), I have a list of all items 'bought' by my users. Given this input data (user,item) X,1 X,2 Y,1 Y,2 Z,2 Z,3 I build a python dictionary {1: ['X','Y'], 2 : ['X','Y','Z'], 3 : ['Z']} From… >>> More
Collaborative filtering in MySQL ?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi I'm trying to develop a site that recommends items(fx. books) to users based on their preferences. So far, I've read O'Reilly's "Collective Intelligence" and numerous other online articles. They all, however, seem to deal with single instances of recommendation, for example if you like book A… >>> More
Best similarity metric for collaborative filtering?

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to decide on the best similarity metric for a product recommendation system using item-based collaborative filtering. This is a shopping basket scenario where ratings are binary valued - the user has either purchased an item or not - there is no explicit rating system (eg, 5-stars). Step… >>> More

Related posts about suggestions

How to introduce web development to non-programmers?

as seen on Programmers - Search for 'Programmers'
Once one of my non-programmer friends asked, "I have a cool website idea that I don't want to share. Rather I want to develop it on my own. So, I want to learn web development. Tell me what to do?" And sometimes many other people asked about how to start with web development as a profession. But… >>> More
How can I find the approximate daily traffic of a site which I don't own?

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I want to find the approximate daily traffic of a site which isn't ours, and the site is located in other country than US (in Greece - hence no Quantcast or Compete.com afaik) and it doesn't use Google Ads (hence no Google Ad Planner). I know about Alexa but the site(s) has/have relatively low traffic… >>> More
Looking for a user suggestion script

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I'm looking for a script that would easily fit or be customized to work as a suggestion script where users can submit suggestions anonymously and have them be displayed on one page and either upvoted or downvoted like digg or reddit without user accounts. Basically i want users to be able to submit… >>> More
What about introduction to programming with C# via LINQPad?

as seen on Programmers - Search for 'Programmers'
From different questions/answers/articles in this and some other sites, I got the idea that the introductory language for programming should be- High level Less verbose C# is one of the heavily used high level languages being used these days. It's also multi-paradigm and descendant of C, the lingua-franca… >>> More
Beginners Tips To Learn Vim

as seen on Programmers - Search for 'Programmers'
I'm the type of developer that only uses GUI fully-featured programmer editor, when I'm at Windows I use Notepad++, at my Mac I use TextMate and at Linux I use GEdit, but now I'm starting to develop inside my Linux server, which doesn't have any window manager installed and I saw this as a beautiful… >>> More