One of
the things that might be surprising in
the LINQ Distinct standard query operator is that it doesn’t automatically work properly on custom classes. There are reasons for this, which I’ll explain shortly.
The example I’ll use in this post focuses on pulling a unique list of names to load into a drop-down list. I’ll explain
the sample application, show you typical first shot at Distinct, explain why it won’t work as you expect, and then demonstrate a solution to make Distinct work with any custom class.
The technologies I’m using are LINQ to Twitter, LINQ to Objects, Telerik Extensions for ASP.NET MVC, ASP.NET MVC 2, and Visual Studio 2010.
The function of
the example program is to show a list of people that I follow. In Twitter API vernacular, these people are called “Friends”; though I’ve never met most of them in real life. This is part of
the ubiquitous language of social networking, and Twitter in particular, so you’ll see my objects named accordingly. Where Distinct comes into play is because I want to have a drop-down list with
the names of
the friends appearing in
the list. Some friends are quite verbose, which means I can’t just extract names from each tweet and populate
the drop-down; otherwise, I would end up with many duplicate names. Therefore, Distinct is
the appropriate operator to eliminate
the extra entries from my friends who tend to be enthusiastic tweeters.
The sample doesn’t do anything with
the drop-down list and I leave that up to imagination for what it’s practical purpose could be; perhaps a filter for
the list if I only want to see a certain person’s tweets or maybe a quick list that I plan to combine with a TextBox and Button to reply to a friend. When
the program runs, you’ll need to authenticate with Twitter, because I’m using OAuth (DotNetOpenAuth), for authentication, and then you’ll see
the drop-down list of names above
the grid with
the most recent tweets from friends. Here’s what
the application looks like when it runs:
As you can see, there is a drop-down list above
the grid.
The drop-down list is where most of
the focus of this article will be. There is some description of
the code before we talk about
the Distinct operator, but we’ll get there soon.
This is an ASP.NET MVC2 application, written with VS 2010. Here’s
the View that produces this screen:
<%@ Page Language="C#" MasterPageFile="~/Views/Shared/Site.Master" Inherits="System.Web.Mvc.ViewPage<TwitterFriendsViewModel>" %>
<%@ Import Namespace="DistinctSelectList.Models" %>
<asp:Content ID="Content1" ContentPlaceHolderID="TitleContent" runat="server">
Home Page
</asp:Content><asp:Content ID="Content2" ContentPlaceHolderID="MainContent" runat="server">
<fieldset>
<legend>Twitter Friends</legend>
<div>
<%= Html.DropDownListFor(
twendVM => twendVM.FriendNames,
Model.FriendNames,
"<All Friends>") %>
</div>
<div>
<% Html.Telerik().Grid<TweetViewModel>(Model.Tweets)
.Name("TwitterFriendsGrid")
.Columns(cols =>
{
cols.Template(col =>
{ %>
<img src="<%= col.ImageUrl %>"
alt="<%= col.ScreenName %>" />
<% });
cols.Bound(col => col.ScreenName);
cols.Bound(col => col.Tweet);
})
.Render(); %>
</div>
</fieldset>
</asp:Content>
As shown above,
the Grid is from Telerik’s Extensions for ASP.NET MVC.
The first column is a template that renders
the user’s Avatar from a URL provided by
the Twitter query. Both
the Grid and DropDownListFor display properties that are collections from a TwitterFriendsViewModel class, shown below:
using System.Collections.Generic;
using System.Web.Mvc;
namespace DistinctSelectList.Models
{
///
/// For finding friend info on screen
///
public class TwitterFriendsViewModel
{
///
/// Display names of friends in drop-down list
///
public List FriendNames { get; set; }
///
/// Display tweets in grid
///
public List Tweets { get; set; }
}
}
I created
the TwitterFreindsViewModel.
The two Lists are what
the View consumes to populate
the DropDownListFor and Grid. Notice that FriendNames is a List of SelectListItem, which is an MVC class. Another custom class I created is
the TweetViewModel (the type of
the Tweets List), shown below:
namespace DistinctSelectList.Models
{
///
/// Info on friend tweets
///
public class TweetViewModel
{
///
/// User's avatar
///
public string ImageUrl { get; set; }
///
/// User's Twitter name
///
public string ScreenName { get; set; }
///
/// Text containing user's tweet
///
public string Tweet { get; set; }
}
}
The initial Twitter query returns much more information than we need for our purposes and this a special class for displaying info in
the View. Now you know about
the View and how it’s constructed. Let’s look at
the controller next.
The controller for this demo performs authentication, data retrieval, data manipulation, and view selection. I’ll skip
the description of
the authentication because it’s a normal part of using OAuth with LINQ to Twitter. Instead, we’ll drill down and focus on
the Distinct operator. However, I’ll show you
the entire controller, below, so that you can see how it all fits together:
using System.Linq;
using System.Web.Mvc;
using DistinctSelectList.Models;
using LinqToTwitter;
namespace DistinctSelectList.Controllers
{
[HandleError]
public class HomeController : Controller
{
private MvcOAuthAuthorization auth;
private TwitterContext twitterCtx;
///
/// Display a list of friends current tweets
///
///
public ActionResult Index()
{
auth = new MvcOAuthAuthorization(InMemoryTokenManager.Instance, InMemoryTokenManager.AccessToken);
string accessToken = auth.CompleteAuthorize();
if (accessToken != null)
{
InMemoryTokenManager.AccessToken = accessToken;
}
if (auth.CachedCredentialsAvailable)
{
auth.SignOn();
}
else
{
return auth.BeginAuthorize();
}
twitterCtx = new TwitterContext(auth);
var friendTweets =
(from tweet in twitterCtx.Status
where tweet.Type == StatusType.Friends
select new TweetViewModel
{
ImageUrl = tweet.User.ProfileImageUrl,
ScreenName = tweet.User.Identifier.ScreenName,
Tweet = tweet.Text
})
.ToList();
var friendNames =
(from tweet in friendTweets
select new SelectListItem
{
Text = tweet.ScreenName,
Value = tweet.ScreenName
})
.Distinct()
.ToList();
var twendsVM = new TwitterFriendsViewModel
{
Tweets = friendTweets,
FriendNames = friendNames
};
return View(twendsVM);
}
public ActionResult About()
{
return View();
}
}
}
The important part of
the listing above are
the LINQ to Twitter queries for friendTweets and friendNames. Both of these results are used in
the subsequent population of
the twendsVM instance that is passed to
the view. Let’s dissect these two statements for clarification and focus on what is happening with Distinct.
The query for friendTweets gets a list of
the 20 most recent tweets (as specified by
the Twitter API for friend queries) and performs a projection into
the custom TweetViewModel class, repeated below for your convenience:
var friendTweets =
(from tweet in twitterCtx.Status
where tweet.Type == StatusType.Friends
select new TweetViewModel
{
ImageUrl = tweet.User.ProfileImageUrl,
ScreenName = tweet.User.Identifier.ScreenName,
Tweet = tweet.Text
})
.ToList();
The LINQ to Twitter query above simplifies what we need to work with in
the View and
the reduces
the amount of information we have to look at in subsequent queries. Given
the friendTweets above,
the next query performs another projection into an MVC SelectListItem, which is required for binding to
the DropDownList. This brings us to
the focus of this blog post, writing a correct query that uses
the Distinct operator.
The query below uses LINQ to Objects, querying
the friendTweets collection to get friendNames:
var friendNames =
(from tweet in friendTweets
select new SelectListItem
{
Text = tweet.ScreenName,
Value = tweet.ScreenName
})
.Distinct()
.ToList();
The above implementation of Distinct seems normal, but it is deceptively incorrect. After running
the query above, by executing
the application, you’ll notice that
the drop-down list contains many duplicates. This will send you back to
the code scratching your head, but there’s a reason why this happens.
To understand
the problem, we must examine how Distinct works in LINQ to Objects. Distinct has two overloads: one without parameters, as shown above, and another that takes a parameter of type IEqualityComparer<T>. In
the case above, no parameters, Distinct will call EqualityComparer<T>.Default
behind the scenes to make comparisons as it iterates through
the list. You don’t have problems with
the built-in types, such as string, int, DateTime, etc, because they all implement IEquatable<T>. However, many .NET Framework classes, such as SelectListItem, don’t implement IEquatable<T>. So, what happens is that EqualityComparer<T>.Default results in a call to Object.Equals, which performs reference equality on reference type objects. You don’t have this problem with value types because
the default implementation of Object.Equals is bitwise equality. However, most of your projections that use Distinct are on classes, just like
the SelectListItem used in this demo application. So,
the reason why Distinct didn’t produce
the results we wanted was because we used a type that doesn’t define its own equality and Distinct used
the default reference equality. This resulted in all objects being included in
the results because they are all separate instances in memory with unique references.
As you might have guessed,
the solution to
the problem is to use
the second overload of Distinct that accepts an IEqualityComparer<T> instance. If you were projecting into your own custom type, you could make that type implement IEqualityComparer<T>, but SelectListItem belongs to
the .NET Framework Class Library. Therefore,
the solution is to create a custom type to implement IEqualityComparer<T>, as in
the SelectListItemComparer class, shown below:
using System.Collections.Generic;
using System.Web.Mvc;
namespace DistinctSelectList.Models
{
public class SelectListItemComparer : EqualityComparer
{
public override bool Equals(SelectListItem x, SelectListItem y)
{
return x.Value.Equals(y.Value);
}
public override int GetHashCode(SelectListItem obj)
{
return obj.Value.GetHashCode();
}
}
}
The SelectListItemComparer class above doesn’t implement IEqualityComparer<SelectListItem>, but rather derives from EqualityComparer<SelectListItem>. Microsoft recommends this approach for consistency with
the behavior of generic collection classes. However, if your custom type already derives from a base class, go ahead and implement IEqualityComparer<T>, which will still work.
EqualityComparer is an abstract class, that implements IEqualityComparer<T> with Equals and GetHashCode abstract methods. For
the purposes of this application,
the SelectListItem.Value property is sufficient to determine if two items are equal. Since SelectListItem.Value is type string,
the code delegates equality to
the string class.
The code also delegates
the GetHashCode operation to
the string class.You might have other criteria in your own object and would need to define what it means for your object to be equal.
Now that we have an IEqualityComparer<SelectListItem>, let’s fix
the problem.
The code below modifies
the query where we want distinct values:
var friendNames =
(from tweet in friendTweets
select new SelectListItem
{
Text = tweet.ScreenName,
Value = tweet.ScreenName
})
.Distinct(new SelectListItemComparer())
.ToList();
Notice how
the code above passes a new instance of SelectListItemComparer as
the parameter to
the Distinct operator. Now, when you run
the application,
the drop-down list will behave as you expect, showing only a unique set of names.
In addition to Distinct, other LINQ Standard Query Operators have overloads that accept IEqualityComparer<T>’s, You can use
the same techniques as shown here, with SelectListItemComparer, with those other operators as well. Now you know how to resolve problems with getting Distinct to work properly and also have a way to fix problems with other operators that require equality comparisons.
@JoeMayo
