Using an alternate search platform in Commerce Server 2009
- by Lewis Benge
Although Microsoft Commerce Server 2009's architecture is built upon Microsoft SQL Server, and has the full power of the SQL Full Text Indexing Search Platform, there are time however when you may require a richer or alternate search platform. One of these scenarios if when you want to implement a faceted (refinement) search into your site, which provides dynamic refinements based on the search results dataset.
Faceted search is becoming popular in most online retail environments as a way of providing an enhanced user experience when browsing a larger catalogue. This is powerful for two reasons, firstly with a traditional search it is down to a user to think of a search term suitable for the product they are trying to find. This typically will not return similar products or help in any way to refine a larger dataset. Faceted searches on the other hand provide a comprehensive list of product properties, grouped together by similarity to help the user narrow down the results returned, as the user progressively restricts the search criteria by selecting additional criteria to search again, these facets needs to continually refresh. The whole experience allows users to explore alternate brands, price-ranges, or find products they hadn't initially thought of or where looking for in a bid to enhance cross sell in the retail environment.
The second advantage of this type of search from a business perspective is also to harvest the search result to start to profile your user. Even though anonymous users may routinely visit your site, and will not necessarily register or complete a transaction to build up marketing data- profiling, you can still achieve the same result by recording search facets used within the search sequence. Below is a faceted search scenario generated from eBay using the search term "server". By creating a search profile of clicking through Computer & Networking -> Servers -> Dell - > New and recording this information against my user profile you can start to predict with a lot more certainty what types of products I am interested in. This will allow you to apply shopping-cart analysis against your search data and provide great cross-sale or advertising opportunity, or personalise the user experience based on your prediction of what the user may be interested in.
This type of search is extremely beneficial in e-Commerce environments but achieving it out of the box with Commerce Server and SQL Full Text indexing can be challenging. In many deployments it is often easier to use an alternate search platform such as Microsoft's FAST,
Apache SOLR, or Endecca, however you still want these products to integrate natively into Commerce Server to ensure that up-to-date inventory information is presented, profile information is generated, and you provide a consistant API. To do so we make the most of the Commerce Server extensibilty points called operation sequence components.
In this example I will be talking about Apache Solr hosted on Apache Tomcat, in this specific example I have used the SolrNet C# library to interface to the Java platform. Also I am not going to talk about Solr configuration of indexing – but in a production envionrment this would typically happen by using Powershell to call the Commerce Server management webservice to export your catalog as XML, apply an XSLT transform to the file to make it conform to SOLR and use a simple HTTP Post to send it to the search enginge for indexing.
Essentially a sequance component is a step in a serial workflow used to call a data repository (which in most cases is usually the Commerce Server pipelines or databases) and map to and from a Commerce Entity object whilst enforcing any business rules. So the first step in the process is to add a new class library to your existing Commerce Server site. You will need to use a new library as Sequence Components will need to be strongly named to be deployed.
Once you are inside of your new project, add a new class file and add a reference to the Microsoft.Commerce.Providers, Microsoft.Commerce.Contracts and the Microsoft.Commerce.Broker assemblies. Now make your new class derive from the base object Microsoft.Commerce.Providers.Components.OperationSequanceComponent and overide the ExecuteQueryMethod. Your screen will then look something similar ot this:
As all we are doing on this component is conducting a search we are only interested in the ExecuteQuery method. This method accepts three arguments, queryOperation, operationCache, and response. The queryOperation will be the object in which we receive our search parameters, the cache allows access to the Commerce Server cache allowing us to store regulary accessed information, and the response object is the object which we will return the result of our search upon. Inside this method is simply where we are going to inject our logic for our third party search platform. As I am not going to explain the inner-workings of actually making a SOLR call, I'll simply provide the sample code here. I would highly recommend however looking at the SolrNet wiki as they have some great explinations of how the API works.
What you will find however is that there are some further extensions required when attempting to integrate a custom search provider. Firstly you out of the box the CommerceQueryOperation you will receive into the method when conducting a search against a catalog is specifically geared towards a SQL Full Text Search with properties such as a Where clause. To make the operation you receive more relevant you will need to create another class, this time derived from Microsoft.Commerce.Contract.Messages.CommerceSearchCriteria and within this you need to detail the properties you will require to allow you to submit as parameters to the SOLR search API. My exmaple looks like this:
[DataContract(Namespace = "http://schemas.microsoft.com/microsoft-multi-channel-commerce-foundation/types/2008/03")]
public
class
CommerceCatalogSolrSearch : CommerceSearchCriteria
{
private
Dictionary<string, string> _facetQueries;
public CommerceCatalogSolrSearch()
{
_facetQueries = new
Dictionary<String, String>();
}
public
Dictionary<String, String> FacetQueries
{
get { return _facetQueries; }
set { _facetQueries = value; }
}
public
String SearchPhrase{ get; set; }
public
int PageIndex { get; set; }
public
int PageSize { get; set; }
public
IEnumerable<String> Facets { get; set; }
public
string Sort { get; set; }
public
new
int FirstItemIndex
{
get
{
return (PageIndex-1)*PageSize;
}
}
public
int LastItemIndex
{
get
{
return FirstItemIndex + PageSize;
}
}
}
To allow you to construct a CommerceQueryOperation call within the API you will also need to construct another class to derived from Microsoft.Commerce.Common.MessageBuilders.CommerceSearchCriteriaBuilder and is simply used to construct an instance of the CommerceQueryOperation you have just created and expose the properties you want set. My Message builder looks like this:
public
class
CommerceCatalogSolrSearchBuilder : CommerceSearchCriteriaBuilder
{
private CommerceCatalogSolrSearch _solrSearch;
public CommerceCatalogSolrSearchBuilder()
{
_solrSearch = new CommerceCatalogSolrSearch();
}
public
String SearchPhrase
{
get { return _solrSearch.SearchPhrase; }
set { _solrSearch.SearchPhrase = value; }
}
public
int PageIndex
{
get { return _solrSearch.PageIndex; }
set { _solrSearch.PageIndex = value; }
}
public
int PageSize
{
get { return _solrSearch.PageSize; }
set { _solrSearch.PageSize = value; }
}
public
Dictionary<String,String> FacetQueries
{
get { return _solrSearch.FacetQueries; }
set { _solrSearch.FacetQueries = value; }
}
public
String[] Facets
{
get { return _solrSearch.Facets.ToArray(); }
set { _solrSearch.Facets = value; }
}
public
override CommerceSearchCriteria ToSearchCriteria()
{
return _solrSearch;
}
}
Once you have these two classes in place you can now safely cast the CommerceOperation you receive as an argument of the overidden ExecuteQuery method in the SequenceComponent to the CommerceCatalogSolrSearch operation you have just created, e.g.
public CommerceCatalogSolrSearch TryGetSearchCriteria(CommerceOperation operation)
{
var searchCriteria = operation as CommerceQueryOperation;
if (searchCriteria == null)
throw
new
Exception("No search criteria present");
var local = (CommerceCatalogSolrSearch) searchCriteria.SearchCriteria;
if (local == null)
throw
new
Exception("Unexpected Search Criteria in Operation");
return local;
}
Now you have all of your search parameters present, you can go off an call the external search platform API. You will of-course get proprietry objects returned, so the next step in the process is to convert the results being returned back into CommerceEntities. You do this via another extensibility point within the Commerce Server API called translatators. Translators are another separate class, this time derived inheriting the interface Microsoft.Commerce.Providers.Translators.IToCommerceEntityTranslator . As you can imaginge this interface is specific for the conversion of the object TO a CommerceEntity, you will need to implement a separate interface if you also need to go in the opposite direction. If you implement the required method for the interace you will get a single translate method which has a source onkect, destination CommerceEntity, and a collection of properties as arguments. For simplicity sake in this example I have hard-coded the mappings, however best practice would dictate you map the objects using your metadatadefintions.xml file . Once complete your translator would look something like the following:
public
class
SolrEntityTranslator : IToCommerceEntityTranslator
{
#region IToCommerceEntityTranslator Members
public
void Translate(object source, CommerceEntity destinationCommerceEntity,
CommercePropertyCollection propertiesToReturn)
{
if (source.GetType().Equals(typeof (SearchProduct)))
{
var searchResult = (SearchProduct) source;
destinationCommerceEntity.Id = searchResult.ProductId;
destinationCommerceEntity.SetPropertyValue("DisplayName", searchResult.Title);
destinationCommerceEntity.ModelName = "Product";
}
}
Once you have a translator in place you can then safely map the results of your search platform into Commerce Entities and attach them on to the CommerceResponse object in a fashion similar to this:
foreach (SearchProduct result in matchingProducts)
{
var destinationEntity = new CommerceEntity(_returnModelName);
Translator.ToCommerceEntity(result, destinationEntity, _queryOperation.Model.Properties);
response.CommerceEntities.Add(destinationEntity);
}
In SOLR I actually have two objects being returned – a product, and a collection of facets so I have an additional translator for facet (which maps to a custom facet CommerceEntity) and my facet response from SOLR is passed into the Translator helper class seperatley.
When all of this is pieced together you have sucessfully completed the extensiblity point coding. You would have created a new OperationSequanceComponent, a custom SearchCritiera object and message builder class, and translators to convert the objects into Commerce Entities. Now you simply need to configure them, and can start calling them in your code. Make sure you sign you assembly, compile it and identiy its signature.
Next you need to put this a reference of your new assembly into the Channel.Config configuration file replacing that of the existing SQL Full Text component:
You will also need to add your translators to the Translators node of your Channel.Config too:
Lastly add any custom CommerceEntities you have developed to your MetaDataDefintions.xml file.
Your configuration is now complete, and you should now be able to happily make a call to the Commerce Foundation API, which will act as a proxy to your third party search platform and return back CommerceEntities of your search results. If you require data to be enriched, or logged, or any other logic applied then simply add further sequence components into the OperationSequence (obviously keeping the search response first) to the node of your Channel.Config file.
Now to call your code you simply request it as per any other CommerceQuery operation, but taking into account you may be receiving multiple types of CommerceEntity returned:
public
KeyValuePair<FacetCollection ,List<Product>> DoFacetedProductQuerySearch(string searchPhrase, string orderKey, string sortOrder,
int recordIndex, int recordsPerPage, Dictionary<string, string> facetQueries, out
int totalItemCount)
{
var products = new
List<Product>();
var query = new CommerceQuery<CatalogEntity, CommerceCatalogSolrSearchBuilder>();
query.SearchCriteria.PageIndex = recordIndex;
query.SearchCriteria.PageSize = recordsPerPage;
query.SearchCriteria.SearchPhrase = searchPhrase;
query.SearchCriteria.FacetQueries = facetQueries;
totalItemCount = 0;
CommerceResponse response = SiteContext.ProcessRequest(query.ToRequest());
var queryResponse = response.OperationResponses[0] as CommerceQueryOperationResponse;
// No results. Return the empty list
if (queryResponse != null && queryResponse.CommerceEntities.Count == 0)
return
new
KeyValuePair<FacetCollection, List<Product>>();
totalItemCount = (int)queryResponse.TotalItemCount;
// Prepare a multi-operation to retrieve the product variants
var multiOperation = new CommerceMultiOperation();
//Add products to results
foreach (Product product in queryResponse.CommerceEntities.Where(x => x.ModelName == "Product"))
{
var productQuery = new CommerceQuery<Product>(Product.ModelNameDefinition);
productQuery.SearchCriteria.Model.Id = product.Id;
productQuery.SearchCriteria.Model.CatalogId = product.CatalogId;
var variantQuery = new CommerceQueryRelatedItem<Variant>(Product.RelationshipName.Variants);
productQuery.RelatedOperations.Add(variantQuery);
multiOperation.Add(productQuery);
}
CommerceResponse variantsResponse = SiteContext.ProcessRequest(multiOperation.ToRequest());
foreach (CommerceQueryOperationResponse queryOpResponse in variantsResponse.OperationResponses)
{
if (queryOpResponse.CommerceEntities.Count() > 0)
products.Add(queryOpResponse.CommerceEntities[0]);
}
//Get facet collection
FacetCollection facetCollection =
queryResponse.CommerceEntities.Where(x => x.ModelName == "FacetCollection").FirstOrDefault();
return
new
KeyValuePair<FacetCollection, List<Product>>(facetCollection, products);
}
..And that is it – simply a few classes and some configuration will allow you to extend the Commerce Server query operations to call a third party search platform, whilst still maintaing a unifed API in the remainder of your code.
This logic stands for any extensibility within CommerceServer, which requires excution in a serial fashioon such as call to LOB systems or web service to validate or enrich data. Feel free to use this example on other applications, and if you have any questions please feel free to e-mail and I'll help out where I can!
