I don't know about you but I frequently need property bags in my applications
to store and possibly cache arbitrary data. Dictionary<T,V> works well for this although I always seem
to be hunting for a more specific generic type that provides a string key based dictionary. There's string dictionary, but it only works with strings. There's Hashset<T> but it uses the actual values as keys. In most key value pair situations for me string is key value
to work off. Dictionary<T,V> works well enough, but there are some issues with serialization of dictionaries in .NET. The .NET framework doesn't do well serializing IDictionary objects out of the box. The XmlSerializer doesn't support serialization of IDictionary via it's default serialization, and while the DataContractSerializer does support IDictionary serialization it produces some pretty atrocious XML. What doesn't work? First off Dictionary serialization with the Xml Serializer doesn't work so the following fails: [TestMethod]
public void DictionaryXmlSerializerTest()
{
var bag = new Dictionary<string, object>();
bag.Add("key", "Value");
bag.Add("Key2", 100.10M);
bag.Add("Key3", Guid.NewGuid());
bag.Add("Key4", DateTime.Now);
bag.Add("Key5", true);
bag.Add("Key7", new byte[3] { 42, 45, 66 });
TestContext.WriteLine(this.ToXml(bag));
}
public string ToXml(object obj)
{
if (obj == null)
return null;
StringWriter sw = new StringWriter();
XmlSerializer ser = new XmlSerializer(obj.GetType());
ser.Serialize(sw, obj);
return sw.ToString();
}
The error you get with this is:
System.NotSupportedException: The type System.Collections.Generic.Dictionary`2[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Object, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] is not supported because it implements IDictionary.
Got it! BTW, the same is true with binary serialization.
Running the same code above against the DataContractSerializer does work:
[TestMethod]
public void DictionaryDataContextSerializerTest()
{
var bag = new Dictionary<string, object>();
bag.Add("key", "Value");
bag.Add("Key2", 100.10M);
bag.Add("Key3", Guid.NewGuid());
bag.Add("Key4", DateTime.Now);
bag.Add("Key5", true);
bag.Add("Key7", new byte[3] { 42, 45, 66 });
TestContext.WriteLine(this.ToXmlDcs(bag));
}
public string ToXmlDcs(object value, bool throwExceptions = false)
{
var ser = new DataContractSerializer(value.GetType(), null, int.MaxValue, true, false, null);
MemoryStream ms = new MemoryStream();
ser.WriteObject(ms, value);
return Encoding.UTF8.GetString(ms.ToArray(), 0, (int)ms.Length);
}
This DOES work but produces some pretty heinous XML (formatted with line breaks and indentation here):
<ArrayOfKeyValueOfstringanyType xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<KeyValueOfstringanyType>
<Key>key</Key>
<Value i:type="a:string" xmlns:a="http://www.w3.org/2001/XMLSchema">Value</Value>
</KeyValueOfstringanyType>
<KeyValueOfstringanyType>
<Key>Key2</Key>
<Value i:type="a:decimal" xmlns:a="http://www.w3.org/2001/XMLSchema">100.10</Value>
</KeyValueOfstringanyType>
<KeyValueOfstringanyType>
<Key>Key3</Key>
<Value i:type="a:guid" xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/">2cd46d2a-a636-4af4-979b-e834d39b6d37</Value>
</KeyValueOfstringanyType>
<KeyValueOfstringanyType>
<Key>Key4</Key>
<Value i:type="a:dateTime" xmlns:a="http://www.w3.org/2001/XMLSchema">2011-09-19T17:17:05.4406999-07:00</Value>
</KeyValueOfstringanyType>
<KeyValueOfstringanyType>
<Key>Key5</Key>
<Value i:type="a:boolean" xmlns:a="http://www.w3.org/2001/XMLSchema">true</Value>
</KeyValueOfstringanyType>
<KeyValueOfstringanyType>
<Key>Key7</Key>
<Value i:type="a:base64Binary" xmlns:a="http://www.w3.org/2001/XMLSchema">Ki1C</Value>
</KeyValueOfstringanyType>
</ArrayOfKeyValueOfstringanyType>
Ouch! That seriously hurts the eye! :-) Worse though it's extremely verbose with all those repetitive namespace declarations.
It's good
to know that it works in a pinch, but for a human readable/editable solution or something lightweight
to store in a database it's not quite ideal.
Why should I care?
As a little background, in one of my applications I have a need for a flexible property bag that is used on a free form database field on an otherwise static entity. Basically what I have is a standard database record
to which arbitrary properties can be added in an XML based string field. I intend
to expose those arbitrary properties as a collection from field data stored in XML. The concept is pretty simple: When loading write the data
to the collection, when the data is saved serialize the data into an XML string and store it into the database. When reading the data pick up the XML and if the collection on the entity is accessed automatically deserialize the XML into the Dictionary. (I'll talk more about this in another post).
While the DataContext Serializer would work, it's verbosity is problematic both for size of the generated XML strings and the fact that users can manually edit this XML based property data in an advanced mode. A clean(er) layout certainly would be preferable and more user friendly.
Custom XMLSerialization with a PropertyBag Class
So… after a bunch of experimentation with different serialization formats I decided
to create a custom PropertyBag class that provides for a serializable Dictionary. It's basically a custom Dictionary<TType,TValue> implementation with the keys always set as string keys. The result are PropertyBag<TValue> and PropertyBag (which defaults
to the object type for values).
The PropertyBag<TType> and PropertyBag classes provide these features:
Subclassed from Dictionary<T,V>
Implements IXmlSerializable with a cleanish XML format
ToXml() and FromXml() methods
to export and import
to and from XML strings
Static CreateFromXml() method
to create an instance
It's simple enough as it's merely a Dictionary<string,object> subclass but that supports serialization
to a - what I think at least - cleaner XML format. The class is super simple
to use:
[TestMethod]
public void PropertyBagTwoWayObjectSerializationTest()
{
var bag = new PropertyBag();
bag.Add("key", "Value");
bag.Add("Key2", 100.10M);
bag.Add("Key3", Guid.NewGuid());
bag.Add("Key4", DateTime.Now);
bag.Add("Key5", true);
bag.Add("Key7", new byte[3] { 42,45,66 } );
bag.Add("Key8", null);
bag.Add("Key9", new ComplexObject()
{
Name = "Rick",
Entered = DateTime.Now,
Count = 10
});
string xml = bag.ToXml();
TestContext.WriteLine(bag.ToXml());
bag.Clear();
bag.FromXml(xml);
Assert.IsTrue(bag["key"] as string == "Value");
Assert.IsInstanceOfType( bag["Key3"], typeof(Guid));
Assert.IsNull(bag["Key8"]);
//Assert.IsNull(bag["Key10"]);
Assert.IsInstanceOfType(bag["Key9"], typeof(ComplexObject));
}
This uses the PropertyBag class which uses a PropertyBag<string,object> - which means it returns untyped values of type object. I suspect for me this will be the most common scenario as I'd want
to store arbitrary values in the PropertyBag rather than one specific type.
The same code with a strongly typed PropertyBag<decimal> looks like this:
[TestMethod]
public void PropertyBagTwoWayValueTypeSerializationTest()
{
var bag = new PropertyBag<decimal>();
bag.Add("key", 10M);
bag.Add("Key1", 100.10M);
bag.Add("Key2", 200.10M);
bag.Add("Key3", 300.10M);
string xml = bag.ToXml();
TestContext.WriteLine(bag.ToXml());
bag.Clear();
bag.FromXml(xml);
Assert.IsTrue(bag.Get("Key1") == 100.10M);
Assert.IsTrue(bag.Get("Key3") == 300.10M);
}
and produces typed results of type decimal. The types can be either value or reference types the combination of which actually proved
to be a little more tricky than anticipated due
to null and specific string value checks required - getting the generic typing right required use of default(T) and Convert.ChangeType()
to trick the compiler into playing nice.
Of course the whole raison d'etre for this class is the XML serialization. You can see in the code above that we're doing a .ToXml() and .FromXml()
to serialize
to and from string. The XML produced for the first example looks like this:
<?xml version="1.0" encoding="utf-8"?>
<properties>
<item>
<key>key</key>
<value>Value</value>
</item>
<item>
<key>Key2</key>
<value type="decimal">100.10</value>
</item>
<item>
<key>Key3</key>
<value type="___System.Guid">
<guid>f7a92032-0c6d-4e9d-9950-b15ff7cd207d</guid>
</value>
</item>
<item>
<key>Key4</key>
<value type="datetime">2011-09-26T17:45:58.5789578-10:00</value>
</item>
<item>
<key>Key5</key>
<value type="boolean">true</value>
</item>
<item>
<key>Key7</key>
<value type="base64Binary">Ki1C</value>
</item>
<item>
<key>Key8</key>
<value type="nil" />
</item>
<item>
<key>Key9</key>
<value type="___Westwind.Tools.Tests.PropertyBagTest+ComplexObject">
<ComplexObject>
<Name>Rick</Name>
<Entered>2011-09-26T17:45:58.5789578-10:00</Entered>
<Count>10</Count>
</ComplexObject>
</value>
</item>
</properties>
The format is a bit cleaner than the DataContractSerializer. Each item is serialized into <key> <value> pairs. If the value is a string no type information is written. Since string tends
to be the most common type this saves space and serialization processing. All other types are attributed. Simple types are mapped
to XML types so things like decimal, datetime, boolean and base64Binary are encoded using their Xml type values. All other types are embedded with a hokey format that describes the .NET type preceded by a three underscores and then are encoded using the XmlSerializer. You can see this best above in the ComplexObject encoding.
For custom types this isn't pretty either, but it's more concise than the DCS and it works as long as you're serializing back and forth between .NET clients at least.
The XML generated from the second example that uses PropertyBag<decimal> looks like this:
<?xml version="1.0" encoding="utf-8"?>
<properties>
<item>
<key>key</key>
<value type="decimal">10</value>
</item>
<item>
<key>Key1</key>
<value type="decimal">100.10</value>
</item>
<item>
<key>Key2</key>
<value type="decimal">200.10</value>
</item>
<item>
<key>Key3</key>
<value type="decimal">300.10</value>
</item>
</properties>
How does it work
As I mentioned there's nothing fancy about this solution - it's little more than a subclass of Dictionary<T,V> that implements custom Xml Serialization and a couple of helper methods that facilitate getting the XML in and out of the class more easily. But it's proven very handy for a number of projects for me where dynamic data storage is required.
Here's the code:
/// <summary>
/// Creates a serializable string/object dictionary that is XML serializable
/// Encodes keys as element names and values as simple values with a type
/// attribute that contains an XML type name. Complex names encode the type
/// name with type='___namespace.classname' format followed by a standard xml
/// serialized format. The latter serialization can be slow so it's not recommended
///
to pass complex types if performance is critical.
/// </summary>
[XmlRoot("properties")]
public class PropertyBag : PropertyBag<object>
{
/// <summary>
/// Creates an instance of a propertybag from an Xml string
/// </summary>
/// <param name="xml">Serialize</param>
/// <returns></returns>
public static PropertyBag CreateFromXml(string xml)
{
var bag = new PropertyBag();
bag.FromXml(xml);
return bag;
}
}
/// <summary>
/// Creates a serializable string for generic types that is XML serializable.
///
/// Encodes keys as element names and values as simple values with a type
/// attribute that contains an XML type name. Complex names encode the type
/// name with type='___namespace.classname' format followed by a standard xml
/// serialized format. The latter serialization can be slow so it's not recommended
///
to pass complex types if performance is critical.
/// </summary>
/// <typeparam name="TValue">Must be a reference type. For value types use type object</typeparam>
[XmlRoot("properties")]
public class PropertyBag<TValue> : Dictionary<string, TValue>, IXmlSerializable
{
/// <summary>
/// Not implemented - this means no schema information is passed
/// so this won't work with ASMX/WCF services.
/// </summary>
/// <returns></returns>
public System.Xml.Schema.XmlSchema GetSchema()
{
return null;
}
/// <summary>
/// Serializes the dictionary
to XML. Keys are
/// serialized
to element names and values as
/// element values. An xml type attribute is embedded
/// for each serialized element - a .NET type
/// element is embedded for each complex type and
/// prefixed with three underscores.
/// </summary>
/// <param name="writer"></param>
public void WriteXml(System.Xml.XmlWriter writer)
{
foreach (string key in this.Keys)
{
TValue value = this[key];
Type type = null;
if (value != null)
type = value.GetType();
writer.WriteStartElement("item");
writer.WriteStartElement("key");
writer.WriteString(key as string);
writer.WriteEndElement();
writer.WriteStartElement("value");
string xmlType = XmlUtils.MapTypeToXmlType(type);
bool isCustom = false;
// Type information attribute if not string
if (value == null)
{
writer.WriteAttributeString("type", "nil");
}
else if (!string.IsNullOrEmpty(xmlType))
{
if (xmlType != "string")
{
writer.WriteStartAttribute("type");
writer.WriteString(xmlType);
writer.WriteEndAttribute();
}
}
else
{
isCustom = true;
xmlType = "___" + value.GetType().FullName;
writer.WriteStartAttribute("type");
writer.WriteString(xmlType);
writer.WriteEndAttribute();
}
// Actual deserialization
if (!isCustom)
{
if (value != null)
writer.WriteValue(value);
}
else
{
XmlSerializer ser = new XmlSerializer(value.GetType());
ser.Serialize(writer, value);
}
writer.WriteEndElement(); // value
writer.WriteEndElement(); // item
}
}
/// <summary>
/// Reads the custom serialized format
/// </summary>
/// <param name="reader"></param>
public void ReadXml(System.Xml.XmlReader reader)
{
this.Clear();
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Element && reader.Name == "key")
{
string xmlType = null;
string name = reader.ReadElementContentAsString();
// item element
reader.ReadToNextSibling("value");
if (reader.MoveToNextAttribute())
xmlType = reader.Value;
reader.MoveToContent();
TValue value;
if (xmlType == "nil")
value = default(TValue); // null
else if (string.IsNullOrEmpty(xmlType))
{
// value is a string or object and we can assign TValue
to value
string strval = reader.ReadElementContentAsString();
value = (TValue) Convert.ChangeType(strval, typeof(TValue));
}
else if (xmlType.StartsWith("___"))
{
while (reader.Read() && reader.NodeType != XmlNodeType.Element)
{ }
Type type = ReflectionUtils.GetTypeFromName(xmlType.Substring(3));
//value = reader.ReadElementContentAs(type,null);
XmlSerializer ser = new XmlSerializer(type);
value = (TValue)ser.Deserialize(reader);
}
else
value = (TValue)reader.ReadElementContentAs(XmlUtils.MapXmlTypeToType(xmlType), null);
this.Add(name, value);
}
}
}
/// <summary>
/// Serializes this dictionary
to an XML string
/// </summary>
/// <returns>XML String or Null if it fails</returns>
public string ToXml()
{
string xml = null;
SerializationUtils.SerializeObject(this, out xml);
return xml;
}
/// <summary>
/// Deserializes from an XML string
/// </summary>
/// <param name="xml"></param>
/// <returns>true or false</returns>
public bool FromXml(string xml)
{
this.Clear();
// if xml string is empty we return an empty dictionary
if (string.IsNullOrEmpty(xml))
return true;
var result = SerializationUtils.DeSerializeObject(xml,
this.GetType()) as PropertyBag<TValue>;
if (result != null)
{
foreach (var item in result)
{
this.Add(item.Key, item.Value);
}
}
else
// null is a failure
return false;
return true;
}
/// <summary>
/// Creates an instance of a propertybag from an Xml string
/// </summary>
/// <param name="xml"></param>
/// <returns></returns>
public static PropertyBag<TValue> CreateFromXml(string xml)
{
var bag = new PropertyBag<TValue>();
bag.FromXml(xml);
return bag;
}
}
}
The code uses a couple of small helper classes SerializationUtils and XmlUtils for mapping Xml types
to and from .NET, both of which are from the WestWind,Utilities project (which is the same project where PropertyBag lives) from the West Wind Web Toolkit. The code implements ReadXml and WriteXml for the IXmlSerializable implementation using old school XmlReaders and XmlWriters (because it's pretty simple stuff - no need for XLinq here).
Then there are two helper methods .ToXml() and .FromXml() that basically allow your code
to easily convert between XML and a PropertyBag object. In my code that's what I use
to actually
to persist
to and from the entity XML property during .Load() and .Save() operations. It's sweet
to be able
to have a string key dictionary and then be able
to turn around with 1 line of code
to persist the whole thing
to XML and back.
Hopefully some of you will find this class as useful as I've found it. It's a simple solution
to a common requirement in my applications and I've used the hell out of it in the short time since I created it.
Resources
You can find the complete code for the two classes plus the helpers in the Subversion repository for Westwind.Utilities. You can grab the source files from there or download the whole project. You can also grab the full Westwind.Utilities assembly from NuGet and add it
to your project if that's easier for you.
PropertyBag Source Code
SerializationUtils and XmlUtils
Westwind.Utilities Assembly on NuGet (add from Visual Studio)
© Rick Strahl, West Wind Technologies, 2005-2011Posted in .NET CSharp
Tweet
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
