I've been working on a request to
the West Wind Application Configuration library to add JSON support.
The config library is a very easy to use code-first approach to configuration: You create a class that holds
the configuration data that inherits from a base configuration class, and then assign a persistence provider at runtime that determines where and how
the configuration data is store. Currently
the library supports .NET Configuration stores (web.config/app.config), XML files, SQL records and string storage.About once a week somebody asks me about JSON support and I've deflected this question for
the longest time because frankly I think that JSON as a configuration store doesn't really buy a heck of a lot over XML. Both formats require
the user to perform some fixup of
the plain configuration data - in XML into XML tags, with JSON using JSON delimiters for properties and property formatting rules. Sure JSON is a little less verbose and maybe a little easier to read if you have hierarchical data, but overall
the differences are pretty minor in my opinion. And yet -
the requests keep rolling in.Hard Link Issues in a Component LibraryAnother reason I've been hesitant is that I really didn't want to pull in a dependency on an external JSON library - in this case JSON.NET - into
the core library. If you're not using JSON.NET elsewhere I don't want a user to have to require a hard dependency on JSON.NET unless they want to use
the JSON feature. JSON.NET is also sensitive to versions and doesn't play nice with multiple versions when hard linked. For example, when you have a reference to V4.4 in your project but
the host application has a reference to version 4.5 you can run into assembly load problems. NuGet's Update-Package can solve some of this *if* you can recompile, but that's not ideal for a component that's supposed to be just plug and play. This is no criticism of JSON.NET - this really applies to any dependency that might change. So hard linking
the DLL can be problematic for a number reasons, but
the primary reason is to not force loading of JSON.NET unless you actually need it when you use
the JSON configuration features of
the library.Enter Dynamic LoadingSo rather than adding an assembly reference to
the project, I decided that it would be better to dynamically load
the DLL at runtime and then use dynamic typing to access various classes. This allows me to run without a hard assembly reference and allows more flexibility with version number differences now and in
the future.But there are also a couple of downsides:No assembly reference means only dynamic access - no compiler type checking or IntellisenseRequirement for
the host application to have reference to JSON.NET or else get runtime errorsThe former is minor, but
the latter can be problematic. Runtime errors are always painful, but in this case I'm willing to live with this. If you want to use JSON configuration settings JSON.NET needs to be loaded in
the project. If this is a Web project, it'll likely be there already.So there are a few things that are needed to make this work:Dynamically create an instance and optionally attempt to load an Assembly (if not loaded)Load types into dynamic variablesUse Reflection for a few tasks like statics/enumsThe dynamic keyword in C# makes
the formerly most difficult Reflection part - method calls and property assignments - fairly painless. But as cool as dynamic is it doesn't handle all aspects of Reflection. Specifically it doesn't deal with object activation, truly dynamic (string based) member activation or accessing of non instance members, so there's still a little bit of work left to do with Reflection.Dynamic Object InstantiationThe first step in getting
the process rolling is to instantiate
the type you need to work with. This might be a two step process - loading
the instance from a string value, since we don't have a hard type reference and potentially having to load
the assembly. Although
the host project might have a reference to JSON.NET, that instance might have not been loaded yet since it hasn't been accessed yet. In ASP.NET this won't be a problem, since ASP.NET preloads all referenced assemblies on AppDomain startup, but in other executable project, assemblies are just in time loaded only when they are accessed.Instantiating a type is a two step process: Finding
the type reference and then activating it. Here's
the generic code out of my ReflectionUtils library I use for this:/// <summary>
/// Creates an instance of a type based on a string. Assumes that
the type's
/// </summary>
/// <param name="typeName">Common
name of
the type</param>
/// <param name="args">Any constructor parameters</param>
/// <returns></returns>
public static object CreateInstanceFromString(string typeName, params object[] args)
{
object instance = null;
Type type = null;
try
{
type = GetTypeFromName(typeName);
if (type == null)
return null;
instance = Activator.CreateInstance(type, args);
}
catch
{
return null;
}
return instance;
}
/// <summary>
/// Helper routine that looks up a type
name and tries to retrieve
the
/// full type reference in
the actively executing assemblies.
/// </summary>
/// <param name="typeName"></param>
/// <returns></returns>
public static Type GetTypeFromName(string typeName)
{
Type type = null;
// Let default
name binding find it
type = Type.GetType(typeName, false);
if (type != null)
return type;
// look through assembly list
var assemblies = AppDomain.CurrentDomain.GetAssemblies();
// try to find manually
foreach (Assembly asm in assemblies)
{
type = asm.GetType(typeName, false);
if (type != null)
break;
}
return type;
}
To use this for loading JSON.NET I have a small factory function that instantiates JSON.NET and sets a bunch of configuration settings on
the generated object.
The startup code also looks for failure and tries loading up
the assembly when it fails since that's
the main reason
the load would fail. Finally it also caches
the loaded instance for reuse (according to James
the JSON.NET instance is thread safe and quite a bit faster when cached). Here's what
the factory function looks like in JsonSerializationUtils:/// <summary>
/// Dynamically creates an instance of JSON.NET
/// </summary>
/// <param name="throwExceptions">If true throws exceptions otherwise returns null</param>
/// <returns>Dynamic JsonSerializer instance</returns>
public static dynamic CreateJsonNet(bool throwExceptions = true)
{
if (JsonNet != null)
return JsonNet;
lock (SyncLock)
{
if (JsonNet != null)
return JsonNet;
// Try to create instance
dynamic json = ReflectionUtils.CreateInstanceFromString("Newtonsoft.Json.JsonSerializer");
if (json == null)
{
try
{
var ass = AppDomain.CurrentDomain.Load("Newtonsoft.Json");
json = ReflectionUtils.CreateInstanceFromString("Newtonsoft.Json.JsonSerializer");
}
catch (Exception ex)
{
if (throwExceptions)
throw;
return null;
}
}
if (json == null)
return null;
json.ReferenceLoopHandling =
(dynamic) ReflectionUtils.GetStaticProperty("Newtonsoft.Json.ReferenceLoopHandling", "Ignore");
// Enums as strings in JSON
dynamic enumConverter = ReflectionUtils.CreateInstanceFromString("Newtonsoft.Json.Converters.StringEnumConverter");
json.Converters.Add(enumConverter);
JsonNet = json;
}
return JsonNet;
}This code's purpose is to return a fully configured JsonSerializer instance. As you can see
the code tries to create an instance and when it fails tries to load
the assembly, and then re-tries loading.Once
the instance is loaded some configuration occurs on it. Specifically I set
the ReferenceLoopHandling option to not blow up immediately when circular references are encountered. There are a host of other small config setting that might be useful to set, but
the default seem to be good enough in recent versions. Note that I'm setting ReferenceLoopHandling which requires an Enum value to be set. There's no real easy way (short of using
the cardinal numeric value) to set a property or pass parameters from static values or enums. This means I still need to use Reflection to make this work. I'm using
the same ReflectionUtils class I previously used to handle this for me.
The function looks up
the type and then uses Type.InvokeMember() to read
the static property.Another feature I need is have Enum values serialized as strings rather than numeric values which is
the default. To do this I can use
the StringEnumConverter to convert enums to strings by adding it to
the Converters collection.As you can see there's still a bit of Reflection to be done even in C# 4+ with dynamic, but with a few helpers this process is relatively painless.Doing
the actual JSON ConversionFinally I need to actually do my JSON conversions. For
the Utility class I need serialization that works for both strings and files so I created four methods that handle these tasks two each for serialization and deserialization for string and file.Here's what
the File Serialization looks like:/// <summary>
/// Serializes an object instance to a JSON file.
/// </summary>
/// <param name="value">the value to serialize</param>
/// <param name="fileName">Full path to
the file to write out with JSON.</param>
/// <param name="throwExceptions">Determines whether exceptions are thrown or false is returned</param>
/// <param name="formatJsonOutput">if true pretty-formats
the JSON with line breaks</param>
/// <returns>true or false</returns>
public static bool SerializeToFile(object value, string fileName, bool throwExceptions = false, bool formatJsonOutput = false)
{
dynamic writer = null;
FileStream fs = null;
try
{
Type type = value.GetType();
var json = CreateJsonNet(throwExceptions);
if (json == null)
return false;
fs = new FileStream(fileName, FileMode.Create);
var sw = new StreamWriter(fs, Encoding.UTF8);
writer = Activator.CreateInstance(JsonTextWriterType, sw);
if (formatJsonOutput)
writer.Formatting = (dynamic)Enum.Parse(FormattingType, "Indented");
writer.QuoteChar = '"';
json.Serialize(writer, value);
}
catch (Exception ex)
{
Debug.WriteLine("JsonSerializer Serialize error: " + ex.Message);
if (throwExceptions)
throw;
return false;
}
finally
{
if (writer != null)
writer.Close();
if (fs != null)
fs.Close();
}
return true;
}You can see more of
the dynamic invocation in this code. First I grab
the dynamic JsonSerializer instance using
the CreateJsonNet() method shown earlier which returns a dynamic. I then create a JsonTextWriter and configure a couple of enum settings on it, and then call Serialize() on
the serializer instance with
the JsonTextWriter that writes
the output to disk. Although this code is dynamic it's still fairly short and readable.For full circle operation here's
the DeserializeFromFile() version:/// <summary>
/// Deserializes an object from file and returns a reference.
/// </summary>
/// <param name="fileName">name of
the file to serialize to</param>
/// <param name="objectType">The Type of
the object. Use typeof(yourobject class)</param>
/// <param name="binarySerialization">determines whether we use Xml or Binary serialization</param>
/// <param name="throwExceptions">determines whether failure will throw rather than return null on failure</param>
/// <returns>Instance of
the deserialized object or null. Must be cast to your object type</returns>
public static object DeserializeFromFile(string fileName, Type objectType, bool throwExceptions = false)
{
dynamic json = CreateJsonNet(throwExceptions);
if (json == null)
return null;
object result = null;
dynamic reader = null;
FileStream fs = null;
try
{
fs = new FileStream(fileName, FileMode.Open, FileAccess.Read);
var sr = new StreamReader(fs, Encoding.UTF8);
reader = Activator.CreateInstance(JsonTextReaderType, sr);
result = json.Deserialize(reader, objectType);
reader.Close();
}
catch (Exception ex)
{
Debug.WriteLine("JsonNetSerialization Deserialization Error: " + ex.Message);
if (throwExceptions)
throw;
return null;
}
finally
{
if (reader != null)
reader.Close();
if (fs != null)
fs.Close();
}
return result;
}This code is a little more compact since there are no prettifying options to set. Here JsonTextReader is created dynamically and it receives
the output from
the Deserialize() operation on
the serializer.You can take a look at
the full JsonSerializationUtils.cs file on GitHub to see
the rest of
the operations, but
the string operations are very similar -
the code is fairly repetitive.These generic serialization utilities isolate
the dynamic serialization logic that has to deal with
the dynamic nature of JSON.NET, and any code that uses these functions is none
the wiser that JSON.NET is dynamically loaded.Using
the JsonSerializationUtils WrapperThe final consumer of
the SerializationUtils wrapper is an actual ConfigurationProvider, that is responsible for handling reading and writing JSON values to and from files.
The provider is simple a small wrapper around
the SerializationUtils component and there's very little code to make this work now:The whole provider looks like this:/// <summary>
/// Reads and Writes configuration settings in .NET config files and
/// sections. Allows reading and writing to default or external files
/// and specification of
the configuration section that settings are
/// applied to.
/// </summary>
public class JsonFileConfigurationProvider<TAppConfiguration> : ConfigurationProviderBase<TAppConfiguration>
where TAppConfiguration: AppConfiguration, new()
{
/// <summary>
/// Optional -
the Configuration file where configuration settings are
/// stored in. If not specified uses
the default Configuration Manager
/// and its default store.
/// </summary>
public string JsonConfigurationFile
{
get { return _JsonConfigurationFile; }
set { _JsonConfigurationFile = value; }
}
private string _JsonConfigurationFile = string.Empty;
public override bool Read(AppConfiguration config)
{
var newConfig = JsonSerializationUtils.DeserializeFromFile(JsonConfigurationFile, typeof(TAppConfiguration)) as TAppConfiguration;
if (newConfig == null)
{
if(Write(config))
return true;
return false;
}
DecryptFields(newConfig);
DataUtils.CopyObjectData(newConfig, config, "Provider,ErrorMessage");
return true;
}
/// <summary>
/// Return
/// </summary>
/// <typeparam name="TAppConfig"></typeparam>
/// <returns></returns>
public override TAppConfig Read<TAppConfig>()
{
var result = JsonSerializationUtils.DeserializeFromFile(JsonConfigurationFile, typeof(TAppConfig)) as TAppConfig;
if (result != null)
DecryptFields(result);
return result;
}
/// <summary>
/// Write configuration to XmlConfigurationFile location
/// </summary>
/// <param name="config"></param>
/// <returns></returns>
public override bool Write(AppConfiguration config)
{
EncryptFields(config);
bool result = JsonSerializationUtils.SerializeToFile(config, JsonConfigurationFile,false,true);
// Have to decrypt again to make sure
the properties are readable afterwards
DecryptFields(config);
return result;
}
}This incidentally demonstrates how easy it is to create a new provider for
the West Wind Application Configuration component. Simply implementing 3 methods will do in most cases.Note this code doesn't have any dynamic dependencies - all that's abstracted away in
the JsonSerializationUtils(). From here on, serializing JSON is just a matter of calling
the static methods on
the SerializationUtils class.Already, there are several other places in some other tools where I use JSON serialization this is coming in very handy. With a couple of lines of code I was able to add JSON.NET support to an older AJAX library that I use replacing quite a bit of code that was previously in use. And for any other manual JSON operations (in a couple of apps I use JSON Serialization for 'blob' like document storage) this is also going to be handy.Performance?Some of you might be thinking that using dynamic and Reflection can't be good for performance. And you'd be right… In performing some informal testing it looks like
the performance of
the native code is nearly twice as fast as
the dynamic code. Most of
the slowness is attributable to type lookups. To test I created a native class that uses an actual reference to JSON.NET and performance was consistently around 85-90% faster with
the referenced code. This will change though depending on
the size of objects serialized -
the larger
the object
the more processing time is spent inside
the actual dynamically activated components and
the less difference there will be. Dynamic code is always slower, but how much it really affects your application primarily depends on how frequently
the dynamic code is called in relation to
the non-dynamic code executing. In most situations where dynamic code is used 'to get
the process rolling' as I do here
the overhead is small enough to not matter.All that being said though - I serialized 10,000 objects in 80ms vs. 45ms so this is hardly slouchy performance. For
the configuration component speed is not that important because both read and write operations typically happen once on first access and then every once in a while. But for other operations -
say a serializer trying to handle AJAX requests on a Web Server one would be well served to create a hard dependency.Dynamic Loading - Worth it?Dynamic loading is not something you need to worry about but on occasion dynamic loading makes sense. But there's a price to be paid in added code and a performance hit which depends on how frequently
the dynamic code is accessed. But for some operations that are not pivotal to a component or application and are only used under certain circumstances dynamic loading can be beneficial to avoid having to ship extra files adding dependencies and loading down distributions. These days when you create new projects in Visual Studio with 30 assemblies before you even add your own code, trying to keep file counts under control seems like a good idea. It's not
the kind of thing you do on a regular basis, but when needed it can be a useful option in your toolset… © Rick Strahl, West Wind Technologies, 2005-2013Posted in .NET C#
Tweet
!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
