I've been working on a request to the West Wind Application Configuration library to add JSON support. The config library is a very easy to use code-first approach to configuration: You create a class that holds the configuration data that inherits from a base configuration class,
and then assign a persistence provider at runtime that determines where
and how the configuration data is store. Currently the library supports .NET Configuration stores (web.config/app.config), XML files, SQL records
and string storage.About once a week somebody asks me about JSON support
and I've deflected this question for the longest time because frankly I think that JSON as a configuration store doesn't really buy a heck of a lot over XML. Both formats require the user to perform some fixup of the plain configuration data - in XML into XML tags, with JSON using JSON delimiters for properties
and property formatting rules. Sure JSON is a little less verbose
and maybe a little easier to read if you have hierarchical data, but overall the differences are pretty minor in my opinion.
And yet - the requests keep rolling in.Hard Link Issues in a Component LibraryAnother reason I've been hesitant is that I really didn't want to pull in a dependency on an external JSON library - in this case JSON.NET - into the core library. If you're not using JSON.NET elsewhere I don't want a user to have to require a hard dependency on JSON.NET unless they want to use the JSON feature. JSON.NET is also sensitive to versions
and doesn't play nice with multiple versions when hard linked. For example, when you have a reference to V4.4 in your project but the host application has a reference to version 4.5 you can run into assembly load problems. NuGet's Update-Package can solve some of this *if* you can recompile, but that's not ideal for a component that's supposed to be just plug
and play. This is no criticism of JSON.NET - this really applies to any dependency that might change. So hard linking the DLL can be problematic for a number reasons, but the primary reason is to not force loading of JSON.NET unless you actually need it when you use the JSON configuration features of the library.Enter Dynamic LoadingSo rather than adding an assembly reference to the project, I decided that it would be better to dynamically load the DLL at runtime
and then use dynamic typing to access various classes. This allows me to run without a hard assembly reference
and allows more flexibility with version number differences now
and in the future.But there are also a couple of downsides:No assembly reference means only dynamic access - no compiler type checking or IntellisenseRequirement for the host application to have reference to JSON.NET or else get runtime errorsThe former is minor, but the latter can be problematic. Runtime errors are always painful, but in this case I'm willing to live with this. If you want to use JSON configuration settings JSON.NET needs to be loaded in the project. If this is a Web project, it'll likely be there already.So there are a few things that are needed to make this work:Dynamically create an instance
and optionally attempt to load an Assembly (if not loaded)Load types into dynamic variablesUse Reflection for a few tasks like statics/enumsThe dynamic keyword in C# makes the formerly most difficult Reflection part - method calls
and property assignments - fairly painless. But as cool as dynamic is it doesn't handle all aspects of Reflection. Specifically it doesn't deal with object activation, truly dynamic (string based) member activation or accessing of non instance members, so there's still a little bit of work left to do with Reflection.Dynamic Object InstantiationThe first step in getting the process rolling is to instantiate the type you need to work with. This might be a two step process - loading the instance from a string value, since we don't have a hard type reference
and potentially having to load the assembly. Although the host project might have a reference to JSON.NET, that instance might have not been loaded yet since it hasn't been accessed yet. In ASP.NET this won't be a problem, since ASP.NET preloads all referenced assemblies on AppDomain startup, but in other executable project, assemblies are just in time loaded only when they are accessed.Instantiating a type is a two step process: Finding the type reference
and then activating it. Here's the generic code out of my ReflectionUtils library I use for this:/// <summary>
/// Creates an instance of a type based on a string. Assumes that the type's
/// </summary>
/// <param name="typeName">Common name of the type</param>
/// <param name="args">Any constructor parameters</param>
/// <returns></returns>
public static object CreateInstanceFromString(string typeName, params object[] args)
{
object instance = null;
Type type = null;
try
{
type = GetTypeFromName(typeName);
if (type == null)
return null;
instance = Activator.CreateInstance(type, args);
}
catch
{
return null;
}
return instance;
}
/// <summary>
/// Helper routine that looks up a type name
and tries to retrieve the
/// full type reference in the actively executing assemblies.
/// </summary>
/// <param name="typeName"></param>
/// <returns></returns>
public static Type GetTypeFromName(string typeName)
{
Type type = null;
// Let default name binding find it
type = Type.GetType(typeName, false);
if (type != null)
return type;
// look through assembly list
var assemblies = AppDomain.CurrentDomain.GetAssemblies();
// try to find manually
foreach (Assembly asm in assemblies)
{
type = asm.GetType(typeName, false);
if (type != null)
break;
}
return type;
}
To use this for loading JSON.NET I have a small factory function that instantiates JSON.NET
and sets a bunch of configuration settings on the generated object. The startup code also looks for failure
and tries loading up the assembly when it fails since that's the main reason the load would fail. Finally it also caches the loaded instance for reuse (according to James the JSON.NET instance is thread safe
and quite a bit faster when cached). Here's what the factory function looks like in JsonSerializationUtils:/// <summary>
/// Dynamically creates an instance of JSON.NET
/// </summary>
/// <param name="throwExceptions">If true throws exceptions otherwise returns null</param>
/// <returns>Dynamic JsonSerializer instance</returns>
public static dynamic CreateJsonNet(bool throwExceptions = true)
{
if (JsonNet != null)
return JsonNet;
lock (SyncLock)
{
if (JsonNet != null)
return JsonNet;
// Try to create instance
dynamic json = ReflectionUtils.CreateInstanceFromString("Newtonsoft.Json.JsonSerializer");
if (json == null)
{
try
{
var ass = AppDomain.CurrentDomain.Load("Newtonsoft.Json");
json = ReflectionUtils.CreateInstanceFromString("Newtonsoft.Json.JsonSerializer");
}
catch (Exception ex)
{
if (throwExceptions)
throw;
return null;
}
}
if (json == null)
return null;
json.ReferenceLoopHandling =
(dynamic) ReflectionUtils.GetStaticProperty("Newtonsoft.Json.ReferenceLoopHandling", "Ignore");
// Enums as strings in JSON
dynamic enumConverter = ReflectionUtils.CreateInstanceFromString("Newtonsoft.Json.Converters.StringEnumConverter");
json.Converters.Add(enumConverter);
JsonNet = json;
}
return JsonNet;
}This code's purpose is to return a fully configured JsonSerializer instance. As you can see the code tries to create an instance
and when it fails tries to load the assembly,
and then re-tries loading.Once the instance is loaded some configuration occurs on it. Specifically I set the ReferenceLoopHandling option to not blow up immediately when circular references are encountered. There are a host of other small config setting that might be useful to set, but the default seem to be good enough in recent versions. Note that I'm setting ReferenceLoopHandling which requires an Enum value to be set. There's no real easy way (short of using the cardinal numeric value) to set a property or pass parameters from static values or enums. This means I still need to use Reflection to make this work. I'm using the same ReflectionUtils class I previously used to handle this for me. The function looks up the type
and then uses Type.InvokeMember() to read the static property.Another feature I need is have Enum values serialized as strings rather than numeric values which is the default. To do this I can use the StringEnumConverter to convert enums to strings by adding it to the Converters collection.As you can see there's still a bit of Reflection to be done even in C# 4+ with dynamic, but with a few helpers this process is relatively painless.Doing the actual JSON ConversionFinally I need to actually do my JSON conversions. For the Utility class I need serialization that works for both strings
and files so I created four methods that handle these tasks two each for serialization
and deserialization for string
and file.Here's what the File Serialization looks like:/// <summary>
/// Serializes an object instance to a JSON file.
/// </summary>
/// <param name="value">the value to serialize</param>
/// <param name="fileName">Full path to the file to write out with JSON.</param>
/// <param name="throwExceptions">Determines whether exceptions are thrown or false is returned</param>
/// <param name="formatJsonOutput">if true pretty-formats the JSON with line breaks</param>
/// <returns>true or false</returns>
public static bool SerializeToFile(object value, string fileName, bool throwExceptions = false, bool formatJsonOutput = false)
{
dynamic writer = null;
FileStream fs = null;
try
{
Type type = value.GetType();
var json = CreateJsonNet(throwExceptions);
if (json == null)
return false;
fs = new FileStream(fileName, FileMode.Create);
var sw = new StreamWriter(fs, Encoding.UTF8);
writer = Activator.CreateInstance(JsonTextWriterType, sw);
if (formatJsonOutput)
writer.Formatting = (dynamic)Enum.Parse(FormattingType, "Indented");
writer.QuoteChar = '"';
json.Serialize(writer, value);
}
catch (Exception ex)
{
Debug.WriteLine("JsonSerializer Serialize error: " + ex.Message);
if (throwExceptions)
throw;
return false;
}
finally
{
if (writer != null)
writer.Close();
if (fs != null)
fs.Close();
}
return true;
}You can see more of the dynamic invocation in this code. First I grab the dynamic JsonSerializer instance using the CreateJsonNet() method shown earlier which returns a dynamic. I then create a JsonTextWriter
and configure a couple of enum settings on it,
and then call Serialize() on the serializer instance with the JsonTextWriter that writes the output to disk. Although this code is dynamic it's still fairly short
and readable.For full circle operation here's the DeserializeFromFile() version:/// <summary>
/// Deserializes an object from file
and returns a reference.
/// </summary>
/// <param name="fileName">name of the file to serialize to</param>
/// <param name="objectType">The Type of the object. Use typeof(yourobject class)</param>
/// <param name="binarySerialization">determines whether we use Xml or Binary serialization</param>
/// <param name="throwExceptions">determines whether failure will throw rather than return null on failure</param>
/// <returns>Instance of the deserialized object or null. Must be cast to your object type</returns>
public static object DeserializeFromFile(string fileName, Type objectType, bool throwExceptions = false)
{
dynamic json = CreateJsonNet(throwExceptions);
if (json == null)
return null;
object result = null;
dynamic reader = null;
FileStream fs = null;
try
{
fs = new FileStream(fileName, FileMode.Open, FileAccess.Read);
var sr = new StreamReader(fs, Encoding.UTF8);
reader = Activator.CreateInstance(JsonTextReaderType, sr);
result = json.Deserialize(reader, objectType);
reader.Close();
}
catch (Exception ex)
{
Debug.WriteLine("JsonNetSerialization Deserialization Error: " + ex.Message);
if (throwExceptions)
throw;
return null;
}
finally
{
if (reader != null)
reader.Close();
if (fs != null)
fs.Close();
}
return result;
}This code is a little more compact since there are no prettifying options to set. Here JsonTextReader is created dynamically
and it receives the output from the Deserialize() operation on the serializer.You can take a look at the full JsonSerializationUtils.cs file on GitHub to see the rest of the
operations, but the string
operations are very similar - the code is fairly repetitive.These generic serialization utilities isolate the dynamic serialization logic that has to deal with the dynamic nature of JSON.NET,
and any code that uses these functions is none the wiser that JSON.NET is dynamically loaded.Using the JsonSerializationUtils WrapperThe final consumer of the SerializationUtils wrapper is an actual ConfigurationProvider, that is responsible for handling reading
and writing JSON values to
and from files. The provider is simple a small wrapper around the SerializationUtils component
and there's very little code to make this work now:The whole provider looks like this:/// <summary>
/// Reads
and Writes configuration settings in .NET config files
and
/// sections. Allows reading
and writing to default or external files
///
and specification of the configuration section that settings are
/// applied to.
/// </summary>
public class JsonFileConfigurationProvider<TAppConfiguration> : ConfigurationProviderBase<TAppConfiguration>
where TAppConfiguration: AppConfiguration, new()
{
/// <summary>
/// Optional - the Configuration file where configuration settings are
/// stored in. If not specified uses the default Configuration Manager
///
and its default store.
/// </summary>
public string JsonConfigurationFile
{
get { return _JsonConfigurationFile; }
set { _JsonConfigurationFile = value; }
}
private string _JsonConfigurationFile = string.Empty;
public override bool Read(AppConfiguration config)
{
var newConfig = JsonSerializationUtils.DeserializeFromFile(JsonConfigurationFile, typeof(TAppConfiguration)) as TAppConfiguration;
if (newConfig == null)
{
if(Write(config))
return true;
return false;
}
DecryptFields(newConfig);
DataUtils.CopyObjectData(newConfig, config, "Provider,ErrorMessage");
return true;
}
/// <summary>
/// Return
/// </summary>
/// <typeparam name="TAppConfig"></typeparam>
/// <returns></returns>
public override TAppConfig Read<TAppConfig>()
{
var result = JsonSerializationUtils.DeserializeFromFile(JsonConfigurationFile, typeof(TAppConfig)) as TAppConfig;
if (result != null)
DecryptFields(result);
return result;
}
/// <summary>
/// Write configuration to XmlConfigurationFile location
/// </summary>
/// <param name="config"></param>
/// <returns></returns>
public override bool Write(AppConfiguration config)
{
EncryptFields(config);
bool result = JsonSerializationUtils.SerializeToFile(config, JsonConfigurationFile,false,true);
// Have to decrypt again to make sure the properties are readable afterwards
DecryptFields(config);
return result;
}
}This incidentally demonstrates how easy it is to create a new provider for the West Wind Application Configuration component. Simply implementing 3 methods will do in most cases.Note this code doesn't have any dynamic dependencies - all that's abstracted away in the JsonSerializationUtils(). From here on, serializing JSON is just a matter of calling the static methods on the SerializationUtils class.Already, there are several other places in some other tools where I use JSON serialization this is coming in very handy. With a couple of lines of code I was able to add JSON.NET support to an older AJAX library that I use replacing quite a bit of code that was previously in use.
And for any other manual JSON
operations (in a couple of apps I use JSON Serialization for 'blob' like document storage) this is also going to be handy.Performance?Some of you might be thinking that using dynamic
and Reflection can't be good for performance.
And you'd be right… In performing some informal testing it looks like the performance of the native code is nearly twice as fast as the dynamic code. Most of the slowness is attributable to type lookups. To test I created a native class that uses an actual reference to JSON.NET
and performance was consistently around 85-90% faster with the referenced code. This will change though depending on the size of objects serialized - the larger the object the more processing time is spent inside the actual dynamically activated components
and the less difference there will be. Dynamic code is always slower, but how much it really affects your application primarily depends on how frequently the dynamic code is called in relation to the non-dynamic code executing. In most situations where dynamic code is used 'to get the process rolling' as I do here the overhead is small enough to not matter.All that being said though - I serialized 10,000 objects in 80ms vs. 45ms so this is hardly slouchy performance. For the configuration component speed is not that important because both read
and write
operations typically happen once on first access
and then every once in a while. But for other
operations - say a serializer trying to handle AJAX requests on a Web Server one would be well served to create a hard dependency.Dynamic Loading - Worth it?Dynamic loading is not something you need to worry about but on occasion dynamic loading makes sense. But there's a price to be paid in added code
and a performance hit which depends on how frequently the dynamic code is accessed. But for some
operations that are not pivotal to a component or application
and are only used under certain circumstances dynamic loading can be beneficial to avoid having to ship extra files adding dependencies
and loading down distributions. These days when you create new projects in Visual Studio with 30 assemblies before you even add your own code, trying to keep file counts under control seems like a good idea. It's not the kind of thing you do on a regular basis, but when needed it can be a useful option in your toolset… © Rick Strahl, West Wind Technologies, 2005-2013Posted in .NET C#
Tweet
!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
