My favorite feature of HTML5, hands-down, is HTML5 local storage (aka DOM storage). By taking advantage of HTML5 local storage, you can dramatically improve
the performance of your data-driven ASP.NET applications by caching data in
the browser persistently. Think of HTML5 local storage like browser cookies, but much better. Like cookies, local storage is persistent. When you add
something to browser local storage, it remains there when
the user returns to
the website (possibly days or months later). Importantly, unlike
the cookie storage limitation of 4KB, you can store up to 10 megabytes in HTML5 local storage. Because HTML5 local storage works with
the latest versions of all modern browsers (IE, Firefox, Chrome, Safari), you can start taking advantage of this HTML5 feature in your applications right now. Why use HTML5 Local Storage? I use HTML5 Local Storage in
the JavaScript Reference application: http://Superexpert.com/JavaScriptReference
The JavaScript Reference application is an HTML5 app that provides an interactive reference
for all of
the syntax elements of JavaScript (You can read more about
the application and download
the source code
for the application here). When you open
the application
for the first time, all of
the entries are transferred from
the server to
the browser (all 300+ entries). All of
the entries are stored in local storage. When you open
the application in
the future, only changes are transferred from
the server to
the browser.
The benefit of this approach is that
the application performs extremely fast. When you click
the details link to view details on a particular entry,
the entry details appear instantly because all of
the entries are stored on
the client machine. When you perform key-up searches, by typing in
the filter textbox, matching entries are displayed very quickly because
the entries are being filtered on
the local machine. This approach can have a dramatic effect on
the performance of any interactive data-driven web application. Interacting with data on
the client is almost always faster than interacting with
the same data on
the server. Retrieving Data from
the Server In
the JavaScript Reference application, I use Microsoft WCF Data Services to expose data to
the browser. WCF Data Services generates a REST interface
for your data automatically. Here are
the steps: Create your database tables in Microsoft SQL Server.
For example, I created a database named ReferenceDB and a database table named Entities. Use
the Entity Framework to generate your data model.
For example, I used
the Entity Framework to generate a class named ReferenceDBEntities and a class named Entities. Expose your data through WCF Data Services. I added a WCF Data Service to my project and modified
the data service class to look like this: using System.Data.Services;
using System.Data.Services.Common;
using System.Web;
using JavaScriptReference.Models;
namespace JavaScriptReference.Services
{
[System.ServiceModel.ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class EntryService : DataService<ReferenceDBEntities> {
// This method is called only once to initialize service-wide policies.
public static void InitializeService(DataServiceConfiguration config) {
config.UseVerboseErrors = true;
config.SetEntitySetAccessRule("*", EntitySetRights.All);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
}
// Define a change interceptor
for the Products entity set.
[ChangeInterceptor("Entries")]
public void OnChangeEntries(Entry entry, UpdateOperations operations) {
if (!HttpContext.Current.Request.IsAuthenticated) {
throw new DataServiceException("Cannot update reference unless authenticated.");
}
}
}
}
The WCF data service is named EntryService. Notice that it derives from DataService<ReferenceEntitites>. Because it derives from DataService<ReferenceEntities>,
the data service exposes
the contents of
the ReferenceEntitiesDB database.
In
the code above, I defined a ChangeInterceptor to prevent un-authenticated users from making changes to
the database. Anyone can retrieve data through
the service, but only authenticated users are allowed to make changes.
After you expose data through a WCF Data Service, you can use jQuery to retrieve
the data by performing an Ajax call.
For example, I am using an Ajax call that looks
something like this to retrieve
the JavaScript entries from
the EntryService.svc data service:
$.ajax({
dataType: "json",
url: “/Services/EntryService.svc/Entries”,
success: function (result) {
var data = callback(result["d"]);
}
});
Notice that you must unwrap
the data using result[“d”]. After you unwrap
the data, you have a JavaScript array of
the entries.
I’m transferring all 300+ entries from
the server to
the client when
the application is opened
for the first time. In other words, I transfer
the entire database from
the server to
the client, once and only once, when
the application is opened
for the first time.
The data is transferred using JSON. Here is a fragment:
{
"d" : [
{
"__metadata": {
"uri": "http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries(1)", "type": "ReferenceDBModel.Entry"
}, "Id": 1, "Name": "Global", "Browsers": "ff3_6,ie8,ie9,c8,sf5,es3,es5", "Syntax": "object", "ShortDescription": "Contains global variables and functions", "FullDescription": "<p>\nThe Global object is determined by
the host environment. In web browsers,
the Global object is
the same as
the windows object.\n</p>\n<p>\nYou can use
the keyword <code>this</code> to refer to
the Global object when in
the global context (outside of any function).\n</p>\n<p>\nThe Global object holds all global variables and functions.
For example,
the following code demonstrates that
the global <code>movieTitle</code> variable refers to
the same thing as <code>window.movieTitle</code> and <code>this.movieTitle</code>.\n</p>\n<pre>\nvar movieTitle = \"Star Wars\";\nconsole.log(movieTitle === this.movieTitle); // true\nconsole.log(movieTitle === window.movieTitle); // true\n</pre>\n", "LastUpdated": "634298578273756641", "IsDeleted": false, "OwnerId": null
}, {
"__metadata": {
"uri": "http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries(2)", "type": "ReferenceDBModel.Entry"
}, "Id": 2, "Name": "eval(string)", "Browsers": "ff3_6,ie8,ie9,c8,sf5,es3,es5", "Syntax": "function", "ShortDescription": "Evaluates and executes JavaScript code dynamically", "FullDescription": "<p>\nThe following code evaluates and executes
the string \"3+5\" at runtime.\n</p>\n<pre>\nvar result = eval(\"3+5\");\nconsole.log(result); // returns 8\n</pre>\n<p>\nYou can rewrite
the code above like this:\n</p>\n<pre>\nvar result;\neval(\"result = 3+5\");\nconsole.log(result);\n</pre>", "LastUpdated": "634298580913817644", "IsDeleted": false, "OwnerId": 1
}
…
]}
I worried about
the amount of time that it would take to transfer
the records. According to Google Chome, it takes about 5 seconds to retrieve all 300+ records on a broadband connection over
the Internet. 5 seconds is a small price to pay to avoid performing any server fetches of
the data in
the future.
And here are
the estimated times using different types of connections using Fiddler:
Notice that using a modem, it takes 33 seconds to download
the database. 33 seconds is a significant chunk of time. So, I would not use
the approach of transferring
the entire database up front if you expect a significant portion of your website audience to connect to your website with a modem.
Adding Data to HTML5 Local Storage
After
the JavaScript entries are retrieved from
the server,
the entries are stored in HTML5 local storage. Here’s
the reference documentation
for HTML5 storage
for Internet Explorer:
http://msdn.microsoft.com/en-us/library/cc197062(VS.85).aspx
You access local storage by accessing
the windows.localStorage object in JavaScript. This object contains key/value pairs.
For example, you can use
the following JavaScript code to add a new item to local storage:
<script type="text/javascript">
window.localStorage.setItem("message", "Hello World!");
</script>
You can use
the Google Chrome Storage tab in
the Developer Tools (hit CTRL-SHIFT I in Chrome) to view items added to local storage:
After you add an item to local storage, you can read it at any time in
the future by using
the window.localStorage.getItem() method:
<script type="text/javascript">
window.localStorage.setItem("message", "Hello World!");
</script>
You only can add strings to local storage and not JavaScript objects such as arrays. Therefore, before adding a JavaScript object to local storage, you need to convert it into a JSON string. In
the JavaScript Reference application, I use a wrapper around local storage that looks
something like this:
function Storage() {
this.get = function (name) {
return JSON.parse(window.localStorage.getItem(name));
};
this.set = function (name, value) {
window.localStorage.setItem(name, JSON.stringify(value));
};
this.clear = function () {
window.localStorage.clear();
};
}
If you use
the wrapper above, then you can add arbitrary JavaScript objects to local storage like this:
var store = new Storage();
// Add array to storage
var products = [
{name:"Fish", price:2.33},
{name:"Bacon", price:1.33}
];
store.set("products", products);
// Retrieve items from storage
var products = store.get("products");
Modern browsers support
the JSON object natively. If you need
the script above to work with older browsers then you should download
the JSON2.js library from:
https://github.com/douglascrockford/JSON-js
The JSON2 library will use
the native JSON object if a browser already supports JSON.
Merging Server Changes with Browser Local Storage
When you first open
the JavaScript Reference application,
the entire database of JavaScript entries is transferred from
the server to
the browser. Two items are added to local storage: entries and entriesLastUpdated.
The first item contains
the entire entries database (a big JSON string of entries).
The second item, a timestamp, represents
the version of
the entries.
Whenever you open
the JavaScript Reference in
the future,
the entriesLastUpdated timestamp is passed to
the server. Only records that have been deleted, updated, or added since entriesLastUpdated are transferred to
the browser.
The OData query to get
the latest updates looks like this:
http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries?$filter=(LastUpdated%20gt%20634301199890494792L)
If you remove URL encoding,
the query looks like this:
http://superexpert.com/javascriptreference/Services/EntryService.svc/Entries?$filter=(LastUpdated gt 634301199890494792L)
This query returns only those entries where
the value of LastUpdated > 634301199890494792 (the version timestamp).
The changes – new JavaScript entries, deleted entries, and updated entries – are merged with
the existing entries in local storage.
The JavaScript code
for performing
the merge is contained in
the EntriesHelper.js file.
The merge() method looks like this:
merge: function (oldEntries, newEntries) {
// concat (this performs
the add)
oldEntries = oldEntries || [];
var mergedEntries = oldEntries.concat(newEntries);
// sort
this.sortByIdThenLastUpdated(mergedEntries);
// prune duplicates (this performs
the update)
mergedEntries = this.pruneDuplicates(mergedEntries);
// delete
mergedEntries = this.removeIsDeleted(mergedEntries);
// Sort
this.sortByName(mergedEntries);
return mergedEntries;
},
The contents of local storage are then updated with
the merged entries.
I spent several hours writing
the merge() method (much longer than I expected). I found two resources to be extremely useful.
First, I wrote extensive unit tests
for the merge() method. I wrote
the unit tests using server-side JavaScript. I describe this approach to writing unit tests in this blog entry.
The unit tests are included in
the JavaScript Reference source code.
Second, I found
the following blog entry to be super useful (thanks Nick!):
http://nicksnettravels.builttoroam.com/post/2010/08/03/OData-Synchronization-with-WCF-Data-Services.aspx
One big challenge that I encountered involved timestamps. I originally tried to store an actual UTC time as
the value of
the entriesLastUpdated item. I quickly discovered that trying to work with dates in JSON turned out to be a big can of worms that I did not want to open. Next, I tried to use a SQL timestamp column. However, I learned that OData cannot handle
the timestamp data type when doing a filter query. Therefore, I ended up using a bigint column in SQL and manually creating
the value when a record is updated.
I overrode
the SaveChanges() method to look
something like this:
public override int SaveChanges(SaveOptions options) {
var changes = this.ObjectStateManager.GetObjectStateEntries(
EntityState.Modified |
EntityState.Added |
EntityState.Deleted);
foreach (var change in changes) {
var entity = change.Entity as IEntityTracking;
if (entity != null) {
entity.LastUpdated = DateTime.Now.Ticks;
}
}
return base.SaveChanges(options);
}
Notice that I assign Date.Now.Ticks to
the entity.LastUpdated property whenever an entry is modified, added, or deleted.
Summary
After building
the JavaScript Reference application, I am convinced that HTML5 local storage can have a dramatic impact on
the performance of any data-driven web application. If you are building a web application that involves extensive interaction with data then I recommend that you take advantage of this new feature included in
the HTML5 standard.
