The goal of this blog entry is to describe how you can create custom data sources which you can use with the controls in the WinJS library. In particular, I explain how you can create an IndexedDbDataSource which you can use to store and retrieve data from an IndexedDB database.

If you want to skip ahead, and ignore all of the fascinating content in-between, I’ve included the complete code for the IndexedDbDataSource at the very bottom of this blog entry.

What is IndexedDB?

IndexedDB is a database in the browser. You can use the IndexedDB API with all modern browsers including Firefox, Chrome, and Internet Explorer 10. And, of course, you can use IndexedDB with Metro style apps written with JavaScript.

If you need to persist data in a Metro style app written with JavaScript then IndexedDB is a good option. Each Metro app can only interact with its own IndexedDB databases. And, IndexedDB provides you with transactions, indices, and cursors – the elements of any modern database.

An IndexedDB database might be different than the type of database that you normally use. An IndexedDB database is an object-oriented database and not a relational database. Instead of storing data in tables, you store data in object stores. You store JavaScript objects in an IndexedDB object store.

You create new IndexedDB object stores by handling the upgradeneeded event when you attempt to open a connection to an IndexedDB database. For example, here’s how you would both open a connection to an existing database named TasksDB and create the TasksDB database when it does not already exist:

var reqOpen = window.indexedDB.open(“TasksDB”, 2);
reqOpen.onupgradeneeded = function (evt) {
   var newDB = evt.target.result;
   newDB.createObjectStore("tasks", { keyPath: "id", autoIncrement: true });
};
reqOpen.onsuccess = function () {
    var db = reqOpen.result;
   // Do something with db
};

When you call window.indexedDB.open(), and the database does not already exist, then the upgradeneeded event is raised. In the code above, the upgradeneeded handler creates a new object store named tasks. The new object store has an auto-increment column named id which acts as the primary key column.

If the database already exists with the right version, and you call window.indexedDB.open(), then the success event is raised. At that point, you have an open connection to the existing database and you can start doing something with the database.

You use asynchronous methods to interact with an IndexedDB database. For example, the following code illustrates how you would add a new object to the tasks object store:

var transaction = db.transaction(“tasks”,  “readwrite”);
var reqAdd = transaction.objectStore(“tasks”).add({
  name: “Feed the dog”
});
reqAdd.onsuccess = function() {
  // Tasks added successfully
};

The code above creates a new database transaction, adds a new task to the tasks object store, and handles the success event. If the new task gets added successfully then the success event is raised.

Creating a WinJS IndexedDbDataSource

The most powerful control in the WinJS library is the ListView control. This is the control that you use to display a collection of items.

If you want to display data with a ListView control, you need to bind the control to a data source. The WinJS library includes two objects which you can use as a data source: the List object and the StorageDataSource object. The List object enables you to represent a JavaScript array as a data source and the StorageDataSource enables you to represent the file system as a data source.

If you want to bind an IndexedDB database to a ListView then you have a choice. You can either dump the items from the IndexedDB database into a List object or you can create a custom data source. I explored the first approach in a previous blog entry. In this blog entry, I explain how you can create a custom IndexedDB data source.

Implementing the IListDataSource Interface

You create a custom data source by implementing the IListDataSource interface. This interface contains the contract for the methods which the ListView needs to interact with a data source.

The easiest way to implement the IListDataSource interface is to derive a new object from the base VirtualizedDataSource object. The VirtualizedDataSource object requires a data adapter which implements the IListDataAdapter interface.

Yes, because of the number of objects involved, this is a little confusing. Your code ends up looking something like this:

var IndexedDbDataSource = WinJS.Class.derive(
        WinJS.UI.VirtualizedDataSource,
        function (dbName, dbVersion, objectStoreName, upgrade, error) {
            this._adapter = new IndexedDbDataAdapter(dbName, dbVersion, objectStoreName, upgrade, error);
            this._baseDataSourceConstructor(this._adapter);
        },
        {
            nuke: function () {
                this._adapter.nuke();
            },
            remove: function (key) {
                this._adapter.removeInternal(key);
            }
        }
    );

The code above is used to create a new class named IndexedDbDataSource which derives from the base VirtualizedDataSource class. In the constructor for the new class, the base class _baseDataSourceConstructor() method is called. A data adapter is passed to the _baseDataSourceConstructor() method.

The code above creates a new method exposed by the IndexedDbDataSource named nuke(). The nuke() method deletes all of the objects from an object store. The code above also overrides a method named remove(). Our derived remove() method accepts any type of key and removes the matching item from the object store.

Almost all of the work of creating a custom data source goes into building the data adapter class. The data adapter class implements the IListDataAdapter interface which contains the following methods:

· change()

· getCount()

· insertAfter()

· insertAtEnd()

· insertAtStart()

· insertBefore()

· itemsFromDescription()

· itemsFromEnd()

· itemsFromIndex()

· itemsFromKey()

· itemsFromStart()

· itemSignature()

· moveAfter()

· moveBefore()

· moveToEnd()

· moveToStart()

· remove()

· setNotificationHandler()

· compareByIdentity

Fortunately, you are not required to implement all of these methods. You only need to implement the methods that you actually need.

In the case of the IndexedDbDataSource, I implemented the getCount(), itemsFromIndex(), insertAtEnd(), and remove() methods. If you are creating a read-only data source then you really only need to implement the getCount() and itemsFromIndex() methods.

Implementing the getCount() Method

The getCount() method returns the total number of items from the data source. So, if you are storing 10,000 items in an object store then this method would return the value 10,000.

Here’s how I implemented the getCount() method:

getCount: function () {
    var that = this;
    return new WinJS.Promise(function (complete, error) {
        that._getObjectStore().then(function (store) {
            var reqCount = store.count();
            reqCount.onerror = that._error;
            reqCount.onsuccess = function (evt) {
                complete(evt.target.result);
            };
        });
    });
}

The first thing that you should notice is that the getCount() method returns a WinJS promise. This is a requirement. The getCount() method is asynchronous which is a good thing because all of the IndexedDB methods (at least the methods implemented in current browsers) are also asynchronous.

The code above retrieves an object store and then uses the IndexedDB count() method to get a count of the items in the object store. The value is returned from the promise by calling complete().

Implementing the itemsFromIndex method

When a ListView displays its items, it calls the itemsFromIndex() method. By default, it calls this method multiple times to get different ranges of items.

Three parameters are passed to the itemsFromIndex() method: the requestIndex, countBefore, and countAfter parameters. The requestIndex indicates the index of the item from the database to show. The countBefore and countAfter parameters represent hints. These are integer values which represent the number of items before and after the requestIndex to retrieve. Again, these are only hints and you can return as many items before and after the request index as you please.

Here’s how I implemented the itemsFromIndex method:

itemsFromIndex: function (requestIndex, countBefore, countAfter) {
    var that = this;
    return new WinJS.Promise(function (complete, error) {
        that.getCount().then(function (count) {
            if (requestIndex >= count) {
                return WinJS.Promise.wrapError(new WinJS.ErrorFromName(WinJS.UI.FetchError.doesNotExist));
            }
            var startIndex = Math.max(0, requestIndex - countBefore);
            var endIndex = Math.min(count, requestIndex + countAfter + 1);

            that._getObjectStore().then(function (store) {
                var index = 0;
                var items = [];
                var req = store.openCursor();
                req.onerror = that._error;
                req.onsuccess = function (evt) {
                    var cursor = evt.target.result;

                    if (index < startIndex) {
                        index = startIndex;
                        cursor.advance(startIndex);
                        return;
                    }

                    if (cursor && index < endIndex) {
                        index++;
                        items.push({
                            key: cursor.value[store.keyPath].toString(),
                            data: cursor.value
                        });
                        cursor.continue();
                        return;
                    }

                    results = {
                        items: items,
                        offset: requestIndex - startIndex,
                        totalCount: count
                    };
                    complete(results);
                };
            });
        });
    });
}

In the code above, a cursor is used to iterate through the objects in an object store. You fetch the next item in the cursor by calling either the cursor.continue() or cursor.advance() method. The continue() method moves forward by one object and the advance() method moves forward a specified number of objects.

Each time you call continue() or advance(), the success event is raised again. If the cursor is null then you know that you have reached the end of the cursor and you can return the results.

Some things to be careful about here. First, the return value from the itemsFromIndex() method must implement the IFetchResult interface. In particular, you must return an object which has an items, offset, and totalCount property.

Second, each item in the items array must implement the IListItem interface. Each item should have a key and a data property.

Implementing the insertAtEnd() Method

When creating the IndexedDbDataSource, I wanted to go beyond creating a simple read-only data source and support inserting and deleting objects. If you want to support adding new items with your data source then you need to implement the insertAtEnd() method.

Here’s how I implemented the insertAtEnd() method for the IndexedDbDataSource:

insertAtEnd:function(unused, data) {
    var that = this;
    return new WinJS.Promise(function (complete, error) {
        that._getObjectStore("readwrite").done(function(store) {
            var reqAdd = store.add(data);
            reqAdd.onerror = that._error;
            reqAdd.onsuccess = function (evt) {
                var reqGet = store.get(evt.target.result);
                reqGet.onerror = that._error;
                reqGet.onsuccess = function (evt) {
                    var newItem = {
                        key:evt.target.result[store.keyPath].toString(),
                        data:evt.target.result
                    }
                    complete(newItem);
                };
            };
        });
    });
}

When implementing the insertAtEnd() method, you need to be careful to return an object which implements the IItem interface. In particular, you should return an object that has a key and a data property. The key must be a string and it uniquely represents the new item added to the data source. The value of the data property represents the new item itself.

Implementing the remove() Method

Finally, you use the remove() method to remove an item from the data source. You call the remove() method with the key of the item which you want to remove.

Implementing the remove() method in the case of the IndexedDbDataSource was a little tricky. The problem is that an IndexedDB object store uses an integer key and the VirtualizedDataSource requires a string key. For that reason, I needed to override the remove() method in the derived IndexedDbDataSource class like this:

var IndexedDbDataSource = WinJS.Class.derive(
        WinJS.UI.VirtualizedDataSource,
        function (dbName, dbVersion, objectStoreName, upgrade, error) {
            this._adapter = new IndexedDbDataAdapter(dbName, dbVersion, objectStoreName, upgrade, error);
            this._baseDataSourceConstructor(this._adapter);
        },
        {
            nuke: function () {
                this._adapter.nuke();
            },
            remove: function (key) {
                this._adapter.removeInternal(key);
            }
        }
    );

When you call remove(), you end up calling a method of the IndexedDbDataAdapter named removeInternal() . Here’s what the removeInternal() method looks like:

setNotificationHandler: function (notificationHandler) {
    this._notificationHandler = notificationHandler;
},

removeInternal: function(key) {
    var that = this;
    return new WinJS.Promise(function (complete, error) {
        that._getObjectStore("readwrite").done(function (store) {
            var reqDelete = store.delete (key);
            reqDelete.onerror = that._error;
            reqDelete.onsuccess = function (evt) {
                that._notificationHandler.removed(key.toString());
                complete();
            };
        });
    });
}

The removeInternal() method calls the IndexedDB delete() method to delete an item from the object store. If the item is deleted successfully then the _notificationHandler.remove() method is called.

Because we are not implementing the standard IListDataAdapter remove() method, we need to notify the data source (and the ListView control bound to the data source) that an item has been removed. The way that you notify the data source is by calling the _notificationHandler.remove() method.

Notice that we get the _notificationHandler in the code above by implementing another method in the IListDataAdapter interface: the setNotificationHandler() method. You can raise the following types of notifications using the _notificationHandler:

· beginNotifications()

· changed()

· endNotifications()

· inserted()

· invalidateAll()

· moved()

· removed()

· reload()

These methods are all part of the IListDataNotificationHandler interface in the WinJS library.

Implementing the nuke() Method

I wanted to implement a method which would remove all of the items from an object store. Therefore, I created a method named nuke() which calls the IndexedDB clear() method:

nuke: function () {
    var that = this;
    return new WinJS.Promise(function (complete, error) {
        that._getObjectStore("readwrite").done(function (store) {
            var reqClear = store.clear();
            reqClear.onerror = that._error;
            reqClear.onsuccess = function (evt) {
                that._notificationHandler.reload();
                complete();
            };
        });
    });
}

Notice that the nuke() method calls the _notificationHandler.reload() method to notify the ListView to reload all of the items from its data source. Because we are implementing a custom method here, we need to use the _notificationHandler to send an update.

Using the IndexedDbDataSource

To illustrate how you can use the IndexedDbDataSource, I created a simple task list app. You can add new tasks, delete existing tasks, and nuke all of the tasks.

You delete an item by selecting an item (swipe or right-click) and clicking the Delete button.

Here’s the HTML page which contains the ListView, the form for adding new tasks, and the buttons for deleting and nuking tasks:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <title>DataSources</title>

    <!-- WinJS references -->
    <link href="//Microsoft.WinJS.1.0.RC/css/ui-dark.css" rel="stylesheet" />
    <script src="//Microsoft.WinJS.1.0.RC/js/base.js"></script>
    <script src="//Microsoft.WinJS.1.0.RC/js/ui.js"></script>

    <!-- DataSources references -->
    <link href="indexedDb.css" rel="stylesheet" />
    <script type="text/javascript" src="indexedDbDataSource.js"></script>
    <script src="indexedDb.js"></script>
</head>
<body>

    <div id="tmplTask" data-win-control="WinJS.Binding.Template">
        <div class="taskItem">
            Id: <span data-win-bind="innerText:id"></span>
            <br /><br />
            Name: <span data-win-bind="innerText:name"></span>
        </div>
    </div>

    <div id="lvTasks"
        data-win-control="WinJS.UI.ListView"
        data-win-options="{
            itemTemplate: select('#tmplTask'),
            selectionMode: 'single'
        }"></div>

    <form id="frmAdd">
        <fieldset>
            <legend>Add Task</legend>
            <label>New Task</label>
            <input id="inputTaskName" required />
            <button>Add</button>
        </fieldset>
    </form>

    <button id="btnNuke">Nuke</button>
    <button id="btnDelete">Delete</button>

</body>
</html>

And here is the JavaScript code for the TaskList app:

/// <reference path="//Microsoft.WinJS.1.0.RC/js/base.js" />
/// <reference path="//Microsoft.WinJS.1.0.RC/js/ui.js" />

function init() {

    WinJS.UI.processAll().done(function () {
        var lvTasks = document.getElementById("lvTasks").winControl;

        // Bind the ListView to its data source
        var tasksDataSource = new DataSources.IndexedDbDataSource("TasksDB", 1, "tasks", upgrade);
        lvTasks.itemDataSource = tasksDataSource;

        // Wire-up Add, Delete, Nuke buttons
        document.getElementById("frmAdd").addEventListener("submit", function (evt) {
            evt.preventDefault();
            tasksDataSource.beginEdits();
            tasksDataSource.insertAtEnd(null, {
                name: document.getElementById("inputTaskName").value
            }).done(function (newItem) {
                tasksDataSource.endEdits();
                document.getElementById("frmAdd").reset();
                lvTasks.ensureVisible(newItem.index);
            });
        });

        document.getElementById("btnDelete").addEventListener("click", function () {
            if (lvTasks.selection.count() == 1) {
                lvTasks.selection.getItems().done(function (items) {
                    tasksDataSource.remove(items[0].data.id);
                });
            }
        });

        document.getElementById("btnNuke").addEventListener("click", function () {
            tasksDataSource.nuke();
        });

        // This method is called to initialize the IndexedDb database
        function upgrade(evt) {
            var newDB = evt.target.result;
            newDB.createObjectStore("tasks", { keyPath: "id", autoIncrement: true });
        }

    });
}

document.addEventListener("DOMContentLoaded", init);

The IndexedDbDataSource is created and bound to the ListView control with the following two lines of code:

var tasksDataSource = new DataSources.IndexedDbDataSource("TasksDB", 1, "tasks", upgrade); 

lvTasks.itemDataSource = tasksDataSource; 

The IndexedDbDataSource is created with four parameters: the name of the database to create, the version of the database to create, the name of the object store to create, and a function which contains code to initialize the new database.

The upgrade function creates a new object store named tasks with an auto-increment property named id:

function upgrade(evt) { 

  var newDB = evt.target.result; 

  newDB.createObjectStore("tasks", { keyPath: "id", autoIncrement: true }); 

} 

The Complete Code for the IndexedDbDataSource

Here’s the complete code for the IndexedDbDataSource:

(function () {

    /************************************************
    * The IndexedDBDataAdapter enables you to work
    * with a HTML5 IndexedDB database.
    *************************************************/

    var IndexedDbDataAdapter = WinJS.Class.define(
        function (dbName, dbVersion, objectStoreName, upgrade, error) {
            this._dbName = dbName;  // database name
            this._dbVersion = dbVersion;  // database version
            this._objectStoreName = objectStoreName; // object store name
            this._upgrade = upgrade; // database upgrade script
            this._error = error || function (evt) { console.log(evt.message); };
        },
        {

            /*******************************************
            *  IListDataAdapter Interface Methods
            ********************************************/

            getCount: function () {
                var that = this;
                return new WinJS.Promise(function (complete, error) {
                    that._getObjectStore().then(function (store) {
                        var reqCount = store.count();
                        reqCount.onerror = that._error;
                        reqCount.onsuccess = function (evt) {
                            complete(evt.target.result);
                        };
                    });
                });
            },

            itemsFromIndex: function (requestIndex, countBefore, countAfter) {
                var that = this;
                return new WinJS.Promise(function (complete, error) {
                    that.getCount().then(function (count) {
                        if (requestIndex >= count) {
                            return WinJS.Promise.wrapError(new WinJS.ErrorFromName(WinJS.UI.FetchError.doesNotExist));
                        }
                        var startIndex = Math.max(0, requestIndex - countBefore);
                        var endIndex = Math.min(count, requestIndex + countAfter + 1);

                        that._getObjectStore().then(function (store) {
                            var index = 0;
                            var items = [];
                            var req = store.openCursor();
                            req.onerror = that._error;
                            req.onsuccess = function (evt) {
                                var cursor = evt.target.result;

                                if (index < startIndex) {
                                    index = startIndex;
                                    cursor.advance(startIndex);
                                    return;
                                }

                                if (cursor && index < endIndex) {
                                    index++;
                                    items.push({
                                        key: cursor.value[store.keyPath].toString(),
                                        data: cursor.value
                                    });
                                    cursor.continue();
                                    return;
                                }

                                results = {
                                    items: items,
                                    offset: requestIndex - startIndex,
                                    totalCount: count
                                };
                                complete(results);
                            };
                        });
                    });
                });
            },

            insertAtEnd:function(unused, data) {
                var that = this;
                return new WinJS.Promise(function (complete, error) {
                    that._getObjectStore("readwrite").done(function(store) {
                        var reqAdd = store.add(data);
                        reqAdd.onerror = that._error;
                        reqAdd.onsuccess = function (evt) {
                            var reqGet = store.get(evt.target.result);
                            reqGet.onerror = that._error;
                            reqGet.onsuccess = function (evt) {
                                var newItem = {
                                    key:evt.target.result[store.keyPath].toString(),
                                    data:evt.target.result
                                }
                                complete(newItem);
                            };
                        };
                    });
                });
            },

            setNotificationHandler: function (notificationHandler) {
                this._notificationHandler = notificationHandler;
            },

            /*****************************************
            *  IndexedDbDataSource Method
            ******************************************/

            removeInternal: function(key) {
                var that = this;
                return new WinJS.Promise(function (complete, error) {
                    that._getObjectStore("readwrite").done(function (store) {
                        var reqDelete = store.delete (key);
                        reqDelete.onerror = that._error;
                        reqDelete.onsuccess = function (evt) {
                            that._notificationHandler.removed(key.toString());
                            complete();
                        };
                    });
                });
            },

            nuke: function () {
                var that = this;
                return new WinJS.Promise(function (complete, error) {
                    that._getObjectStore("readwrite").done(function (store) {
                        var reqClear = store.clear();
                        reqClear.onerror = that._error;
                        reqClear.onsuccess = function (evt) {
                            that._notificationHandler.reload();
                            complete();
                        };
                    });
                });
            },

            /*******************************************
            *  Private Methods
            ********************************************/

            _ensureDbOpen: function () {
                var that = this;

                // Try to get cached Db
                if (that._cachedDb) {
                    return WinJS.Promise.wrap(that._cachedDb);
                }

                // Otherwise, open the database
                return new WinJS.Promise(function (complete, error, progress) {
                    var reqOpen = window.indexedDB.open(that._dbName, that._dbVersion);
                    reqOpen.onerror = function (evt) {
                        error();
                    };
                    reqOpen.onupgradeneeded = function (evt) {
                        that._upgrade(evt);
                        that._notificationHandler.invalidateAll();
                    };
                    reqOpen.onsuccess = function () {
                        that._cachedDb = reqOpen.result;
                        complete(that._cachedDb);
                    };
                });
            },

            _getObjectStore: function (type) {
                type = type || "readonly";
                var that = this;
                return new WinJS.Promise(function (complete, error) {
                    that._ensureDbOpen().then(function (db) {
                        var transaction = db.transaction(that._objectStoreName, type);
                        complete(transaction.objectStore(that._objectStoreName));
                    });
                });
            },

            _get: function (key) {
                return new WinJS.Promise(function (complete, error) {
                    that._getObjectStore().done(function (store) {
                        var reqGet = store.get(key);
                        reqGet.onerror = that._error;
                        reqGet.onsuccess = function (item) {
                            complete(item);
                        };
                    });
                });
            }

        }
    );

    var IndexedDbDataSource = WinJS.Class.derive(
        WinJS.UI.VirtualizedDataSource,
        function (dbName, dbVersion, objectStoreName, upgrade, error) {
            this._adapter = new IndexedDbDataAdapter(dbName, dbVersion, objectStoreName, upgrade, error);
            this._baseDataSourceConstructor(this._adapter);
        },
        {
            nuke: function () {
                this._adapter.nuke();
            },

            remove: function (key) {
                this._adapter.removeInternal(key);
            }

        }
    );

    WinJS.Namespace.define("DataSources", {
        IndexedDbDataSource: IndexedDbDataSource
    });

})();

Summary

In this blog post, I provided an overview of how you can create a new data source which you can use with the WinJS library. I described how you can create an IndexedDbDataSource which you can use to bind a ListView control to an IndexedDB database.

While describing how you can create a custom data source, I explained how you can implement the IListDataAdapter interface. You also learned how to raise notifications — such as a removed or invalidateAll notification — by taking advantage of the methods of the IListDataNotificationHandler interface.