Modern ASP.NET web applications take full advantage of client-side JavaScript to provide better interactivity and responsiveness. If you are building an ASP.NET application in
the right way, you quickly end up with lots and lots of JavaScript code. When writing server code, you should be writing unit tests. One big advantage of unit tests is that they provide you with a safety net that enable you to safely modify your existing code – for example, fix bugs, add new features, and make performance enhancements -- without breaking your existing code. Every time you modify your code, you can execute your unit tests to verify that you have not broken anything. For
the same reason that you should write unit tests for your server code, you should write unit tests for your client code. JavaScript is just as susceptible to bugs as C#. There is no shortage of unit testing frameworks for JavaScript. Each of
the major JavaScript libraries has its own unit testing framework. For example, jQuery has QUnit, Prototype has UnitTestJS, YUI has YUI Test, and Dojo has Dojo Objective Harness (DOH).
The challenge is integrating a JavaScript unit testing framework with Visual Studio. Visual Studio and Visual Studio ALM provide fantastic support for server-side unit tests. You can easily view
the results of running your unit tests in
the Visual Studio Test Results window. You can set up a check-in policy which requires that all unit tests pass before your source code can be committed to
the source code repository. In addition, you can set up Team Build to execute your unit tests automatically. Unfortunately, Visual Studio does not provide “out-of-the-box” support for JavaScript unit tests. MS Test,
the unit testing framework included in Visual Studio, does not support JavaScript unit tests. As soon as you leave
the server world, you are left on your own.
The goal of this blog entry is to describe one approach to integrating JavaScript unit tests with MS Test so that you can execute your JavaScript unit tests side-by-side with your C# unit tests.
The goal is to enable you to execute JavaScript unit tests in exactly
the same way as server-side unit tests. You can download
the source code described by this project by scrolling to
the end of this blog entry. Rejected Approach: Browser Launchers One popular approach to executing JavaScript unit tests is to use a browser as a test-driver. When you use a browser as a test-driver, you open up a browser window to execute and view
the results of executing your JavaScript unit tests. For example, QUnit –
the unit testing framework for jQuery – takes this approach.
The following HTML page illustrates how you can use QUnit to create a unit test for a function named addNumbers(). <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Using QUnit</title>
<link rel="stylesheet" href="http://github.com/jquery/qunit/raw/master/qunit/qunit.css" type="text/css" />
</head>
<body>
<h1 id="qunit-header">QUnit example</h1>
<h2 id="qunit-banner"></h2>
<div id="qunit-testrunner-toolbar"></div>
<h2 id="qunit-userAgent"></h2>
<ol id="qunit-tests"></ol>
<div id="qunit-fixture">test markup, will be hidden</div>
<script type="text/javascript" src="http://code.jquery.com/jquery-latest.js"></script>
<script type="text/javascript" src="http://github.com/jquery/qunit/raw/master/qunit/qunit.js"></script>
<script type="text/javascript">
//
The function to test
function addNumbers(a, b) {
return a+b;
}
//
The unit test
test("Test of addNumbers", function () {
equals(4, addNumbers(1,3), "1+3 should be 4");
});
</script>
</body>
</html>
This test verifies that calling addNumbers(1,3) returns
the expected value 4. When you open this page in a browser, you can see that this test does, in fact, pass.
The idea is that you can quickly refresh this QUnit HTML JavaScript test driver page in your browser whenever you modify your JavaScript code. In other words, you can keep a browser window open and keep refreshing it over and over while you are developing your application. That way, you can know very quickly whenever you have broken your JavaScript code.
While easy to setup, there are several big disadvantages to this approach to executing JavaScript unit tests:
You must view your JavaScript unit test results in a different location than your server unit test results.
The JavaScript unit test results appear in
the browser and
the server unit test results appear in
the Visual Studio Test Results window. Because all of your unit test results don’t appear in a single location, you are more likely to introduce bugs into your code without noticing it.
Because your unit tests are not integrated with Visual Studio – in particular, MS Test -- you cannot easily include your JavaScript unit tests when setting up check-in policies or when performing automated builds with Team Build.
A more sophisticated approach to using a browser as a test-driver is to automate
the web browser. Instead of launching
the browser and loading
the test code yourself, you use a framework to automate this process. There are several different testing frameworks that support this approach:
· Selenium – Selenium is a very powerful framework for automating browser tests. You can create your tests by recording a Firefox session or by writing
the test driver code in server code such as C#. You can learn more about Selenium at http://seleniumhq.org/.
LTAF –
The ASP.NET team uses
the Lightweight Test Automation Framework to test JavaScript code in
the ASP.NET framework. You can learn more about LTAF by visiting
the project home at CodePlex: http://aspnet.codeplex.com/releases/view/35501
jsTestDriver – This framework uses Java to automate
the browser. jsTestDriver creates a server which can be used to automate multiple browsers simultaneously. This project is located at http://code.google.com/p/js-test-driver/
TestSwam – This framework, created by John Resig, uses PHP to automate
the browser. Like jsTestDriver,
the framework creates a test server. You can open multiple browsers that are automated by
the test server. Learn more about TestSwarm by visiting
the following address: https://github.com/jeresig/testswarm/wiki
Yeti – This is
the framework introduced by Yahoo for automating browser tests. Yeti uses server-side JavaScript and depends on Node.js. Learn more about Yeti at http://www.yuiblog.com/blog/2010/08/25/introducing-yeti-the-yui-easy-testing-interface/
All of these frameworks are great for integration tests – however, they are not
the best frameworks to use for unit tests. In one way or another, all of these frameworks depend on executing tests within
the context of a “living and breathing” browser.
If you create an ASP.NET Unit Test then Visual Studio will launch a web server before executing
the unit test. Why is launching a web server so bad? It is not
the worst thing in
the world. However, it does introduce dependencies that prevent your code from being tested in isolation. One of
the defining features of a unit test -- versus an integration test – is that a unit test tests code in isolation.
Another problem with launching a web server when performing unit tests is that launching a web server can be slow. If you cannot execute your unit tests quickly, you are less likely to execute your unit tests each and every time you make a code change. You are much more likely to fall into
the pit of failure.
Launching a browser when performing a JavaScript unit test has all of
the same disadvantages as launching a web server when performing an ASP.NET unit test. Instead of testing a unit of JavaScript code in isolation, you are testing JavaScript code within
the context of a particular browser.
Using
the frameworks listed above for integration tests makes perfect sense. However, I want to consider a different approach for creating unit tests for JavaScript code.
Using Server-Side JavaScript for JavaScript Unit Tests
A completely different approach to executing JavaScript unit tests is to perform
the tests outside of any browser. If you really want to test JavaScript then you should test JavaScript and leave
the browser out of
the testing process.
There are several ways that you can execute JavaScript on
the server outside
the context of any browser:
Rhino – Rhino is an implementation of JavaScript written in Java.
The Rhino project is maintained by
the Mozilla project. Learn more about Rhino at http://www.mozilla.org/rhino/
V8 – V8 is
the open-source Google JavaScript engine written in C++. This is
the JavaScript engine used by
the Chrome web browser. You can download V8 and embed it in your project by visiting http://code.google.com/p/v8/
JScript – JScript is
the JavaScript Script Engine used by Internet Explorer (up to but not including Internet Explorer 9), Windows Script Host, and Active Server Pages.
Internet Explorer is still
the most popular web browser. Therefore, I decided to focus on using
the JScript Script Engine to execute JavaScript unit tests.
Using
the Microsoft Script Control
There are two basic ways that you can pass JavaScript to
the JScript Script Engine and execute
the code: use
the Microsoft Windows Script Interfaces or use
the Microsoft Script Control.
The difficult and proper way to execute JavaScript using
the JScript Script Engine is to use
the Microsoft Windows Script Interfaces. You can learn more about
the Script Interfaces by visiting http://msdn.microsoft.com/en-us/library/t9d4xf28(VS.85).aspx
The main disadvantage of using
the Script Interfaces is that they are difficult to use from .NET. There is a great series of articles on using
the Script Interfaces from C# located at http://www.drdobbs.com/184406028.
I picked
the easier alternative and used
the Microsoft Script Control.
The Microsoft Script Control is an ActiveX control that provides a higher level abstraction over
the Window Script Interfaces.
You can download
the Microsoft Script Control from here:
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d7e31492-2595-49e6-8c02-1426fec693ac
After you download
the Microsoft Script Control, you need to add a reference to it to your project. Select
the Visual Studio menu option Project, Add Reference to open
the Add Reference dialog. Select
the COM tab and add
the Microsoft Script Control 1.0.
Using
the Script Control is easy. You call
the Script Control AddCode() method to add JavaScript code to
the Script Engine. Next, you call
the Script Control Run() method to run a particular JavaScript function.
The reference
documentation for
the Microsoft Script Control is located at
the MSDN website:
http://msdn.microsoft.com/en-us/library/aa227633%28v=vs.60%29.aspx
Creating
the JavaScript Code to Test
To keep things simple, let’s imagine that you want to test
the following JavaScript function named addNumbers() which simply adds two numbers together:
MvcApplication1\Scripts\Math.js
function addNumbers(a, b) {
return 5;
}
Notice that
the addNumbers() method always returns
the value 5. Right-now, it will not pass a good unit test.
Create this file and save it in your project with
the name Math.js in your MVC project’s Scripts folder (Save
the file in your actual MVC application and not your MVC test application).
Creating
the JavaScript Test Helper Class
To make it easier to use
the Microsoft Script Control in unit tests, we can create a helper class. This class contains two methods:
LoadFile() – Loads a JavaScript file. Use this method to load
the JavaScript file being tested or
the JavaScript file containing
the unit tests.
ExecuteTest() – Executes
the JavaScript code. Use this method to execute a JavaScript unit test.
Here’s
the code for
the JavaScriptTestHelper class:
JavaScriptTestHelper.cs
using System;
using System.IO;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSScriptControl;
namespace MvcApplication1.Tests {
public class JavaScriptTestHelper : IDisposable {
private ScriptControl _sc;
private TestContext _context;
/// <summary>
/// You need to use this helper with Unit Tests and not
/// Basic Unit Tests because you need a Test Context
/// </summary>
/// <param name="testContext">Unit Test Test Context</param>
public JavaScriptTestHelper(TestContext testContext) {
if (testContext == null) {
throw new ArgumentNullException("TestContext");
}
_context = testContext;
_sc = new ScriptControl();
_sc.Language = "JScript";
_sc.AllowUI = false;
}
/// <summary>
/// Load
the contents of a JavaScript file into
the
/// Script Engine.
/// </summary>
/// <param name="path">Path to JavaScript file</param>
public void LoadFile(string path) {
var fileContents = File.ReadAllText(path);
_sc.AddCode(fileContents);
}
/// <summary>
/// Pass
the path of
the test that you want to execute.
/// </summary>
/// <param name="testMethodName">JavaScript function name</param>
public void ExecuteTest(string testMethodName) {
dynamic result = null;
try {
result = _sc.Run(testMethodName, new object[] { });
} catch {
var error = ((IScriptControl)_sc).Error;
if (error != null) {
var description = error.Description;
var line = error.Line;
var column = error.Column;
var text = error.Text;
var source = error.Source;
if (_context != null) {
var details = String.Format("{0} \r\nLine: {1} Column: {2}", source, line, column);
_context.WriteLine(details);
}
}
throw new AssertFailedException(error.Description);
}
}
public void Dispose() {
_sc = null;
}
}
}
Notice that
the JavaScriptTestHelper class requires a Test Context to be instantiated. For this reason, you can use
the JavaScriptTestHelper only with a Visual Studio Unit Test and not a Basic Unit Test (These are two different types of Visual Studio project items).
Add
the JavaScriptTestHelper file to your MVC test application (for example, MvcApplication1.Tests).
Creating
the JavaScript Unit Test
Next, we need to create
the JavaScript unit test function that we will use to test
the addNumbers() function. Create a folder in your MVC test project named JavaScriptTests and add
the following JavaScript file to this folder:
MvcApplication1.Tests\JavaScriptTests\MathTest.js
/// <reference path="JavaScriptUnitTestFramework.js"/>
function testAddNumbers() {
// Act
var result = addNumbers(1, 3);
// Assert
assert.areEqual(4, result, "addNumbers did not return right value!");
}
The testAddNumbers() function takes advantage of another JavaScript library named JavaScriptUnitTestFramework.js. This library contains all of
the code necessary to make assertions. Add
the following JavaScriptnitTestFramework.js to
the same folder as
the MathTest.js file:
MvcApplication1.Tests\JavaScriptTests\JavaScriptUnitTestFramework.js
var assert = {
areEqual: function (expected, actual, message) {
if (expected !== actual) {
throw new Error("Expected value " + expected
+ " is not equal to " + actual + ". " + message);
}
}
};
There is only one type of assertion supported by this file:
the areEqual() assertion. Most likely, you would want to add additional types of assertions to this file to make it easier to write your JavaScript unit tests.
Deploying
the JavaScript Test Files
This step is non-intuitive. When you use Visual Studio to run unit tests, Visual Studio creates a new folder and executes a copy of
the files in your project. After you run your unit tests, your Visual Studio Solution will contain a new folder named TestResults that includes a subfolder for each test run.
You need to configure Visual Studio to deploy your JavaScript files to
the test run folder or Visual Studio won’t be able to find your JavaScript files when you execute your unit tests. You will get an error that looks something like this when you attempt to execute your unit tests:
You can configure Visual Studio to deploy your JavaScript files by adding a Test Settings file to your Visual Studio Solution. It is important to understand that you need to add this file to your Visual Studio Solution and not a particular Visual Studio project. Right-click your Solution in
the Solution Explorer window and select
the menu option Add, New Item. Select
the Test Settings item and click
the Add button.
After you create a Test Settings file for your solution, you can indicate that you want a particular folder to be deployed whenever you perform a test run. Select
the menu option Test, Edit Test Settings to edit your test configuration file. Select
the Deployment tab and select your MVC test project’s JavaScriptTest folder to deploy.
Click
the Apply button and
the Close button to save
the changes and close
the dialog.
Creating
the Visual Studio Unit Test
The very last step is to create
the Visual Studio unit test (the MS Test unit test). Add a new unit test to your MVC test project by selecting
the menu option Add New Item and selecting
the Unit Test project item (Do not select
the Basic Unit Test project item):
The difference between a Basic Unit Test and a Unit Test is that a Unit Test includes a Test Context. We need this Test Context to use
the JavaScriptTestHelper class that we created earlier.
Enter
the following test method for
the new unit test:
[TestMethod]
public void TestAddNumbers() {
var jsHelper = new JavaScriptTestHelper(this.TestContext);
// Load JavaScript files
jsHelper.LoadFile("JavaScriptUnitTestFramework.js");
jsHelper.LoadFile(@"..\..\..\MvcApplication1\Scripts\Math.js");
jsHelper.LoadFile("MathTest.js");
// Execute JavaScript Test
jsHelper.ExecuteTest("testAddNumbers");
}
This code uses
the JavaScriptTestHelper to load three files:
JavaScripUnitTestFramework.js – Contains
the assert functions.
Math.js – Contains
the addNumbers() function from your MVC application which is being tested.
MathTest.js – Contains
the JavaScript unit test function.
Next,
the test method calls
the JavaScriptTestHelper ExecuteTest() method to execute
the testAddNumbers() JavaScript function.
Running
the Visual Studio JavaScript Unit Test
After you complete all of
the steps described above, you can execute
the JavaScript unit test just like any other unit test. You can use
the keyboard combination CTRL-R, CTRL-A to run all of
the tests in
the current Visual Studio Solution. Alternatively, you can use
the buttons in
the Visual Studio toolbar to run
the tests:
(Unfortunately,
the Run All Impacted Tests button won’t work correctly because Visual Studio won’t detect that your JavaScript code has changed. Therefore, you should use either
the Run Tests in Current Context or Run All Tests in Solution options instead.)
The results of running
the JavaScript tests appear side-by-side with
the results of running
the server tests in
the Test Results window. For example, if you Run All Tests in Solution then you will get
the following results:
Notice that
the TestAddNumbers() JavaScript test has failed. That is good because our addNumbers() function is hard-coded to always return
the value 5.
If you double-click
the failing JavaScript test, you can view additional details such as
the JavaScript error message and
the line number of
the JavaScript code that failed:
Summary
The goal of this blog entry was to explain an approach to creating JavaScript unit tests that can be easily integrated with Visual Studio and Visual Studio ALM. I described how you can use
the Microsoft Script Control to execute JavaScript on
the server. By taking advantage of
the Microsoft Script Control, we were able to execute our JavaScript unit tests side-by-side with all of our other unit tests and view
the results in
the standard Visual Studio Test Results window.
You can download
the code discussed in this blog entry from here:
http://StephenWalther.com/downloads/Blog/JavaScriptUnitTesting/JavaScriptUnitTests.zip
Before running this code, you need to first install
the Microsoft Script Control which you can download from here:
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d7e31492-2595-49e6-8c02-1426fec693ac
