Modern ASP.NET web applications take full advantage of client-side JavaScript
to provide better interactivity and responsiveness. If you are building
an ASP.NET application in the right way, you quickly end up with lots and lots of JavaScript code. When writing server code, you should be writing unit tests. One big advantage of unit tests is that they provide you with a safety net that enable you
to safely modify your existing code – for example, fix bugs, add new features, and make performance enhancements -- without breaking your existing code. Every time you modify your code, you can execute your unit tests
to verify that you have not broken anything. For the same reason that you should write unit tests for your server code, you should write unit tests for your client code. JavaScript is just as susceptible
to bugs as C#. There is no shortage of unit testing frameworks for JavaScript. Each of the major JavaScript libraries has its own unit testing framework. For example, jQuery has QUnit, Prototype has UnitTestJS, YUI has YUI Test, and Dojo has Dojo Objective Harness (DOH). The challenge is integrating a JavaScript unit testing framework with Visual Studio. Visual Studio and Visual Studio ALM provide fantastic support for server-side unit tests. You can easily view the results of running your unit tests in the Visual Studio Test Results window. You can set up a check-in policy which requires that all unit tests pass before your source code can be committed
to the source code repository. In addition, you can set up Team Build
to execute your unit tests automatically. Unfortunately, Visual Studio does not provide “out-of-the-box” support for JavaScript unit tests. MS Test, the unit testing framework included in Visual Studio, does not support JavaScript unit tests. As soon as you leave the server world, you are left on your own. The goal of this blog entry is
to describe one approach
to integrating JavaScript unit tests with MS Test so that you can execute your JavaScript unit tests side-by-side with your C# unit tests. The goal is
to enable you
to execute JavaScript unit tests in exactly the same way as server-side unit tests. You can download the source code described by this project by scrolling
to the end of this blog entry. Rejected Approach: Browser Launchers One popular approach
to executing JavaScript unit tests is
to use a browser as a test-driver. When you use a browser as a test-driver, you open up a browser window
to execute and view the results of executing your JavaScript unit tests. For example, QUnit – the unit testing framework for jQuery – takes this approach. The following HTML page illustrates
how you can use QUnit
to create a unit test for a function named addNumbers(). <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Using QUnit</title>
<link rel="stylesheet" href="http://github.com/jquery/qunit/raw/master/qunit/qunit.css" type="text/css" />
</head>
<body>
<h1 id="qunit-header">QUnit example</h1>
<h2 id="qunit-banner"></h2>
<div id="qunit-testrunner-toolbar"></div>
<h2 id="qunit-userAgent"></h2>
<ol id="qunit-tests"></ol>
<div id="qunit-fixture">test markup, will be hidden</div>
<script type="text/javascript" src="http://code.jquery.com/jquery-latest.js"></script>
<script type="text/javascript" src="http://github.com/jquery/qunit/raw/master/qunit/qunit.js"></script>
<script type="text/javascript">
// The function
to test
function addNumbers(a, b) {
return a+b;
}
// The unit test
test("Test of addNumbers", function () {
equals(4, addNumbers(1,3), "1+3 should be 4");
});
</script>
</body>
</html>
This test verifies that calling addNumbers(1,3) returns the expected value 4. When you open this page in a browser, you can see that this test does, in fact, pass.
The idea is that you can quickly refresh this QUnit HTML JavaScript test driver page in your browser whenever you modify your JavaScript code. In other words, you can keep a browser window open and keep refreshing it over and over while you are developing your application. That way, you can know very quickly whenever you have broken your JavaScript code.
While easy
to setup, there are several big disadvantages
to this approach
to executing JavaScript unit tests:
You must view your JavaScript unit test results in a different location than your server unit test results. The JavaScript unit test results appear in the browser and the server unit test results appear in the Visual Studio Test Results window. Because all of your unit test results don’t appear in a single location, you are more likely
to introduce bugs into your code without noticing it.
Because your unit tests are not integrated with Visual Studio – in particular, MS Test -- you cannot easily include your JavaScript unit tests when setting up check-in policies or when performing automated builds with Team Build.
A more sophisticated approach
to using a browser as a test-driver is
to automate the web browser. Instead of launching the browser and loading the test code yourself, you use a framework
to automate this process. There are several different testing frameworks that support this approach:
· Selenium – Selenium is a very powerful framework for automating browser tests. You can
create your tests by recording a Firefox session or by writing the test driver code in server code such as C#. You can learn more about Selenium at http://seleniumhq.org/.
LTAF – The ASP.NET team uses the Lightweight Test Automation Framework
to test JavaScript code in the ASP.NET framework. You can learn more about LTAF by visiting the project home at CodePlex: http://aspnet.codeplex.com/releases/view/35501
jsTestDriver – This framework uses Java
to automate the browser. jsTestDriver creates a server which can be used
to automate multiple browsers simultaneously. This project is located at http://code.google.com/p/js-test-driver/
TestSwam – This framework, created by John Resig, uses PHP
to automate the browser. Like jsTestDriver, the framework creates a test server. You can open multiple browsers that are automated by the test server. Learn more about TestSwarm by visiting the following address: https://github.com/jeresig/testswarm/wiki
Yeti – This is the framework introduced by Yahoo for automating browser tests. Yeti uses server-side JavaScript and depends on Node.js. Learn more about Yeti at http://www.yuiblog.com/blog/2010/08/25/introducing-yeti-the-yui-easy-testing-interface/
All of these frameworks are great for integration tests – however, they are not the best frameworks
to use for unit tests. In one way or another, all of these frameworks depend on executing tests within the context of a “living and breathing” browser.
If you
create an ASP.NET Unit Test then Visual Studio will launch a web server before executing the unit test. Why is launching a web server so bad? It is not the worst thing in the world. However, it does introduce dependencies that prevent your code from being tested in isolation. One of the defining features of a unit test -- versus
an integration test – is that a unit test tests code in isolation.
Another problem with launching a web server when performing unit tests is that launching a web server can be slow. If you cannot execute your unit tests quickly, you are less likely
to execute your unit tests each and every time you make a code change. You are much more likely
to fall into the pit of failure.
Launching a browser when performing a JavaScript unit test has all of the same disadvantages as launching a web server when performing
an ASP.NET unit test. Instead of testing a unit of JavaScript code in isolation, you are testing JavaScript code within the context of a particular browser.
Using the frameworks listed above for integration tests makes perfect sense. However, I want
to consider a different approach for creating unit tests for JavaScript code.
Using Server-Side JavaScript for JavaScript Unit Tests
A completely different approach
to executing JavaScript unit tests is
to perform the tests outside of any browser. If you really want
to test JavaScript then you should test JavaScript and leave the browser out of the testing process.
There are several ways that you can execute JavaScript on the server outside the context of any browser:
Rhino – Rhino is
an implementation of JavaScript written in Java. The Rhino project is maintained by the Mozilla project. Learn more about Rhino at http://www.mozilla.org/rhino/
V8 – V8 is the open-source Google JavaScript engine written in C++. This is the JavaScript engine used by the Chrome web browser. You can download V8 and embed it in your project by visiting http://code.google.com/p/v8/
JScript – JScript is the JavaScript Script Engine used by Internet Explorer (up
to but not including Internet Explorer 9), Windows Script Host, and Active Server Pages.
Internet Explorer is still the most popular web browser. Therefore, I decided
to focus on using the JScript Script Engine
to execute JavaScript unit tests.
Using the Microsoft Script Control
There are two basic ways that you can pass JavaScript
to the JScript Script Engine and execute the code: use the Microsoft Windows Script Interfaces or use the Microsoft Script Control.
The difficult and proper way
to execute JavaScript using the JScript Script Engine is
to use the Microsoft Windows Script Interfaces. You can learn more about the Script Interfaces by visiting http://msdn.microsoft.com/en-us/library/t9d4xf28(VS.85).aspx
The main disadvantage of using the Script Interfaces is that they are difficult
to use from .NET. There is a great series of articles on using the Script Interfaces from C# located at http://www.drdobbs.com/184406028.
I picked the easier alternative and used the Microsoft Script Control. The Microsoft Script Control is
an ActiveX control that provides a higher level abstraction over the Window Script Interfaces.
You can download the Microsoft Script Control from here:
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d7e31492-2595-49e6-8c02-1426fec693ac
After you download the Microsoft Script Control, you need
to add a reference
to it
to your project. Select the Visual Studio menu option Project, Add Reference
to open the Add Reference dialog. Select the COM tab and add the Microsoft Script Control 1.0.
Using the Script Control is easy. You call the Script Control AddCode() method
to add JavaScript code
to the Script Engine. Next, you call the Script Control Run() method
to run a particular JavaScript function.
The reference documentation for the Microsoft Script Control is located at the MSDN website:
http://msdn.microsoft.com/en-us/library/aa227633%28v=vs.60%29.aspx
Creating the JavaScript Code
to Test
To keep things simple, let’s imagine that you want
to test the following JavaScript function named addNumbers() which simply adds two numbers together:
MvcApplication1\Scripts\Math.js
function addNumbers(a, b) {
return 5;
}
Notice that the addNumbers() method always returns the value 5. Right-now, it will not pass a good unit test.
Create this file and save it in your project with the name Math.js in your MVC project’s Scripts folder (Save the file in your actual MVC application and not your MVC test application).
Creating the JavaScript Test Helper Class
To make it easier
to use the Microsoft Script Control in unit tests, we can
create a helper class. This class contains two methods:
LoadFile() – Loads a JavaScript file. Use this method
to load the JavaScript file being tested or the JavaScript file containing the unit tests.
ExecuteTest() – Executes the JavaScript code. Use this method
to execute a JavaScript unit test.
Here’s the code for the JavaScriptTestHelper class:
JavaScriptTestHelper.cs
using System;
using System.IO;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSScriptControl;
namespace MvcApplication1.Tests {
public class JavaScriptTestHelper : IDisposable {
private ScriptControl _sc;
private TestContext _context;
/// <summary>
/// You need
to use this helper with Unit Tests and not
/// Basic Unit Tests because you need a Test Context
/// </summary>
/// <param name="testContext">Unit Test Test Context</param>
public JavaScriptTestHelper(TestContext testContext) {
if (testContext == null) {
throw new ArgumentNullException("TestContext");
}
_context = testContext;
_sc = new ScriptControl();
_sc.Language = "JScript";
_sc.AllowUI = false;
}
/// <summary>
/// Load the contents of a JavaScript file into the
/// Script Engine.
/// </summary>
/// <param name="path">Path
to JavaScript file</param>
public void LoadFile(string path) {
var fileContents = File.ReadAllText(path);
_sc.AddCode(fileContents);
}
/// <summary>
/// Pass the path of the test that you want
to execute.
/// </summary>
/// <param name="testMethodName">JavaScript function name</param>
public void ExecuteTest(string testMethodName) {
dynamic result = null;
try {
result = _sc.Run(testMethodName, new object[] { });
} catch {
var error = ((IScriptControl)_sc).Error;
if (error != null) {
var description = error.Description;
var line = error.Line;
var column = error.Column;
var text = error.Text;
var source = error.Source;
if (_context != null) {
var details = String.Format("{0} \r\nLine: {1} Column: {2}", source, line, column);
_context.WriteLine(details);
}
}
throw new AssertFailedException(error.Description);
}
}
public void Dispose() {
_sc = null;
}
}
}
Notice that the JavaScriptTestHelper class requires a Test Context
to be instantiated. For this reason, you can use the JavaScriptTestHelper only with a Visual Studio Unit Test and not a Basic Unit Test (These are two different types of Visual Studio project items).
Add the JavaScriptTestHelper file
to your MVC test application (for example, MvcApplication1.Tests).
Creating the JavaScript Unit Test
Next, we need
to create the JavaScript unit test function that we will use
to test the addNumbers() function.
Create a folder in your MVC test project named JavaScriptTests and add the following JavaScript file
to this folder:
MvcApplication1.Tests\JavaScriptTests\MathTest.js
/// <reference path="JavaScriptUnitTestFramework.js"/>
function testAddNumbers() {
// Act
var result = addNumbers(1, 3);
// Assert
assert.areEqual(4, result, "addNumbers did not return right value!");
}
The testAddNumbers() function takes advantage of another JavaScript library named JavaScriptUnitTestFramework.js. This library contains all of the code necessary
to make assertions. Add the following JavaScriptnitTestFramework.js
to the same folder as the MathTest.js file:
MvcApplication1.Tests\JavaScriptTests\JavaScriptUnitTestFramework.js
var assert = {
areEqual: function (expected, actual, message) {
if (expected !== actual) {
throw new Error("Expected value " + expected
+ " is not equal
to " + actual + ". " + message);
}
}
};
There is only one type of assertion supported by this file: the areEqual() assertion. Most likely, you would want
to add additional types of assertions
to this file
to make it easier
to write your JavaScript unit tests.
Deploying the JavaScript Test Files
This step is non-intuitive. When you use Visual Studio
to run unit tests, Visual Studio creates a new folder and executes a copy of the files in your project. After you run your unit tests, your Visual Studio Solution will contain a new folder named TestResults that includes a subfolder for each test run.
You need
to configure Visual Studio
to deploy your JavaScript files
to the test run folder or Visual Studio won’t be able
to find your JavaScript files when you execute your unit tests. You will get
an error that looks something like this when you attempt
to execute your unit tests:
You can configure Visual Studio
to deploy your JavaScript files by adding a Test Settings file
to your Visual Studio Solution. It is important
to understand that you need
to add this file
to your Visual Studio Solution and not a particular Visual Studio project. Right-click your Solution in the Solution Explorer window and select the menu option Add, New Item. Select the Test Settings item and click the Add button.
After you
create a Test Settings file for your solution, you can indicate that you want a particular folder
to be deployed whenever you perform a test run. Select the menu option Test, Edit Test Settings
to edit your test configuration file. Select the Deployment tab and select your MVC test project’s JavaScriptTest folder
to deploy.
Click the Apply button and the Close button
to save the changes and close the dialog.
Creating the Visual Studio Unit Test
The very last step is
to create the Visual Studio unit test (the MS Test unit test). Add a new unit test
to your MVC test project by selecting the menu option Add New Item and selecting the Unit Test project item (Do not select the Basic Unit Test project item):
The difference between a Basic Unit Test and a Unit Test is that a Unit Test includes a Test Context. We need this Test Context
to use the JavaScriptTestHelper class that we created earlier.
Enter the following test method for the new unit test:
[TestMethod]
public void TestAddNumbers() {
var jsHelper = new JavaScriptTestHelper(this.TestContext);
// Load JavaScript files
jsHelper.LoadFile("JavaScriptUnitTestFramework.js");
jsHelper.LoadFile(@"..\..\..\MvcApplication1\Scripts\Math.js");
jsHelper.LoadFile("MathTest.js");
// Execute JavaScript Test
jsHelper.ExecuteTest("testAddNumbers");
}
This code uses the JavaScriptTestHelper
to load three files:
JavaScripUnitTestFramework.js – Contains the assert functions.
Math.js – Contains the addNumbers() function from your MVC application which is being tested.
MathTest.js – Contains the JavaScript unit test function.
Next, the test method calls the JavaScriptTestHelper ExecuteTest() method
to execute the testAddNumbers() JavaScript function.
Running the Visual Studio JavaScript Unit Test
After you complete all of the steps described above, you can execute the JavaScript unit test just like any other unit test. You can use the keyboard combination CTRL-R, CTRL-A
to run all of the tests in the current Visual Studio Solution. Alternatively, you can use the buttons in the Visual Studio toolbar
to run the tests:
(Unfortunately, the Run All Impacted Tests button won’t work correctly because Visual Studio won’t detect that your JavaScript code has changed. Therefore, you should use either the Run Tests in Current Context or Run All Tests in Solution options instead.)
The results of running the JavaScript tests appear side-by-side with the results of running the server tests in the Test Results window. For example, if you Run All Tests in Solution then you will get the following results:
Notice that the TestAddNumbers() JavaScript test has failed. That is good because our addNumbers() function is hard-coded
to always return the value 5.
If you double-click the failing JavaScript test, you can view additional details such as the JavaScript error message and the line number of the JavaScript code that failed:
Summary
The goal of this blog entry was
to explain
an approach
to creating JavaScript unit tests that can be easily integrated with Visual Studio and Visual Studio ALM. I described
how you can use the Microsoft Script Control
to execute JavaScript on the server. By taking advantage of the Microsoft Script Control, we were able
to execute our JavaScript unit tests side-by-side with all of our other unit tests and view the results in the standard Visual Studio Test Results window.
You can download the code discussed in this blog entry from here:
http://StephenWalther.com/downloads/Blog/JavaScriptUnitTesting/JavaScriptUnitTests.zip
Before running this code, you need
to first install the Microsoft Script Control which you can download from here:
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d7e31492-2595-49e6-8c02-1426fec693ac
