Recently I was working on updating a legacy application
to MVC 4 that included free form text input. When I set up the new site my initial approach was
to not allow any rich HTML input, only simple text formatting that would respect a few simple HTML commands for bold, lists etc. and automatically handles line break processing for new lines and paragraphs. This is typical for what I do with most multi-line text input in my apps and it works very well with very little development effort involved. Then the client sprung another note: Oh by the way we have a bunch of customers (real estate agents) who need
to post complete HTML documents. Oh uh! There goes the simple theory. After some discussion and pleading on my part (<snicker>)
to try and
avoid this type of raw HTML input because of potential XSS issues, the client decided
to go ahead and allow raw HTML input anyway. There has been lots of discussions on this subject on StackOverFlow (and here and here) but
to after reading through some of the solutions I didn't really find anything that would work even closely for what I needed. Specifically we need
to be able
to allow just about any HTML markup, with the exception of script code. Remote CSS and Images need
to be loaded, links need
to work and so. While the 'legit' HTML posted by these agents is basic in nature it does span most of the full gamut of HTML (4). Most of the solutions XSS prevention/sanitizer solutions I found were way
to aggressive and rendered the posted output unusable mostly because they tend
to strip any externally loaded content. In short I needed a custom solution. I thought the best solution
to this would be
to use an HTML parser - in this case the Html Agility Pack - and then
to run through all the HTML markup provided and remove any of the blacklisted tags and a number of attributes that are prone
to JavaScript injection. There's much discussion on whether
to use blacklists vs. whitelists in the discussions mentioned above, but I found that whitelists can make sense in simple scenarios where you might allow manual HTML input, but when you need
to allow a larger array of HTML functionality a blacklist is probably easier
to manage as the vast majority of elements and attributes could be allowed. Also white listing gets a bit more complex with HTML5 and the new proliferation of new HTML tags and most new tags generally don't affect XSS issues directly. Pure whitelisting based on elements and attributes also doesn't capture many edge cases (see some of the XSS cheat sheets listed below) so even with a white list, custom logic is still required
to handle many of those edge cases. The Microsoft Web Protection Library (AntiXSS) My first thought was
to check out the Microsoft AntiXSS library. Microsoft has an HTML Encoding and Sanitation library in the Microsoft Web Protection Library (formerly AntiXSS Library) on CodePlex, which provides stricter functions for whitelist encoding and sanitation. Initially I thought the Sanitation class and its static members would do the trick for me,but I found that this library is way too restrictive for my needs. Specifically the Sanitation class strips out images and links which rendered the full HTML from our real estate clients completely useless. I didn't spend much time with it, but apparently I'm not alone if feeling this library is not really useful without some way
to configure operation.
To give you an example of what didn't work for me with the library here's a small and simple HTML fragment that includes script, img and anchor tags. I would expect the script
to be stripped and everything else
to be left intact. Here's the original HTML:var value = "<b>Here</b> <script>alert('hello')</script> we go. Visit the " + "<a href='http://west-wind.com'>West Wind</a> site. " + "<img src='http://west-wind.com/images/new.gif' /> " ;
and the code
to sanitize it with the AntiXSS Sanitize class:@Html.Raw(Microsoft.Security.Application.Sanitizer.GetSafeHtmlFragment(value))
This produced a not so useful sanitized string:
Here we go. Visit the <a>West Wind</a> site.
While it removed the <script> tag (good) it also removed the href from the link and the image tag altogether (bad). In some situations this might be useful, but for most tasks I doubt this is the desired behavior. While links can contain javascript: references and images can 'broadcast' information
to a server, without configuration
to tell the library what
to restrict this becomes useless
to me. I couldn't find any way
to customize the white list, nor is there code available in this 'open source' library on CodePlex.
Using Html Agility Pack for HTML Parsing
The WPL library wasn't going
to cut it. After doing a bit of research I decided the best approach for a custom solution would be
to use an HTML parser and inspect the HTML fragment/document I'm trying
to import. I've used the HTML Agility Pack before for a number of apps where I needed an HTML parser without requiring an instance of a full browser like the Internet Explorer Application object which is inadequate in Web apps. In case you haven't checked out the Html Agility Pack before, it's a powerful HTML parser library that you can use from your .NET code. It provides a simple, parsable HTML DOM model
to full HTML documents or HTML fragments that let you walk through each of the elements in your document. If you've used the HTML or XML DOM in a browser before you'll feel right at home with the Agility Pack.
Blacklist based HTML Parsing
to strip XSS Code
For my purposes of HTML sanitation, the process involved is
to walk the HTML document one element at a time and then check each element and attribute against a blacklist. There's quite a bit of argument of what's better: A whitelist of allowed items or a blacklist of denied items. While whitelists tend
to be more secure, they also require a lot more configuration. In the case of HTML5 a whitelist could be very extensive. For what I need, I only want
to ensure that no JavaScript is executed, so a blacklist includes the obvious <script> tag plus any tag that allows loading of external content including <iframe>, <object>, <embed> and <link> etc. <form> is also excluded
to avoid posting content
to a different location. I also disallow <head> and <meta> tags in particular for my case, since I'm only allowing posting of HTML fragments. There is also some internal logic
to exclude some attributes or attributes that include references
to JavaScript or CSS expressions.
The default tag blacklist reflects my use case, but is customizable and can be added
to.
Here's my HtmlSanitizer implementation:using System.Collections.Generic;
using System.IO;
using System.Xml;
using HtmlAgilityPack;
namespace Westwind.Web.Utilities
{
public class HtmlSanitizer
{
public HashSet<string> BlackList = new HashSet<string>()
{
{ "script" },
{ "iframe" },
{ "form" },
{ "object" },
{ "embed" },
{ "link" },
{ "head" },
{ "meta" }
};
/// <summary>
/// Cleans up an HTML string and removes HTML tags in blacklist
/// </summary>
/// <param name="html"></param>
/// <returns></returns>
public static string SanitizeHtml(string html, params string[] blackList)
{
var sanitizer = new HtmlSanitizer();
if (blackList != null && blackList.Length > 0)
{
sanitizer.BlackList.Clear();
foreach (string item in blackList)
sanitizer.BlackList.Add(item);
}
return sanitizer.Sanitize(html);
}
/// <summary>
/// Cleans up an HTML string by removing elements
/// on the blacklist and all elements that start
/// with onXXX .
/// </summary>
/// <param name="html"></param>
/// <returns></returns>
public string Sanitize(string html)
{
var doc = new HtmlDocument();
doc.LoadHtml(html);
SanitizeHtmlNode(doc.DocumentNode);
//return doc.DocumentNode.WriteTo();
string output = null;
// Use an XmlTextWriter
to create self-closing tags
using (StringWriter sw = new StringWriter())
{
XmlWriter writer = new XmlTextWriter(sw);
doc.DocumentNode.WriteTo(writer);
output = sw.ToString();
// strip off XML doc header
if (!string.IsNullOrEmpty(output))
{
int at = output.IndexOf("?>");
output = output.Substring(at + 2);
}
writer.Close();
}
doc = null;
return output;
}
private void SanitizeHtmlNode(HtmlNode node)
{
if (node.NodeType == HtmlNodeType.Element)
{
// check for blacklist items and remove
if (BlackList.Contains(node.Name))
{
node.Remove();
return;
}
// remove CSS Expressions and embedded script links
if (node.Name == "style")
{
if (string.IsNullOrEmpty(node.InnerText))
{
if (node.InnerHtml.Contains("expression") || node.InnerHtml.Contains("javascript:"))
node.ParentNode.RemoveChild(node);
}
}
// remove script attributes
if (node.HasAttributes)
{
for (int i = node.Attributes.Count - 1; i >= 0; i--)
{
HtmlAttribute currentAttribute = node.Attributes[i];
var attr = currentAttribute.Name.ToLower();
var val = currentAttribute.Value.ToLower();
span style="background: white; color: green">// remove event handlers
if (attr.StartsWith("on"))
node.Attributes.Remove(currentAttribute);
// remove script links
else if (
//(attr == "href" || attr== "src" || attr == "dynsrc" || attr == "lowsrc") &&
val != null &&
val.Contains("javascript:"))
node.Attributes.Remove(currentAttribute);
// Remove CSS Expressions
else if (attr == "style" &&
val != null &&
val.Contains("expression") || val.Contains("javascript:") || val.Contains("vbscript:"))
node.Attributes.Remove(currentAttribute);
}
}
}
// Look through child nodes recursively
if (node.HasChildNodes)
{
for (int i = node.ChildNodes.Count - 1; i >= 0; i--)
{
SanitizeHtmlNode(node.ChildNodes[i]);
}
}
}
}
}
Please note: Use this as a starting point only for your own parsing and review the code for your specific use case! If your needs are less lenient than mine were you can you can make this much stricter by not allowing src and href attributes or CSS links if your HTML doesn't allow it. You can also check links for external URLs and disallow those - lots of options. The code is simple enough
to make it easy
to extend
to fit your use cases more specifically. It's also quite easy
to make this code work using a WhiteList approach if you want
to go that route. The code above is semi-generic for allowing full featured HTML fragments that only disallow script related content.
The Sanitize method walks through each node of the document and then recursively drills into all of its children until the entire document has been traversed. Note that the code here uses an XmlTextWriter
to write output - this is done
to preserve XHTML style self-closing tags which are otherwise left as non-self-closing tags.
The sanitizer code scans for blacklist elements and removes those elements not allowed. Note that the blacklist is configurable either in the instance class as a property or in the static method via the string parameter list. Additionally the code goes through each element's attributes and looks for a host of rules gleaned from some of the XSS cheat sheets listed at the end of the post. Clearly there are a lot more XSS vulnerabilities, but a lot of them apply
to ancient browsers (IE6 and versions of Netscape) - many of these glaring holes (like CSS expressions - WTF IE?) have been removed in modern browsers.
What a Pain
To be honest this is NOT a piece of code that I wanted
to write. I think building anything related
to XSS is better left
to people who have far more knowledge of the topic than I do. Unfortunately, I was unable
to find a tool that worked even closely for me, or even provided a working base. For the project I was working on I had no choice and I'm sharing the code here merely as a base line
to start with and potentially expand on for specific needs. It's sad that Microsoft Web Protection Library is currently such a train wreck - this is really something that should come from Microsoft as the systems vendor or possibly a third party that provides security tools.
Luckily for my application we are dealing with a authenticated and validated users so the user base is fairly well known, and relatively small - this is not a wide open Internet application that's directly public facing. As I mentioned earlier in the post, if I had my way I would simply not allow this type of raw HTML input in the first place, and instead rely on a more controlled HTML input mechanism like MarkDown or even a good HTML Edit control that can provide some limits on what types of input are allowed. Alas in this case I was overridden and we had
to go forward and allow *any* raw HTML posted.
Sometimes I really feel sad that it's come this far - how many good applications and tools have been thwarted by fear of XSS (or worse) attacks? So many things that could be done *if* we had a more secure browser experience and didn't have
to deal with every little script twerp trying
to hack into Web pages and obscure browser bugs. So much time wasted building secure apps, so much time wasted by others trying
to hack apps… We're a funny species - no other species manages
to waste as much time, effort and resources as we humans do :-)
Resources
Code on GitHub
Html Agility Pack
XSS Cheat Sheet
XSS Prevention Cheat Sheet
Microsoft Web Protection Library (AntiXss)
StackOverflow Links:
http://stackoverflow.com/questions/341872/html-sanitizer-for-net
http://blog.stackoverflow.com/2008/06/safe-html-and-xss/
http://code.google.com/p/subsonicforums/source/browse/trunk/SubSonic.Forums.Data/HtmlScrubber.cs?r=61© Rick Strahl, West Wind Technologies, 2005-2012Posted in Security HTML ASP.NET JavaScript
Tweet
!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
