I've written an XML text editor that provides 2 view options for the same XML text, one indented (virtually), the other left-justified. The motivation for the left-justified view is to help users 'see' the whitespace characters they're using for indentation of plain-text or XPath code without interference from indentation that is an automated side-effect of the XML context.

I want to provide visual clues (in the non-editable part of the editor) for the left-justified mode that will help the user, but without getting too elaborate.

I tried just using connecting lines, but that seemed too busy. The best I've come up with so far is shown in a mocked up screenshot of the editor below, but I'm seeking better/simpler alternatives (that don't require too much code).

[Edit]

Taking the heatmap idea (from: @jimp) I get this and 3 alternatives - labelled a, b and c:

The following section describes the accepted answer as a proposal, bringing together ideas from a number of other answers and comments. As this question is now community wiki, please feel free to update this.

NestView

The name for this idea which provides a visual method to improve the readability of nested code without using indentation.

Contour Lines

The name for the differently shaded lines within the NestView

The image above shows the NestView used to help visualise an XML snippet. Though XML is used for this illustration, any other code syntax that uses nesting could have been used for this illustration.

An Overview:

1. The contour lines are shaded (as in a heatmap) to convey nesting level

2. The contour lines are angled to show when a nesting level is being either opened or closed.

3. A contour line links the start of a nesting level to the corresponding end.

4. The combined width of contour lines give a visual impression of nesting level, in addition to the heatmap.

5. The width of the NestView may be manually resizable, but should not change as the code changes. Contour lines can either be compressed or truncated to keep acheive this.

6. Blank lines are sometimes used code to break up text into more digestable chunks. Such lines could trigger special behaviour in the NestView. For example the heatmap could be reset or a background color contour line used, or both.

7. One or more contour lines associated with the currently selected code can be highlighted. The contour line associated with the selected code level would be emphasized the most, but other contour lines could also 'light up' in addition to help highlight the containing nested group

8. Different behaviors (such as code folding or code selection) can be associated with clicking/double-clicking on a Contour Line.

9. Different parts of a contour line (leading, middle or trailing edge) may have different dynamic behaviors associated.

10. Tooltips can be shown on a mouse hover event over a contour line

11. The NestView is updated continously as the code is edited. Where nesting is not well-balanced assumptions can be made where the nesting level should end, but the associated temporary contour lines must be highlighted in some way as a warning.

12. Drag and drop behaviors of Contour Lines can be supported. Behaviour may vary according to the part of the contour line being dragged.

13. Features commonly found in the left margin such as line numbering and colour highlighting for errors and change state could overlay the NestView.

Additional Functionality

The proposal addresses a range of additional issues - many are outside the scope of the original question, but a useful side-effect.

Visually linking the start and end of a nested region

The contour lines connect the start and end of each nested level

Highlighting the context of the currently selected line

As code is selected, the associated nest-level in the NestView can be highlighted

Differentiating between code regions at the same nesting level

In the case of XML different hues could be used for different namespaces. Programming languages (such as c#) support named regions that could be used in a similar way.

Dividing areas within a nesting area into different visual blocks

Extra lines are often inserted into code to aid readability. Such empty lines could be used to reset the saturation level of the NestView's contour lines.

Multi-Column Code View

Code without indentation makes the use of a multi-column view more effective because word-wrap or horizontal scrolling is less likely to be required. In this view, once code has reach the bottom of one column, it flows into the next one:

Usage beyond merely providing a visual aid

As proposed in the overview, the NestView could provide a range of editing and selection features which would be broadly in line with what is expected from a TreeView control. The key difference is that a typical TreeView node has 2 parts: an expander and the node icon. A NestView contour line can have as many as 3 parts: an opener (sloping), a connector (vertical) and a close (sloping).

On Indentation

The NestView presented alongside non-indented code complements, but is unlikely to replace, the conventional indented code view.

It's likely that any solutions adopting a NestView, will provide a method to switch seamlessly between indented and non-indented code views without affecting any of the code text itself - including whitespace characters. One technique for the indented view would be 'Virtual Formatting' - where a dynamic left-margin is used in lieu of tab or space characters. The same nesting-level data used to dynamically render the NestView could also used for the more conventional-looking indented view.

Printing

Indentation will be important for the readability of printed code. Here, the absence of tab/space characters and a dynamic left-margin means that the text can wrap at the right-margin and still maintain the integrity of the indented view. Line numbers can be used as visual markers that indicate where code is word-wrapped and also the exact position of indentation:

Screen Real-Estate: Flat Vs Indented

Addressing the question of whether the NestView uses up valuable screen real-estate:

Contour lines work well with a width the same as the code editor's character width. A NestView width of 12 character widths can therefore accommodate 12 levels of nesting before contour lines are truncated/compressed.

If an indented view uses 3 character-widths for each nesting level then space is saved until nesting reaches 4 levels of nesting, after this nesting level the flat view has a space-saving advantage that increases with each nesting level.

Note: A minimum indentation of 4 character widths is often recommended for code, however XML often manages with less. Also, Virtual Formatting permits less indentation to be used because there's no risk of alignment issues

A comparison of the 2 views is shown below:

Based on the above, its probably fair to conclude that view style choice will be based on factors other than screen real-estate. The one exception is where screen space is at a premium, for example on a Netbook/Tablet or when multiple code windows are open. In these cases, the resizable NestView would seem to be a clear winner.

Use Cases

Examples of real-world examples where NestView may be a useful option:

1. Where screen real-estate is at a premium

   a. On devices such as tablets, notepads and smartphones

   b. When showing code on websites

   c. When multiple code windows need to be visible on the desktop simultaneously

2. Where consistent whitespace indentation of text within code is a priority

3. For reviewing deeply nested code. For example where sub-languages (e.g. Linq in C# or XPath in XSLT) might cause high levels of nesting.

Accessibility

Resizing and color options must be provided to aid those with visual impairments, and also to suit environmental conditions and personal preferences:

Compatability of edited code with other systems

A solution incorporating a NestView option should ideally be capable of stripping leading tab and space characters (identified as only having a formatting role) from imported code. Then, once stripped, the code could be rendered neatly in both the left-justified and indented views without change. For many users relying on systems such as merging and diff tools that are not whitespace-aware this will be a major concern (if not a complete show-stopper).

Other Works:

Visualisation of Overlapping Markup

Published research by Wendell Piez, dated from 2004, addresses the issue of the visualisation of overlapping markup, specifically LMNL. This includes SVG graphics with significant similarities to the NestView proposal, as such, they are acknowledged here.

The visual differences are clear in the images (below), the key functional distinction is that NestView is intended only for well-nested XML or code, whereas Wendell Piez's graphics are designed to represent overlapped nesting.

The graphics above were reproduced - with kind permission - from http://www.piez.org

Sources:

1. Towards Hermenutic Markup
2. Half-steps toward LMNL