Enabling Enhanced Whitespace Handling

Enabling Enhanced Whitespace Handling

Whitespace processing is often a key consideration when XML documents are compared. This document outlines how DeltaNova’s whitespace-processing features can be utilised.

See a more complete description of whitespace processing in the following sections.

• Managing Whitespace

• Lexical Preservation (see references to ignorableWhitespace)

Ignorable Whitespace

LexicalPreservation

Ignorable whitespace refers to text nodes with whitespace-only characters found in the XML tree in places where text is not allowed (these are most frequently added for formatting purposes). DeltaNova's LexicalPreservation filter can identify and treat such nodes specially:

• DeltaNova exploits a referenced XML Schema to identify ignorable whitespace nodes, in addition to referenced DTDs.

• If preserveIgnorableWhitespace is set to true, text nodes identified as ignorableWhitespace are wrapped in a preserve:ignorable element.

• There are ProcessingMode and OutputType LexicalPreservation options for specifying how ignorable whitespace nodes are treated in the output pipeline.

• A grammar attribute is added to the root element of each input XML document, with a value of dtd or schema. This is added to indicate whether the LexicalPreservation filter exploits DTD or XML Schema information respectively for whitespace processing.

• To prevent normalization by a subsequent NormalizeSpace filter, a mixed-content attribute with a value of true is added to any element where text nodes are allowed by the DTD or Schema.

NormalizeSpace

The NormalizeSpace filter will not normalize whitespace-only text nodes if a mixed-content attribute with a value of true is found on the parent element. All other whitespace-only text nodes are treated as ignorable whitespace and removed. The behaviour has been enhanced in the following ways for cases where a DTD or XML Schema for the input XML has not been loaded.

• For each element name found in the input XML, a pre-normalization filter determines whether non-whitespace text can occur in immediate child text nodes. All elements identified as such are marked with a mixed-content="true" attribute.

• If no DTD or XML Schema information was added by LexicalPreservation, and no grammar attribute is found on the root element of the input XML, the pre-normalization filter adds a grammar attribute with a value of inferred.

Preserving Whitespace in Text Content

The NormalizeSpace filter reduces any sequence of whitespace characters in significant text content to a single space character unless an xml:space="preserve" attribute was found on an ancestor element. This behaviour has also extends as follows:

• The pre-normalization filter analyzes whitespace occurring in significant text content in the input XML. A deltaxml:space="preserve" attribute is then added to the parent element if the whitespace characteristics are significantly different to that which would be expected for a normally indented text node in formatted XML.

• Whitespace-only text nodes that are significant (i.e. not ignorable) are considered to have special meaning and are therefore marked by the pre-normalization filter with a deltaxml:space="preserve" attribute to prevent normalization.

Keeping Whitespace Metadata

As described in the previous sections, information is added by the LexicalPreservation and NormalizeSpace filters to the input XML to assist with whitespace processing. This information, kept as grammar, mixed-content, and space attributes, may in some cases be useful for formatting the comparison result (DeltaNova’s folding DiffReport exploits this).

For this reason there is now a new LexicalPreservation PreserveContentModel setting:

• When the setPreserveContentModel setting is true, the grammar, mixed-content, and space attributes are preserved throughout the pipeline, otherwise they will be removed.

• The PreserveContentModel setting works by selecting which namespace is used for the content model attributes. The deltaxml namespace is used when PreserveContentModel is false otherwise, the preserve namespace is used.

• Attributes used for content model information are likely to be removed by the Compare pipeline if they are in the deltaxml namespace, but will always be preserved if in the preserve namespace.

• The option of using preserve and deltaxml namespaces also provides compatibility with the mixed-content attribute which can be added by a custom input filter. The preserve and deltaxml namespaces are:

  • http://www.deltaxml.com/ns/preserve

  • http://www.deltaxml.com/ns/well-formed-delta-v1

Whitespace Processing In Practice

The type of comparator used for comparison determines whether whitespace-processing filters are controlled implicitly through comparator properties or by adding the filters explicitly. This is summarised below.

Filters affecting whitespace processing - '✓' indicates implicit control via comparator properties

DocumentComparator (or DCP)

The whitespace processing changes are fully integrated into the DocumentComparator.

PipelinedComparator (or DXP)

For LexicalPreservation, whitespace processing is managed automatically through LexicalPreservationConfig properties.

Because the NormalizeSpace filter is added to the pipeline explicitly, the pre-normalization filter should be added immediately before this (unless an XML Schema or DTD will always be loaded). This is added as the resource XSLT filter whitespace-detection.xsl.