Regression Testing Concepts

Test Suite

All authored tests are contained within a Test Suite.  A Test Suite provides a programmatic interface for the adding, removing and finding of tests contained within the suite, provides some test execution settings and sets up the association with a Rule Application.

A persistence provider can be used with the Test Suite to load and save it from a data store.

The NUnit analog of a test suite is a unit test assembly.

Data States

Data States represent the initial state of a test, prior to rule execution.  The XML data contained within them is loaded into their associated Entities when the tests are executed.  Data States may either represent an irVerify TestScenario (serialized RuleSessionState) or a discrete EntityState.

Individual Data States may be shared by one or more Tests.

Tests

Tests represent a collection of Assertions.  They act on a specific context such as an Entity or Independent RuleSet.  They govern whether the RuleSession during Test execution will either ApplyRules, or execute an explicit RuleSet.

If more than one Data State is used by the test, it also handles the mapping of the Data States to temporary linked Entity fields or Independent RuleSet’s Entity arguments.

There is always a new RuleSession and RuleServiceConnection created for each Test’s execution. The following procedure occurs during Test execution:

• A new RuleSession and RuleServiceConnection are created for the test, setting up any EndPoint Overrides that may exist.
• Initial state is loaded into Entities or Independent RuleSets from the Data States.
• The RuleSession calls ApplyRules or executes an explicit RuleSet.
• The Assertions are processed against the final state, producing results. 

The NUnit analog of a Regression Testing Test is a unit test.

Assertions

Assertions validate individual output state values from fields, calculations, collections, notifications or rule elements against expected values asserted at authoring time.  There are many different types of Assertion to choose from.

When they are executed, they generate either a pass or fail, and highlight their expected and actual values.  For a Test to pass, all of its constituent Assertions must also pass.

The NUnit analog of an Assertion is the Assert statement.

Folders

Folders act as both a user interface feature and a means to group common Tests together.  While folders may contain any Regression Testing construct, they are primarily used to provide a hierarchical grouping for Tests.

All Data States are contained within a root Data State folder on the Test Suite.  Although an SDK author may use folders to group Data States hierarchically, this is prohibited in the user interface as it serves no purpose.

There is also a root Test folder on the Test Suite.  This may contain a mixture of Tests and other Folders which may contain their own Folders and Tests.

Tests may be executed at the folder level so that only a specific group of Tests is executed.

The NUnit analog of a Folder is a TestFixture.

Data State Overrides

These provide the ability to override individual state values at Test execution time.  By applying them to a Test, the value of a single field may be modified, which overrides its original input state defined in the Test’s Data State.

This feature may be useful if some of the Rules change in a Rule Application, and the Test’s DataState requires fine-grained adjustment in order to satisfy the Test’s Assertions.