Elementary comparison testing

Elementary comparison testing (ECT) is a formal white-box control flow test design method. Its purpose is detailed testing of important and complex functionality. Tests to assess the coverage of conditions are based on the code or pseudocode behind the functionality being tested. Restricted from multiple-condition coverage and basis path testing, coverage of all independent isolated condition paths is reached through modified condition/decision coverage (MC/DC). Isolated conditions are forming connected situations creating test cases. The independence of a condition is demonstrated by changing the condition value of each condition in isolation. Each relevant condition value is covered by test cases.

A test case

A test case consists of a logical consonant path from start to end passing one or many decisions. Contradicting situations are excluded and deduced from the test case matrix. The MC/DC approach isolates every condition neglecting all possible subpath combinations and path coverage

$T=n+1$

where

• T is the amount of test cases per decision, and
• n the amount of conditions.

The decision $d_{i}$ consists of a combination of elementary conditions

${\begin{aligned}\Sigma &=\{0,1\}\\C&=\{c_{0},c_{1},c_{2},c_{3},...,c_{n}\}\end{aligned}}$

$\epsilon :C\rightarrow \Sigma \times C$

$D\subseteq C^{*};d_{i}\in D.$

The transition function $\alpha$ is defined as

$\alpha :D\times \Sigma ^{*}\rightarrow \Sigma \times D.$

Given the transition $\vdash$

$\vdash \subseteq (\Sigma \times D\times \Sigma ^{*})\times (\Sigma \times D\times \Sigma ^{*})$

$S_{j}=(b_{j},d_{m},v_{j})\vdash (b_{j+1},d_{n},v_{j+1})$
$E_{j}=(a_{j},c_{j})\vdash (a_{j+1},c_{k})$
$(b_{j+1},d_{n})=\alpha (d_{m},v_{j});(b_{j+1},c_{k})=\epsilon (c_{j});a_{j}\in \Sigma ,$

the isolated test path $P_{m}$ consists of

${\begin{aligned}P_{m}&=(b_{0},d_{0},v_{0})\vdash ...\vdash (b_{i},d_{i},v_{i})\vdash ^{*}(b_{n},d_{n},v_{n})\\&=(b_{0},c_{0})\vdash ...\vdash (b_{m},c_{m})\vdash ^{*}(b_{n},c_{n})\end{aligned}}$

$b_{i}\in \Sigma ;c_{m}\in d_{i};v\in C^{*};d_{0}=S;d_{n}=E.$

Test case graph

A test case graph illustrates all the necessary independent paths (test cases) to cover all isolated conditions. Conditions are represented by nodes and condition values (situations) by edges. All program situations are addressed by an edge. Each situation is connected to one preceding and successive condition. Test cases might overlap due to isolated conditions.

Inductive Proof Isolated Condition Paths

The elementary comparison testing method can be proofed inductively.

Figure 2: ECT Inductive Proof Anchor

There are $r=2^{n}$ possible condition value combinations

$\forall {i}\in \{1,...,n\}\ c_{i}\mapsto \{0,\ 1\}$.

When each condition $c_{i}$ is isolated, the amount of relevant test cases $T$ per decision is:

$T=\log _{2}(r)+1=n+1.$

Figure 3: ECT Inductive Proof End

$\forall {i}\in \{1,...,n\}$ there are $0<e<i+1$ edges connecting from preceding nodes $c_{i}$ and $s=2$ edges towards succeeding nodes from $c_{i}$.

Each individual condition $c_{i}$ connects with one path $\forall {i}\in \{1,...,n-1\}\ c_{i}\mapsto \{0,\ 1\}$ from the maximal possible $n$ connecting to $c_{n}$ isolating $c_{n}$. All predecessor conditions $c_{i};\ i<n$ and respective paths are isolated, therefore when one node (condition) is added the total amount of paths from start to end (test cases) increases by:

${\begin{aligned}T&=n-1+2\\&=n+1.\end{aligned}}$

$\ q.\ e.\ d.$

Test Case Design Steps

1. Identify decisions
2. Determine test situations per decision point (Modified Condition / Decision Coverage)
3. Create logical test case matrix
4. Create physical test cases matrix

Example

This example explains in detail ETC applied to a holiday booking system. The discount system offers reduced vacations. The offered discounts are $-10\%$ for average customers, $-20\%$ for expensive vacations and members otherwise $0\%$ discount. The example has undertaken detailed testing creating logical and physical test cases isolating all conditions.

Figure 4: ECT Example Control Flow Graph

Pseudocode

if days > 15 or price > 1000 or member
  return -0.2
else if (days > 8 and days < 15 or price>500 and price < 1000) and workday
  return -0.1
else
  return 0.0


Factors

• Number of days: $<8;\ 8-15;\ >15$
• Price (euros): $<500;\ 500-1000;\ >1000$
• Membership card: $none;\ silver;\ gold;\ platinum$
• Departure date: $workday;\ weekend;\ holiday$

$T=3\times 3\times 4\times 3=108$ possible combinations (test cases).

Step 1 Decisions

Figure 5: ECT Example D2 Conditions

${\begin{aligned}d_{1}&=days>15\ or\ price>1000\ Eur\ or\ member\\c_{1}&=days>15\\c_{2}&=price>1000\\c_{3}&=member\\\end{aligned}}$

${\begin{aligned}d_{2}&=(8<days<15\ or\ 500<price<1000\ Eur)\ and\ workday\\c_{4}&=8<days<15\\c_{5}&=500<price<1000\ Eur\\c_{6}&=workday\\\end{aligned}}$

Step 2: MC/DC Matrix

Table 1: Example D1 MC/DC

		Outcome
Decision D1		1			0
Conditions		c1	c2	c3	c1	c2	c3
c1	$days>15$	1	0	0	0	0	0
c2	$price>1000$	0	1	0	0	0	0
c3	$member$	0	0	1	0	0	0

Table 2: Example D2 MC/DC

		Outcome
Decision D2		1			0
Conditions		c4	c5	c6	c4	c5	c6
c4	$8<days<15$	1	0	1	0	0	1
c5	$500<price<1000$	0	1	1	0	0	1
c6	$workday$	1	0	1	1	0	0

The highlighted diagonals in the MC/DC Matrix are describing the isolated conditions:

$(c_{i},c_{i})\mapsto \{1,0\}$

all duplicate situations are striked like proven.

Step 3: Logical Test Case Matrix

Table 3: Example Logical Test Case Matrix

Situation $S_{j}$	$T_{1}$	$T_{2}$	$T_{3}$	$T_{4}$	$T_{5}$	$T_{6}$	$T_{7}$
$\alpha (d_{1},\mathbf {1} 00)\mapsto (1,E)$	x
$\alpha (d_{1},\mathbf {0} 00)\mapsto (0,d_{2})$		x			x	x	x
$\alpha (d_{1},0\mathbf {1} 0)\mapsto (1,E)$			x
$\alpha (d_{1},00\mathbf {1} )\mapsto (1,E)$				x
$\alpha (d_{2},\mathbf {1} 01)\mapsto (1,E)$		x
$\alpha (d_{2},\mathbf {0} 01)\mapsto (1,E)$						x
$\alpha (d_{2},0\mathbf {1} 1)\mapsto (1,E)$					x
$\alpha (d_{2},11\mathbf {0} )\mapsto (0,E)$							x

Test cases are formed by marking situations. For every decision $d_{i};\ 0<i<n+1$ a succeeding and preceding consonant subpath is searched till every connected path has a start $S$ and an end $E$:

${\begin{aligned}T_{1}&=(d_{1},100)\vdash (1,E)\\T_{2}&=(d_{1},000)\vdash (0,d_{2},100)\vdash (1,E)\\T_{3}&=(d_{1},010)\vdash (1,E)\\\end{aligned}}$
$...$
$T_{n+1}$

Step 4: Physical Test Case Matrix

Table 4: Example Physical Test Cases

Factor\Test Case	$T_{1}$	$T_{2}$	$T_{3}$	$T_{4}$	$T_{5}$	$T_{6}$	$T_{7}$
days	16	14			8	8	8
price			1100			600
departure							sa
member				silver
Result
0							0
-10		1			1	1
-20	1		1	1

Physical test cases are created from logical test cases by filling in actual value representations and their respective result.

Test Case Graph

Figure 6: ECT Example Test Case Graph

In the example test case graph, all test cases and their isolated conditions are marked by colors and the remaining paths are implicitly passed.

See also

• Multiple condition coverage
• Control flow graph
• Decision to decision path