Structural Coverage (Code Coverage) Analysis :

Test coverage analysis is a two step process, involving requirements-based coverage analysis and structural
coverage analysis. The first step analyzes the test cases in relation to the software requirements to confirm that
the selected test cases satisfy the specified criteria. The second step confirms that the requirements-based test
procedures exercised the code structure. Structural coverage analysis may not satisfy the specified criteria.
Additional guidelines are provided for resolution of such situations as dead code.

Code coverage analysis is the process of:

• Finding areas of a program not exercised by a set of test cases,
• Creating additional test cases to increase coverage, and
• Determining a quantitative measure of code coverage, which is an indirect measure of quality.

An optional aspect of code coverage analysis is:

• Identifying redundant test cases that do not increase coverage.

Code coverage analysis is a structural testing technique (AKA glass box testing and white box testing).
Structural testing compares test program behavior against the apparent intention of the source code.
This contrasts with functional testing (AKA black-box testing), which compares test program behavior against a
requirements specification. Structural testing examines how the program works, taking into account possible
pitfalls in the structure and logic. Functional testing examines what the program accomplishes, without
regard to how it works internally.

The objective of this analysis is to determine which code structure was not exercised by the requirements-based
test procedures. The requirements-based test cases may not have completely exercised the code structure, so
structural coverage analysis is performed and additional verification produced to provide structural coverage.
Guidance includes:

• The analysis should confirm the degree of structural coverage appropriate to the software level.
• The structural coverage analysis may be performed on the Source Code, unless the software level is A
  and the compiler generates object code that is not directly traceable to Source Code statements. Then,
  additional verification should be performed on the object code to establish the correctness of such
  generated code sequences. A compiler-generated array-bound check in the object code is an example
  of object code that is not directly traceable to the Source Code.
• The analysis should confirm the data coupling and control coupling between the code components.

The purpose of structural coverage analysis with the associated structural coverage analysis resolution is to
complement requirements-based testing as follows:

• Provide evidence that the code structure is verified to the degree required for the applicable
  software level
• Provide a means to support demonstration of absence of unintended functions
• Establish the thoroughness of requirements-based testing.

The following sub-categories of tests are defined to achieve the above mentioned objectives:

1. Statement Coverage
2. Decision Coverage
3. Modified Condition / Decision Coverage (MC / DC)

Statement Coverage:

This measure reports whether each executable statement is encountered. To achieve statement coverage, every
executable statement in the program is invoked at least once during software testing. Achieving statement
coverage shows that all code statements are reachable (in the context of DO-178B, reachable based on test
cases developed from the requirements). In summary, this measure is affected more by computational
statements than by decisions.

Consider the following code segment:

By choosing x = 2, y = 0, and z = 4 as input to this code segment, every statement is executed at least once.
Statement coverage by default also include

• Condition Coverage and
• Multiple Conditions Coverage.
• Loop Coverage

Statement coverage is calculated as follows

      Number of Executable Statements Executed
Statement Coverage = ————————————————— * 100 %
                                                    Total Number of Executable Statements

Advantages :

• The chief advantage of this measure is that it can be applied directly to object code and does not require
  processing source code.

Limitations :

• Statement coverage does not report whether loops reach their termination condition – only whether the
  loop body was executed. With C, C++, and Java, this limitation affects loops that contain break
  statements.
• Since do-while loops always execute at least once, statement coverage considers them the same rank
  as non-branching statements.
• Statement coverage is completely insensitive to the logical operators (|| and &&).
• Statement coverage cannot distinguish consecutive switch labels.

Condition Coverage:

Condition coverage reports the true or false outcome of each boolean sub-expression, separated by logical-and
and logical-or if they occur. Condition coverage measures the sub-expressions independently of each other. This
measure is similar to decision coverage but has better sensitivity to the control flow. Condition coverage means
that every condition has been made to take true and false.

The branch condition coverage is calculated as follows

Number of Boolean Operand Values Executed
Branch Condition Coverage = —————————————————- * 100 %
                                                       Total Number of Boolean Operand Values

Multiple Condition Coverage:

Multiple condition coverage reports whether every possible combination of boolean sub-expressions occurs. As
with condition coverage, the sub-expressions are separated by logical-and and logical-or, when present. The
test cases required for full multiple condition coverage of a condition are given by the logical operator truth table
for the condition. Multiple conditions coverage means that every possible combination of the logical condition
must be executed during at least once and every possible condition must evaluated to both true and false.
However, full condition coverage does not guarantee full decision coverage

Limitations :

• A disadvantage of this measure is that it can be tedious to determine the minimum set of test cases
  required, especially for very complex boolean expressions.
• An additional disadvantage of this measure is that the number of test cases required could vary
  substantially among conditions that have similar complexity.

The multiple condition combination coverage is calculated as follows

Multiple Condition Combination Coverage =
Number of Boolean Operand Value Combinations Executed
—————————————————————— * 100 %
Total Number of Boolean Operand Value Combinations

Loop Coverage:

This measure reports whether you executed each loop body zero times, exactly once, and more than once
(consecutively). For do-while loops loop coverage reports whether you executed the body exactly once, and
more than once. The valuable aspect of this measure is determining whether while-loops and for-loops execute
more than once, information not reported by others measure.

Loops shall be tested to verify that the loop executes appropriately.
– If the loop executes for a fixed number of iterations this will be automatically achieved as long as the
loop is executed.
– If the loop executes for a variable number of cycles, test cases shall be included that test the loop at
the minimum and maximum number of cycles.

Where loops set array values, test cases shall be included that use the lowest and highest array indices that
the loop can affect. Every array value that can be affected by the test case shall be examined and the
number of array values shall be identically equal to the number that are set by the loop.
– If this cannot be done, then the unit shall be rejected as not testable. (For example, a unit with a loop
is not testable if during the execution of the loop a single array value is set more than once.)

In case of a static loop (Number of iterations in the loop is controlled by a constant) like
for (I=0; I<10; I++)  this loop would not have 100% loop coverage since it cannot undergo 0 iteration and 1
iteration. At any instant only 2 or more iterations are possible. So the loop coverage will be 1/3  33.33%.

Decision Coverage:

This measure reports whether Boolean expressions tested in control structures (such as the if-statement and
while-statement) evaluated to both true and false. The entire Boolean expression is considered one true-or-false
predicate regardless of whether it contains logical-and or logical-or operators. Additionally, this measure includes
coverage of switch-statement cases, exception handlers, and interrupts handlers. Also known as: branch
coverage, all-edges coverage, basis path coverage, and decision-decision-path testing. “Basis path” testing
selects paths that achieve decision coverage.

Advantages :

• Simplicity without the problems of statement coverage.

Limitations :

• A disadvantage of this measure ignores branches within boolean expressions which occur due to shortcircuit
  operators.

Decision coverage requires two test cases: one for a true outcome and another for a false outcome. For simple
decisions (i.e., decisions with a single condition), decision coverage ensures complete testing of control
constructs. For the decision (A or B), test cases (TF) and (FF) will toggle the decision outcome between true and
false. However, the effect of B is not tested; that is, those test cases cannot distinguish between the decision (A
or B) and the decision A.

The decision coverage is calculated as follows

Number of Executed Decision Outcomes
Decision Coverage = ———————————————- * 100 %
                                      Total Number of Decision Outcomes