Skip to content

Ch 3: Neural Networks ​

Official Reference

ISTQB® CT-AI Syllabus v2.0, Chapter 3 (375 minutes), section 3.4, Learning Objectives (LOs) AI-3.4.1 and AI-3.4.3 (K2), and Hands-on Objective HO-3.4.2 (H1). Official syllabus and documents

Exam weight: part of the pooled Chapter 3 K2 questions. The official sample exam asks "what happens next in training after the loss is calculated?" (AI-3.4.1). An additional sample question asks why structural coverage alone is not sufficient (AI-3.4.3). The three coverage keywords are K1 definitions you must know.

Artificial neural networks were originally designed to mimic the human brain, a network of interconnected biological neurons. You don't need the mathematics for the exam, but you do need to know how information flows through the network, how training adjusts it, and how testers can measure which parts of it a test suite has exercised.

Neural network: A network of primitive processing elements connected by weighted links with adjustable weights, in which each element produces a value by applying a nonlinear function to its input values, and transmits it to other elements or presents it as an output value.
Perceptron: A neural network with just one layer and one neuron.
Deep neural network: A neural network comprised of several layers of neurons. (Syllabus Chapter 9 adds the synonym: multi-layer perceptron)
Source: ISTQB® Glossary

The single-layer perceptron is one of the earliest artificial neural networks. It can be used for supervised learning of binary classifiers for linearly separable problems, i.e., deciding whether an input belongs to a class or not. Example: separating spam from non-spam emails by learning to divide the two categories' features with a straight line in the input space. Most current neural networks are deep neural networks because they have several layers. Fully connected networks can be considered multi-layer perceptrons.

3.4.1 Structure and Working of a Deep Neural Network (K2) ​

mermaid
flowchart LR
  subgraph IN["Input layer"]
    I1(("x1"))
    I2(("x2"))
    I3(("x3"))
  end
  subgraph H1["Hidden layer 1"]
    A1(("n")) 
    A2(("n"))
    A3(("n"))
    A4(("n"))
  end
  subgraph H2["Hidden layer 2"]
    B1(("n"))
    B2(("n"))
    B3(("n"))
  end
  subgraph OUT["Output layer"]
    O1(("y"))
  end
  I1 & I2 & I3 --> A1 & A2 & A3 & A4
  A1 & A2 & A3 & A4 --> B1 & B2 & B3
  B1 & B2 & B3 --> O1

In a fully connected network every neuron feeds every neuron in the next layer, and layers can have different numbers of neurons; information flows left to right, from raw inputs to an increasingly abstract representation and finally to the output.

LayerRole (syllabus)
Input layerReceives the inputs, e.g., pixel values from a camera
Hidden layersMade of artificial neurons (nodes) that compute and pass information on, gradually transforming the input into increasingly abstract representations
Output layerProvides results to the outside world, e.g., a value giving the likelihood that the image is a cat

What one neuron computes ​

mermaid
flowchart TB
  P1["activation a1<br/>from previous layer"] -->|"× w1"| S["Σ weighted sum<br/>+ bias b"]
  P2["activation a2"] -->|"× w2"| S
  P3["activation a3"] -->|"× w3"| S
  S --> F["activation function<br/>(non-linear)"]
  F --> AV["activation value<br/>passed to next layer"]

Each connection has its own weight and each neuron its own bias; changing a weight or bias changes the activation value, and that is exactly what training does.

activation value = activation function( w1·a1 + w2·a2 + … + wn·an + b )

  • Each connection has its own independent weight. Each neuron has its own bias.
  • The activation function is non-linear. Different activation functions yield different activation values.
  • This bias is a number inside the neuron. It is unrelated to the bias in 5.1.2 (unfair treatment of groups).
Neuron: A node in a neural network, usually receiving multiple input values and generating an activation value.
Activation function: The formula associated with a neuron in a neural network that determines the output of the neuron from the inputs to the neuron.
Weight: An internal variable of a connection between neurons in a neural network that affects how it computes its outputs and that changes as the neural network is trained.
Epoch: An iteration of ML training on the whole training dataset.
Source: ISTQB® Glossary

How the network learns (training loop) ​

mermaid
flowchart TB
  I["Initialize weights and biases<br/>small random values (biases sometimes 0)"] --> F["Pass training data forward<br/>(each neuron runs its activation function)"]
  F --> C["Compare output with known correct result<br/>→ error (loss)"]
  C --> B["Feed the loss back through the network<br/>adjust weights and biases to reduce it"]
  B --> E{"Output good<br/>enough?"}
  E -->|"no: next data / next epoch"| F
  E -->|"yes"| D["End training"]

Initialization happens once; after that the loop is always forward pass → loss → adjust weights and biases, repeated over many epochs until the output is good enough.

Exam Distinction

After the loss is calculated, the next step is to adjust weights and biases. It is not to re-run the same data "to confirm the loss", not to change the activation functions (a design decision), and not to reset the weights (that happens only at initialization).

Hands-on exercise HO-3.4.2 (H1): Experience the implementation of a perceptron

Goal (syllabus): watch a perceptron learn a simple function such as AND, modifying its weights and bias over several epochs until the error is zero. A spreadsheet or a simulation can be used.

Rules used here (a standard textbook perceptron, supplementary): output = 1 if w1·x1 + w2·x2 + b > 0, else 0. Error e = target − output. If e ≠ 0: w1 += e·x1, w2 += e·x2, b += e (learning rate 1). Start with w1 = w2 = b = 0.

EpochErrors in the epochw1, w2, b at the end of the epoch
11 (on 1,1)1, 1, 1
232, 1, 0
332, 1, −1
422, 2, −1
512, 1, −2
60: learned2, 1, −2

Check the final perceptron: (0,0) → −2 → 0 · (0,1) → −1 → 0 · (1,0) → 0 → 0 · (1,1) → 1 → 1. That is exactly AND.

python
data = [((0, 0), 0), ((0, 1), 0), ((1, 0), 0), ((1, 1), 1)]   # the AND function
w1 = w2 = b = 0.0
for epoch in range(1, 21):
    errors = 0
    for (x1, x2), target in data:
        output = 1 if w1 * x1 + w2 * x2 + b > 0 else 0         # step activation
        e = target - output
        if e:
            errors += 1
            w1, w2, b = w1 + e * x1, w2 + e * x2, b + e         # adjust weights and bias
    print(f"epoch {epoch}: errors={errors} w1={w1} w2={w2} b={b}")
    if errors == 0:
        break

Try: change the data to XOR (0,1 → 1; 1,0 → 1; others → 0). The perceptron never reaches zero errors, because XOR is not linearly separable, and that is why multi-layer networks are needed.

3.4.3 Coverage Measures for Neural Networks (K2) ​

Neural networks do not follow explicitly coded paths; their behavior comes from learned weights, bias values and activations. So code coverage does not work, and specialized structural coverage measures assess how far test inputs have activated different parts of the network.

MeasureDefinition (syllabus)Mini example (supplementary)
Neuron coverageProportion of neurons whose output exceeds a specified threshold during testing10 neurons, threshold 0.5; 7 exceed it for at least one test → 70%
k-Multisection Neuron Coverage (kMNC)Each neuron's possible output range is divided into k sections; kMNC is the proportion of these sections activated during testing10 neurons, k = 4 → 40 sections; 26 hit → 65%
Neuron Boundary Coverage (NBC)Proportion of neurons whose output during testing exceeds the maximum achieved in training, or is less than the minimum achieved in training2 of 10 neurons went outside their training range → 20%
Neuron coverage: The coverage of neurons in a neural network where their output exceeds a specified threshold.
K-multisection neuron coverage: The coverage of sections of the output range of each neuron in a neural network, where the output range is divided into k sections.
Neuron boundary coverage: The coverage of neurons in a neural network where their output either exceeds the maximum achieved in training, or is less than the minimum achieved in training.
Source: ISTQB® Glossary (all three are Chapter 3 keywords)

Why testers use them: they reveal untested areas of the model, which may hide defects or unlearned behaviors. If some neurons or layers never activate during testing, performance near the related decision boundaries is questionable, and testers can design additional test inputs or test conditions for those parts. Commercial tools for these measures are currently limited.

Their limits: structural coverage alone does not guarantee that a network will generalize well or handle real-world variation. Networks can learn spurious correlations, giving correct activations for incorrect reasons. Testers should therefore also use other techniques such as adversarial testing and metamorphic testing (Chapters 5–7).

Testing analogy

Neuron coverage is to a neural network what statement coverage is to code. 100% tells you every part was exercised, not that it was right. A model that recognizes "wolf" by spotting snow in the background can have full coverage and still be wrong.

Exam Distinction

Neuron coverage asks whether the output went above a threshold. kMNC asks how much of each neuron's output range was exercised (k sections). NBC asks whether the output went outside the range seen in training (corner-case behavior).

Unit Quiz ​

Unit Quiz: Chapter 3, page 3 (3.4)

Q1. How does a neuron in a hidden layer compute its activation value?

AIt copies the largest activation value from the previous layer
BIt calculates the weighted sum of the activation values from the connected neurons in the previous layer, adds its bias, and passes the result through a non-linear activation function
CIt applies the same fixed weight to every input and outputs the average
DIt compares its input with the correct result and outputs the error

Q2. During training, what happens with the error (loss) after the network's output has been compared with the known correct result?

AIt is fed back through the network to adjust the weights and biases so that the difference is minimized
BIt is stored, and training stops immediately
CIt is used to choose a different activation function for each neuron
DIt is used to reset all weights to new random values

Q3. What is an epoch?

AOne neuron's activation calculation
BThe number of hidden layers in a network
COne pass of the whole training dataset through the network during training
DThe time needed to make a single prediction

Q4. Testing a neural network shows that 18 of its 24 neurons produced an output above the specified threshold for at least one test input. What is the neuron coverage?

A25%
B18%
C33%
D75%

Q5. Why does the syllabus say structural coverage alone is not enough for testing neural networks?

ABecause coverage measures can only be calculated for perceptrons
BBecause commercial tools make coverage testing too expensive
CBecause neural networks can learn spurious correlations, giving correct activations for incorrect reasons, so other techniques such as adversarial and metamorphic testing are also needed
DBecause coverage measures replace the need to evaluate the model's outputs

0 / 5 answered

Topic Drills: 3.4.1 · 3.4.3 · Next: Chapter 4 →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).