Appearance
Topic Drill 3.4.3: Coverage Measures for Neural Networks
About this drill
LO AI-3.4.3 (K2): Describe the different coverage measures for neural networks.
Lesson: Ch 3, section 3.4.3
This drill covers every point in section 3.4.3:
- Why neural networks need specialized structural coverage measures
- Neuron coverage, k-multisection neuron coverage, neuron boundary coverage (definitions and simple calculations (calculations are supplementary))
- Benefits: revealing untested areas, guiding additional tests; limited commercial tools
- Limits: spurious correlations, generalization; combining with adversarial and metamorphic testing
Topic Drill: 3.4.3
Q1. Why are specialized structural coverage measures needed for neural networks?
ABecause neural networks are always too small for statement coverage
BBecause neural networks do not follow explicitly coded paths; their behavior is dictated by learned weights, bias values and activations
CBecause regulators forbid code coverage for AI
DBecause neural networks have no source code at all
Q2. During testing, 30 of the 40 neurons in a network produce an output above the specified threshold for at least one test input. What is the neuron coverage?
A25%
B30%
C75%
D40%
Q3. A small network has 5 neurons. Each neuron's output range is divided into k = 10 sections, and the tests activated 35 of the sections. What is the k-multisection neuron coverage?
A70%
B35%
C50%
D7%
Q4. Which description matches NEURON BOUNDARY COVERAGE (NBC)?
AThe proportion of neurons whose output exceeds a specified threshold
BThe proportion of the k sections of each neuron's output range that are activated
CThe proportion of layers that contain at least one active neuron
DThe proportion of neurons whose output during testing exceeds the maximum, or falls below the minimum, achieved in training
Q5. Coverage analysis shows that a group of neurons in one layer never activated during testing. What does the syllabus suggest the tester should do?
ARemove those neurons from the network
BDesign additional test inputs or test conditions to exercise the under-explored parts of the network
CIgnore it, because coverage measures are not reliable
DRetrain the network with fewer epochs
Q6. A network reaches high neuron coverage, yet investigation shows it identifies huskies by the snow in the background rather than by the dog. Which limitation does this illustrate?
ANeural networks can learn spurious correlations, giving correct activations for incorrect reasons, so structural coverage alone does not guarantee good generalization
BNeuron coverage cannot be calculated for image models
CThe threshold for neuron coverage was set too low
DThe network needs more hidden layers
Q7. What does the syllabus say about tool support for neural network coverage measures?
AAll ML development frameworks measure them automatically
BThey are mandated by ISO/IEC 25059
CThey can only be measured manually
DCommercial tools supporting these specific coverage measures are currently limited
Q8. Which TWO of the following are neural network coverage measures described in the syllabus?
Select 2 options.
AStatement coverage
BNeuron coverage
CBranch coverage
DNeuron boundary coverage
EModified condition/decision coverage
Q9. A test suite made only of typical, everyday inputs achieves 0% neuron boundary coverage. What does this MOST likely indicate?
AThe network has no neurons
BThe network is overfitted
CNo test drove any neuron outside the output range seen in training, so corner-case behavior has not been exercised
DAll neurons exceeded the threshold
0 / 9 answered
← 3.4.1 Deep Neural Networks · Next drill: 4.1.1 Locked and Adaptive →