Skip to content

Topic Drill 6.1.1: ML Model Risks and Mitigations ​

About this drill

LO AI-6.1.1 (K2): Give examples of test approaches used for risk mitigation of ML models.
Lesson: Ch 6, section 6.1.1

This drill covers every point in section 6.1 / 6.1.1:

  • Functional, non-functional and deployment risks
  • Every row of the risk table, including the ones with several mitigations (test oracle problem, poor requirements, robustness) and the four update/deployment rows

Topic Drill: 6.1.1

Q1. A reinforcement learning cleaning robot is rewarded for "no visible dirt". It learns to cover dirt with a rug instead of cleaning it. Which mitigation does the 6.1.1 table give for this risk?

ASide-effects testing
BReward hacking testing
CDrift testing
DSmoke testing

Q2. A video recommender achieves its engagement target, but it also leads some users to watch for many hours a day, which nobody intended. Which mitigation does the 6.1.1 table map to "model causes side-effects"?

AReward hacking testing
BA/B testing
CFuzz testing
DSide-effects testing

Q3. Which mitigation does the table list ALONGSIDE adversarial testing for "lack of model AI robustness due to unexpected inputs"?

AFuzz testing
BSmoke testing
CRegression testing
DRequirements review

Q4. After a model update, some predictions that used to be correct are now wrong, and the team wants to find these new defects. Which mitigation does the table map to "model updates introduce defects"?

AA/B testing
BDrift testing
CBack-to-back testing
DPerformance testing

Q5. Right after an updated model is deployed, the service fails to start at all. Which mitigation does the table map to "deployment of updated model causes immediate failure"?

ARegression testing
BSmoke testing
CMetamorphic testing
DModel documentation review

Q6. Testers cannot tell from the model's documentation what its interface expects or what accuracy it achieved. Which mitigation applies?

AAPI testing
BExploratory testing
CFuzz testing
DModel documentation review

Q7. Which mitigation does the table map to "model API defect"?

AAPI testing (see 7.1.2)
BBack-to-back testing
CRed teaming
DDrift testing

Q8. A model makes correct predictions but takes 4 seconds each, and the requirement is 200 ms. Which mitigation applies to "inadequate ML model performance efficiency"?

AML functional performance testing
BTesting for underfitting
CPerformance testing
DA/B testing

Q9. Which TWO mitigations does the table list for the TEST ORACLE PROBLEM?

Select 2 options.

ASmoke testing
BMetamorphic testing
CFuzz testing
DBack-to-back testing
EAPI testing

Q10. Which TWO mitigations does the table list for POOR SYSTEM REQUIREMENTS?

Select 2 options.

ARequirements review
BA/B testing
CExploratory testing
DDrift testing
ESmoke testing

Q11. Which mitigation does the table give for an "unethical model"?

ATesting for bias
BEthical system testing
CRed teaming
DSide-effects testing

Q12. The introduction to 6.1 groups ML model risks into functional, non-functional and deployment risks. Which of the following is given as a NON-functional model risk?

ABias
BOverfitting
CAdversarial vulnerabilities
DLack of AI robustness

0 / 12 answered

← 5.1.6 Label Correctness · Next drill: 6.1.2 Model Documentation →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).