Appearance
Topic Drill 6.1.1: ML Model Risks and Mitigations
About this drill
LO AI-6.1.1 (K2): Give examples of test approaches used for risk mitigation of ML models.
Lesson: Ch 6, section 6.1.1
This drill covers every point in section 6.1 / 6.1.1:
- Functional, non-functional and deployment risks
- Every row of the risk table, including the ones with several mitigations (test oracle problem, poor requirements, robustness) and the four update/deployment rows
Topic Drill: 6.1.1
Q1. A reinforcement learning cleaning robot is rewarded for "no visible dirt". It learns to cover dirt with a rug instead of cleaning it. Which mitigation does the 6.1.1 table give for this risk?
ASide-effects testing
BReward hacking testing
CDrift testing
DSmoke testing
Q2. A video recommender achieves its engagement target, but it also leads some users to watch for many hours a day, which nobody intended. Which mitigation does the 6.1.1 table map to "model causes side-effects"?
AReward hacking testing
BA/B testing
CFuzz testing
DSide-effects testing
Q3. Which mitigation does the table list ALONGSIDE adversarial testing for "lack of model AI robustness due to unexpected inputs"?
AFuzz testing
BSmoke testing
CRegression testing
DRequirements review
Q4. After a model update, some predictions that used to be correct are now wrong, and the team wants to find these new defects. Which mitigation does the table map to "model updates introduce defects"?
AA/B testing
BDrift testing
CBack-to-back testing
DPerformance testing
Q5. Right after an updated model is deployed, the service fails to start at all. Which mitigation does the table map to "deployment of updated model causes immediate failure"?
ARegression testing
BSmoke testing
CMetamorphic testing
DModel documentation review
Q6. Testers cannot tell from the model's documentation what its interface expects or what accuracy it achieved. Which mitigation applies?
AAPI testing
BExploratory testing
CFuzz testing
DModel documentation review
Q7. Which mitigation does the table map to "model API defect"?
AAPI testing (see 7.1.2)
BBack-to-back testing
CRed teaming
DDrift testing
Q8. A model makes correct predictions but takes 4 seconds each, and the requirement is 200 ms. Which mitigation applies to "inadequate ML model performance efficiency"?
AML functional performance testing
BTesting for underfitting
CPerformance testing
DA/B testing
Q9. Which TWO mitigations does the table list for the TEST ORACLE PROBLEM?
Select 2 options.
ASmoke testing
BMetamorphic testing
CFuzz testing
DBack-to-back testing
EAPI testing
Q10. Which TWO mitigations does the table list for POOR SYSTEM REQUIREMENTS?
Select 2 options.
ARequirements review
BA/B testing
CExploratory testing
DDrift testing
ESmoke testing
Q11. Which mitigation does the table give for an "unethical model"?
ATesting for bias
BEthical system testing
CRed teaming
DSide-effects testing
Q12. The introduction to 6.1 groups ML model risks into functional, non-functional and deployment risks. Which of the following is given as a NON-functional model risk?
ABias
BOverfitting
CAdversarial vulnerabilities
DLack of AI robustness
0 / 12 answered
← 5.1.6 Label Correctness · Next drill: 6.1.2 Model Documentation →