Skip to content

Topic Drill 6.1.7: Drift Testing ​

About this drill

LO AI-6.1.7 (K2): Explain how drift testing is used on operational machine learning systems.
Lesson: Ch 6, section 6.1.7

This drill covers every point in section 6.1.7:

  • Data drift (input statistics change; spam/phishing example) vs. concept drift (input-to-correct-output relationship changes; regulation example)
  • Dynamic drift testing: user feedback as current ground truth, threshold; direct vs. indirect feedback
  • Static drift testing: input and predicted output distributions, Kolmogorov-Smirnov

Topic Drill: 6.1.7

Q1. A spam filter starts missing a new kind of phishing email that did not exist when it was trained. Which type of drift does the syllabus use this example for?

AData drift
BConcept drift
CLabel drift
DModel drift

Q2. After a public health crisis, users' search queries look much the same, but what they consider a RELEVANT result has changed, so the model's learned ranking rules no longer fit. Which type of drift is this?

AData drift
BOverfitting
CConcept drift
DUnderfitting

Q3. What does DYNAMIC drift testing rely on?

AOnly the statistical distribution of the inputs
BFeedback from users, which provides the current ground truth to compare with the model's output against a threshold
CRetraining the model daily
DA pseudo-oracle built by another team

Q4. In a film recommendation system, which of the following is DIRECT user feedback?

AData on which films the user actually watched
BThe time of day the user logs in
CThe distribution of genres in the catalog
DThe user rating a recommendation

Q5. For a news recommender, which of the following would be INDIRECT user feedback that could support dynamic drift testing?

AA thumbs-up button pressed on a recommended article
BA satisfaction survey about recommendations
CHow long users actually spend reading the recommended articles
DThe statistical distribution of article categories in the input feed

Q6. Which statistical test does the syllabus name for STATIC drift testing?

AKolmogorov-Smirnov
BCohen's Kappa
CMann-Whitney U
DChi-squared only

Q7. What does static drift testing compare?

AThe model's predictions with user ratings
BTwo versions of the model on live traffic
CThe training and validation accuracy
DThe statistical properties of the input and predicted output data distributions; a significant difference indicates drift

Q8. According to the ISTQB® Glossary, drift testing is which type of testing?

AAcceptance testing
BMaintenance testing
CComponent testing
DStatic testing

Q9. Which TWO statements about drift are CORRECT?

Select 2 options.

AStatic drift testing requires the current ground truth
BConcept drift makes the model's learned decision boundaries outdated, which reduces its predictive accuracy
CDynamic drift testing needs no feedback
DDrift only affects adaptive (self-learning) systems
EData drift is a change over time in the statistical properties of the operational input data

0 / 9 answered

← 6.1.5 Metamorphic Testing · Next drill: 6.1.8 Overfitting and Underfitting →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).