Skip to content

Topic Drill 6.1.8: Testing for Overfitting and Underfitting ​

About this drill

LO AI-6.1.8 (K2): Explain how overfitting and underfitting are detected by testing.
Lesson: Ch 6, section 6.1.8

This drill covers every point in section 6.1.8:

  • Three outcomes; when to test
  • Overfitting: cause, independent test data with less common examples, test vs. validation gap
  • Underfitting: two causes, low metrics on both training and validation, learning curves

Topic Drill: 6.1.8

Q1. What are the three possible outcomes of generating an ML model, according to 6.1.8?

ALocked, adaptive and hybrid
BData drift, concept drift and no drift
COverfitting, underfitting and a "right-fitting" model
DTraining, validation and testing

Q2. When should testing for overfitting and underfitting occur, according to the syllabus?

ADuring training, evaluation and tuning
BOnly after deployment
COnly during acceptance testing
DOnly when users complain

Q3. What causes overfitting, according to the syllabus?

AThe model is too simple for the data
BThe training data lacks important features
CThe operational data changes over time
DThe model learns the training data too well, capturing noise rather than the underlying pattern

Q4. How should a team test a model for overfitting?

AEvaluate it on the training data only
BEvaluate it on a separate test dataset not used in training, including some less common examples, and check whether it performs significantly worse than on the validation dataset
CCompare its training time with other models
DCheck its neuron coverage

Q5. A house-price model performs poorly on both its training and its validation data. Investigation shows the dataset contains no information about location, which strongly drives prices. What is the MOST likely diagnosis?

AOverfitting
BConcept drift
CUnderfitting, because the training data lacks features that reflect an important relationship between inputs and outputs
DA right-fitting model with bad luck

Q6. Which visual technique does the syllabus mention for detecting UNDERFITTING?

AConfusion matrix heat maps
BHistograms of the input features
CNeuron activation maps
DLearning curves: training and validation errors that stay high and close together without significant improvement

Q7. A model scores 99% on training data and 97% on validation data, but only 78% on an independent test dataset that includes many rare cases. What is the MOST likely issue?

AUnderfitting
BOverfitting
CData drift
DLow acceptance criteria

Q8. Which TWO statements about overfitting and underfitting are CORRECT?

Select 2 options.

AOverfitting shows up as poor performance on the training data
BUnderfitting results in poor ML functional performance on both the training and the validation datasets
CA right-fitting model is not a possible outcome
DThe test dataset used to detect overfitting should include some less common examples unlikely to have been used during training
EUnderfitting can only be detected after deployment

0 / 8 answered

← 6.1.7 Drift Testing · Next drill: 6.1.9 A/B Testing →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).