Appearance
Topic Drill 5.1.4: Testing for Data Representativeness
About this drill
LO AI-5.1.4 (K2): Explain how to test for data representativeness.
Lesson: Ch 5, section 5.1.4
This drill covers every point in section 5.1.4:
- What representativeness testing determines; the forms of misrepresentation it addresses
- Step 1: target population, use cases, users and environments, distributions and edge cases (experts, existing systems, trusted benchmarks), stratified reference data
- Step 2: EDA of both datasets, feature correlations, anomalies and gaps
- Step 3: Chi-squared and Kolmogorov-Smirnov tests, imbalance, typical and edge cases
- Timing: before training; monitoring for drift after deployment
Topic Drill: 5.1.4
Q1. A credit model will be used nationwide, but its training data comes only from urban branches. What does data representativeness testing determine in this situation?
AWhether the labels in the data are correct
BHow closely the characteristics of the training, validation and test datasets match the real-world data the model will encounter in operation
CWhether the data pipeline transforms the data correctly
DWhether the data came from a trusted source
Q2. Which of the following is NOT listed as a way to identify expected operational data distributions and critical edge cases when defining the target population?
AConsulting domain experts
BAnalyzing data from existing systems or similar applications
CUsing benchmark datasets from trusted sources (e.g., NIST)
DUsing the trained model's own predictions as the reference
Q3. In step 2 (analyze data characteristics), which datasets is EDA applied to?
ABoth the training/test datasets being evaluated AND the reference dataset that represents the expected operational data
BOnly the training dataset
COnly the reference dataset
DOnly the production data after deployment
Q4. Why does step 2 include examining feature relationships, especially correlations?
ATo remove all correlated features
BTo compute the model's accuracy
CTo identify patterns that should be preserved
DTo generate labels
Q5. Which formal statistical tests does the syllabus name for comparing distributions in step 3?
At-test and z-test
BChi-squared and Kolmogorov-Smirnov
CCohen's Kappa and percentage agreement
DAccuracy and F1-score
Q6. A smart-thermostat model will be sold in the Nordic countries, but its dataset contains no readings below −20 °C. Which part of the representativeness testing steps should reveal this?
AStep 3: verify adequate coverage of both typical scenarios and edge/boundary cases
BConfiguration management review
CLabel correctness testing
DDataset constraint testing of the type of each attribute
Q7. What does the syllabus recommend AFTER deployment, regarding representativeness?
ANothing further: representativeness is only checked once
BRetraining the model every day
CContinuously monitoring the properties of operational input data to detect changes that might indicate data drift from the training data distributions
DReplacing the reference dataset with production data
Q8. Which TWO forms of misrepresentation does the syllabus say data representativeness testing addresses?
Select 2 options.
AImbalanced class representation
BInconsistent labels
CWrong data types
DInadequate coverage of operational scenarios
EFramework security vulnerabilities
Q9. What is the role of the reference dataset in data representativeness testing?
AIt is the final hold-out dataset for model testing
BIt is a universal benchmark that applies to all ML systems
CIt replaces the training dataset
DIt is a baseline representing the expected operational data, against which the training data's distributions, correlations and coverage are compared
0 / 9 answered
← 5.1.3 Data Pipeline Testing · Next drill: 5.1.5 Dataset Constraint Testing →