Appearance
Ch 5: Input Data Risks, Bias and Data Pipelines
Official Reference
ISTQB® CT-AI Syllabus v2.0, Chapter 5 (180 minutes), sections 5.1.1–5.1.3, Learning Objectives (LOs) AI-5.1.1, AI-5.1.2, AI-5.1.3 (all K2). Official syllabus and documents
Exam weight (whole Chapter 5): ~16% of points (6 questions, 7 points): one question per LO. Dataset constraint testing (AI-5.1.5, page 2) is the 2-point K3 question. Expect "which test approach BEST addresses this risk?" scenarios on this page.
Input data testing is one of the two ML-specific test levels (4.3.1). Its objective is to confirm that the data used by the ML system (MLS) for training, testing and prediction is of sufficient quality (see 3.2). It includes reviews, statistical techniques (e.g., testing data for bias), exploratory data analysis (EDA) of the training data, and static and dynamic testing of the data pipeline. For a tester this is familiar ground: think of it as test data management and interface testing, with the data itself as the test object.
Chapter 5 keywords (know name and definition, K1): data pipeline testing, data representativeness testing, dataset constraint testing, input data testing, label correctness testing, review, testing for bias, disparate impact analysis, multiple annotation.
Input data testing: A test level that focuses on the quality of data used for training an ML model. (The syllabus extends this to production data, see 4.3.1)
Review: A type of static testing in which the quality of a work product or process is evaluated by individuals.
Source: ISTQB® Glossary
Input data testing: A test level that focuses on the quality of data used for training an ML model. (The syllabus extends this to production data, see 4.3.1)
Review: A type of static testing in which the quality of a work product or process is evaluated by individuals.
Source: ISTQB® Glossary
5.1.1 Input Data Risks and Mitigations (K2)
The syllabus gives this table of example input data risks and the testing that can mitigate them:
| Potential risks | Possible risk mitigations |
|---|---|
| Defects in the training data leading to bias · Problems within the algorithm, model or ML development framework that introduce systemic unfairness | Testing for bias (5.1.2) |
| Training data acquired from untrustworthy sources · Data is poorly managed | Data provenance testing |
| Poisoned training data | A/B testing (6.1.9) · Data provenance testing · EDA (3.2.1) · Attacks as part of red teaming (4.2.2) |
| Internally inconsistent dataset · Out-of-range data · Wrong data types | Dataset constraint testing (5.1.5) |
| Sub-optimal feature selection | Feature testing |
| Imbalanced dataset due to insufficient coverage of all target classes · Skewed dataset through data augmentation · Missing data · Training data focused on a subset of all use cases · Full range of values not covered in the dataset | Data representativeness testing (5.1.4) |
| Poor labelling guidelines · Ambiguous data · Poor annotation leading to inaccurate or inconsistent labels | Label correctness testing (5.1.6) |
| Poor design or integration leading to data pipeline failures · Data quality defects compromising data pipeline outputs · Performance degradation in operational use of the pipeline · Security breaches or uncontrolled changes impacting the pipeline | Data pipeline testing (5.1.3) |
Data provenance testing: A test type to determine if data is sourced, managed, protected, and used in accordance with an organization's policies, standards, and regulatory obligations.
Feature testing: A test type to determine if an AI model training dataset contains an appropriate set of features.
Source: ISTQB® Glossary
Feature testing: A test type to determine if an AI model training dataset contains an appropriate set of features.
Source: ISTQB® Glossary
How to answer "which mitigation?" questions
Ask what the risk is about:
- where the data came from or how it was handled → provenance
- whether the values obey rules (ranges, types, consistency) → constraints
- whether the data covers reality (classes, use cases, value ranges, missing data) → representativeness
- whether the labels are right → label correctness
- the machinery that moves and transforms the data → pipeline testing
- unfair treatment → testing for bias
Poisoning is the one risk with four mitigations.
5.1.2 Testing for Bias (K2)
Bias in an MLS is non-random, unfair differences in treatment based on sensitive attributes such as gender, age or race. It is often unlawful and makes the system discriminatory.
Bias: The systematic difference in treatment of certain objects, people or groups in comparison to others. (Syllabus Chapter 9, after ISO/IEC TR 24027)
Sensitive attributes: Characteristics that define legally or ethically protected groups and individuals that must be controlled to prevent discrimination.
Testing for bias: A test type to detect bias in input data, an AI model, or an AI-based system.
Data bias: A systematic error caused by inaccurate, incomplete, or unrepresentative training data that leads to unfair outcomes in ML models.
Algorithmic bias: A type of bias caused by the ML algorithm.
Source: ISTQB® Glossary unless noted
Sensitive attributes: Characteristics that define legally or ethically protected groups and individuals that must be controlled to prevent discrimination.
Testing for bias: A test type to detect bias in input data, an AI model, or an AI-based system.
Data bias: A systematic error caused by inaccurate, incomplete, or unrepresentative training data that leads to unfair outcomes in ML models.
Algorithmic bias: A type of bias caused by the ML algorithm.
Source: ISTQB® Glossary unless noted
The two main sources of bias:
| Source | Syllabus examples |
|---|---|
| Data bias: defects in the training data | Unrepresentativeness, historical skews, deliberate poisoning |
| Algorithmic bias: defects in the algorithm, model or development framework that introduce systemic unfairness | An algorithm using a decision threshold for credit scores in a loan approval system |
Test approaches for detecting bias (seven):
| Approach | Kind (supplementary) | What it does (syllabus) |
|---|---|---|
| Reviews of the overall ML workflow, especially data preparation | Static | Identify their potential for introducing bias |
| Reviews of dataset documentation | Static | Examine how data was collected and annotated, and which populations are represented |
| Static analysis of data preparation programs and model implementation code | Static | Find anti-patterns or mishandling of sensitive attributes |
| EDA of training data | Analysis | Visualization and clustering reveal imbalanced data, skewed distributions, anomalous groupings across sensitive attributes |
| Dynamic testing | Dynamic | Feed a known unbiased, representative dataset through the system and look for statistically significant differences in outcomes across sensitive groups. It finds bias in outputs regardless of whether it came from the data or from model development |
| Label correctness testing (5.1.6) | Analysis | Find mislabeling that teaches incorrect associations for specific sensitive attributes |
| Disparate impact analysis | Dynamic | Counterfactual testing (below) |
Disparate impact analysis: A technique for detecting bias by comparing decisions on original scenarios with their counterfactual versions, where sensitive attributes are swapped. (ISTQB® Glossary)
mermaid
flowchart TB
S1["1. Identify the sensitive attributes<br/>for the model (e.g., gender, age)"] --> S2["2. Generate counterfactuals<br/>e.g., same loan application,<br/>gender changed from male to female"]
S2 --> S3["3. Generate model output<br/>for the counterfactuals"]
S3 --> S4["4. Analyze many tests for a<br/>statistically significant result:<br/>does changing a sensitive attribute<br/>change the outcome? → bias"]Disparate impact analysis is a controlled experiment: change only the sensitive attribute, keep everything else equal, and repeat often enough for the difference to be statistically meaningful.
Two notes from the syllabus:
- It can be applied to combinations of sensitive attributes, revealing hidden bias associated with attribute combinations.
- Make sure counterfactuals are realistic. Otherwise the model may react to the unrealistic example rather than to any underlying bias.
Example (illustrative): an insurer's pricing model. Take 2,000 real quotes, create a copy of each with only gender flipped, and compare premiums. Then flip age band and gender together to look for combination bias. Avoid impossible combinations such as a "17-year-old with 30 years of driving experience": the model would react to the nonsense, not to bias.
Exam Distinction: "detect bias in the training data EARLY"
Early, data-focused → reviews of the ML workflow and data preparation, dataset documentation reviews, and EDA. Output-focused (bias in the model's predictions, whatever its origin) → dynamic testing with a representative dataset and disparate impact analysis. Code-focused → static analysis of preparation programs and model code.
5.1.3 Data Pipeline Testing (K2)
Data pipeline: The implementation of data preparation activities to provide input data to support training by an ML algorithm or prediction by an ML model.
Data pipeline testing: A test type that covers the training and operational data pipelines of a machine learning system.
Source: ISTQB® Glossary
Data pipeline testing: A test type that covers the training and operational data pipelines of a machine learning system.
Source: ISTQB® Glossary
Effective data pipeline testing confirms reliability and performance and maintains high data quality throughout the ML workflow. The syllabus describes a layered approach:
mermaid
flowchart TB
D["Design phase<br/>data-pipeline design reviews"] --> C["Component testing<br/>ingestion, transformation scripts, sensor interfaces"]
C --> CI["Component integration testing<br/>data flow across internal interfaces"]
CI --> S["System testing<br/>smoke · functional · non-functional ·<br/>fault injection · back-to-back"]
S --> SI["System integration testing<br/>external sources, storage, monitoring,<br/>downstream consumers (ML models)"]
SI --> P["Testing in production<br/>back-to-back vs. previous versions · A/B ·<br/>continuous monitoring"]
CM["Configuration management reviews<br/>correct code versions, configs, datasets<br/>across training, testing, production"] -.-> C
CM -.-> S
CM -.-> PEach layer catches a different class of pipeline defect, from bad design to bad hand-offs to bad behavior in production; configuration management reviews cut across all of them.
| Layer | What is tested (syllabus) |
|---|---|
| Design reviews | Data-pipeline design, during the design phase |
| Component testing | Data ingestion components, transformation scripts, sensor interfaces, using code reviews, static analysis and hardware-specific tests to verify reliable data capture. Validate transformation logic, data validation rules, error handling, and vulnerabilities that could be exploited to introduce malware or poisoned data |
| Component integration testing | Seamless data flow across internal interfaces and correct interpretation of data as it moves through the pipeline; finds mismatched interfaces or incorrect assumptions between components |
| System testing | The fully assembled pipeline: smoke testing first (basic functionality); functional testing (transformations, data routing); non-functional testing (performance under load, scalability, security); fault injection (simulated defective inputs, robustness of data integrity); back-to-back testing of the operational pipeline against the training pipeline for consistent functionality |
| System integration testing | Interaction with external systems or services: data sources, storage platforms, monitoring tools, downstream consumers such as ML models |
| Testing in production | Back-to-back testing against previous versions (consistent or improved performance); A/B testing of new pipeline iterations against baselines; tools that continuously monitor model behavior, performance and defects in real time |
| Configuration management reviews | The correct pipeline code versions, configurations and datasets are used across training, testing and production |
Align the strategy with the pipeline's purpose:
| Training pipelines | Operational pipelines |
|---|---|
| Often exploratory prototypes | Must be robust |
| Testing may focus on data integrity | Testing prioritizes reliability, performance, maintainability |
Exam Distinction
Back-to-back testing appears twice: in system testing it compares the operational pipeline with the training pipeline; in production it compares the pipeline with previous versions. And both pipeline types benefit from the layered approach and from configuration management reviews. The difference is in priorities, not in "which tests are allowed".
Unit Quiz
Unit Quiz: Chapter 5, page 1 (5.1.1–5.1.3)
Q1. A team receives a training dataset from a data broker it has never worked with and cannot confirm where the data came from or how it was handled. Which test approach from the syllabus BEST mitigates this risk?
ADataset constraint testing
BFeature testing
CData provenance testing
DLabel correctness testing
Q2. Which risk does the syllabus map to DATASET CONSTRAINT TESTING?
AInternally inconsistent dataset, out-of-range data and wrong data types
BTraining data focused on a subset of all use cases
CPoor labelling guidelines
DSecurity breaches impacting the data pipeline
Q3. A loan model uses a hard-coded credit-score threshold that systematically disadvantages one group of applicants, even though the training data is representative. What kind of bias is this?
AData bias
BLabeling bias
CSampling bias
DAlgorithmic bias
Q4. In disparate impact analysis, what is the tester's FIRST step?
AIdentify the sensitive attributes for the model
BGenerate counterfactuals by changing sensitive attributes
CAnalyze the results for statistical significance
DRetrain the model without the sensitive attributes
Q5. During system testing of a data pipeline, the team deliberately feeds corrupted and malformed records into it to see whether data integrity is maintained. What is this called?
ASmoke testing
BBack-to-back testing
CFault injection testing
DConfiguration management review
Q6. What is the KEY difference in test focus between a training data pipeline and an operational data pipeline?
ATraining pipelines need no testing because they are prototypes
BTesting of training pipelines may focus on data integrity, while operational pipeline testing prioritizes reliability, performance and maintainability
COnly operational pipelines need configuration management reviews
DTraining pipelines are only tested with fault injection
0 / 6 answered
Topic Drills: 5.1.1 · 5.1.2 · 5.1.3 · Next page: Representativeness, Constraints and Labels →