Skip to content

Topic Drill 4.3.2: Risk-Based Testing of Machine Learning Systems ​

About this drill

LO AI-4.3.2 (K2): Explain how risk-based testing is applied to machine learning systems.
Lesson: Ch 4, section 4.3.2

This drill covers every point in section 4.3.2:

  • Risk-based testing for all systems; regulatory frameworks require it; AI has unique risks; no standard categorization
  • Project vs. product risks; the three workflow areas (development, input data, model) with their example risks and chapters
  • Alternative categorization by ISO/IEC 25059 characteristics; ML-specific forms of testing

Topic Drill: 4.3.2

Q1. To which systems does the syllabus say risk-based testing should be applied?

AOnly to safety-critical AI systems
BOnly to systems that use ML
CTo all systems, regardless of whether they contain AI components or not
DOnly to adaptive AI-based systems

Q2. What does the syllabus say about regulatory frameworks and risk-based approaches for AI?

AMost regulatory frameworks, published or in development, require a risk-based approach to the development and management of AI-based systems
BRegulators prohibit risk-based testing for AI
COnly the OECD requires risk-based testing
DRegulators require AI systems to be tested exhaustively instead

Q3. Using the ML workflow categorization, "biased training data" is which kind of risk?

AA development-area project risk
BA model-area product risk
CNot an ML-specific risk
DAn input-data-area product risk

Q4. Using the ML workflow categorization, an "overfitted model" is which kind of risk?

AA development-area project risk
BA model-area product risk
CAn input-data-area product risk
DA generic usability risk

Q5. Using the ML workflow categorization, "security vulnerabilities in the ML development framework" is which kind of risk?

AAn input-data-area product risk
BA model-area product risk
CA development-area project risk
DNot a risk the syllabus mentions

Q6. Besides the ML workflow, which other way of categorizing AI risks does the syllabus mention?

ABy the quality characteristics defined in ISO/IEC 25059
BBy programming language
CBy the number of model parameters
DBy hosting location (cloud or on-premises)

Q7. Which of the following does the syllabus give as forms of testing specifically designed to address ML system risks?

AUsability testing, accessibility testing and localization testing
BBoundary value analysis, decision table testing and state transition testing
CStatement, branch and condition coverage
DData pipeline testing, adversarial testing, and review of algorithm/model suitability

Q8. Which TWO statements about ML risks are CORRECT according to section 4.3.2?

Select 2 options.

AML-specific risks are only product risks
BThere is no standard way to categorize ML risks
CData-pipeline defects are a development-area project risk
DML-specific risks are associated with both the development of the MLS (project risks) and the MLS itself (product risks)
ERisk-based testing is only needed for AI systems

Q9. A team evaluated its fraud model only by overall accuracy, on a dataset where fraud is very rare. Using the ML workflow categorization, which risk does this choice represent?

AAn input-data-area product risk: biased training data
BA development-area project risk: poor selection of evaluation approach
CA model-area product risk: susceptibility to adversarial examples
DNo risk: accuracy is always sufficient

0 / 9 answered

← 4.3.1 Test Levels · Next drill: 5.1.1 Input Data Risks →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).