Appearance
Topic Drill 4.3.2: Risk-Based Testing of Machine Learning Systems
About this drill
LO AI-4.3.2 (K2): Explain how risk-based testing is applied to machine learning systems.
Lesson: Ch 4, section 4.3.2
This drill covers every point in section 4.3.2:
- Risk-based testing for all systems; regulatory frameworks require it; AI has unique risks; no standard categorization
- Project vs. product risks; the three workflow areas (development, input data, model) with their example risks and chapters
- Alternative categorization by ISO/IEC 25059 characteristics; ML-specific forms of testing
Topic Drill: 4.3.2
Q1. To which systems does the syllabus say risk-based testing should be applied?
AOnly to safety-critical AI systems
BOnly to systems that use ML
CTo all systems, regardless of whether they contain AI components or not
DOnly to adaptive AI-based systems
Q2. What does the syllabus say about regulatory frameworks and risk-based approaches for AI?
AMost regulatory frameworks, published or in development, require a risk-based approach to the development and management of AI-based systems
BRegulators prohibit risk-based testing for AI
COnly the OECD requires risk-based testing
DRegulators require AI systems to be tested exhaustively instead
Q3. Using the ML workflow categorization, "biased training data" is which kind of risk?
AA development-area project risk
BA model-area product risk
CNot an ML-specific risk
DAn input-data-area product risk
Q4. Using the ML workflow categorization, an "overfitted model" is which kind of risk?
AA development-area project risk
BA model-area product risk
CAn input-data-area product risk
DA generic usability risk
Q5. Using the ML workflow categorization, "security vulnerabilities in the ML development framework" is which kind of risk?
AAn input-data-area product risk
BA model-area product risk
CA development-area project risk
DNot a risk the syllabus mentions
Q6. Besides the ML workflow, which other way of categorizing AI risks does the syllabus mention?
ABy the quality characteristics defined in ISO/IEC 25059
BBy programming language
CBy the number of model parameters
DBy hosting location (cloud or on-premises)
Q7. Which of the following does the syllabus give as forms of testing specifically designed to address ML system risks?
AUsability testing, accessibility testing and localization testing
BBoundary value analysis, decision table testing and state transition testing
CStatement, branch and condition coverage
DData pipeline testing, adversarial testing, and review of algorithm/model suitability
Q8. Which TWO statements about ML risks are CORRECT according to section 4.3.2?
Select 2 options.
AML-specific risks are only product risks
BThere is no standard way to categorize ML risks
CData-pipeline defects are a development-area project risk
DML-specific risks are associated with both the development of the MLS (project risks) and the MLS itself (product risks)
ERisk-based testing is only needed for AI systems
Q9. A team evaluated its fraud model only by overall accuracy, on a dataset where fraud is very rare. Using the ML workflow categorization, which risk does this choice represent?
AAn input-data-area product risk: biased training data
BA development-area project risk: poor selection of evaluation approach
CA model-area product risk: susceptibility to adversarial examples
DNo risk: accuracy is always sufficient
0 / 9 answered