Skip to content

Topic Drill 4.3.1: Test Levels for Machine Learning Systems ​

About this drill

LO AI-4.3.1 (K2): Summarize the test levels used to develop machine learning systems.
Lesson: Ch 4, section 4.3.1

This drill covers every point in section 4.3 / 4.3.1:

  • The two ML-specific test levels: input data testing (training and production data) and ML model testing
  • Conventional test levels as risks require: component (non-AI components), component integration (pipeline ↔ model ↔ UI; API testing of AIaaS), system (performance once embedded, compressed models, non-functional), system integration (external systems), acceptance (suitability of AI services)

Topic Drill: 4.3.1

Q1. A speech recognition system works well for most users but regularly fails for speakers with a regional accent that is rare in the training data. Which test level should have been performed MORE thoroughly to catch this earliest?

AComponent testing
BInput data testing
CSystem integration testing
DAcceptance testing

Q2. An application uses an external AI-as-a-Service (AIaaS) sentiment API. At which test level does the syllabus place API testing of that service?

AInput data testing
BML model testing
CAcceptance testing
DComponent integration testing

Q3. At which test level are the user interface and communication components of an ML system tested on their own?

AComponent testing
BML model testing
CSystem integration testing
DInput data testing

Q4. The data pipeline sends temperatures in Fahrenheit, but the model expects Celsius, so predictions are wrong. Which test level focuses on detecting this kind of defect?

AML model testing
BAcceptance testing
CComponent integration testing, which checks that inputs from the data pipeline are received as expected by the model
DInput data testing of the training data only

Q5. At which test level does the syllabus mention non-functional testing of the time required to deliver a prediction from the complete AI-based system?

AComponent testing
BSystem testing
CInput data testing
DComponent integration testing

Q6. An AI-based loan system exchanges data with an external credit bureau and a payment provider. Which test level verifies these interfaces and data exchanges in an environment representative of operational conditions?

AComponent testing
BML model testing
CInput data testing
DSystem integration testing

Q7. An organization plans to buy a third-party AI service and must decide whether it is suitable for its system and whether the agreed ML functional performance criteria are achieved. Which test level is this?

AAcceptance testing
BComponent testing
CInput data testing
DML model testing

Q8. What is ML model testing concerned with, according to section 4.3.1?

AThe training data and production data
BThe user interface of the ML system
CThe ML models, which are the ultimate output of the ML workflow
DInterfaces with external systems

Q9. Which TWO statements about test levels for ML systems are CORRECT?

Select 2 options.

AInput data testing only covers the training data
BThe two ML-specific test levels cannot address all ML risks; conventional test levels are also typically required, depending on the perceived risks
CSystem testing replaces ML model testing
DAcceptance testing is never needed for AI provided as a service
EComponent testing applies to the data pipeline

0 / 9 answered

← 4.2.2 Red Teaming · Next drill: 4.3.2 Risk-Based Testing →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).