Skip to content

Topic Drill 3.1.2: Machine Learning Workflow ​

About this drill

LO AI-3.1.2 (K2): Summarize the workflow used to create an ML system.
Lesson: Ch 3, section 3.1.2

This drill covers every point in section 3.1.2:

  • Every activity: understand objectives, select framework, select & build algorithm, prepare & test data, train, evaluate, tune, test, deploy, use, monitor & tune
  • Model vs. algorithm hyperparameters; model generation = train + evaluate + tune
  • Validation dataset (evaluate) vs. independent test dataset (test); who performs model testing
  • Iterative application; integration with non-ML components; A/B testing for new models

Topic Drill: 3.1.2

Q1. In which ML workflow activity are the acceptance criteria, including the ML functional performance metrics, defined for the model?

AUnderstand the objectives
BEvaluate the model
CTest the model
DMonitor & tune the model

Q2. On what basis is the ML development framework selected in the ML workflow?

AThe size of the test team
BThe available hardware only
CThe objectives, the acceptance criteria and the business priorities
DThe results of testing the model

Q3. Which statement about "Select & build the algorithm" is CORRECT?

AThe algorithm must always be hand-coded by the development team
BThe algorithm is selected based on factors including the objectives, acceptance criteria and available data, and is often retrieved from a software library
CThe algorithm is selected after the model has been deployed
DThe algorithm is selected using the test dataset

Q4. A data scientist sets the number of training iterations for a neural network. In the syllabus's terms, what kind of parameter is this?

AA model hyperparameter
BA weight
CAn acceptance criterion
DAn algorithm hyperparameter

Q5. A team trains a random forest, an SVM and a neural network on several training datasets. How is the final combination chosen, according to the syllabus?

ABased on the evaluation results against the agreed ML functional performance metrics (using the validation dataset)
BBy choosing the model that trained fastest
CBy testing all of them on the independent test dataset and keeping the best
DBy asking end users to vote

Q6. Which workflow activity matches this description: "The tuned model is converted to run on an embedded device and then checked again against its acceptance criteria"?

ATest the model
BUse the model
CDeploy the model
DTune the model

Q7. Who typically performs the "Test the model" activity, according to the syllabus?

AOnly end users during acceptance testing
BTypically data engineers or scientists; testers with sufficient domain knowledge and access to the relevant resources can also do it
COnly certified test managers
DNobody: the framework tests the model automatically

Q8. How can a deployed model be used, according to the "Use the model" activity?

AOnly in isolation, never integrated with other components
BOnly for retraining
COnly through manual requests by data scientists
DIntegrated into a larger operational system, running scheduled batch predictions or real-time predictions on request

Q9. An operational model is regularly evaluated against its acceptance criteria, a drop is found, and a retrained model is built. How does the syllabus suggest comparing the new model with the existing one?

AUsing a form of A/B testing
BUsing neuron coverage
CRe-running the original validation only
DDeploying the new model without comparison

Q10. Which TWO statements about the ML workflow are CORRECT?

Select 2 options.

AThe workflow shown includes the integration of the ML model with the non-ML parts of the system
BIn practice, the workflow is applied iteratively, with steps repeated
CModel testing is always performed only by data scientists
DThe data used to train, evaluate, tune and test the model must be representative of the operational data
EThe "Evaluate the model" activity uses the independent test dataset

Q11. In which workflow activity are the data and any automated data preparation steps tested?

AUnderstand the objectives
BTrain the model
CPrepare & test data
DDeploy the model

0 / 11 answered

← 3.1.1 Forms of ML · Next drill: 3.1.4 Pretrained Models, Fine-Tuning, RAG →

Unofficial study guide for the ISTQB® Certified Tester AI Testing (CT-AI) exam. Not affiliated with International Software Testing Qualifications Board (ISTQB®).