Appearance
Topic Drill 2.2.1: Acceptance Criteria for AI-Based Systems
About this drill
LO AI-2.2.1 (K2): Give examples of acceptance criteria for AI-based systems.
Lesson: Ch 2, section 2.2.1
This drill covers every point in section 2.2 / 2.2.1:
- Why criteria are statistical, probabilistic or threshold-based rather than binary
- Considering both functional and non-functional characteristics; ISO/IEC 25010 and 25059 as the framework; focus on AI-specific characteristics and safety
- Recognizing criteria for every characteristic: AI functional correctness, functional adaptability, user controllability, transparency, AI robustness, intervenability, societal and ethical risk mitigation, and safety
- Spotting generic (non-AI-specific) criteria
Topic Drill: 2.2.1
Q1. A product owner insists that the acceptance criterion for a new image classifier must be "the system classifies every image correctly". Based on the syllabus, what is the BEST response from the tester?
AAgree, because binary criteria are always easier to test
BAgree, but only test the criterion with a small set of images
CPropose a statistical or threshold-based criterion (e.g., accuracy of 95%), because AI-based systems cannot guarantee perfect accuracy
DRemove the criterion, because AI-based systems need no acceptance criteria
Q2. Acceptance criterion: "The voice assistant keeps its word error rate below 8% with background noise of up to 70 dB." Which characteristic does it target?
AAI robustness
BFunctional adaptability
CUser controllability
DTransparency
Q3. Acceptance criterion: "When the inspection drone reports low positioning confidence, a supervisor can take manual control from the ground station within 1 second." Which characteristic does it MOST likely target?
AAI functional correctness
BTransparency
CSocietal and ethical risk mitigation
DUser controllability
Q4. Acceptance criterion: "Before deployment, the model card must state the sources of the training data, the model version and its known limitations." Which characteristic does it target?
AIntervenability
BTransparency
CAI robustness
DFunctional adaptability
Q5. Acceptance criterion: "After a user hides three sports articles in a session, the news feed shows no more than 10% sports content for the rest of that session." Which characteristic does it target?
AAI robustness
BIntervenability
CFunctional adaptability
DAI functional correctness
Q6. Acceptance criterion: "The difference in selection rates between gender groups shall not exceed the threshold of the specified fairness metric." Which characteristic does it target?
ASocietal and ethical risk mitigation
BAI functional correctness
CUser controllability
DSafety
Q7. Acceptance criterion: "The fraud detection model achieves a recall of at least 92% on the independent test dataset." Which characteristic does it target?
ATransparency
BAI robustness
CIntervenability
DAI functional correctness
Q8. Which of the following is LEAST likely to be an acceptance criterion for an AI-specific quality characteristic from ISO/IEC 25059?
AA plant operator can veto any AI-proposed valve change classified as critical within a 20-second window before it executes
BThe administration console loads in under 2 seconds on a standard office PC
CThe diagnostic tool displays the five input features that most influenced each prediction
DThe parcel router keeps a misrouting rate below 1% while the barcode scanner is partly obscured
Q9. Which of the following examples does the syllabus give for SAFETY rather than for one of the ISO/IEC 25059 characteristics?
AA supervisor can take control of an autonomous drone within 0.5 seconds of a distress signal
BA chatbot passes an internal red teaming assessment with a score of 95% or higher
CNon-AI components of an AI-based steering control system comply with ISO 26262-6 at ASIL C
DAn API endpoint returns the unique version identifier of the deployed prediction model
Q10. Which TWO statements about acceptance criteria for AI-based systems are CORRECT according to section 2.2?
Select 2 options.
ABoth functional and non-functional quality characteristics should be considered
BAcceptance criteria must always be binary pass/fail statements
CISO/IEC 25059 replaces ISO/IEC 25010 when defining acceptance criteria
DThe section focuses on criteria for the AI-specific characteristics of ISO/IEC 25059 and on safety
EAcceptance criteria apply only to AI functional correctness
Q11. Acceptance criterion: "If the AI scheduling system proposes taking a hospital ward's oxygen supply offline, the duty engineer has 60 seconds to veto the action before it executes." Which characteristic does it target?
AUser controllability
BIntervenability
CTransparency
DFunctional adaptability
0 / 11 answered