Appearance
Topic Drill 2.1.2: AI and Safety
About this drill
LO AI-2.1.2 (K2): Explain the special considerations that arise when AI is used in safety-related systems.
Lesson: Ch 2, section 2.1.2
This drill covers every point in section 2.1.2:
- What a safety-related system is; non-AI safety systems are effortful but feasible
- Specifications: vague goals, requirements implicit in training data, inadequate traceability
- Non-determinism: random number generation, slight input variations
- Self-learning: drifting from the tested behavior; managing learning and data; safety guards (prompt filtering)
- Explainability and transparency: explainable AI and LIME, and their limitations
- Evolving regulations: mature safety standards exclude or prohibit AI; the EU AI Act treats safety components as high-risk
Topic Drill: 2.1.2
Q1. According to the syllabus, what makes a system "safety-related"?
AIt has the potential to cause injury or harm to people, property or the environment
BIt contains at least one AI component
CIt is classified as high-risk under the EU AI Act
DIt processes personal data
Q2. The requirements for an autonomous campus shuttle say "drive safely and comfortably". Most of the actual driving behavior was learned from recorded driving data. The safety assessor cannot trace individual requirements to the implementation. Which challenge does this illustrate?
ANon-determinism
BEvolving regulations
CSpecifications
DSelf-learning
Q3. A thoroughly tested pedestrian-detection model occasionally gives a different result when the input image differs by only a few pixels, or when a random seed changes. Which challenge is this?
ASpecifications
BNon-determinism
CExplainability and transparency
DEvolving regulations
Q4. A medical device keeps learning from patient data after it has been certified. Which mitigation does the syllabus mention to stop it from learning or making decisions that could compromise safety?
ARemoving all monitoring to avoid influencing the learning
BSwitching the device to a rule-based expert system
CRe-certifying the device only once a year
DImplementing safety guards, such as a content moderation component that filters prompts
Q5. What does the syllabus say about explainable AI techniques such as LIME in safety-related systems?
AThey can provide insights into the system's reasoning, but are not widely available and may compromise system performance
BThey guarantee full transparency of any neural network
CThey are mandatory under all functional safety standards
DThey remove the need for testing
Q6. How do mature functional safety international standards currently treat AI, according to the syllabus?
AThey contain detailed AI-specific testing requirements
BThey allow AI only for edge devices
CThey do not include the use of AI, and some even prohibit its use in such systems
DThey recommend AI for all safety-critical functions
Q7. How does the EU AI Act classify AI systems used as safety components, for example in aviation, medical devices or automotive?
AAs minimal risk, because safety standards already apply
BAs high-risk, imposing strict requirements on their development and testing
CAs unacceptable risk, so they are banned
DThey are excluded from the EU AI Act
Q8. Which TWO of the following are among the additional challenges the syllabus lists for AI in safety-related systems?
Select 2 options.
AHigh development cost
BNon-determinism
CLack of test automation tools
DShort release cycles
EEvolving regulations
Q9. What does the syllabus say about developing and testing NON-AI safety-related systems?
AIt is impossible to do with today's standards
BIt is as challenging as for AI-based systems
CIt needs no special effort
DIt can take a lot of effort, but it is feasible
Q10. After a near-miss, an investigator asks why an AI emergency-braking system decided NOT to brake. The development team cannot reconstruct the reason from the model. Which challenge does this illustrate?
ASpecifications
BExplainability and transparency
CSelf-learning
DEvolving regulations
0 / 10 answered
← 2.1.1 AI-Specific Quality Characteristics · Next drill: 2.2.1 Acceptance Criteria →