Design: statements 9 - 12
The design stage includes concept development, requirements engineering, and solution design.
Designing AI systems that are effective, efficient, and ethical involves being clear on the problem, understanding the impacts of technical decisions, taking a design approach with humans at the centre and having a clear definition of success.
In the design stage agencies consider how the AI system will operate with and impact existing processes, people, data, and technology. This includes considering potential malfunctions and harms.
Without appropriate design an AI system could:
- cause harm due to incorrect information, caused by AI hallucinations, false positives, or false negatives
- be used beyond their purpose
- perpetuate existing injustices
- be misused, misunderstood, or abused
- be susceptible to malfunctions of another interacting system
- experience behaviour and performance issues caused by other external factors.
At the design stage agencies also determine the performance and reliability measures relevant to their AI system’s tasks. Considerations when selecting metrics include business, performance, safety, reliability, explainability, and transparency.
Notes:
Under the Digital Experience Policy agencies must meet design standards for digital services.
The Voluntary AI Safety Standard outlines the need to establish and implement a risk management process to identify and mitigate risks.