Appendix C: In-scope AI use cases
Criteria and areas of consideration
At a minimum, an AI use case is in scope of this policy if any of the following apply:
- The use, misuse or failure of AI could lead to more than insignificant harm to individuals, communities, organisations, the environment or the collective rights of cultural groups including First Nations peoples.
- The use of AI will materially influence administrative decisions that affect individuals, communities, organisations, the environment or the collective rights of cultural groups including First Nations peoples.
- It is possible the public will directly interact with, or be significantly impacted by, the AI or its outputs without human review.
- The AI is designed to use personal or sensitive data[1] or security classified information[2].
- It is deemed an elevated risk AI use case as directed by the DTA.
Agencies may wish to apply this policy to AI use cases that do not meet the above criteria. This includes use cases with specific characteristics or factors unique to an agency's operating environment that may benefit from applying an impact assessment and governance actions.
This policy has been designed to exclude incidental and lower risk uses of AI that do not meet the criteria. Incidental uses of AI may include off-the-shelf software with AI features such as grammar checks and internet searches with AI functionality. The policy recognises that incidental usage of AI will grow over time and focuses on uses that require additional oversight and governance.
In assessing whether a use case is in scope, agencies should also carefully consider AI use in the following areas:
- recruitment and other employment-related decision making
- automated decision making of discretionary decisions
- administration of justice and democratic processes
- law enforcement, profiling individuals, and border control
- health
- education
- critical infrastructure.
While use cases in these areas are not automatically high-risk, they are more likely to involve risks that require careful attention through an impact assessment.
Experimentation
For the avoidance of doubt, agencies are not required to apply this policy if they are doing early-stage experimentation which does not:
- commit to proceeding with a use case or to any design decisions that would affect implementation later
- risk harming anyone
- introduce or exacerbate any privacy or security risks.
If there is a likelihood of proceeding with the AI use case while experimenting in this phase, agencies should apply the policy. Agencies should also apply the Australian Government AI technical standard, which provides relevant information for developing use cases at each stage of the AI lifecycle.