Search

Agencies must

• Criterion 6: Identify and assign AI roles to ensure a diverse team of business and technology professionals with specialised skills.

  Specialist roles may include, noting that an individual may perform one or more of these roles:

  • AI accountable official: A senior executive accountable for their agency’s implementation of the Policy for the responsible use of AI in government
  • Data scientists and analysts: Professionals who collect, process, and analyse datasets to inform AI models. They will have expertise in statistical analysis which supports the development of reliable AI systems
  • AI integration engineers: Professionals responsible for planning, designing and implementing all components requiring integration in an AI system. The role includes reviewing client needs, developing and testing specifications and documenting outputs
  • AI and machine language engineers: Specialists who design, build, and maintain AI models and algorithms. They work closely with data scientists to implement scalable AI systems
  • AI test engineers: Specialists who verify and validate AI systems against business and technical requirements
  • Ethics and compliance officers: Specialists who ensure that AI systems adhere to legal standards and ethical guidelines, mitigating risks associated with AI systems
  • Domain experts: Individuals with specialised knowledge in specific fields, such as healthcare or finance, who provide context and insights to ensure that AI systems are relevant and effective within their respective domain.

• Criterion 7: Build and maintain AI capabilities by undertaking regular training and education of end users, staff, and stakeholders.

  This may involve:

  • agencies should provide regular training programs keep staff updated on the latest tools, methodologies, ethical guidelines and regulatory requirements
  • consider how to tailor training to the knowledge requirements of each role and provide staff involved in procurement, design, development, testing, and deployment of AI systems with specialised training. For example, individuals responsible for managing and operating AI decision-making systems should undergo specific AI ethics training
  • consider tailoring training for people with disability
  • consider interactive workshops, simulations, case study walk-throughs and computing sandpit environments to provide more immersive and real-world-like experiences especially for more complex aspects of AI.

Agencies should:

• Criterion 8: Mitigate staff over reliance, under reliance, and aversion of AI.

  This may involve:

  • perform periodic technology-specific training, performance assessments, peer reviews, or random audits
  • implement a regular feedback loop for incorrect AI outcomes.
     

Statement 4: Enable AI auditing

Agencies must: 

• Criterion 9: Provide end-to-end auditability.

  End-to-end AI auditability refers to the ability to trace and inspect the decisions and processes involved in the AI system lifecycle. This enables internal and external scrutiny. Publishing audit results enables public accountability, transparency, and trust.

  This may include:

  • establishing documentation across the AI system lifecycle as agreed with the accountable official. This should demonstrate conformance with the AI technical standard, and compliance with relevant legislation and regulations.
  • establishing traceability of decisions and changes from requirements through to operational impacts
  • ensuring accessibility, availability, and explainability of technical and non-technical information to assist audits

  • ensuring audit logging of the AI tools and systems are configured appropriately

    This may include:

    • enabling or disabling the capture of system inputs and outputs
    • detect and record modifications to the system’s operation or performance
    • record who made the modification, under what authority, and the rationale for the modification
    • record system version and any other critical system information.
  • reviewing of audit logs
  • ensuring independence and avoiding conflict of interest when undertaking AI audits.

• Criterion 10: Perform ongoing data-specific checks across the AI lifecycle.

  This should address:

  • data quality for AI training, capabilities, and limitations
  • how data was evaluated for bias
  • controls to detect and manage data poisoning
  • legislative compliance.

• Criterion 11: Perform ongoing model-specific checks across the AI lifecycle.

  This should address:

  • track and maintain experiments with new models and algorithms to ensure reproducibility, achieving similar model performance with the same dataset
  • output flaws such as factually incorrect, nonsensical, or misleading information, which may be referred to as AI hallucinations
  • bias and potential harms, such as ensuring fair treatment of all demographic groups
  • model explainability
  • controls to detect and manage model poisoning
  • legislative compliance.

Statement 5: Provide explainability based on the use case

Agencies must

• Criterion 12: Explain the AI system and technology used, including the limitations and capabilities of the system.

  AI algorithms and technologies such as deep learning models, are often seen as 'black boxes'. This can make it difficult to understand how they work and the factors that generate outcomes. Providing clear and understandable explanations of AI outputs helps maintain trust and transparency with AI systems.

  Explainability on the specific context of the use case ensures clear understanding and reasoning behind AI system output. This supports accountability, trust, and ethical considerations.

  This may include:

  • explaining the AI system such as:
    • consideration of trade-offs such as cost and performance
    • what changes are made with AI system updates
    • how feedback is used in improving AI system performance
    • whether the AI system is static or learns from user behaviour
    • whether AI techniques would provide clearer explanations and validate AI actions and decisions.
  • use cases that are impacted by legislation, regulation, rules, or third-party involvement
  • explain how the system operates including situations that require human intervention
  • explain technical and governance mechanisms that ensure ethical outcomes from the use of an AI system
  • inform stakeholders when changes are made to the system
  • persona level explainability adhering to need-to-know principles.

Agencies should:

• Criterion 13: Explain outputs made by the AI system to end users.

  This typically includes:

  • explaining:
    • AI outputs that have serious consequences
    • how outputs are based on the data used
    • consequences of system actions and user interactions
    • errors
    • high-risk situations
    • avoid explanations that are confusing or misleading
  • using a variety of methods to explain outputs.

• Criterion 14: Explain how data is used and shared by the AI system.

  This includes:

  • how personal and organisational data is used and shared between the AI system and other applications
  • who can access the data
  • where identified data has been used, or will be used, for AI system training.
     

Statement 6: Manage system bias