Search

The challenges for government use of AI are complex and linked with other governance considerations, such as:

• the APS Code of Conduct
• data governance
• cyber security
• ICT infrastructure
• privacy
• sourcing and procurement
• copyright
• ethics practices.

Across the lifecycle stages, agencies should consider:

• technology operations – to ensure compliance, efficiency, and ethical standards
• reference architecture – to provide structured frameworks that guide the design, development, and management of AI solutions
• people capabilities – having the specialised skills required for successful implementation
• auditability – enabling external scrutiny, supporting transparency, and accountability
• explainability – identifying what needs to be explained and when, making complex AI processes transparent and trustworthy
• system bias – maintaining the role of positive bias in delivering meaningful outcomes, while mitigating the source and impacts of problematic bias
• version control – tracking and managing changes to information to inform stakeholder decision-making
• watermarking – to embed visual or hidden markers into generated content so that its creation details can be identified.

Designing AI systems that are effective, efficient, and ethical involves being clear on the problem, understanding the impacts of technical decisions, taking a design approach with humans at the centre and having a clear definition of success.

In the design stage agencies consider how the AI system will operate with and impact existing processes, people, data, and technology. This includes considering potential malfunctions and harms.

Without appropriate design an AI system could:

• cause harm due to incorrect information,  caused by AI hallucinations, false positives, or false negatives
• be used beyond their purpose
• perpetuate existing injustices
• be misused, misunderstood, or abused
• be susceptible to malfunctions of another interacting system
• experience behaviour and performance issues caused by other external factors.

At the design stage agencies also determine the performance and reliability measures relevant to their AI system’s tasks. Considerations when selecting metrics include business, performance, safety, reliability, explainability, and transparency.
 

Data used by an AI system can be classified into development and deployment data.

Development data includes all inputs and outputs (and reference data for GenAI) used to develop the AI system. The dataset is made up of smaller datasets – train dataset, validation dataset, and test dataset.

• Train dataset – this dataset is used to train the AI system. The AI system learns patterns in the train dataset. The train dataset is the largest subset of the modelling dataset. For GenAI, the train dataset may also include reference or contextual datasets such as retrieval-augmented generation (RAG) datasets and prompt datasets
• Validation dataset – this dataset is used to evaluate the model's performance during model training. It is used to fine-tune and select the best-performing model, such as through cross validation
• Test dataset – this dataset is used to evaluate the final model's performance on previously unseen data. This dataset helps provide unbiased evaluation of model performance.

Deployment data includes AI system inputs such as live production data, user input data, configuration data, and AI system outputs such as predictions, recommendations, classifications, logs, and system health data. Deployment stage inputs are new and previously unseen by the AI system. 

The performance of an AI system is dependent on robust management of data quality and the availability of data. 

Key workstreams within this stage include: 

• data orchestration – establishing central oversight of and planning the flow of data to an AI system from across datasets
• data transformation – converting and optimising data for use by the AI system
• feature engineering – methods to improve AI model training to better identify and learn patterns in the data
• data quality – measuring dimensions of a dataset associated with greater performance and reliability
• data validation – testing the consistency, accuracy, and reliability of the data to ensure it meets the requirements of the AI system
• data integration and fusion – combining data from multiple sources to synchronise the flow of data to the AI system
• data sharing – promoting reuse, reducing resources required for collection and analysis, and helping to build interoperability between systems and datasets
• model dataset establishment – using real-world production data to build, refine, and contextualise a high-quality AI model.
   

AI training involves processing large amounts of data to enable AI models to recognise patterns, make predictions, draw inferences, and generate content. This process creates a mathematical model with parameters that can range from a few to trillions. Training an AI model might require adjustment of these parameters, entailing increased processing power and storage. 

Training a model can be compute-heavy, relying on infrastructure that may be significantly expensive. The model architecture, including choice of the AI algorithm and learning strategy, together with the size of the model dataset, will influence the infrastructure requirements for the training environment.

The AI Model encapsulates a complex mathematical relationship between input and output data that it derives from patterns in a modelling dataset. AI models can be chained together to provide more complex capabilities.

Pre-processing and post-processing augment the capabilities of the AI model. Application, platform, and infrastructure components are shown here as well as they all contribute to the overall behaviour and performance of the whole AI system.

Due to the number of mathematical computations involved and time taken to execute them, training can be a highly intensive stage of the AI lifecycle. This will depend on the infrastructure resources available, the algorithms used to train the AI model and the size of the training datasets.

Key considerations during this stage include:

• the model architecture, including the AI model and how components within the model interact, as well as the use of off-the-shelf or pre-trained models
• selection and development of the algorithms and learning strategies used to train the AI model
• an iterative process of implementing model architecture, setting hyperparameters, and training on model datasets
• model validation tests, supplemented by human evaluation, which evaluate whether the model is fit-for-purpose and reliable
• trained model selection assessments, which streamline development and enhance capabilities by comparing various models for the AI system
• continuous improvement frameworks which set processes for measuring model outputs, business, and user feedback to manage model performance.

If after multiple attempts of refinement, the model does not meet requirements or success criteria, a new model may need to be created, business requirements updated, or the model is retired.

See the Design lifecycle stage for details on measuring model outputs, as well as business and user feedback, to manage AI model performance.

See the Apply version control practices statement in the Whole of AI lifecycle section for detail on tracking changes to training models, trained models, algorithms, learning types, and hyperparameters.