Statement 1: Define an operational model
Agencies should:
Criterion 1: Identify a suitable operational model to design, develop, and deliver the system securely and efficiently.
Implementing effective operational models for AI systems needs careful consideration to ensure compliance, efficiency, and ethical standards. They also provide tools for traceability, reproducibility, and modularity.
Existing operational models can be used or extended for AI systems. Operational models can streamline the iterative nature of design, and develop and deliver AI systems more securely, efficiently, and reliably. Some examples include:
- Model operations (ModelOps) – set of practices and technologies to streamline lifecycle management for decision models, using interdisciplinary approaches and automation tools
- Machine learning operations (MLOps) – like ModelOps but for machine learning
- Large language model operations (LLMOps) – like ModelOps but for large language models
- Data operations (DataOps) – practices to streamline lifecycle management for data using interdisciplinary approaches and automated pipelines
- Development operations (DevOps) – a software development methodology combining software development and ICT operations for streamlined workflows.
The above list contains examples that are at varying levels of abstraction. For example, LLMOps is a type of MLOps as it inherits many of the same properties.
Ensure governance and security are integrated into the operational model.
Criterion 2: Consider the technology impacts of the operating model.
These include:
- the resources required for system development and maintenance, including computational power and data storage
- AI requirements including potential harm and bias, human oversight and intervention, AI model configuration, and pre and post processing options for fine-tuning models
- the data requirements of the system including sourcing and usage, provenance training, data diversity, data used in pre-trained models, and intellectual property rights.
Note: The source of the impacts listed will be tied to selection decisions of the data and model, and additional training applied to the model.
Criterion 3: Consider suitable technology hosting strategies.
The hosting strategy can involve one of the following models:
- Infrastructure as a service (IaaS) – for maximum control and flexibility; generally suitable for complex AI
- Platform as a service (PaaS) – generally for AI experimentation and development; no in-house infrastructure management required
- Software as a service (SaaS) – generally for ready-made AI solutions; no in-house infrastructure management and no in-house AI system development required.
The strategy to adopt should consider:
- use case and enterprise needs
- flexibility, scalability, control, computational performance
- AI development and support costs
- customisation
- vendor lock-in
- security and privacy considerations.