Statement 24: Select trained models
Agencies should
Criterion 88: Assess a pool of trained models against acceptance metrics to select a model for the AI system.
This involves:
- defining clear needs and expectations
- comparing multiple trained models, usually generated based on different configurations
- prioritising based on metrics such as ‘simplest’ or ‘most effective’
- documenting the rationale for selection based on results from training models with various model architectures, learning strategies, and configurations
- any risk and mitigation plans
- a model refresh and re-training plan and register
- implementing mechanisms for explainability of model outputs to system users
- feedback channels and mechanisms implemented for monitoring and managing model performance
- an audit plan
- documenting a method for retiring the model.