Statement 14: Implement data orchestration processes
Agencies must
Criterion 48: Implement processes to enable data access and retrieval, encompassing the sharing, archiving, and deletion of data.
Considerations include:
- security classifications and permissions of the data
- speed or mode of the data, such as streaming or batch data
- alignment to Guidelines for data transfers | Cyber.gov.au.
Agencies should
Criterion 49: Establish standard operating procedures for data orchestration.
This includes:
- defining responsibilities between business areas and identifying mutual outcomes to be managed across teams. This is particularly important for business areas that are owners of datasets
- considering inclusion of infrastructure arrangements and use of cloud arrangements for data storage or processing.
Practices to be defined include:
- data governance
- data testing
- security and access controls.
Criterion 50: Configure integration processes to integrate data in increments.
This includes:
- enabling agencies to better manage incident identification and intervention during data integration
- ensuring risks of creating personal identifiable information from data integration are managed appropriately.
- Criterion 51: Implement automation processes to orchestrate the reliable flow of data between systems and platforms.
Criterion 52: Perform oversight and regular testing of task dependencies.
This should involve having comprehensive backup plans in place to handle potential outages or incidents.
The following should be considered:
- regular backups of critical data
- failover mechanisms
- detailed recovery procedures to minimise downtime and data loss.
Criterion 53: Establish and maintain data exchange processes.
This includes:
- how often will data need to be accessed by the system
- at what points will the frequency, magnitude, or speed of access change
- how will security processes adapt when data is exposed to new risks across the AI system
- how will data be monitored for changes to accessibility or completeness
- will the sensitivity of the data change once processed or analysed
- how to validate data trust and authenticity.