Human agency, oversight and control
This domain focuses on enabling meaningful human control of AI systems and helps agencies design for meaningful accountability in AI system development and operation. This includes patterns that make risk and system behaviour understandable, support human attention, enable timely and effective intervention, and preserve traceability between AI system behaviour and human decisions and actions.
It helps agencies architect design-time controls built into the system and run-time requirements that remain with human operators (Zhu, Lu, Ding, Lee, & Wang, 2026). For design-time controls, HAOC does not mean designing systems where human approval becomes an unnecessary bottleneck in every loop. HAOC helps agencies design automation to operate within limits that humans can observe, understand, define, approve, pause, revise and revoke in time. Circuit breakers, service-level remediation triggers, anomaly-driven model rollback and other safety logic may proceed without per-action approval where their activation conditions have been authorised in advance. Similarly, automated verification and validation mechanisms may continuously assess system behaviour against predefined requirements, providing a scalable form of oversight that complements human review and supports continuous improvement.
For run-time controls, a complete architecture covers the roles and skills required to operate the system safely and effectively. For example, an agentic system may require an agent supervisor role to judge whether the agents are performing in alignment with their objectives or executing irreversible actions.
For advisory and generative systems that co-produce work with a human, such as drafting, summarisation or analytical assistance, this domain is concerned with enabling human verification and preserving genuine authorship.
This domain is informed by behavioural science and human factors research. These disciplines provide the foundation for understanding how people interact with AI systems, including managing risks such as automation bias, vigilance decrement and skill erosion.
Typical considerations for the layer include:
- decision support – AI interactions that help humans understand, assess and act on information without undermining their judgement or accountability
- alerting – notify users of significant events, risks, threshold breaches or required actions
- anomaly highlighting – surface unusual patterns, unexpected behaviour or low-confidence outcomes for review
- escalation routing – direct complex, high-risk or unresolved situations to the appropriate decision-makers
- intervention – mechanisms that allow humans to influence or control AI system behaviour when required
- human override – enable authorised users to modify, reject or override AI-generated recommendations or actions
- circuit breakers – pause or restrict operations when predefined safety, risk or performance thresholds are exceeded
- irreversibility guardrails – require heightened review or approval before irreversible, high-impact or sensitive actions occur
- transfer of control – support safe handover between automated and human-led operation when required.