Role of Human Oversight

Human oversight of AI agents is not a binary choice between full automation and manual review. It is a design decision—one that must be made deliberately, at the right granularity, for every consequential action an agent can take. The two dominant patterns, Human-in-the-Loop (HITL) and Human-on-the-Loop (HOTL), are not competing philosophies. They are complementary controls that apply at different points in the autonomy spectrum, and leading organizations deploy both in a risk-tiered architecture.

HITL: The Agent Pauses for Human Decision

In the Human-in-the-Loop pattern, the agent reaches a checkpoint and halts execution until a human reviews and approves before proceeding. This is the appropriate pattern for any action that is irreversible, high-impact, involves regulated data, or crosses a defined risk threshold.

The technical implementation in LangGraph uses the interrupt() primitive. When an agent node calls interrupt(), the graph execution suspends, the current state is persisted to a checkpoint store, and control is handed to an external review workflow. The human reviewer receives the agent’s proposed action with full context—what decision the agent is about to make, what data it used, and what the downstream consequences are. Upon approval or rejection (with optional modification), execution resumes from the checkpoint with the human’s decision incorporated into the agent’s state.

from langgraph.types import interrupt

def review_action(state: AgentState) -> AgentState:
    proposed_action = state["proposed_action"]
    # Agent pauses here; state is persisted to checkpoint store
    human_decision = interrupt({
        "action": proposed_action,
        "context": state["reasoning_trace"],
        "risk_tier": state["risk_classification"],
        "reviewer_instructions": "Approve, reject, or modify the proposed action."
    })
    return {**state, "approved_action": human_decision}

The checkpoint store enables durable interrupts—the agent does not need to be running while waiting for human review, which can take minutes or hours. This makes HITL practical in asynchronous workflows where reviewers operate on different schedules than the agent.

HOTL: The Human Monitors and Can Override

In the Human-on-the-Loop pattern, the agent acts autonomously in real time while a human supervisor monitors its activity and retains the authority to intervene. The human does not approve each action but watches for anomalies, policy violations, or situations outside the agent’s competence.

This pattern is appropriate for high-volume, lower-risk decisions where the cost of HITL approval at every step is prohibitive but complete autonomy carries unacceptable risk. A security operations agent that triage-classifies thousands of alerts per day cannot realistically pause for human approval on each one—but a human analyst should be monitoring the classification patterns and can escalate, override, or pause the agent when anomalies appear.

Effective HOTL requires an observability layer that gives supervisors real-time visibility into agent state, decision rationale, and anomaly signals. Tools like LangSmith, Arize AI, and Datadog LLM Observability can surface agent traces, flag unusual patterns, and alert supervisors when the agent encounters situations outside its training distribution.

Risk-Tiered Oversight: The Truist Bank Model

Truist Bank has pioneered a risk-tiered oversight architecture that blends HITL and HOTL based on transaction risk level. Low-risk, routine decisions (e.g., standard balance inquiries, FAQ responses) operate under HOTL with passive monitoring. Medium-risk decisions (e.g., dispute initiation, account updates) trigger HOTL with active alerting—a human can intervene within a defined window before the action completes. High-risk decisions (e.g., large fund transfers, credit limit changes, account closures) invoke HITL with a mandatory approval step.

This tiered model is directly aligned with the EU AI Act’s proportionality principle: oversight requirements scale with the risk level of the AI application. It is also consistent with the NIST AI RMF’s Manage function, which calls for risk response strategies that are appropriate to the risk level and context.

Formalizing Oversight Roles

Oversight patterns only work if responsibility is clearly assigned. Designate an AI Operations role (sometimes called an AI Controller or Agent Supervisor) with defined duties: reviewing HITL queues within SLA windows, monitoring HOTL dashboards, escalating anomalies, and maintaining a log of override decisions with rationale. This role is accountable for the day-to-day health of agent behavior, analogous to how a trading desk supervisor is accountable for trader activity.

At the organizational level, an AI Governance Committee should receive periodic reports from AI Operations and review aggregate patterns: Are override rates increasing for a specific agent? Are HITL queues being processed within SLA? Are there repeat incidents in a specific domain? These signals inform decisions about agent retraining, constraint tightening, or use case scope reduction.

The EU AI Act Article 14 requires that natural persons assigned to human oversight have the competence, authority, and resources to understand the AI system’s capabilities and limitations. This is an auditable requirement—simply assigning someone to a queue is insufficient. Oversight personnel must be trained, empowered, and given tools that make the agent’s reasoning interpretable.

Graceful Degradation

Oversight design must include failure scenarios. If the HITL review queue backs up and actions time out, what happens? If the HOTL monitoring dashboard is unavailable, does the agent continue unchecked? Design for graceful degradation: define timeout behaviors for HITL checkpoints (pause the workflow, not proceed autonomously), establish offline alerting for HOTL monitoring gaps, and document the manual fallback process for each agent capability when oversight infrastructure is unavailable.

Make It Your Own

Key questions to ask in the context of your organization:

  • Have you defined explicit risk tiers for your agents’ actions and mapped each tier to the appropriate oversight pattern (HITL, HOTL, or hybrid), with thresholds documented and enforced in code?
  • For HITL workflows, have you implemented LangGraph interrupt() with a durable checkpoint store so that agent state is preserved during asynchronous human review—without requiring the agent to remain running?
  • Have you designated named individuals to AI Operations roles with defined SLA obligations for HITL queue processing and HOTL anomaly response, with these obligations reflected in their job descriptions?
  • Does your HOTL observability layer provide reviewers with interpretable agent traces—not just action outcomes—so that supervisors can evaluate the quality of the agent’s reasoning, not just its conclusions?
  • Have you modeled what happens when oversight infrastructure fails, and do your agents have explicit graceful degradation behaviors (pause, alert, fallback to manual) rather than defaulting to autonomous action?
  • Are your HITL reviewers trained to the standard required by EU AI Act Article 14—do they understand the agent’s capabilities, limitations, and the conditions under which they should override?