Ongoing Considerations

The four implementation phases of this roadmap have discrete entry criteria, defined activities, and exit gates. The ongoing considerations that follow do not. They are continuous disciplines that begin in Phase 1 and persist indefinitely — evolving in scope and maturity as the enterprise’s agentic AI program advances, but never concluding. Treating these as phase-specific activities, or as activities that can be deferred until a later phase, is a failure mode that consistently surfaces in post-implementation reviews of programs that stalled or were canceled.

The Gartner projection that more than 40% of agentic AI projects will be canceled by 2027 is not primarily a commentary on technology readiness — it is a commentary on organizational readiness. The programs at highest risk are those that completed a successful pilot, declared victory, and then allowed the continuous disciplines to atrophy as the organization’s attention moved to the next initiative. The enterprises that reach Phase 4 and sustain it are those that treated ongoing considerations as standing operational commitments, not as transitional activities.

Change Management as a Permanent Function

Change management for agentic AI is not a project with a start and end date. It is an ongoing organizational function that adapts its focus and methods as the program matures. In Phase 1, the message is “we are experimenting carefully.” In Phase 2, it shifts to “this is working and here is the evidence.” In Phase 3, it becomes “agents are now standard infrastructure, and here is how we use them responsibly.” In Phase 4, the narrative addresses the workforce implications of optimization — including honest conversations about how task automation at scale reshapes roles and career paths.

The communication channels, the internal champions network, and the feedback mechanisms established in early phases must be maintained and refreshed continuously. Internal success stories — specific, quantified accounts of business impact from real deployments — are the most effective communication tool across all phases. Publishing these stories regularly, rather than accumulating them for a periodic report, maintains organizational awareness and builds the institutional confidence that sustains investment through the inevitable rough patches.

Leadership visibility is non-negotiable. Programs whose executive sponsors disengage after Phase 1 success consistently underperform in later phases. The AI governance committee established in Phase 3 should produce a quarterly report to senior leadership that covers performance metrics, cost trends, incident summary, and forward investment priorities. This creates a standing accountability structure that prevents the program from drifting.

Continuous Training and Reskilling

Training is not an event — it is an ongoing investment. New employees require onboarding to the organization’s agentic AI infrastructure as a standard component of their orientation. Existing employees whose roles have been most affected by agent deployment require periodic reskilling to develop the supervision, exception handling, and process design skills that become more valuable as agents take on more routine tasks.

The nature of required training evolves across phases. In Phases 1 and 2, training focuses on how to interact with specific agents and how to recognize when to escalate. In Phase 3, training for managers and team leads must include how to evaluate agent performance, how to identify systemic issues versus one-off errors, and how to make the case for agent modification or retirement when performance is not meeting requirements. In Phase 4, training for engineering and operations staff must include cost optimization techniques, model evaluation methodology, and multi-agent orchestration patterns.

Maintain a training effectiveness measurement program. Track completion rates, assess knowledge retention at 30 and 90 days post-training, and correlate training participation with escalation rates and user satisfaction scores in agent-assisted workflows. This data provides the evidence base for training investment decisions and for identifying where additional support is needed.

Feedback Loop Architecture

A feedback loop is only as valuable as its response rate. An organization that collects user feedback on agent performance but takes six weeks to investigate and respond has a feedback channel, not a feedback loop. The loop requires a complete cycle: collection, triage, investigation, resolution, and communication back to the submitter.

Feedback sources should be diversified. User-initiated reports (voluntary submissions through the designated feedback channel) capture the issues that affect individual users. Automated anomaly detection captures systemic performance degradation that may not surface in individual reports until it becomes severe. The human-in-the-loop audit sampling program captures quality issues that users may not recognize as errors. The governance committee’s quarterly review captures strategic misalignments between agent behavior and organizational goals that are invisible at the task level.

All feedback sources feed into a unified backlog, maintained by the agent platform team, that is triaged weekly and prioritized against a defined scoring framework. The framework should weight severity (the potential for harm from the identified issue), frequency (how often the issue affects users), and strategic relevance (whether the issue is symptomatic of a broader design problem that will recur). This prioritization discipline prevents the backlog from being dominated by low-severity, high-frequency nuisances while high-severity, low-frequency risks go unaddressed.

Model and System Maintenance

Foundation models evolve, deprecate, and change behavior with version updates. The enterprise’s dependency on specific model versions must be managed with the same discipline applied to any software dependency — documented, versioned, tested before updates are applied to production, and monitored for behavioral drift after updates.

Establish a model maintenance schedule that includes: quarterly review of model version currency against available updates, regression testing before any model version change using the domain-specific test sets built in Phase 3 and Phase 4, and a rollback procedure that can be executed within one hour if a model update introduces performance regression in production. Model provider deprecation timelines must be tracked, and migration plans must be initiated at least 90 days before any deprecation date.

The knowledge bases, retrieval corpora, and structured data sources that agents access also require maintenance. Content that was accurate when the agent was deployed becomes stale. Establish document review cycles calibrated to the expected rate of change of the underlying information — quarterly for regulatory and policy content, annually for stable reference material, event-driven for content tied to specific business processes that change with organizational decisions.

Governance Evolution

The AI governance committee established in Phase 3 must evolve its scope and sophistication as the program matures. In Phase 3, governance focuses on approving use cases and establishing standards. In Phase 4 and beyond, governance must address more complex questions: how to handle agent behavior in novel situations not covered by existing policy, how to evaluate the ethical implications of optimization decisions that trade accuracy for cost, how to respond to regulatory changes that affect deployed agents, and how to govern the organization’s relationships with AI model providers whose practices affect the enterprise’s risk profile.

The NIST AI RMF’s Measure function provides the framework for operationalizing ongoing risk assessment: quantitative metrics for each identified risk, monitoring cadences calibrated to risk severity, and documented thresholds that trigger escalation to the governance committee. Organizations should review and update their NIST AI RMF implementation annually at minimum, incorporating lessons from production operations and changes in the external risk environment.

Make It Your Own

Key questions to ask in the context of your organization:

Have you assigned organizational ownership — a named team or function with budget and accountability — for each of the continuous disciplines described in this section, or do they exist as shared responsibilities that will default to no one’s priority when competing demands arise?
Does your change management program include a defined executive reporting cadence — not just ad hoc briefings — that creates a standing accountability structure for the AI governance committee and senior leadership?
Is your training program designed with a measurement component that tracks retention and correlates participation with operational outcomes, so that training investment decisions are evidence-based rather than intuitive?
Does your feedback loop have a documented, committed response timeline at each stage — triage, investigation, resolution, communication — and is compliance with that timeline tracked and reported to the governance committee?
Have you established a model maintenance schedule that includes regression testing before production updates and a tested rollback procedure, and does that schedule account for the deprecation timelines of every model version currently in production?
Is your AI governance committee’s charter scoped to evolve beyond use case approval — covering ethical implications of optimization decisions, regulatory change response, and model provider relationship governance — in a way that will remain relevant as your program matures into Phase 4 and beyond?