September 28, 2026, (Inside AI) — A global bank approved an AI system in late 2022 to automate routine credit decisions. Pilots showed faster processing, fewer errors, and clear cost savings. Risk and compliance signed off. The CEO promoted it publicly. Six months later, most applications still passed through human review. Exceptions multiplied, and temporary safeguards became permanent steps. The system scored and recommended, but rarely made final calls.
The technology worked. The economics were clear. What the system never received was authority: the right to decide within agreed boundaries with full organizational backing. This pattern repeats across industries. A manufacturer deploys AI demand forecasting, yet planners treat it as a second opinion. A healthcare network uses AI triage that reduces wait times, but clinicians manually review most recommendations. The system becomes a documentation tool, not a decision engine.
The problem is not failed pilots. It is that accountability and legitimacy structures have not evolved to support AI decisions. Legitimacy means a shared understanding of who can make a decision and have it trusted. Too many institutions accumulate intelligence without reorganizing around it.
Why Overrides Are Rational Self-Protection
When an AI system and a human disagree, the human usually wins quietly. Professionals add review steps, lower thresholds, create documented overrides, and flag sensitive cases. Each adjustment seems prudent. Together they are ruinous. A dispatcher at a global logistics firm explained the logic plainly. "If the system makes the call and something goes wrong, it is my name on the report. If I make the call, at least I can explain it." (Dispatcher, global logistics firm, speaking on condition of anonymity).
The dispatcher was not confused or afraid. He read the accountability structure accurately. Research on professional identity threats shows automation anxiety can shape behavior. But this pattern emerges even where change management is strong, psychological safety is high, and employees admit the system outperforms them. What remains is structural exposure: the person asked to follow the system absorbs the career cost when it errs.
This is not new. Enterprise resource planning systems in the 1990s promised centralized visibility. Nestlé USA's divisions operated autonomously, each accountable for local production and costs. When ERP required surrendering discretion to a common process, employees kept making decisions themselves. As Tom Davenport of Babson College noted, the company treated the system as a technology installation rather than a general-management redesign. Automated credit scoring followed the same path. Studies of auto lending show algorithmic underwriting produced higher profits and lower defaults than human underwriting. The biggest performance gap appeared where loan officers had the strongest incentives to preserve discretion. Other research found officers manipulated inputs to push borderline applications over approval thresholds for volume incentives.
The airline industry offers a contrast. Revenue management systems gained lasting authority because airlines reassigned accountability. Carriers clarified when human intervention was appropriate, structured escalation, changed performance evaluations, and made adhering to the system professionally defensible. Authority shifted not because algorithms became perfect but because governance changed.
UPS's ORION routing system illustrates the tension. When rollout began in 2013, many drivers treated computer-generated routes as suggestions. Over time UPS changed the scorecard. Drivers were measured against ORION's prescribed route rather than their own past performance. Following the algorithm became the professionally defensible act.
Agentic AI systems intensify the problem. Recommender systems preserve the fiction of human control: the system proposes, the human decides. Agentic systems act. They book, approve, route, contract, and escalate within parameters someone authorized but no one continuously monitors. When an agent acts within its parameters and the outcome is bad, accountability becomes genuinely difficult. Was the parameter wrong? Was authorization too broad? Did the agent encounter an unanticipated situation? These questions will surface in boardrooms, courtrooms, and regulatory filings. When AI recommends, the accountability gap produces workarounds. When an agentic system acts, it can produce liability ambiguity on a large scale.
Amazon's Hands off the Wheel initiative shows how authority transfers. Begun in 2012 as Project Yoda, it embedded forecasting, pricing, and purchasing predictions in workflow tools. The organization separated input authority from output authority. Business teams shaped data and parameters, but routine overrides were discouraged and sometimes required approval. Vendor managers shifted from individual buying and pricing to auditing inputs, exceptions, and outcomes. The transfer stuck because accountability moved with authority. When results missed expectations, the question was not which person made the wrong call but which input, rule, or system design failed and who was responsible for correction.
Read: Verizon to invest $70 million in AI training effort
Leaders must ask four governance questions before scaling any AI system. (1) Who loses discretion if this AI system works? (2) Who gains authority and over which decisions? (3) Who explains outcomes when the system is followed? (4) Who absorbs transition risk during the learning curve? If these questions are not answered explicitly at approval, they will be answered implicitly afterward by individual operators preserving their existing authority.
The pressure to align authority with accountability will come from outside. The AFL-CIO's Workers First Initiative on AI, launched in October 2025, called for human review of automated decisions, transparency in AI use, and appeal rights for workers subject to AI-driven employment decisions. Organizations that have not formally aligned authority with accountability will increasingly be asked to do so by employees, lawmakers, or both.
IKEA and the Ingka Group offer an example. As Ingka's AI chatbot Billie resolved simple inquiries, the company reskilled call-center workers in remote interior design, digital retail sales, relationship building, and complex problem-solving. Ingka made new work visible while scaling AI, helping employees see transition as growth rather than managed decline. Salesforce offers a parallel under its 4Rs framework: redesign, reskill, redeploy, and rebalance. As Agentforce absorbed routine support work, Salesforce helped employees move into new roles. According to the company, 51% of hires in its first quarter of fiscal 2026 were internal.
When these elements come together, professional authority does not disappear. It concentrates. The underwriter no longer reviews routine cases the model already scored. She works on cases outside the model's confidence boundary, portfolio patterns suggesting drift, and edge cases informing future parameters. The dispatcher monitors the exception queue, manages client relationships the model cannot capture, and escalates situations the system was not designed for. Both know what they are responsible for. Both can explain a decision that went wrong without beginning, "I overrode the system because...."
Most organizations track adoption through usage dashboards. They rarely track behavioral signatures of quiet containment: whether exceptions are rising or falling, whether override thresholds have shifted, whether escalation has added steps. During ORION rollout, UPS's critical governance move was not tracking whether drivers used the system but whether they followed it. That shift changed what was managed.
The measure of success is not usage rates or smooth rollouts. Ask instead: If the system were turned off tomorrow, would the decisions that matter actually change? Or would people continue making decisions as they always have, only without the interface open? If decisions would not change, authority has not moved. And if authority has not moved, adoption has not occurred, no matter how advanced the model or disciplined the implementation.
