The defining challenge of enterprise AI is no longer capability, and not even governance. It is the allocation of judgment authority — and most organizations have not designed for it.
There is a particular kind of signal worth paying attention to: when the people building a technology start arguing, in public, that it should be slowed down. Not their critics. Not regulators. The builders themselves.
On June 4, 2026, two of the most influential firms in artificial intelligence began making exactly that argument — a development reported by the Japanese business daily Nikkei. Anthropic, the developer of the Claude models, suggested that under certain conditions it could be desirable to deliberately slow the pace of frontier AI development — to buy society time to respond. OpenAI, its principal rival, took a different but converging position: voluntary industry measures, it argued, are no longer sufficient, and stronger governmental oversight is warranted.
This is not how the industry usually talks. The commercial logic of a frontier AI company points in one direction — faster models, broader deployment, a lead over competitors that has to be defended quarter by quarter. For the firms whose entire identity is built on acceleration to start sketching the design of a brake is, on its face, a contradiction.
It is worth resisting the temptation to read this as a story about runaway machines. The more useful question is narrower and more practical: why would the companies closest to the technology conclude that the current trajectory needs a governor — and what does that tell organizations that are not building frontier models, but merely trying to use them responsibly?
This is not, fundamentally, a capability problem
The reflexive interpretation of any "AI companies want to slow down" headline is the familiar one: the systems are becoming too powerful for their creators to control. There is a thread of that concern in the public record. In its essay "When AI Builds Itself," Anthropic sketches a set of possible trajectories for AI development, including one it treats with particular caution — a scenario of recursive self-improvement, in which AI systems increasingly design and refine their successors, and the pace of progress is bounded mainly by available computing resources rather than by human effort.
But fixating on that scenario misreads the more revealing part of the argument.
The trajectory Anthropic describes as most likely is not the dramatic one. It is a near-term world in which most of the work of AI development becomes automated — AI assisting in the creation of better AI — while humans continue to hold the strategic judgment: setting research direction, interpreting results, deciding what matters and what does not. The provocative figure the company offers is of organizations of roughly a hundred people producing the output of tens of thousands. The hands move faster. What stays human is the judgment about where to point them.
Read carefully, this is not a thesis about intelligence. It is a thesis about authority. The thing the builders are trying to protect is not human capability — they readily concede that machines will outpace it in domain after domain — but human judgment: the act of deciding, and of being answerable for the decision.
And that concern does not belong to some distant future of superintelligence. It is already operating inside ordinary organizations, today, in the most mundane processes imaginable.
The vanishing decision-maker
Consider a scene that plays out thousands of times a day across the financial sector.
A lending application arrives. An AI system ingests the applicant's information and transaction history and produces a recommendation: approve. A loan officer reviews the recommendation, sees nothing amiss, and approves. A manager initials the file in the final-review field. The process moves cleanly. No one has done anything wrong.
Now ask a simple question: who decided to extend this loan?
The AI will say it only produced a recommendation. The officer will say the recommendation looked sound, so the approval was reasonable. The manager will say the file had already been reviewed by the officer. Every individual action is defensible — even responsible. And yet the subject of the decision has quietly disappeared. There is a decision, and there is no decider.
When nothing goes wrong, this ambiguity stays invisible. It surfaces only later — when the loan is questioned, when a regulator asks how the judgment was made, when a pattern of outcomes turns out to be skewed. At that point, everyone involved arrives at the same question from different directions: whose decision was that, exactly?
The pattern is not confined to banking. It is, if anything, sharper in the public sector. Take the administration of grants and subsidies — a domain governments worldwide are eager to make more efficient. The work decomposes into recognizable stages: a formal eligibility check, a substantive review of the merits, and a final determination to award or decline. Each of these has traditionally been performed by people. Increasingly, autonomous AI agents are entering the earlier stages, and in the context of public-sector digital transformation, that is broadly welcomed: routine verification is exactly the kind of work organizations want to delegate.
The same dynamic appears in enterprise approvals of every kind — procurement, hiring shortlists, claims adjudication, access requests. AI recommends, an employee approves, a manager signs. The workflow looks identical to the pre-AI version. What has changed is invisible from the outside: as the recommendation becomes more sophisticated and more reliably correct, the human review becomes more perfunctory. The better the model, the more the "approval" collapses into a formality. Capability rises, and the locus of judgment grows harder to find.
This is the uncomfortable thing the frontier firms appear to be seeing in a more advanced form. The problem is not that AI is too intelligent. It is that, as AI participates in more decisions, who is actually deciding becomes progressively harder to identify.
Why "human in the loop" is not enough
The instinctive policy response — and the correct one, as far as it goes — is to insist on human involvement.
Japan's regulators have done precisely this. The AI Business Guidelines, Version 1.2, issued jointly by the Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry on March 31, 2026, advance a clear principle: AI should not be left to decide alone. Organizations are expected to consider use patterns in which human judgment is interposed at appropriate points, with particular attention to outcomes touching fairness, privacy, and safety. As autonomous AI agents proliferate, the Japanese government is increasingly emphasizing that they should operate within mechanisms that require meaningful human judgment, particularly in areas involving privacy, safety, and consequential outcomes.
This is sound guidance. But it also exposes the real difficulty, rather than resolving it.
Telling an organization to "keep a human in the loop" answers the question of whether a person is involved. It does not answer the questions that actually determine accountability: which human, deciding what, at which point, and under what authority. Without those answers, the mandate to insert a human produces exactly the lending scenario described above — a process in which several people are involved, a human is unmistakably "in the loop," and yet no one can say who decided.
A human in the loop is a necessary condition. It is not a sufficient one. Human presence is not the same thing as human authority. A person standing inside a process they do not have the authority, the information, or the mandate to overturn is not a decision-maker. They are a witness with a signature.
The gap, then, is not between "AI decides" and "humans decide." It is between processes where judgment authority is defined and processes where it is merely present. Closing that gap is not a matter of inserting people. It is a matter of design.
We are, collectively, spending enormous energy debating how to manage AI: how to evaluate it, how much oversight to require, whether and when to slow it down. These are real questions. But beneath them sits a prior one that rarely gets named directly.
The question is not, in the end, how we manage AI.
It is who retains the authority to decide when AI participates in the decision — and whether that authority has been designed, or merely assumed.
A different layer: Decision Design
The reader who has followed the argument this far may reasonably conclude that this is, after all, a governance problem. It is not — or not only. Governance describes part of the answer, but it is not the vocabulary that captures the shape of the problem itself. What is missing is a discipline that takes the act of judgment as its explicit object.
That discipline is Decision Design.
Decision Design treats judgment — the act of deciding, and of being answerable for it — as something that can and must be designed, rather than left to accumulate by habit and default. In a pre-AI organization, judgment could be treated as a property of an individual: a person looked at a situation and decided, and accountability followed the person. Once AI enters the process, that assumption breaks. Judgment is no longer an event inside one head; it is a structure distributed across a recommending model, a reviewing employee, and an approving manager. A structure can be designed. And if it is not designed, it defaults — into the lending scenario, where everyone participates and no one decides.
It is important to be precise about what Decision Design is and is not, because it is easily confused with adjacent fields it does not replace.
Decision Design is not about improving decisions alone; it is about designing the authority structure within which decisions become institutionally legitimate. A better model produces better recommendations. It does not, by itself, establish who is entitled to act on them, or who answers for the result. Those are questions of authority, not accuracy.
This is also where Decision Design separates itself from the frameworks executives already know.
Governance, on its own, is insufficient. Governance establishes how an organization controls and supervises its use of AI and manages the associated risk. It is essential infrastructure. But governance frameworks rarely descend to the level of specifying who, precisely, holds judgment authority over a given class of decision — and that is exactly where accountability is won or lost.
Digital transformation, on its own, is insufficient. DX reshapes processes through technology, but it does not interrogate the location of judgment. If anything, successful DX tends to bury judgment deeper inside automated flows, making the decision-maker harder to find rather than easier.
Automation, on its own, is insufficient. Automation works by removing judgment from a process. Decision Design is concerned with the judgment that cannot — and must not — be removed: the consequential calls for which an institution must remain answerable.
AI ethics, on its own, is insufficient. Ethics asks what the right decision is. Decision Design asks a different question that ethics presupposes but does not resolve: who holds the legitimate authority to make that decision, and to bear its consequences.
None of this is an attack on those disciplines. Decision Design does not compete with governance, DX, automation, or AI ethics; it operates at a different layer beneath them and across them. Every governance document, every transformation roadmap, every automation specification contains, implicitly, an answer to the question "who decides here?" — usually an unexamined one. Decision Design is the practice of making that answer explicit and deliberate. It is a complementary governance layer, addressing four things the others leave under-specified: the allocation of authority, the continuity of accountability, the design of escalation, and the institutional legitimacy of the resulting decisions.
The Decision Boundary
At the center of Decision Design sits a single, load-bearing concept: the Decision Boundary.
A Decision Boundary is the line that separates what is delegated to AI from what is retained by humans, and that specifies, for a given class of decision, who holds the authority to decide. Crucially, Decision Boundaries are not operational thresholds; they are institutional demarcations of legitimate authority. A confidence score above which a model auto-approves is an operational threshold — a tuning parameter. A Decision Boundary is a statement about who is entitled to bind the institution, and from what point a human must take ownership. The two are easily conflated, and the conflation is where accountability leaks away.
The abstraction becomes concrete when applied. Return to the lending case, and consider how it looks once a Decision Boundary has been designed into it rather than left to emerge by accident.
AI responsibility. The system performs consistency checks on the application, reconciles it against transaction history, and computes a quantitative risk score. For applications below a defined exposure threshold and above a defined score, the AI's assessment is treated as a provisional first-pass conclusion — but explicitly not as a binding decision.
Human responsibility. Applications involving qualitative circumstances, scores within a defined borderline band, or first-time counterparties are owned by a named human role, whose judgment is the operative decision. In these cases the model's output is positioned as one input among several, not as the answer.
Escalation conditions. Three triggers move the decision upward to a manager with the corresponding authority: exposure above the defined threshold; a material divergence between the model's score and the officer's assessment; or a history of exceptional handling on the account. When any of these fires, the locus of judgment shifts, and that shift is itself part of the design rather than an ad hoc reaction.
Decision Log requirements. Each consequential decision records who held final authority, how the AI's recommendation was weighted, and the basis for the conclusion reached. The point is not data capture for its own sake. Decision Logs do not merely record outputs; they preserve accountability continuity across distributed judgment processes — so that when the question "who decided?" arrives, months later, it has an answer that can be traced rather than reconstructed.
The moment this structure is in place, the vanishing decision-maker reappears. From the outside, the workflow is unchanged: AI recommends, a human approves. But the boundary between the model's conclusion and the human's judgment is now explicit, the conditions under which authority moves are pre-defined, and the place to look when something goes wrong was decided before anything went wrong.
This is also, in operational terms, what regulators are reaching for. When the Japanese guidelines call for mechanisms that interpose meaningful human judgment, the instruction only becomes real at the level of the Decision Boundary. Drawing that line is how a regulatory principle — "do not let AI decide alone" — is translated into a process that actually behaves that way.
The judgment question of the decade
The argument that began with two AI companies reaching for the brake, the regulatory insistence on human judgment, and the quiet disappearance of the decision-maker inside ordinary workflows all converge on the same point. The intelligence of these systems will continue to grow, and there is little to be gained from wishing otherwise. The question that matters is what happens to the authority to decide as the capability around it expands.
For a generation of executives, the governing question of enterprise AI has been some version of "Can the AI do this?" That question is rapidly answering itself in the affirmative across one domain after another. It is being replaced by a harder one, and organizations that have not yet noticed the substitution are the ones most exposed by it.
The defining governance question of the AI era is no longer "Can AI decide?"
It is "Who retains legitimate authority when AI participates in decision-making?"
What organizations increasingly need is not another oversight mechanism, but an architecture for judgment itself. The challenge is architectural before it is operational.
That question will not be answered by better models, faster transformation, broader automation, or clearer ethical principles — though it will require all of them. It will be answered, or left dangerously unanswered, by whether organizations choose to design the architecture of judgment itself, or continue to assume it.
Sources referenced: Anthropic, "When AI Builds Itself" (June 4, 2026), arguing that slowing frontier AI development may be desirable under certain conditions; Nikkei, reporting (June 2026) on Anthropic's argument and on OpenAI's position that voluntary industry measures are insufficient and that stronger governmental oversight is warranted; Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry (Japan), AI Business Guidelines Version 1.2 (March 31, 2026).
Decision Design is a judgment architecture framework proposed by Ryoji Morii, founder of Insynergy Inc., for structuring authority, accountability, and decision boundaries in AI-augmented organizations.