Requiring humans to stay in the loop is the right instinct. It is not, by itself, a governance design.
Ryoji Morii / Insynergy Inc.
Why This Change Matters
In March 2026, Japan's Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry jointly published a draft revision of the country's AI Business Guidelines. The revision, reported by Nikkei CrossTech on March 16, marks a meaningful shift in how governments are beginning to think about autonomous AI systems.
The core move is this: the Japanese government is moving to require developers and deploying firms to build mechanisms that make human judgment mandatory for autonomous AI agents, in light of malfunction and privacy risks. The draft introduces formal definitions for AI agents and physical AI systems, adopts a risk-based approach to governance, and recommends a principle of minimum necessary authority for AI agents — limiting what autonomous systems are permitted to act upon without human involvement.
The draft also addresses specific failure modes: unintended transaction execution, deletion of critical data, physical malfunctions, and risks to professional and educational domains where AI substitution could erode human judgment at scale.
This is not a compliance update. It is a signal that the era of deploying AI agents without institutionally defined boundaries is ending — at least at the level of regulatory expectation.
The question is whether the concept at the center of this shift — "human judgment in the loop" — is sufficient to meet the governance challenge it is trying to address.
It is not. And understanding why requires a different vocabulary.
Human-in-the-Loop Is a Start, Not an Answer
Human-in-the-Loop, as a design principle, holds that autonomous AI systems should include human checkpoints — moments where a person reviews, approves, or overrides AI-generated outputs or actions. The principle is sound. In high-stakes domains, the alternative — full AI autonomy without human recourse — is not governable by any reasonable definition.
But the principle has a structural limitation that becomes visible as soon as you try to implement it in a real organization.
Consider two scenarios. In the first, a credit analyst signs off on a risk score generated by an AI model. In the second, a legal reviewer reads through a contract draft produced by an AI system. Both involve a human in the loop. Both satisfy the formal condition the Japanese government's draft guidelines are moving toward.
Now ask a different set of questions. In the first scenario: does the analyst have the authority to override the model? Is there a defined condition under which the score must be escalated? Is the basis for the sign-off recorded anywhere? In the second: what does "review" mean — format checking, substantive legal analysis, or final authorization? Who bears accountability if the contract contains an error?
These questions do not have answers in the Human-in-the-Loop framework, because that framework is concerned with presence, not structure. It asks whether a human is there. It does not ask what that human is authorized to decide, on what basis, within what constraints, and with what accountability trail.
This gap is not a criticism of the Japanese government's direction. It is a description of what the next layer of governance work must address.
The Real Problem Is Authority, Not Presence
The governance problem that emerges as AI agents become operationally embedded is not primarily a problem of oversight. It is a problem of authority.
In traditional organizational structures, authority over decisions is relatively legible. Approval chains exist. Escalation paths are defined. Accountability is anchored to roles and documented through process. These structures are imperfect, but they are structures.
When AI agents enter the picture, authority over decisions becomes ambiguous in a specific way. The AI performs analysis, generates recommendations, or in agentic configurations, executes actions directly. A human is present somewhere in the process. But the distribution of authority between the AI's judgment and the human's judgment is rarely made explicit.
What results is a form of accountability diffusion: the human has technically approved something, but without a defined scope of review, without a clear escalation condition, and without a record of the reasoning behind the approval. The form of oversight is present. The substance of judgment is not.
The Japanese government's draft guidelines are trying to address the form. Governance design must also address the substance.
Decision Design as Judgment Architecture
The conceptual framework that addresses this gap is Decision Design.
Decision Design is not a tool for improving individual decisions. It is a discipline for designing the authority structure within which decisions are made, transferred, and made accountable across human-AI systems.
To be precise about what it is and what it is not:
Decision Design is the practice of explicitly defining which decisions belong to AI systems, which belong to human agents, how authority transfers between them, and how the exercise of that authority is recorded. It treats judgment not as a natural human act that merely needs to be preserved, but as a structural element of organizational governance that must be deliberately designed.
Decision Design is not a decision-support tool. It is not a dashboard or an alert system. It is not a set of ethical principles for AI behavior. And it is not equivalent to Human-in-the-Loop implementation. All of these operate within a decision structure. Decision Design is concerned with the structure itself.
The problem Decision Design addresses is the one that emerges when AI augments or replaces human judgment in consequential processes, but the authority structure governing that augmentation is left undefined. Approval workflows exist. Humans are present. But accountability is diffuse, authority is implicit, and the institutional legitimacy of decisions — the quality of being made by someone with the right authority, on the right basis, within the right constraints — cannot be demonstrated.
Decision Design is not about improving decisions alone; it is about designing the authority structure within which decisions become institutionally legitimate.
Decision Boundaries and Institutional Legitimacy
The operational concept at the center of Decision Design is the Decision Boundary.
A Decision Boundary is the explicit demarcation of where AI judgment ends and human judgment begins — or more precisely, where one form of legitimate authority ends and another begins. Decision Boundaries are not static thresholds or performance cutoffs. They are defined in terms of risk type, consequence reversibility, domain-specific accountability requirements, and organizational authority structures.
Decision Boundaries are not operational thresholds; they are institutional demarcations of legitimate authority.
This distinction matters because operational thresholds are typically set by technical teams and adjusted as model performance improves. Institutional demarcations, by contrast, are set by governance structures and reflect organizational accountability, legal exposure, and the domain-specific requirements of what a decision means in context.
In practical terms, designing Decision Boundaries requires decomposing what is often called "human review" into distinct acts: formal review (does the output conform to required specifications?), substantive review (is the content of the output appropriate given the full context?), and final authorization (does the accountable authority affirm the decision and accept responsibility for its consequences?). Each of these is a different judgment act, requires different expertise and authority, and carries different accountability implications. Conflating them under "human approval" is where governance collapses into compliance theater.
Decision Boundaries must also include explicit escalation conditions — the defined circumstances under which an AI system must halt autonomous action and route to human judgment — and explicit stopping conditions for agentic systems operating in high-stakes environments. The Japanese government's draft guidelines implicitly call for both; Decision Design provides the structural vocabulary to specify them.
Why Decision Logs Matter
A governance architecture that defines authority structures but does not record their exercise is incomplete.
This is the function of Decision Logs.
A Decision Log is not an output record. It is not a system audit trail that records what an AI produced. A Decision Log records the exercise of judgment: who held authority over a given decision, what basis they used, how their judgment related to the AI's output, and what accountability they accepted in making the decision.
Decision Logs do not merely record outputs; they preserve accountability continuity across distributed judgment processes.
In AI-augmented workflows, accountability is distributed across multiple actors and multiple moments: the AI generates, a reviewer assesses, an authorizing agent approves, and an executing system acts. Without a Decision Log that captures the judgment exercise at each stage, the accountability chain cannot be reconstructed after the fact. This matters not only for internal governance but for regulatory scrutiny, legal exposure, and organizational learning.
The difference between a sign-off and a Decision Log entry is the difference between recording that a human was present and recording what that human decided, on what basis, and within what defined scope of authority. The first satisfies the form of Human-in-the-Loop. The second is what makes governance real.
What This Means for Organizations
For organizations implementing AI agents in consequential workflows, the Japanese government's policy direction — wherever they operate — is a useful prompt for a more fundamental question: have we actually designed our judgment structure, or have we assumed that human presence is sufficient?
A structured approach to Decision Design involves several steps that go beyond compliance:
Map judgment points. Identify every point in a workflow where a decision is made — by AI, by a human, or through some combination. Make the implicit explicit.
Classify by authority type. Distinguish between decisions that can be delegated to AI systems, decisions that require human substantive judgment, and decisions that require formal human authorization. Apply the risk-based logic the Japanese guidelines recommend, but extend it to authority, not just risk level.
Decompose "review." Separate formal review, substantive review, and final authorization. Define what each requires, who holds the authority to perform it, and what a record of its exercise must contain.
Define escalation and stopping conditions. For every AI-autonomous action, define the conditions under which the system must stop and route to human authority. Make these conditions explicit, not inferred.
Design Decision Logs. Build record structures that capture the exercise of judgment, not just the outputs of AI processes. Ensure these logs are usable for accountability reconstruction, not merely for audit compliance.
Build review cycles. Decision Boundaries are not permanent. As AI capability changes, as organizational contexts shift, and as regulatory environments evolve, the appropriate demarcations of authority must be revisited. Build that review into governance structure, not left to ad hoc reassessment.
Conclusion
Japan's revised AI Business Guidelines represent a meaningful step: a government formally requiring that human judgment be built into the governance of autonomous AI systems. The direction is right.
But requiring human presence in a process is the beginning of a governance design, not its completion. The harder problem — the one that organizations will increasingly face as AI agents become embedded in consequential workflows — is designing the authority structure within which human judgment actually functions as judgment.
That is the domain of Decision Design. Its core operational concept, Decision Boundary, makes the authority question structurally legible: not merely whether a human is present, but where their authority begins, what it covers, how it is exercised, and how its exercise is preserved in Decision Logs for accountability purposes.
The next phase of AI governance is not primarily about oversight. It is about the design of judgment architecture — making explicit what has been left implicit, and making accountable what has been left diffuse.
The institutions that address this question with structural seriousness, rather than compliance sufficiency, will be better positioned for what follows.
Ryoji Morii is the Founder and Representative Director of Insynergy Inc., a Tokyo-based management consulting firm specializing in AI governance and organizational decision architecture. This article draws on the framework developed in "Decision Design as Judgment Architecture" (SSRN Working Paper, Abstract ID: 6341998).
Primary source: Nikkei CrossTech, "AI autonomous execution: Government to require human judgment intervention, revising AI Business Guidelines," March 16, 2026; Ministry of Internal Affairs and Communications / Ministry of Economy, Trade and Industry, Draft Revision of AI Business Guidelines, March 2026.