By Ryoji Morii, Insynergy Inc.
The Policy Signal Is Right. The Framing Is Not Enough.
Governments are beginning to move. The Japanese government is moving toward requiring mechanisms in which human judgment remains necessary for autonomous AI agents, in light of risks such as malfunction and privacy intrusion. Japan's Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry are updating their AI Business Guidelines—originally issued in 2024 and revised annually—to address autonomous AI agents and physical AI systems that operate in the real world through robotic interfaces. The updated guidelines, substantially approved at an expert advisory meeting in March 2026, ask developers to establish mechanisms that make human judgment mandatory at specified points in AI-driven processes.
The directional intent is sound. As AI agents become capable of chaining actions across tools, systems, and external environments—and as physical AI begins to act on the material world rather than merely producing text—the risks of unmediated autonomous operation are real and growing. A policy response is appropriate.
But a policy signal is not a design answer. And the specific framing of this one—requiring that organizations identify "matters requiring judgment" and "select appropriate targets according to their importance"—leaves the harder problem almost entirely unaddressed.
The harder problem is not which decisions to flag. It is whether the judgment structure itself has been designed at all.
"Human in the Loop" Is Not a Design Answer
The phrase "human-in-the-loop" has become a near-universal governance shorthand. It sounds precise. It is not.
The core ambiguity is this: a human can be present in a process without exercising any meaningful judgment. When a staff member reviews a queue of AI-generated outputs and clicks approve, they are technically in the loop. But what have they actually decided? If the volume is high, the outputs plausible, the context opaque, and the time short, the approval is ratification—not judgment. The human presence produces a record of oversight without the substance of it.
This is not a hypothetical failure mode. It is already the default in many organizations deploying generative AI at scale. The higher the quality of AI output, the stronger the pull toward ratification. A response that looks right is harder to question. An output that almost never fails is harder to scrutinize. And so the human in the loop gradually becomes a rubber stamp on decisions the AI has already made.
There is a meaningful distinction between formal review, substantive review, and final authority. These are not the same act. Conflating them produces what might be called the hollow oversight problem: governance appearance without governance substance. The record shows a human approved it. The reality is that no human decided anything.
In agentic systems, the problem compounds structurally. When AI agents chain actions—where the output of one agent becomes the input of the next—the opportunity for meaningful human intervention may never exist in the first place. The speed and complexity of multi-agent interaction outpaces human supervisory capacity by design. Requiring "human judgment" under these conditions, without specifying where, by whom, on what basis, and with what authority, is a policy that cannot be operationalized.
Selecting decision points is not the same as designing a judgment structure. Human presence is not the same as accountable authority. Governance appearance is not governance substance.
The Real Problem Is Institutional Judgment Design
What organizations deploying AI agents actually need to determine is not a list of flagged decision types. They need answers to a more fundamental set of questions.
Who holds judgment authority at each point in an AI-driven process? What conditions trigger a return of authority to a human actor? What information must be present for that human to exercise genuine judgment—rather than ratify an output they cannot meaningfully evaluate? What happens when a situation falls outside the anticipated range? Who is accountable when the chain of AI actions produces an outcome no individual in the organization recognized as a decision?
These questions are not operational. They are architectural. They concern the institutional structure within which AI systems act and humans remain—or fail to remain—accountable.
The gap between identifying "important decisions" and designing judgment architecture is where governance fails quietly. Organizations build approval workflows, issue usage guidelines, and designate reviewers. None of this constitutes judgment design. A workflow tells you the sequence. A guideline tells you the rules. Neither tells you who actually holds authority, under what conditions that authority transfers, or what record preserves accountability when judgment is distributed across humans, models, and systems operating at different speeds.
This is the design gap that current policy framing does not address—and that organizations cannot close by adding a human to a process that was not designed for human judgment in the first place.
Decision Design
Decision Design is the practice of making judgment structure explicit, intentional, and institutionally legible.
It is not a decision-quality framework. It is not a process improvement methodology. Decision Design is not about improving decisions alone; it is about designing the authority structure within which decisions become institutionally legitimate.
That distinction matters. Legitimacy, in this context, is not a philosophical abstraction. It is the organizational condition under which a decision can be attributed to a responsible actor, reviewed against a defined standard, escalated when necessary, and defended when challenged. Without it, a decision may be made—outputs are produced, actions are taken—but no one has actually decided anything in an institutionally meaningful sense.
In practice, Decision Design requires organizations to define, for each consequential judgment point in an AI-enabled process:
Judgment authority: Who holds the authority to decide at this point—a human, a system operating within defined parameters, or a combination? Does that actor have both the information and the standing to exercise genuine authority?
Conditions for AI autonomy: Under what circumstances may the system continue to act without human involvement? What constitutes a situation within scope, and what constitutes a situation that exceeds it?
Conditions for human return: What triggers the handoff of authority back to a human actor? Is this triggered by output type, confidence level, consequence threshold, or some combination?
Escalation logic: When a situation exceeds both AI capacity and the authority of the designated human reviewer, what is the defined escalation path? Who receives it, with what information, and within what timeframe?
Responsibility transfer: When authority moves from one actor to another—from system to human, from reviewer to manager, from team to governance layer—how is that transfer recorded, and who is accountable for what afterward?
These elements do not exist as separate policies. They function as a unified judgment architecture. Absent that architecture, each element erodes: authority becomes ambiguous, escalation becomes ad hoc, and accountability becomes impossible to trace.
Decision Boundaries
A Decision Boundary is the point at which legitimate authority must change hands.
This is not a score threshold. It is not a workflow gate. It is not an operational checkpoint configured in a system prompt. Decision Boundaries are not operational thresholds; they are institutional demarcations of legitimate authority.
The distinction is significant. An operational threshold is a technical parameter—a confidence score below which a system flags a response for review, a dollar value above which a transaction requires sign-off. These thresholds serve operational purposes, and they are useful. But they do not, by themselves, constitute governance. A threshold tells a system when to pause. A Decision Boundary defines who holds authority on the other side of that pause, and on what basis they are entitled to exercise it.
In the context of AI agent deployment, Decision Boundaries need to be designed across three levels.
The first is the boundary between autonomous AI action and human review. Below this boundary, the system operates within its authorized scope. Above it, authority returns to a designated human actor with defined standing and access to defined information.
The second is the boundary between routine human review and governance-level escalation. Not every judgment that exceeds AI autonomy is appropriate for a frontline reviewer. Some situations require institutional authority—a risk committee, a compliance function, senior leadership. The conditions under which this escalation is required must be designed in advance, not improvised in the moment.
The third is the boundary between normal operation and suspension. When conditions arise that neither the AI system nor the designated human reviewers can handle within defined parameters, there must be a designed stopping point—a condition under which the process pauses pending institutional review. The absence of this boundary is one of the more serious governance gaps in current AI deployment practice.
Mapping these three boundaries—and specifying the authority, information, and accountability structure at each—is the operational work of Decision Boundary design.
Decision Logs
The third element of this framework is the one most often reduced to something it is not.
A log, in most organizational contexts, is understood as a record of outputs: what the system did, what was approved, what was declined. This is audit trail thinking, and it is necessary but insufficient for AI governance purposes.
Decision Logs do not merely record outputs; they preserve accountability continuity across distributed judgment processes.
The distinction becomes critical when judgment is distributed—when multiple agents act in sequence, when human reviewers intervene at different points, when escalation moves a decision through several organizational layers before resolution. In these conditions, an output record tells you what happened. It does not tell you who held authority at each stage, on what basis each handoff occurred, or whether the accountability chain remained intact throughout.
Accountability continuity is the property of being able to trace, after the fact, not only what was decided but who was authorized to decide it and whether the conditions for that authority were met. Without it, post-hoc review—whether for internal learning, regulatory response, or legal accountability—cannot establish what actually occurred in an institutionally meaningful sense.
Designing Decision Logs means specifying, at each judgment point, what must be captured: the actor, the authority basis, the information available at the time of decision, the conditions that triggered the handoff, and the outcome. This is not a documentation burden. It is the infrastructure that makes distributed judgment governable.
What Organizations Actually Need to Design
For organizations deploying AI agents in consequential workflows, the practical design work follows from these three concepts.
Start with a judgment map, not a risk list. Rather than identifying "high-risk decisions" in the abstract, map every point in the workflow where a judgment occurs—including those currently made implicitly by the AI system. For each, determine the authority structure: who decides, under what conditions, with what information.
Design Decision Boundaries before deployment, not after. The three-level boundary structure—autonomous operation, human review, governance escalation—should be defined as part of system design, not patched in after the first incident. For each boundary, specify the triggering conditions, the authority of the actor receiving the handoff, and the information they require to exercise genuine judgment.
Distinguish ratification from judgment in your review processes. If reviewers are being asked to approve AI outputs under conditions that make substantive review impossible—insufficient time, insufficient context, insufficient expertise—the review process is producing governance appearance, not governance substance. Design for the conditions under which real judgment is possible, or do not claim that judgment is occurring.
Design escalation paths for situations outside the anticipated range. AI agents will encounter situations their designers did not anticipate. The governance question is not whether this will happen but what the organization's response structure looks like when it does. An undefined escalation path is an accountability gap.
Build Decision Logs that capture accountability continuity, not just output history. For each judgment point, define what the log must contain to make the accountability chain traceable. This includes the authority basis for each decision and the conditions that triggered each handoff—not only the outcome.
The shift this requires is not primarily technical. It is conceptual. It means moving from a model in which humans are inserted into AI processes to a model in which judgment authority, escalation logic, and accountability continuity are designed as the foundation on which AI processes operate.
From Human Involvement to Judgment Architecture
The policy move toward requiring human judgment in AI agent deployments is a necessary step. The risks of fully autonomous operation—in systems that act on the real world, chain actions across multiple agents, and move faster than human supervisory capacity—are not theoretical. Governance responses are appropriate.
But the framing of "human involvement" remains conceptually insufficient. A human can be present without exercising authority. Approval can occur without judgment. Oversight can exist in form while failing entirely in substance. And as AI output quality improves, the pull toward ratification will only strengthen.
The governance question for AI agents is not whether a human appears somewhere in the process. It is whether judgment, authority, escalation, and accountability have been intentionally designed—whether the organization can specify, for any given point in an AI-enabled workflow, who holds authority, on what basis, and what record preserves that accountability forward.
This is the work of Decision Design. It is not a quality improvement initiative. It is not a compliance exercise. It is the institutional architecture that makes human judgment in AI systems something more than a procedural formality.
Organizations that design this architecture will be able to govern AI agents with genuine accountability. Those that do not will find themselves with governance records that document approval without authority, oversight without judgment, and accountability that cannot be traced when it is most needed.
The question is not whether to involve humans. The question is whether judgment has been designed.
Ryoji Morii is the Founder and Representative Director of Insynergy Inc., a Tokyo-based management consulting firm specializing in AI governance and Decision Design. He is the author of a working paper on Decision Design published on SSRN (Abstract ID: 6341998).