Why the Pope's warning about AI and human dignity is, beneath the theology, a governance problem we have no language for
When Leo XIV chose his name, he was pointing at something. Popes do this deliberately. The last Leo, Leo XIII, reigned through the second industrial revolution and wrote Rerum Novarum, the encyclical that confronted what factories were doing to people — not the machines themselves, but the way the machinery reorganized human beings into inputs, into hands, into measurable units of labor. He insisted there was a residue of the person that no economic system was entitled to consume.
So when the new Leo took that name and then, early in his pontificate, named artificial intelligence as one of his defining concerns, the gesture carried a thesis inside it. He was not reacting to a technology. He was reacting to a pattern. The pattern of human beings being quietly reorganized again — this time not into labor, but into something stranger.
According to reporting in the Nikkei, Leo XIV has framed AI as a matter of human dignity. Not safety. Not ethics, exactly. Dignity. It is worth pausing on that word, because it is easy to file it under religious vocabulary and move on. Most coverage did exactly that. A pope said a pope-like thing about a new technology; the world nodded and scrolled.
But dignity is not a soft word. In the tradition Leo is drawing from, dignity has a precise function: it marks the line past which a person may not be treated purely as a means. You may not reduce a human being to an instrument, a part, a component in someone else's process. That is the whole content of the term.
Hold onto that, because the rest of this essay is going to argue that the Pope was not making a theological observation at all. He was describing — in the only vocabulary his office gives him — a failure of institutional design. And the failure is spreading fastest in exactly the places that believe they have solved it.
I want to leave the question of why dignity open for now. We will come back to it. By the end, I think it resolves into something quite concrete, and not at all spiritual.
The most reassuring sentence in technology
There is a sentence that appears, almost ritually, wherever AI touches a consequential decision. You have heard it. You have probably said it.
A human makes the final call.
It shows up in self-driving cars and diagnostic imaging, in credit decisions and welfare eligibility, in content moderation and, most gravely, in weapons systems. The AI assists; the human decides. The sentence is designed to reassure, and it works. It works so well that almost no one examines what it actually guarantees.
Look closely at the place where the stakes are highest, because that is where the architecture shows its bones. In the long international argument over autonomous weapons — the debate that the press insists on calling the "killer robot" question — there is a phrase everyone reaches for: meaningful human control. The trigger, the consensus holds, must finally be pulled by a person. Responsibility must rest with a human hand.
Now picture the operational reality the phrase is supposed to govern. A screen populated with dozens, sometimes hundreds, of candidate targets. An AI system scoring each by threat level, ranking them, surfacing recommendations with attached confidence values. The operator has seconds. What the operator is permitted to do is approve the recommendation, or not.
Ask the uncomfortable question. Is that a decision?
In form, yes. A human pressed the button. The chain of responsibility terminates in a person who can be named, blamed, court-martialed. But the substance of the judgment — the weighing, the discrimination between target and not-target, the assessment of proportionality — has already migrated upstream into the system. What remains with the human is the right to approve and the liability for the outcome. The cognitive work has gone elsewhere.
This asymmetry is the thing. Authority and accountability stay with the person. Judgment moves to the machine. And almost no one has noticed that these have come apart, because the paperwork still says a human was in control.
The ritual of confirmation
This is not a battlefield problem. It is a structural one, and the battlefield merely makes it visible. The same shape appears in places with no weapons anywhere near them.
Consider the concept that the entire field has adopted as its safety mechanism: human-in-the-loop. The reasoning is intuitive. Insert a person somewhere in the AI's processing chain, and you have a check against runaway behavior. A human is in the loop, therefore the system is governed. Nearly every AI governance document leans on this idea, often as its load-bearing assurance.
But being in the loop and exercising judgment are not the same act, and the gap between them is where institutions quietly hollow out.
Take credit underwriting, a domain that has lived with this longer than most. There was a time when a loan officer read the financials, sat across from the borrower, weighed the condition of the industry, and decided. Today a model produces a score, and the officer "reviews" it. But the officer works inside a set of incentives. There is pressure to clear volume. And there is an asymmetry buried in the accountability structure: if she overrides the model and the loan sours, the deviation is hers to defend; if she follows the model and the loan sours, the outcome diffuses into the system. Rational behavior, under those conditions, is to ratify.
So review becomes ritual. The officer looks at the score, does not genuinely re-derive the judgment, and approves. On paper, a human reviewed the file. What she actually did was let the model's conclusion pass through her, like a customs gate that stamps without inspecting.
The pattern replicates across the institutional landscape. In public administration, AI now triages subsidy and benefit applications, and a civil servant confirms the determination. In medicine, imaging AI generates findings, and the radiologist confirms. In education, automated systems grade, and the teacher confirms. Confirm, confirm, confirm. Everywhere you look, a human is confirming. Nowhere, increasingly, is a human deciding.
What has happened is a substitution so quiet it leaves no trace. The act of judgment has been replaced by the act of confirmation, and because the documentation is identical — "reviewed and approved by [name]" reads the same whether the review was real or ceremonial — the hollowing-out is invisible from inside the records. The form is preserved perfectly. The substance has evaporated.
Governments are starting to feel the gap — without being able to name it
What is genuinely interesting is that regulators have begun to sense this, even if they lack the vocabulary to describe what they are sensing.
Japan offers a clean example. The Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry jointly maintain a set of AI Business Guidelines, and Version 1.2 turns its attention to autonomous AI agents specifically. It states that such agents — systems that act on their own initiative — require mechanisms ensuring meaningful human judgment, precisely because of risks like malfunction and privacy violation. In a world where an AI agent can invoke other agents and chain tasks together without anyone explicitly authorizing the sequence, the guidelines insist that human judgment be retained somewhere in the structure.
The instinct is correct. When agents call agents and processes spawn processes, the failure mode is no longer a single bad output; it is an entire cascade that no one intended and no one observed beginning. Demanding that human judgment be preserved in that environment is the right reflex.
But notice where the language stops. When a guideline requires "meaningful human judgment," that judgment cannot mean pressing a confirmation button. It cannot mean ratifying a score. And yet the document does not — cannot — specify what the judgment concretely consists of, who exactly inherits it, or under what conditions a human is required to interrupt the machine. That specification is not the regulator's to write. It belongs to each institution, each workflow, each agent deployment, designed case by case.
So the state can say keep human judgment in the system. What it cannot say is here is how that judgment is structured so it does not collapse into ceremony.
That is the gap. And it is not Japan's gap alone. The EU's AI Act demands human oversight. National frameworks everywhere converge on the same requirement: a human must supervise. None of them supplies the architecture that keeps the supervision from becoming theater. The demand exists. The design does not.
The requirement is loud. The structure is missing.
A quiet interval before the turn
Let me set the cases side by side, because the resemblance is the argument.
The weapons operator with seconds to approve. The underwriter ratifying a score against her own incentives. The civil servant confirming a subsidy determination. The radiologist signing off on a finding. Each of them sits inside the loop. Each is, on the institution's books, the locus of judgment. Each carries the responsibility.
And each of them, in a growing share of cases, is not judging. They are confirming.
When something goes wrong — and something always eventually goes wrong — the liability descends onto the human. The AI does not absorb it. It cannot. So responsibility settles on the person while the substance of the decision has already drifted into the system. The human inherits the blame for a judgment he did not, in any meaningful sense, make.
We told ourselves humans were still deciding. In a great many systems, humans are only inheriting responsibility.
The problem here is not the AI. I want to be exact about this, because it is the hinge of everything that follows. The AI is fast, consistent, tireless, and in many domains it produces better recommendations than the humans it sits beside. The trouble is not the quality of the machine's output.
The trouble is that who inherits the judgment has been left undesigned. The sentence "a human makes the final call" distributes reassurance while specifying nothing — not who, not how far, not under what conditions, not with what authority. It is a blank space wearing the costume of a safeguard.
And when the Pope said dignity, I suspect what he was looking at was that blank space.
There is a difference between leaving a space empty by oversight and confronting it as something to be deliberately built. Almost everyone is doing the former. What the moment calls for is a discipline that treats the question — who carries the judgment — not as something settled by habit or accident, but as an object of design.
That discipline is what I call Decision Design.
Decision Design treats the act of judgment itself as the thing to be engineered. Its central construct is the Decision Boundary: the line between what is delegated to the machine and what a human is required to take up and own.
Who decides. How much is handed over, and from where does a human take responsibility back. That line, left implicit in nearly every institution running AI today, is what Decision Design exists to draw on purpose.
What follows takes the concept down to the level where it can actually be built — not as theology, not as ethics, but as institutional architecture. And it returns, at the end, to the Pope's word, because I think dignity turns out to name something surprisingly precise once you have the rest of the vocabulary in hand.
What Decision Design actually designs
Let me fix the object before describing the method, because the term collapses into adjacent concepts the moment it is left vague.
Decision Design is not about improving decisions alone; it is about designing the authority structure within which decisions become institutionally legitimate. That distinction is the whole point. A better model produces better recommendations. It does nothing to clarify who is entitled to own the resulting judgment, who must intervene and when, or where accountability lands when the recommendation and the outcome diverge. Those are questions of authority, not accuracy, and they have to be designed separately.
The material Decision Design works with breaks down into a handful of structural elements.
There is decision authority — the explicit allocation of who holds the right to make a given determination. Human or machine. And where both are present, which one prevails, under what circumstances. Most institutions have never written this down for the decisions AI now touches; the authority exists only as accreted habit.
There is the Decision Boundary itself, the construct everything else organizes around. Decision Boundaries are not operational thresholds; they are institutional demarcations of legitimate authority. This is a subtle but load-bearing claim. A threshold is a number in a workflow — process up to X automatically, escalate above it. A boundary is a statement about who is entitled to decide what, and therefore who is answerable for it. The number may implement the boundary, but the boundary is the institutional fact, and confusing the two is how organizations end up with thresholds that no one is actually accountable for setting.
There are escalation conditions — the precise circumstances under which a judgment must be lifted out of the machine and handed to a person. Confidence below a stated value. A pattern with no precedent. An impact exceeding a defined magnitude. These conditions have to be specified as logic, not left to the intuition of whoever happens to be watching the screen.
There are override conditions — the terms and procedures under which a human may reverse the system's conclusion. What is required to override. Whether a written rationale is mandatory. Where the override is recorded. And, critically, where accountability for a wrong override lands. Saying that humans can override is empty; the override has to carry a designed responsibility structure or it is just another button.
There is accountability continuity — the requirement that responsibility remain traceable from the moment a judgment is made to the moment its consequences arrive, with no point at which it dissolves into "the system handled that part." When a decision passes from AI to human and back, the thread of who-owned-what must not be cut.
And there are Decision Logs, which deserve their own treatment, but for now: the durable record of who judged what, on what basis, in agreement with or in defiance of the machine.
These are not separate features bolted onto a workflow. They are a single act — judgment — decomposed into the parts that can be deliberately built.
What Decision Design is not
The concept is most useful when its borders are sharp, so let me say plainly what it excludes. Every item on this list is something Decision Design is constantly mistaken for.
It is not UX design. How clean the approval screen is, where the confirm button sits — none of that is the subject. If anything, a beautifully designed confirmation interface accelerates the ritualization of confirmation, because it makes the empty gesture frictionless.
It is not workflow optimization. Speeding up the path from input to output is an efficiency concern. Decision Design is an authority concern. They can point in opposite directions; the most "optimized" workflow is often the one that has stripped human judgment out most completely.
It is not an AI ethics slogan. "Human-centered AI," "responsible AI" — these name a direction, a set of values. They do not specify who intervenes under which conditions, and a value that cannot be operationalized into a boundary does no governing work.
It is not an HITL checkbox. Putting a human in the loop is not the goal and never was. As we have seen, a human can occupy the loop while owning nothing inside it. The question is not the human's position in the process but the human's authority within it.
It is not a generic governance framework. A policy document stating that the organization will exercise oversight is a statement of intent. Decision Design is the structure that makes the oversight real. The framework says oversight will happen; Decision Design specifies how it does not hollow out.
And it is not an AI adoption strategy. Adoption strategy asks where to insert AI to capture value. Decision Design frequently asks the inverse — where AI must not be permitted to own the judgment, and where a human is structurally required to take it up. The two disciplines are not opposed, but they are emphatically not the same, and an organization that treats Decision Design as a subroutine of adoption has already misunderstood it.
Stacking the negations sharpens the outline. Decision Design concerns itself with the allocation of judgment authority and the design of its boundaries. Nothing more, and nothing less.
Why the existing layers are not enough
Here is the claim the whole essay has been building toward, stated without hedging.
Governance is insufficient. Digital transformation alone is insufficient. Automation is insufficient. AI Ethics is insufficient. Not wrong — insufficient. Each is necessary; none is enough; and the reason is the same in every case.
Governance sets policy and direction. It declares that the organization will be accountable, will supervise, will manage risk. But policy does not allocate the judgment. It says oversight shall occur without specifying who holds which decision and under what conditions they must take it up.
Digital transformation re-platforms the organization and rewires its processes for a digital substrate. It makes things faster, more integrated, more data-driven. It is silent on the question of who owns the judgments those faster processes now produce.
Automation hands execution to machines. That is its entire purpose and its genuine value. But automation by design removes the human from the act; it does not architect where the human must remain. Left alone, automation's trajectory is to keep eating judgment until only the confirmation gesture is left.
AI Ethics articulates principles — fairness, transparency, human dignity. Indispensable as orientation. But a principle is not an authority structure. "Human dignity" tells you nothing about which civil servant must personally own which subsidy denial, or under what confidence value a targeting recommendation must be lifted to a senior officer.
Each of these operates on a portion of the problem. Each leaves the center — who inherits the judgment — as a blank. Four robust layers, and the load-bearing question falls through the seam between them, owned by none of them.
Decision Design is the name for the layer that has been missing. It is governance-layer architecture, but specifically the architecture of judgment authority itself: who is allocated the right to decide, how decisions escalate, how accountability stays continuous, how the institution's judgment is structured rather than assumed. It does not replace the other four. It occupies the empty seat at the table they have all been gathered around without noticing it was empty.
Bringing it down to the build
If this stayed at the level of concept, it would be one more interesting idea. The test of Decision Design is whether it descends into implementation. It does. Here is the descent.
Escalation conditions, written as logic. For an internal AI agent, the specification reads concretely: if the transaction value exceeds five million yen, or if no comparable precedent exists within the prior six months, halt automatic processing and route to the designated authority. Not "important cases get human review" — that sentence escalates nothing, because "important" is undefined and therefore unfalsifiable. The firing conditions are stated in values and logic that either trigger or do not.
Confidence thresholds, owned by a named person. The model attaches a confidence value to each recommendation. A line is drawn; recommendations below it do not auto-execute and are routed to human judgment. The part organizations miss is that the threshold itself is a decision. Where the line sits is a judgment about acceptable risk, and someone must own that judgment, sign it, and have it recorded. The threshold is not a technical parameter; it is a Decision Boundary expressed numerically, and like every boundary it needs an accountable author.
Override authority, with its responsibility designed in. When a human reverses the machine, the procedure is defined in advance: who is permitted to override, whether a written justification is compulsory, and where accountability rests if the override proves wrong. An override capability without a designed responsibility structure is not governance — it is just a second button beside the first.
Approval routing that refuses to concentrate. Cases are branched to different authorities by their nature rather than funneled to a single approver. Funnel everything to one person and you have manufactured a confirmation ritual by design; that person cannot possibly re-derive every judgment and will default to ratification. Split the routes by the character of the decision, and name the owner of each route.
Decision Logs as accountability infrastructure. Decision Logs do not merely record outputs; they preserve accountability continuity across distributed judgment processes. The log captures the model's recommendation, its confidence value, the human's final determination, whether the two agreed, and the identity and timestamp of the decider — in a form resistant to later alteration. When something fails, the question "who actually inherited this judgment, and where" has a traceable answer. This is what makes accountability continuity real rather than aspirational; the thread we spoke of earlier is physically preserved here.
Authority maps. A complete inventory of which judgments in the organization are owned by which humans or which systems. Most institutions have never drawn this. The first time they do, the discovery is reliably the same: a large number of consequential decisions that no one clearly owns — judgments that fell into the seam, ratified by whoever was nearest the screen. Those blanks are the first thing to design.
Boundary visualization. The line between the machine's autonomous domain and the human's owned domain, rendered as a picture. A boundary that cannot be seen cannot be defended or debated. Drawing it is what makes the question "is this line in the right place?" answerable at all.
Set these against a concrete case. In subsidy screening, the AI checks the formal requirements of each application; high-confidence applications matching established patterns clear an automated first stage. Applications below the confidence threshold, or those representing business forms with no precedent, are lifted to a human examiner. When the examiner rules, the rationale enters the Decision Log. Cases where the human and the machine disagreed are flagged for separate review.
What has changed there is not throughput. The examiner's role has been redefined — from "formally check every file" to "explicitly own the judgments the AI cannot inherit." He is no longer the person who presses the confirm button. He is the person who owns what lies on the far side of the boundary. That is a different job, with a different relationship to responsibility, and it was created by an act of design.
The military case translates the same way. Meaningful human control — a phrase, a value, an aspiration — becomes a concrete Decision Boundary. At which threat levels, which confidence values, which margins of available time may the system act autonomously, and from where is human ownership mandatory? That line is drawn before the engagement, in the calm, not improvised in the seconds when an operator is being pressed to approve. A boundary designed under deliberation is the only kind that survives contact with urgency.
Healthcare, autonomous agents, public administration, underwriting — the structure ports across all of them, because the underlying object is identical in each. It is always the same question wearing different domain costumes: where is the boundary, and who owns the far side of it.
What dignity turns out to mean
Now the question I left open at the start.
Why dignity. Why, of all the available words — risk, safety, harm, bias — did the Pope reach for that one.
Dignity, in the tradition Leo invoked through his name, forbids treating a person purely as a means. It marks the human as something that may not be reduced to an instrument inside another's process.
Return to the confirmation human one last time. He is not the author of the judgment. He is a gate the machine's conclusion passes through on its way to becoming official. He exists, structurally, to generate the signature that lets the output be recorded as human-reviewed. He is, in the most literal sense, an instrument — a person-shaped component whose function is to convert a machine's output into an institutionally legitimate decision. The judgment does not pass through his judgment. It passes around it.
This is the reduction of a human being from the subject of a decision to a part within someone else's process. In Leo XIII's century, the machinery reduced people to labor — to hands, to inputs. In ours, a subtler machinery reduces them to liability-bearers: components retained in the system not to judge but to absorb the responsibility that the system cannot hold. The substance of judgment is taken; the responsibility is left behind, fastened to a person who no longer has the authority that responsibility presupposes.
There is no condition further from dignity than that. To be held answerable for a judgment you were not, in any real sense, permitted to make.
I think that is what the Pope was seeing. Not that AI is dangerous, not that it takes jobs, but that human beings are being quietly written out of the act that makes them more than instruments — and made to keep the liability as a parting gift. The asymmetry has a moral name, and the name is the loss of dignity.
Which means that protecting dignity here is not a matter of principle alone. It is a matter of architecture. It is the deliberate design of who owns which judgment, under what conditions, with what authority — drawn so that the judgments a human is meant to inherit are not allowed to decay into ritual, but are structured so a person can genuinely take them up and answer for them.
Dignity, translated into the language of institutions, is a Decision Boundary in the right place, owned by a named person, recorded in a log that keeps the accountability from drifting away.
Governance will set the policy. Digital transformation will deliver the efficiency. Automation will take the execution. AI Ethics will raise the principle. Every one of them is needed. Not one of them designs who inherits the judgment.
Naming that empty seat, and building a discipline to fill it on purpose — that is the single thing Decision Design exists to do.
Closing
We have spent several years asking whether AI can be trusted to decide. It is the wrong question, or at least a question that arrives too late. In most consequential systems the machine is already deciding, in every sense that matters operationally, and the human beside it is increasingly there to ratify and to be held responsible. The trust question assumes a human decider who, in practice, we have quietly stopped designing for.
The right question is older and harder: who holds the authority to judge, and how is that authority structured so it cannot evaporate while the paperwork still says it is intact. That question does not belong to AI Ethics or to governance policy or to the engineering of the models. It belongs to a layer of institutional reality we have been operating inside without a name for it.
Decision Design is an attempt to give that layer a name, and a set of tools, before the seam between our existing frameworks swallows any more of our judgment. The Pope, in his vocabulary, called the stakes dignity. In ours, the stakes are whether the human chair at the center of these systems is occupied by someone who genuinely decides — or left empty, with a nameplate on it and a button to press.
Author
Ryoji Morii is the founder of Insynergy Inc., a Tokyo-based consultancy working at the intersection of AI governance and organizational decision architecture. He is the originator of the concepts of Decision Design, the Decision Boundary, and Judgment Architecture, and writes on how institutions allocate, structure, and preserve judgment authority in the age of autonomous systems.