Insynergy
← Back to Insights

Shadow AI and the EU AI Act- The Real Exposure Is Unmanaged Decision Authority

Shadow AI creates more than a security and compliance gap. It obscures which judgments employees delegate to AI, who holds legitimate decision authority, and how organizations preserve accountability under the EU AI Act.

A recruiter pastes a candidate's resume into a generative AI service the company has never approved and asks whether the candidate should advance to an interview. This is Shadow AI: employees using AI tools or features that their organization has neither approved nor inventoried. The immediate risks include personal data, security, and compliance under laws such as the EU AI Act. The deeper exposure is unmanaged decision authority. The organization cannot identify which judgment the employee delegated, who had authority to make it, or how anyone could reconstruct it later.

Shadow AI therefore deserves attention as a warning, not as an automatic legal category or a universal answer to which EU AI Act risk matters most. It reveals AI-assisted judgments that an organization has failed to classify, authorize, supervise, and record.

A Hiring Decision the Organization Cannot Explain

The recruiter wants to save time and obtain a second opinion before a full day of screening. In the seconds between paste and send, personal data leaves the company for an external service, an opaque assessment enters the hiring process, and the organization may lose the evidence needed to trace that influence.

If the candidate does not advance, who made the decision? The recruiter who entered the prompt, the model that assessed the resume, or the organization that allowed this practice to develop without defining its limits?

Entering personal data into a generative AI service does not necessarily cause a data breach. Contract terms, retention settings, training policies, and technical controls change the exposure. Leadership still needs to answer a narrower question: under which conditions, using which data, and to what extent did the organization permit AI to influence the judgment?

Shadow AI Includes AI Embedded in Everyday Software

Shadow AI is the use of AI tools or AI features in work processes without formal organizational approval or visibility.

The category extends beyond a standalone chatbot accessed through a personal account. It includes assistants embedded in browsers and search tools, drafting and summarization features in office software, and predictive or generative functions inside CRM systems and other SaaS products. An employee may delegate part of a judgment through summarization, scoring, ranking, or recommendation without recognizing the feature as a separate AI system.

Darren Williams, Founder and CEO of BlackFog, presents Shadow AI as a blind spot for EU AI Act readiness in “Is Shadow AI your Biggest EU AI Act Compliance Risk?”, published by The European Financial Review. His useful claim is that an organization cannot classify the purpose of an AI system it has not identified, place that system under meaningful human oversight, or retain the records that a regulated use may require.

Williams cites BlackFog research reporting that nearly half of employees at larger organizations share work data with unsanctioned AI tools and that 85 percent continue using Shadow AI when an approved alternative exists. The provenance matters. BlackFog is a security vendor, Williams leads the company, and the figures come from its research rather than an independent public dataset. They indicate a problem worth investigating; they do not establish its prevalence across every workforce or jurisdiction.

The EU AI Act Classifies AI by Purpose and Use

Shadow AI is not a legal category in Regulation (EU) 2024/1689, the Artificial Intelligence Act. Unauthorized use does not by itself turn an AI system into a high-risk system or establish a violation. Regulatory treatment depends on the system's intended purpose and the context in which an organization uses it.

The Regulation entered into force on 1 August 2024. Prohibitions on certain AI practices and the AI literacy obligation have applied since 2 February 2025. The original text set 2 August 2026 as the application date for many remaining provisions, including rules for high-risk systems listed in Annex III.

The EU's co-legislators have since adopted an amending regulation under the Digital Omnibus on AI. The European Parliament adopted its first-reading position on 16 June 2026, and the Council approved the legislative act on 29 June. The Presidents of the two institutions signed it on 8 July. As of 16 July 2026, the measure awaits publication in the Official Journal of the European Union and will enter into force on the third day after publication. The adopted text moves the application date for stand-alone high-risk AI systems to 2 December 2027 and for high-risk AI systems embedded in regulated products to 2 August 2028. The amending regulation postpones those obligations; it does not remove them.

Recruitment shows why purpose and use matter. Annex III covers AI systems intended for recruiting or selecting people, including systems used to analyze and filter applications or evaluate candidates. A provider of such a high-risk system carries obligations that include risk management, data governance, technical documentation, and conformity assessment. A deployer using the system in hiring has a different role. Article 26 requires deployers, among other duties, to assign competent people to human oversight, monitor operation, keep automatically generated logs under their control, and ensure that input data under their control is relevant and sufficiently representative. Provider and deployer duties cannot be treated as interchangeable.

The Regulation can also reach organizations outside the European Union. Article 2 covers providers and deployers established in a third country when an AI system's output is used in the Union. Personal data may bring the General Data Protection Regulation into the same situation, but the two regimes impose distinct obligations. Compliance with one does not establish compliance with the other.

These distinctions limit the claim. An unapproved AI use does not automatically become high-risk, and AI use in HR does not receive one classification regardless of purpose. Leaders need visibility into the use before they can determine which rules apply.

Shadow AI Conceals Transfers of Judgment

Most Shadow AI programs begin with a tool inventory: which systems employees use, what data they enter, and whether the organization can retrieve the logs. Access management, data controls, and system logging remain necessary because they establish what employees used.

The inventory should also identify the judgment. Did the AI summarize evidence, rank candidates, or recommend rejection? Did a reviewer assess the output or approve it by habit? Who could override the recommendation, and who must explain the result to the affected person?

An organization may complete its tool inventory and still miss the transfer of authority taking place through those tools. Each discipline addresses part of that transfer. Governance sets organization-wide rules, oversight, and accountability. Digital Transformation redesigns work and value delivery. Automation executes defined processes. AI Ethics articulates principles such as fairness, transparency, and accountability. A recurring decision still requires someone to specify who may make the judgment, under which conditions, with what evidence, and who owns the outcome.

Japan's Ministry of Internal Affairs and Communications and Ministry of Economy, Trade and Industry point toward the same requirement in their AI Guidelines for Business, Version 1.2. The guidelines caution organizations against over-reliance on AI and automation bias, and they call for human involvement, safety, and privacy protections proportionate to the use and its risks. They provide non-binding governance guidance, not EU law or a uniform requirement for human approval of every AI process. An organization must still convert those principles into the design of each consequential judgment.

Human Review Requires Authority to Reject the AI

Placing a person in a process does not prove that the person exercised judgment. A reviewer may approve an AI recommendation in seconds, dozens of times a day. The record then shows a human name even though the model supplied the conclusion and the organization preserved no evidence of independent review.

Substantive human oversight requires the authority to reject, modify, or reverse the output. The reviewer also needs the relevant evidence, time to consider it, skill to test it, and a defined escalation route. The organization must allow the reviewer to disagree with the model without penalty. Without those conditions, human-in-the-loop assigns a signature rather than judgment.

Decision Design Makes Authority Explicit

Decision Design is a framework that treats organizational judgment as an object of design. Decision Design is not about improving decisions alone; it is about designing the authority structure within which decisions become institutionally legitimate.

For a recurring judgment, Decision Design names the authorized decision-maker, assigns a limited role to AI, and defines the authority retained by a person or the organization. It specifies permitted data and evidence, adoption and stopping conditions, exceptions, escalation, records, explanatory responsibility, and the events that trigger review. Teams then implement that design through workflow, permissions, system configuration, and documentation.

This organizational scope distinguishes Decision Design from a general AI-use policy, a compliance checklist, or a method for improving individual cognition. It does not require a person to make every decision or assume that AI is untrustworthy. Nor is it an established legal doctrine, public standard, or academic consensus. It addresses a specific gap: AI distributes judgment across tools and teams while obscuring the person authorized to decide and the person accountable for the outcome.

Decision Design complements established disciplines. Governance supplies rules and oversight. Digital Transformation redesigns work. Automation runs defined processes, while AI Ethics supplies the principles those processes should respect. Decision Design carries those commitments into a particular judgment by defining legitimate authority, stopping conditions, escalation, and records. It depends on law, governance, ethics, risk management, and technical controls rather than replacing them.

Decision Boundaries Define Legitimate Authority

A Decision Boundary defines the institutional line between judgments delegated to AI and judgments retained or accepted by a human or organization. Decision Boundaries are not operational thresholds; they are institutional demarcations of legitimate authority.

AI participation varies by degree. A system may retrieve information, summarize evidence, generate options, evaluate or rank them, recommend an action, or execute one under stated conditions. The organization decides how far AI may proceed for each judgment.

The boundary should account for the effect on a person's rights or opportunities, reversibility, potential harm, data sensitivity and quality, model uncertainty, legal exposure, and access to explanation or appeal. A consequential and irreversible judgment affecting a person's rights calls for tighter limits on delegated authority. A reversible administrative step with limited impact may support greater autonomy.

A Decision Boundary Map for Recruitment

A Decision Boundary Map documents the path of one recurring judgment from evidence to outcome. In recruitment, the map begins with the decision at issue and the candidates whose opportunities it affects. It assigns the AI's permitted role, the reviewer's responsibilities, and a named owner for the final decision. It also identifies allowed data, conditions for considering an AI output, events that stop the process, and the team that receives an escalation.

The record completes the map. The organization specifies which evidence, changes, reasons, and approvals it will preserve, along with the events that force it to revisit the boundary. Those events may include a model change, a data-quality failure, complaints, a legal change, or evidence of disparate outcomes.

Applied to the opening case, an unauthorized general-purpose chatbot may not determine whether the candidate advances. A governed system could support narrower tasks such as structuring resume information, identifying missing details, or preparing interview questions. Using AI to evaluate or rank candidates requires a separate assessment of purpose, regulatory classification, data, bias, oversight, and recordkeeping.

The reviewer has an operational role, not a ceremonial one. The recruiter must be able to inspect the evidence, reject the recommendation, record a different conclusion, and escalate concerns about fairness or data quality. A system that records only an approval click cannot demonstrate that any of those actions occurred.

Decision Logs Preserve Accountability Across the Process

A Decision Log records who made a judgment, which authority permitted it, what evidence supported it, and how the organization reached the outcome. Decision Logs do not merely record outputs; they preserve accountability continuity across distributed judgment processes.

A technical log helps an engineer trace requests, events, and errors. A Decision Log preserves the evidence and criteria considered, the AI output, the relevant model or rule version, the reviewer's changes, the final decision owner, and any escalation or override. It also records the reason for the outcome so the organization can explain the judgment later.

The log creates its own exposure when it concentrates personal data and consequential decisions. Its design therefore needs a defined purpose, access controls, retention and deletion rules, and security measures. Capturing every keystroke does not create accountability if the record omits who held authority and why that person accepted the outcome.

The Organization Must Be Able to Name the Decision-Maker

Return to the recruiter and the rejected candidate. With a Decision Boundary Map, the organization can state how far the AI was permitted to go, who held final authority, and where it recorded the reason. Perhaps the AI structured information and identified omissions; a named hiring manager decided not to advance the candidate; the Decision Log preserved the evidence and reasoning; and concerns about fairness or data quality would have stopped the process and moved the case to HR and legal.

Who rejected the candidate? The answer should identify a person acting within authority the organization granted and on evidence the organization retained. If the organization cannot provide that account, it did not design the judgment it allowed AI to influence.

Shadow AI Controls Should Cover Judgments as Well as Tools

Existing controls can extend from technology to authority. A tool inventory can also list the judgments each system touches. Data controls can define which evidence may inform each decision. Access controls can govern decision rights, while technical logs can connect to records of the decision path. AI literacy can teach employees when to question, correct, or reject an output. Incident response can include the authority to suspend or narrow a Decision Boundary.

Approving an enterprise AI tool will not end informal use when the sanctioned process is slow, the approved tool does not fit the work, or performance targets reward shortcuts. Leaders should examine why employees leave the formal process and redesign it so that responsible use remains practical.

A leadership team can begin with seven questions:

Shadow AI exposes more than an unauthorized tool. It exposes decisions whose authority, limits, and ownership the organization cannot explain. Responsible adoption begins when the organization can identify who holds each judgment before AI influences the outcome.

FAQ

Does using Shadow AI automatically violate the EU AI Act?

No. Shadow AI is not a category in Regulation (EU) 2024/1689, and unauthorized use does not by itself create a high-risk system or establish a violation. The system's intended purpose and use determine its regulatory treatment. An unknown use can still expose an organization to non-compliance when employees apply AI in a regulated context without the required oversight, controls, or records.

Is an approved enterprise AI tool enough to control Shadow AI risk?

No. Tool approval does not define which judgments employees may delegate or who owns the outcome. An effective program combines usable approved tools with explicit Decision Boundaries, decision ownership, data rules, escalation, and records.

How does a Decision Boundary differ from human-in-the-loop oversight?

Human-in-the-loop describes a person's placement in a process. A Decision Boundary specifies the judgment that person owns, the conditions under which AI may contribute, and the point at which the person must intervene. The boundary gives human oversight its authority and scope.

References

Decision Design is a judgment architecture framework proposed by Ryoji Morii, founder of Insynergy Inc., for structuring authority, accountability, and decision boundaries in AI-augmented organizations.

Japanese version is available on note.

Open Japanese version →