Disclosure is a control, not just a compliance checkbox
Disclosure is usually filed under legal compliance. Treated as a control, it works: a person who knows they are dealing with an autonomous system, or hearing a synthetic voice, applies more scrutiny before acting on it.
By Piotr Zenon Żabrowski · Published · Updated
Why disclosure changes attacker economics
Social engineering depends on a target not questioning a message's source. Voice cloning and synthetic video make familiar voices and faces easier to imitate at scale, weakening cues recipients use to question a message. A clear signal that content is synthetic, or an interaction is automated, restores the doubt attackers erase — a control on the human side of the chain, not paperwork beside it.
What the law actually requires
Subject to Article 50's exceptions, providers of AI systems meant to interact directly with natural persons must ensure users are informed they are dealing with AI no later than first interaction, through means the person will notice. The duty lifts where the AI nature is already obvious to a reasonably observant person, and a narrow law-enforcement exception exists. A generic "assistant" label or a line in terms of service does not satisfy this. AI agents get Commission guidance calling for stronger verification, given the range of actions an agent can take on someone's behalf. The article also requires labeling of deepfakes — synthetic image, audio, or video resembling a real person or event — and machine-readable marking of generative content.
| Duty | What it discloses | Attack it undercuts |
|---|---|---|
| System disclosure | You are talking to an AI, not a person | Pretexting an agent as a trusted human contact |
| Content disclosure | This audio/video/image is synthetic | Voice-clone and deepfake-driven fraud |
Agents are the harder case
The gap between an assistant and an autonomous agent is a continuum: more steps without a human, broader tool access, and persistent state push a system further along it. That matters for disclosure because a system able to act — sending messages, moving money, changing records — is worth impersonating in a way a static chatbot is not. If someone cannot tell whether they face a scripted bot or an agent able to execute what it describes, they cannot calibrate how much verification it deserves.
Disclosure as a working control
- AI interaction is disclosed at first contact, in the interface itself, not in terms of service.
- Agents that can take action are disclosed as agents, not labeled generically as an 'assistant.'
- Synthetic audio, video, and images are marked in a way a recipient will notice.
- Staff treat an unexpected voice or video request as unverified until confirmed through a separate channel.
Frequently asked questions
Does disclosure alone stop AI-driven fraud?
No. It removes one advantage attackers rely on — the target not questioning the source — but should sit alongside verification procedures for high-risk actions, not replace them.
Do internal-only agents need disclosure too?
Employees are natural persons, so internal-facing interactions can fall within Article 50. Providers must design the system to inform them unless the AI nature is already obvious in context or another statutory exception applies.
Related
AI security
How to secure GenAI, LLM and agentic AI systems: current OWASP threat classes, layered controls, agent authority, retrieval isolation and monitoring.
Compliance
Turning DORA, NIS2, ISO 27001, GDPR and the EU AI Act into one control set with reusable evidence and aligned reporting clocks.
Agentic threat models
Why the shift from generative query-response to an autonomous agentic loop changes the threat model of an AI system from bad output to unintended action.
AI security assessment
Security assessment of GenAI and LLM systems: prompt injection, tool and agent abuse, data leakage, model supply chain and monitoring gaps.
AI governance & AI Act readiness
Build a workable AI governance framework: system inventory, risk classification, acceptable-use policy, review gates and AI Act evidence.
Compliance readiness
Gap assessment and remediation planning for DORA, NIS2, ISO 27001, GDPR and the EU AI Act — mapped to controls and evidence, not paperwork.