The EU AI Act Just Made AI Disclosure Mandatory. It Says Nothing About Whether the AI Was Right.
Why "It Cites Its Sources" Isn't Verification
3 min read
From today, the EU AI Act requires disclosing when AI wrote something — it says nothing about whether AI got it right. Two objections keep coming up when that gap gets pointed out: "our AI already cites its sources," and "isn't this just evaluations?" Neither is verification. A citation points at a source without checking whether it says what the sentence claims; evaluations grade a system in aggregate, before any of today's specific outputs exist. Disclosure and accuracy are two different problems, and only one of them is now mandatory.
Two objections come up every time this argument gets made in public. The first: "our AI already cites its sources." The second: "isn't this just evaluations?" Both are reasonable things to ask. Both miss the actual claim.
A citation is a pointer. It isn't a check.
A citation attaches a reference to a sentence so a human could go and check it themselves. That's genuinely useful, and it solves a real problem. It is not the same problem this series is about, and it's worth being exact about the three things a citation never does.
It doesn't verify. It never opens the table to confirm the source actually says what the sentence claims it says. It doesn't weigh. It has no view on whether the source is strong enough to carry the claim — nine respondents cite exactly as neatly as four hundred. It doesn't refuse. It has no mechanism to say no. A sentence with a bad citation ships identically to a sentence with a good one.
A real check does three different things: it grades the claim against the actual record and scores how well it matches, instead of just noting a record exists. It weighs whether the evidence can bear the claim's breadth — base size, subgroup, how far the statement reaches. And it refuses — below a bar, the claim is held back rather than shipped with a footnote. A citation is a footnote. A check is an audit, and the difference is whether the system is ever allowed to say no.
Isn't this just evaluations?
Evaluations — often shortened to "evals" — run at build time, on a sampled test set, and hand you an aggregate score: a system passes 92% of its test suite. That's real, and it's how you tune a system before it ships. But an eval never tells you which of today's outputs is the 8%, and it never stops one — by the time an eval runs, the thing it's grading already exists in a test harness, not in front of a client.
Verification runs on this exact output, at the moment it matters, and it has the authority to refuse. Evals tune the engine; verification inspects what actually comes off it — one output at a time, with a veto. A team that has evals and thinks it has verification is exactly the team that ships the confident, well-tested, wrong thing.
The law just picked a side, sort of
Starting today, Article 50 of the EU AI Act requires deployers to disclose when AI-generated text is published to inform the public on matters of public interest. There's a carve-out: the obligation lifts where the content has had substantive human review and a named person or company holds editorial responsibility. Exposure for getting it wrong runs up to €15 million or 3% of worldwide annual turnover.
Article 50 is disclosure and labelling, end to end. There is no obligation anywhere in it that AI-generated output be factually correct. The law will make you say AI wrote it. It won't make it right.
Which is the actual commercial point. Once disclosure is mandatory, "AI-assisted" starts appearing on the deliverable, and the client's very next question is how you know it's sound. That question arrives whether or not anyone in the room has a real answer ready.
What this means in practice
None of this replaces a human's judgment on interpretation or strategy — no data verifies a reading, and anyone claiming otherwise is selling you the hallucination problem twice. What it replaces is the current default, which is: a citation ships, a disclosure label gets added, and nobody actually re-opens the table to see if the sentence is true. That gap is exactly what's still open after both of these boxes get checked.
Let me know on LinkedIn or on Substack if you've had this exact "but it cites its sources" conversation — I'd bet most people building with AI right now have had it at least once.