Accessibility Scanners Only Catch 25% of Issues, They Said. Until Today.

Accessibility scanners only catch 25% of issues, they said. Until today. Let me show you.

Ask how much of accessibility an automated checker can catch and you will hear a quarter, maybe a third. The figure has real sources behind it. When the UK Government Digital Service built a deliberately broken page in 2017 and ran ten tools over it, 29% of the barriers were found by none of them. Deque, who make axe-core, measured their own coverage across 2,000 audits in 2021 and put it at 57% of issues by volume, while noting the industry belief was 20 to 30%.

The usual explanation, put well in a cyber security FAQ that now ranks for the question, is that scanners find structure and miss experience. A rule can see that a label exists. It cannot tell whether focus lands inside a dialog when it opens, whether a status message is ever spoken, or whether a menu only opens for a mouse.

We built 13 pages with 27 of exactly those failures. axe-core, the rule engine inside most free checkers, caught 1. Inclusivio's AI scan caught 24.

Our test suite has a set of small web pages, each built to fail in one way a rule engine cannot see: a dialog that leaves focus behind, a keyboard trap, a menu that only opens on hover, an animation that ignores reduced motion, a form error nobody hears. 13 of the pages carry 27 of these planted failures between them. Two more pages are controls: one clean, one built only to be read aloud.

On November 6, 2026 we ran two scans over all 15 pages. First axe-core 4.11, the open-source rule engine inside most free checkers, with every WCAG 2.0, 2.1 and 2.2 A and AA rule switched on. Then Inclusivio's AI scan with the live models, the same code that runs when you press AI Analyze: it tabs through the page, opens what opens, listens to what a screen reader would announce, watches motion over time, shrinks the viewport to 320 pixels and asks a model to judge what it saw.

Ten of the 15 pages came back with zero violations. Four more reported only that some buttons were smaller than 24 pixels, which is true and has nothing to do with the failures we planted. The one hit was on the keyboard page, where axe correctly flagged a button with no accessible name. The same page also hides its focus ring, puts a control out of reach of the Tab key and lets a sticky header cover the focused element. axe passed all three.

A dialog that steals focus has valid markup. So does a page that flashes four times a second. A rule engine reads markup, so it passes both.

Each test page has a written answer key: the exact failures it was built with. We ran the live scan and checked its findings against that key. 24 matched. The AI scan finds these by doing what a user does and checking the result.

A status message that appears after a button press but is never announced. A dedicated check for this caught it in our test suite and passed it in the live run. A 'Saved' message on the journeys page that disappears before a screen reader can reach it. And 'Load more' on the same page, which inserts content without telling anyone. The scan did flag that button, but for a different reason (it exposes no expanded state), so we do not count it.

All three are announcements: something changes on screen and the question is whether a listener hears it. That is next on our list.

The pages are our own test suite. A finding counts when the scan rates it at 50% or more, the same threshold the product uses to show an issue, and it matches the failure planted on that page. The clean control page had no planted failures and got none. The raw file lists every finding with its probability.

25% describes rule engines. It no longer describes automated testing. If your current checker passes a page, that tells you the markup is valid. It does not tell you whether a keyboard user can close your cookie dialog or whether your checkout error is ever read out.

Run the free scan on your own site, then press AI Analyze on the page that makes you money. The raw results from this test, the 15 pages and the script that ran them are in our repository, and contact@thea11ylabs.com will send the JSON to anyone who asks.