We say this on our own results page, so it's worth saying at length here too: a clean automated scan is not proof your site is accessible. Not because our scanner (or anyone else's) is badly built, but because a meaningful share of WCAG success criteria genuinely can't be evaluated by software — they require a human judgment about meaning, context, or experience that a machine can't make. This page is about exactly where that line sits, and why.
The headline number, and where it actually comes from
We tell users our automated check covers roughly 25–33% of the issues on a page. That figure isn't ours — it traces back to accessibility-tooling vendors' own research into their tools' coverage, and it's worth understanding the two different ways that number gets calculated, because they produce different results and vendors don't always say which one they're using.
Method one: count testable success criteria. Deque's own Automated Accessibility Coverage Report found that its axe engine could partially or fully test 15–16 of the 50 WCAG 2.1 AA success criteria — roughly 30%. A separate, independently run UK Government Digital Service comparison of 13 automated accessibility checkers against a single page with 142 known, documented barriers found the best tool (SortSite) caught 40%, WAVE caught 30%, and axe caught 29% — cited directly in the UK's DWP Accessibility Manual, which references the original 2018 GDS study. (We also found a reference to a 2025 academic study reporting similar ~30% figures via a ResearchGate listing, but the paper itself sat behind an access wall we couldn't get past to verify the numbers directly — flagging that one as unconfirmed rather than citing it as a source.)
Method two: count issue volume rather than criteria. Deque has since pushed back on the criteria-counting method as understating real-world value, arguing that a handful of high-frequency, easy-to-detect issue types (color contrast is the classic example) account for a disproportionate share of actual issues found on real pages. Using that framing, their own analysis of over 13,000 first-time page audits found 57% of total issues were automatically detectable. Both numbers are real and both come from the same vendor's own research — they're just answering different questions ("what fraction of the rulebook can a machine check?" vs. "what fraction of issues on a typical page can a machine catch?"). We use the lower, criteria-based figure on our own site because it's the more conservative, more defensible claim to make to someone deciding whether to trust a clean result — and because issue-volume framing can make a scanner look more complete than it is on a page whose remaining problems are exactly the less-common, harder-to-detect kind.
Either way — and this is the part worth sitting with — even within the WCAG criteria that automated tools can partially test, accuracy varies enormously. Deque's own analysis found some testable criteria caught over 90% of real issues, while others in the same "automatable" bucket caught under 7%. "Automatable" doesn't mean "reliably caught."
What's actually in the untested majority
Here's what "the machine can't check this" looks like concretely, grouped by why it's hard:
*Things that require knowing what content means, not just how it's marked up:*
- Whether alt text on an image is accurate — a scanner can confirm an `alt` attribute exists and isn't empty, but not whether "image1234.jpg" was replaced with a genuinely descriptive alternative or with filler text like "photo"
- Whether link text ("click here," "read more") makes sense out of context to someone navigating by a list of links, which is a common screen-reader workflow
- Whether a CAPTCHA or a complex visual element has an alternative that actually conveys equivalent information, not just some alternative text
Things that require a person to actually try the interaction:
- Whether keyboard focus moves in an order that makes logical sense as you tab through a page (a scanner can find missing `tabindex` or unreachable elements, but not "the order is technically present but nonsensical") — the UK Government Digital Service study found keyboard navigation issues among the least reliably automatable category
- Whether keyboard focus gets trapped somewhere a user can't escape (a modal that opens and won't close via keyboard, for instance)
- Whether a screen reader announces dynamic content changes (like a form validation error appearing) at the moment they happen, not just whether the markup exists to theoretically support that
Things that require judging quality, not presence:
- Whether captions on video content are accurate and synced, versus a caption track merely existing
- Whether an audio description track actually conveys the visual information a sighted viewer gets
- Whether an error message is genuinely helpful ("this field is required" is barely useful; "enter your postcode in the format SW1A 1AA" is) — WCAG requires error identification, not error quality, so both pass automated testing identically
Things that are entirely outside WCAG's test criteria but core to real usability:
- Whether someone using only a keyboard or only a screen reader can actually complete your checkout flow, not just whether each individual page technically conforms
- Cognitive load, plain-language quality, and whether instructions make sense to someone unfamiliar with your product
None of this is a knock on automated tools specifically — it's a structural limit of what's checkable by pattern-matching against markup, and every automated scanner (ours, axe-core-based competitors, and paid enterprise tools alike) runs into the same wall for the same reasons.
What this means practically
A clean automated result tells you something real and useful: the mechanically-checkable failures — missing alt attributes, insufficient color contrast, unlabelled form fields, invalid ARIA usage, and a few dozen other concrete, rule-based checks — aren't present. That's worth knowing, and it's worth fixing before anything else, because these issues are usually cheap to fix and block real users outright. But it is not, and was never designed to be, a substitute for a human accessibility audit, and it's not a compliance certification or legal sign-off — see our note on the EAA's actual requirements if that's what you're trying to satisfy.
If the automated pass comes back clean, the next step for anyone who needs real assurance — not just a Tireless Labs recommendation, this is standard accessibility-industry guidance — is manual testing: someone navigating your key flows with only a keyboard, someone testing with an actual screen reader (NVDA, JAWS, or VoiceOver), and ideally, feedback from people who use assistive technology day to day.