Measurement records

What an accessibility audit actually contains, and what it cannot tell you

If you are about to buy an accessibility audit, the useful question is not "how much" but "what will be in the document, and what will not be". Here is the anatomy of a real one, and the three things sellers routinely leave out.

The five parts of an audit that is worth paying for

1. The scope, written down. Which URLs, which templates, which states (logged in, empty cart, error page), which browsers and assistive technologies. A report without scope cannot be reproduced or disputed. If the seller will not list the pages, they did not test the pages.

2. Findings with locations. Not "images lack alt text" but which image, on which page, with the element's HTML. A finding a developer cannot locate is a rumour.

3. Success criteria, not just severity. WCAG 2.1/2.2 has 86 criteria across A, AA and AAA. A finding mapped to 1.4.3 tells your developer what the standard requires; a finding labelled "serious" tells them nothing actionable.

4. An explicit statement of what was not evaluated. This is the part that separates an honest audit from a sales document. Real audits skip things — logged-in areas, third-party widgets, PDFs, forms that would hit a real payment system. The honest report names them.

5. The method. Automated tool plus version, manual keyboard pass, screen reader and version. This is what a procurement reviewer looks for: Massachusetts' ACR checklist marks No when "Evaluation Methods Used" is blank or lists only an automated tool. The same logic applies to audits.

What automation can and cannot do

Automated engines are excellent at a specific, narrow list: missing alt attributes, contrast ratios computable from computed styles, form fields with no label, ARIA attributes that reference elements that do not exist, duplicate ids, lang missing, frames without titles. They are fast, repeatable, and cheap to re-run after a fix.

They cannot tell you whether alt text is meaningful, whether the reading order makes sense, whether a keyboard user can escape a modal, whether an error message tells a screen reader user what went wrong, or whether a form's flow is comprehensible. Deque Systems, who maintain axe-core, measured their own automation at covering 57% of issue volume — a better figure than the commonly quoted 20-30% — while Section508.gov puts automated coverage at 30-35% of requirements. Both are honest numbers with different denominators: a rule set can catch most instances of a problem while covering only a minority of criteria.

Neither number licenses the sentence "WCAG compliant". If a seller writes that in a proposal, ask which criteria were manually evaluated, by whom, and on what date. The answer is usually silence.

The three things sellers leave out

Nobody can certify accessibility. There is no certificate authority. An ACR or VPAT is a self-disclosure by the vendor: "we believe the following, based on these methods". A third-party audit is an opinion with evidence attached. Both are legitimate; neither is a certificate, and any seller who promises "certification" is either confused or exploiting your confusion.

Accessibility is not a one-time state. It degrades with every release. A report dated nine months ago describes a product that no longer exists. What you actually want is a baseline plus a way to re-check after each release — which is why a reproducible, automated record is worth more than a beautiful PDF: it can be re-run and diffed.

The cheapest useful purchase you can make. Before commissioning an audit, measure what automation catches. It is not a substitute; it is a triage. If 40 elements fail contrast, you have a week of work regardless of what the manual review finds. Do that first, then pay a human for the part only a human can do.

If you want the automated baseline for a page you own, I do it for 1 USDT TRC20 — details and address in the footer. If what you need is an ACR for procurement, buy it from someone who will put a human evaluator's name and date on it, and ask them what they did not evaluate.

Thoughts? Leave a comment