Does UserWay make a site WCAG compliant? We break down exactly what the widget can and cannot change at runtime — and how a two-pass axe-core test shows you where the gap is.
UserWay is one of the most widely deployed accessibility overlay widgets on the web. Its pitch is hard to resist: drop in a single script tag, gain a floating toolbar, and meet your ADA and WCAG obligations. The widget does do real things — it injects ARIA attributes into the DOM, offers a settings panel with font-size and contrast controls, and adds keyboard-navigation shortcuts. The question worth asking precisely is: which WCAG success criteria does that runtime activity actually resolve, and which ones does it leave untouched?
This post works through the mechanics, not the marketing. We will describe what a runtime overlay can and cannot change, then walk through how a before/after witness run using axe-core reveals that boundary on your specific page.
What a runtime overlay can change
JavaScript running in the browser has genuine access to the live DOM. An overlay can do the following in the time between page load and a user interaction:
- Add or patch aria-label attributes on elements that have none in source.
- Inject role attributes to announce landmarks that were missing from the original markup.
- Apply inline CSS to boost text contrast or increase font size when a user activates that setting.
- Suppress motion via prefers-reduced-motion overrides in injected stylesheets.
- Intercept focus events and attempt to enforce a logical tab sequence.
These are legitimate runtime capabilities. Some axe-core rules, under some conditions, will read differently once those changes are applied. That is real coverage, and it is worth acknowledging precisely because the fair question is always one of degree and reliability.
What a runtime overlay cannot change
The hard boundary is the source. An overlay cannot modify the HTML your server sends, rewrite your CSS stylesheet on disk, or alter how native browser controls are built. The consequence is that entire categories of WCAG failures are structurally out of reach:
- Images embedded with a CSS background-image property carry no alt equivalent in the DOM — the overlay has no element to annotate.
- PDF and Office documents linked from the page are unaffected by any page-level JavaScript.
- Custom interactive components built without semantic HTML — a div acting as a button, a span handling click events — may resist ARIA patching in ways the overlay cannot fully predict.
- Video captions and audio description tracks are server-side media assets; the overlay has no write path to them.
- Focus order that is dictated by DOM order cannot be truly reorganized without DOM mutation; overlays that attempt this can introduce more focus bugs than they fix.
- Color contrast in SVG files, icon fonts, or background images computed from CSS variables at render time is often outside what a script can reliably measure and correct.
Even for rules the overlay targets, the fix is applied per-page-load, after a script-load delay. Automated checks running in a real browser and assistive technologies interacting with the page do not always observe the same post-injection state. A rule can appear held up in one test run and fail in the next if the overlay script is slow, blocked by an ad blocker, or subject to a network hiccup.
How a two-pass test shows you the actual boundary
OverlayRiskWitness loads the target page twice in a real hosted browser. The first pass blocks the overlay script at the network layer so axe-core sees the page as the source ships it. The second pass loads the same URL with the overlay active and gives it time to complete its injections. The two axe-core result sets are then compared rule by rule.
Each rule lands in one of three finding states. A rule that had violations in pass 1 and has zero in pass 2 held up — the overlay genuinely resolved it on this page. A rule that still shows violations in pass 2 did not hold up. A rule the engine could not evaluate in one of the passes is marked not testable, which is an evidence gap rather than a pass.
A single axe scan with the overlay active tells you whether the page passes a rule with the widget on. It cannot tell you whether the overlay is the reason, or whether the underlying page would pass anyway. The two-pass diff answers that question directly: no_effect means the overlay made no measurable difference to that rule on this page.
The specific problem with public compliance claims
Most sites that install UserWay also publish an accessibility statement. Common language includes assertions like "this site conforms to WCAG 2.1 Level AA" or "all pages are keyboard accessible." Those are public claims about specific technical standards.
A witness run extracts those claim sentences and places them next to the finding for each rule they name. If the accessibility statement says the site meets 1.4.3 (minimum contrast) and the two-pass diff shows color-contrast violations did not hold up with the overlay active, those two facts are in the same exhibit — timestamped and hashed. That pairing is the evidence. What it means legally is a separate question for counsel, not for the witness.
Automated checks cover a well-defined subset of WCAG success criteria. Screen-reader behavior, cognitive load, and task completion by actual users with disabilities require manual and user testing that no automated tool — overlay or otherwise — can substitute for. A passed axe-core run is a necessary but not sufficient condition for real accessibility.
What this means in practice
None of this is an argument that UserWay has no value. For sites with simple ARIA gaps on well-structured HTML, the widget can produce genuine improvements that show up in automated testing. The question is whether the specific rules that matter for your specific pages held up under a real check — and whether your public accessibility statement accurately reflects that finding.
The practical steps are straightforward. Run a witness on the pages your accessibility statement covers. Read the per-rule diff. Where rules did not hold up, you have two choices: fix the underlying source issue, or revise the claim. Either is a defensible path. What is not defensible is a public claim that an independent test does not support, sitting unexamined.
If stakeholders keep pointing to a high accessibility score as if it closes the conversation, Website Accessibility Scores: What a 0–100 Number Can Show — and What It Still Can't Prove explains why the number can guide triage without replacing a dated page-level comparison.
If the conversation inside your team has already shifted from one widget to broader badge or guarantee language, pair this page with Accessibility guarantee vs. independent evidence to separate runtime improvements from the evidence record your site still needs. For the exact timestamp, snapshot-hash, and claim-quoted exhibit structure, use How to document website accessibility evidence that holds up. If you are rewriting the public claim after that review, use Writing an accessibility statement when you use an overlay to keep the statement scoped to what the live page actually supports.
- Free witness: one-page before/after diff showing the first finding — no signup required.
- Risk Packet ($49 one-time): full evidence file for 5 to 10 pages, claims quoted back, UTC timestamps and snapshot hashes on every exhibit.
- Drift Monitor ($99/month): re-runs the witness on a schedule and alerts when a finding state changes — overlay updates and site deploys can flip a held-up rule without warning.
- Agency Watch ($249/month): the same monitoring surface scaled to client portfolios.
A single witness run takes under two minutes on most pages. The output is not a legal opinion and not a compliance certificate — it is a factual record of what axe-core observed, under both conditions, at a specific moment in time. That is the starting point for any honest conversation about what your overlay is and is not doing.