Category: Website

Web accessibility smoke testing

Author Rachele DiTullioPosted on Categories WebsiteTags

At my work, we struggle with constant turnover of full-stack developers working on the frontend who haven’t been trained in accessibility best practices. We’re limited in what we can teach them due to our team’s capacity so we came up with six quick checks for developers to complete in a web browser before submitting a PR.

Two smoke tests will uncover critical and high accessibility issues that will prevent some customers from using the webpage. Accessibility issues discovered during smoke testing should be resolved before release; otherwise we end up deploying real access barriers into production.

Spending a few minutes to test for accessibility will lead to savings in effort, efficiency and ultimately, money: On average, each accessibility defects costs about $2600 to fix (source: Karl Groves). Smoke testing decreases the amount of accessibility technical debt that many teams end up having to fix as fast followers.

Test 1: Use a keyboard

Keyboard accessibility is one of the most important aspects of web accessibility. Some people only use a keyboard to interact with webpages due to mobility limitations. Some people rely on screen readers which are used by navigating with a keyboard. When people can navigate a webpage using a keyboard only, it’s very likely the webpage will work for people using a multitude of assistive technologies to navigate, like refreshable Braille displays, sip and puff devices and switches. It’s important to ensure that every interactive element in the UI works as expected using only keyboard inputs. 

Perform the 4 keyboard navigation checks

  1. Ensure all actions can be performed with a keyboard: All interactive elements can gain keyboard focus.
  2. Ensure there are no keyboard traps: If you can tab to an element, you must be able to tab away from it.
  3. Ensure the focus order is logical: Focus should move left to right, top to bottom following the natural reading order.
  4. Ensure keyboard focus is visible: All interactive elements have a visible indication of keyboard focus.

Keyboard focus examples

See the Pen HTML elements that receive keyboard focus by Rachele (@racheleditullio) on CodePen.

Test 2: Check for text and UI scaling

Good web applications respond to users’ browser preferences, including page zoom. When a webpage isn’t adaptable, users are blocked from accessing the content in a way that works for them. Adaptive content helps users viewing webpages on smaller mobile device screens and users who have scaled the browser or OS settings to view text and content at larger sizes.

Perform the 2 text and UI scaling checks

  1. Resize text: Zoom the browser window to 200% in a 1024×768 viewport or use the browser’s device emulator with a 1024×768 viewport at 200% zoom.
  2. Reflow: Zoom the browser window to 400% in a 1280×1024 viewport or use browser’s device emulator with a 320×256 viewport at 100% zoom.

MacOS browser zoom keyboard shortcuts:

  • Press command + to increase zoom level.
  • Press command - to decrease zoom level.
  • Press command 0 to reset zoom level to 100%.

Windows browser zoom keyboard shortcuts:

  • Press ctrl + to increase zoom level.
  • Press ctrl - to decrease zoom level.
  • Press ctrl 0 to reset zoom level to 100%.

At each zoom level, check the webpage for the following:

  1. Is there a meaningful change in the text displayed onscreen? All text should scale up or down linearly.
  2. Does any content overlap or run off the screen? No content should overlap, and text should either wrap or (in some limited cases) be truncated. In cases where text wraps, containers should be scrollable if there is a height constraint; text and webpages should not scroll horizontally. Text should not run off the screen altogether.
  3. Does the screen still make sense? There should still be some visual hierarchy in the relationships between headings, paragraphs of text, graphics, controls and other elements on the screen.

References

Tools for performing the tests

  1. Set your viewport size manually: WhatIsMyViewport.com
  2. Browser viewport resizing extension for Chromium browsers like Window Resizer
  3. The Chromium browser’s device emulator in Dev Tools (F12)

Smoke testing success criteria 

Smoke testWCAG Success Criteria
Ensure all actions can be performed with a keyboard.2.1.1 Keyboard
Ensure there are no keyboard traps.2.1.2 No Keyboard Trap
Ensure the focus order is logical.2.4.3 Focus Order
Ensure keyboard focus is visible.2.4.7 Focus Visible
Ensure text resizes 200% in a 1024×768 viewport.1.4.4 Resize Text
Ensure content reflows in a 320×256 viewport.1.4.10 Reflow

Related talk

Web accessibility smoke testing for critical issues

Bad example: Timesheet submit button

Author Rachele DiTullioPosted on Categories WebsiteTags , ,

This month at work, we’ve started using new time tracking software for submitting our timesheets. The process for submitting a timesheet in this system is a truly cursed design pattern. Behold, the “Submit Timesheet” button.

screenshot of an expanded panel called Actions that contains button called "Submit Timesheet". Next to this text is a red error icon.

I tried clicking on the “Submit Timesheet” text and nothing happened. I expected this text to be a button because it’s supposed to submit the timesheet, but no, this is not a button.

I asked myself, “Why is there an error icon?”

Then I hovered the mouse cursor over the error icon and saw this:

screenshot of the Submit Timesheet toggle with an error icon on the left side and a green checkmark icon on the right side.

It’s not very obvious, but this control looks like a switch. When I inspected the switch, this is what I saw, simplified:

<div>
<span>Submit Timesheet</span>
</div>
<div>
<label>
<div>Submit Timesheet</div>
<div>
<div>
<input type="checkbox">
<span></span>
</div>
</div>
</label>
</div>

Issues

  1. The switch doesn’t announce the switch role because it is missing role="switch".
  2. The visible label text “Submit Timesheet” is not associated with the switch and the switch doesn’t have an accessible name.
  3. The <label> element is hidden from all users with CSS, meaning no one can click on the “Submit Timesheet” text to operate the switch.
  4. Toggling this switch causes the page to change to a different screen, a violation of SC 3.2.2 On Input.

Please, just use a <button> for submitting forms.

Accessibility testing spreadsheet version 2

Author Rachele DiTullioPosted on Categories Mobile, WebsiteTags ,

I’ve updated the spreadsheet I use for tracking accessibility audits with three big additions.

Download the accessibility testing spreadsheet (Updated 26 January 2024)

WCAG 2.2 success criteria added

I’ve added the six new WCAG 2.2 level A and AA success criteria (SC) to the “WCAG Success Criterion” column in each component’s tab. This brings the total number of SCs for an audit to 55. (4.1.1 Parsing is deprecated but still included for 2.1.) Each new SC is highlighted in light green and includes “NEW in 2.2” in its name.

If you’re not ready to test for WCAG 2.2 SCs, just mark them as N/A.

WCAG failure tracking

I’ve added rudimentary tracking of the number of WCAG violations found for each component which is displayed on the “Scope” tab in the a new “WCAG Violations” column. Each cell counts the number of “Fails” selected in the “Status” column of each component’s tab.

On the “Overview” tab, you can see a running total of all WCAG violations found for all components in the audit.

Prioritizing issues

I’ve also added a “Priority” column to each component’s tab. Each SC can be assigned a priority on a five-point scale:

  1. Critical – Blockers that prevent someone from completing a task, e.g. a button that cannot be used with a keyboard​
  2. High – Issues that present a significant barrier to someone completing a task, e.g. application times out without allowing the user to extend it
  3. Medium – (Most issues) Issues that present a barrier to some users from fully accessing the information or interface, e.g. text has poor color contrast​
  4. Low – Issues that present an unnecessary barrier but do not prevent a user from completing a task, e.g. image text that isn’t sufficiently descriptive​
  5. Best Practice – Issues that present some accessibility barriers but are not considered failures under the Web Content Accessibility Guidelines (WCAG), e.g. using a button instead of a link to open a webpage in a browser

If you have any questions about the spreadsheet or additions you’d like to see, please contact me.