Accessibil-IT: Automated Accessibility Testing

Accessibil-IT blog: Automated Accessibility TestingLast month, we focused on the benefits of manual accessibility testing and why manual tests were more effective in determining the true level of accessibility and usability of a website for individuals with disabilities.

While manual accessibility tests provide a more thorough approach, such techniques do not scale across multiple websites and thousands of pages.

Automated accessibility tools can streamline the evaluation process and provide the scalability necessary to support larger websites.

There are two basic types of automated accessibility tools: browser-based add-ons, and scanning and reporting applications.

Two Types Of Tools

Browser-based add-ons allow for a quick check of a single web page and can use developer features inherent in the browser to highlight issues. While most browser-based evaluation tools are limited to single-page evaluations, developers or page authors have the benefit of reviewing errors within the context of the rendered page. This can aid in the learning process and help reviewers understand what caused the error and how to avoid it in the future.

Browser-based add-ons can also be effective when evaluating pages behind firewalls or other authentication schemes, in that the accessibility evaluation is performed directly within the browser itself and does not require connection to an external service.

Scanning and reporting applications offer a more robust solution when reviewing a large set of web pages or an entire website. Most scanning and reporting applications can perform a website crawl to identify issues existing on specific pages as well as provide an overview of the entire site.

Developers and page authors can focus remediation efforts on those pages identified as having errors without wasting time searching every single page manually. Additionally, by providing an overview of the website, scanning and reporting applications can highlight the most common accessibility errors found. This allows for institutional staff to target training efforts in an attempt to minimize the recurrence of such accessibility errors site-wide.

Advantages, Limitations

A major advantage of automated accessibility tools is the ability to review a web page or website quickly for accessibility errors. For example, browser-based add-ons can identify accessibility issues present in a page within seconds and indicate the significance of those errors. This allows developers and page authors the opportunity to assess at a technical level if accessibility issues are present prior to performing manual accessibility checks. Websites and web pages should pass automated tests before undergoing manual accessibility testing.

It is important to recognize, however, that automated tools are limited to checking accessibility criteria that can be automated (i.e., “pass” or “fail”) and do not require human review. An automated test can review if an image contains an alt attribute, but it cannot determine if that alt attribute is appropriate in describing the purpose and/or function of the image. Using an automated accessibility evaluation tool is effective in identifying some errors quickly, but automated testing alone should not be the final indicator in determining accessibility.

Automated tools can also suffer from false-positive reports indicating accessibility errors when in fact there are none. Automated tools may generate accessibility errors and warnings on pages and sites that are not reflective of actual accessibility failures. When evaluating web pages using automated tools, having an understanding of accessibility standards and practices can minimize the risk in mistaking false-positive reports for actual accessibility errors.

Testing Automated Tools

With the various automated accessibility tools available, learning how an automated tool reports accessibility issues is important to recognizing actual errors and not chasing false-positive failures. The Accessible University project is a fictional university web page that contains numerous accessibility issues along with a separate web page with those same issue corrected. This offers a stable test page by which developers and page authors can try out different automated tools and evaluate which works best for their needs. Another test suite that has emerged is the Accessibility Tool Audit made available by the Government Digital Service Accessibility Team of GOV.UK.

Regardless of which accessibility tool is used, automated tools are a part of the accessibility evaluation and review process. Using automated accessibility tools can streamline evaluations and target where accessibility issues exist in a web page or website. Incorporating automated accessibility tests along with manual testing will result in more accurate accessibility evaluations and, ideally, more efficient remediation strategies for content and interface designs.

Additional Reading

Choosing an Automated Accessibility Testing Tool: 13 Questions you should ask (2013)
Accessibility Testing Tools – updated (2014)
My Post-CSUN Comparison of Web Accessibility Checkers (2016)
The Experts have Spoken! Our Favorite Accessibility Tools (2016)
What we found when we tested tools on the world’s least-accessible webpage (2017) *Note - results are not without some controversy.


Sean Keegan is Director of the California Community Colleges Accessibility Center


 

Add comment