Tool independence for the web accessibility quantitative metric

Abstract

The Web Accessibility Quantitative Metric (WAQM) aims at accurately measuring the accessibility of web pages. One of the main features of WAQM among others is that it is evaluation tool independent for ranking and accessibility monitoring scenarios. This article proposes a method to attain evaluation tool independence for all foreseeable scenarios. After demonstrating that homepages have a more similar error profile than any other web page in a given web site, 15 homepages were measured with 10,000 different values of WAQM parameters using EvalAccess and LIFT, two automatic evaluation tools for accessibility. A similar procedure was followed with random pages and with several test files obtaining several tuples that minimise the difference between both tools. One thousand four hundred forty-nine web pages from 15 web sites were measured with these tuples and those values that minimised the difference between the tools were selected. Once the WAQM was tuned, the accessibility of 15 web sites was measured with two metrics for web sites, concluding that even if similar values can be produced, obtaining the same scores is undesirable since evaluation tools behave in a different way.

Keywords:

• Web accessibility
• measurement methods
• accessibility metrics

Notes

1. The Web Accessibility Initiative, WAI. Available at http://www.w3.org/WAI/

2. Available at http://labs.google.com/accessible/

3. Further explanations are not given in this regard because the way that the normalise incidence figure is calculated is quite confusing.

4. Available at http://www.wabcluster.org/

5. Available at http://www.w3.org/TR/2006/WD-WCAG20-20060427/appendixD.html

6. Using a PC with Pentium 4 3GHz processor and 1GB of RAM it took almost 22 hours to process 15 pages 10000 times.