Weblogs: Web Accessibility
Public Sector Forums drops 'misleading' SiteMorse League Tables
Wednesday, July 27, 2005SiteMorse's overly-aggressive PR campaign has clearly backfired again. The Public Sector Forums, a highly respected forum for government organisations has decided to stop publishing the SiteMorse league tables. This comes as a serious blow to SiteMorse, as this forum is known to be the source of many prospective local government customers for SiteMorse.
One of the main problems of SiteMorse's press release campaign was attacking organisations on the basis of an automated report (their own SiteMorse tool) without manually checking the results first. Their sales and marketing department should never have publicly attacked organisations without double checking their evidence. At least a peer review of the findings with a web accessibility expert.
PSF statement
In a post on their website, titled "Why we are discontinuing to run the SiteMorse league tables":
Subsequent to discussions both with the company itself [SiteMorse] and a number of accepted experts in the field we have concluded the accessibility element of the tables is extremely misleading, rendering the positions and rankings that make up the league as a whole, meaningless.
A snippet of the post quoted on accessifyforum:
After over two years of bringing you the loved and loathed tables we've decided to call time, basically because of the company's insistence on including the accessibility element which we now - after much deliberation - accept makes the whole exercise utterly meaningless. We therefore feel these are no longer of value to anyone other than SiteMorse itself, and certainly not to our users.
The mounting evidence
Two pieces of information severely damaged the credibility of SiteMorse's accessibility claims. The first was an excellent article by Grant Broome, titled "Automated testing - How useful is it?" published by GAWDS which highlighted how flawed automated testing is on a number of Priority 1 and 2 checkpoints.
The second point, conceeded by SiteMorse on the public sector forums and accessifyforum, was that their tool could only cover, at best, 40% of the checkpoints. Such a low level essentially made the end result meaningless when assessing overall levels of accessibility (especially since the accessibility score then only made up 20 percent of the SiteMorse score), and so it showed that the league tables themselves offered no practical, or useful indication as to the levels of web accessibility achieved.
As Jon Ribbens, Director of SiteMorse, stated on accessifyforum:
As far as I am aware all the league tables including a note to the effect that automated tested does not cover all or even most of the guidelines. A very important point that people seem to be missing also is that it is not an accessibility league table! Accessibility (as measured by automated testing, of course) is only one of the factors that is taken into consideration when ranking the sites.
SiteMorse setback
I'm not particularly overjoyed with the turn of events, but considering SiteMorse's belligerent response to criticism, it was hard to see a way forward to resolving this problem. In hindsight, its obvious that SiteMorse should have been more open, and more forthcoming with details about their product, or at least alleviate the negativity of an "us versus them" situation. In the final analysis, the rationale for dropping SiteMorse's league tables was correct - it is not a realistic measure of the accessibility of the websites concerned.
SiteMorse are especially proud of the inroads they have made into the public sector. They recently claimed to have over two-thirds of Local Government organisations as clients. So they will be especially sensitive to this largely self-inflicted wound.
I hope SiteMorse will reconsider their position, and consider working with the accessibility community. We are supportive of organisations and people genuinely interested in furthering web accessibility. So to are the WaSP Accessibility Task Force, led by the very capable and talented Andy Clarke.
Related Reading
[ Weblog | Categories and feeds | 2011 | 2010 | 2009 | 2008 | 2007 | 2006 | 2005 | 2004 | 2003 | 2002 ]