He who casts the first stone
By Mark Pilgrim | October 24th, 2003 | Filed in Accessibility
Skip to comment formWatchFire, who bought out the Bobby accessibility checker last year, is back in the news with a new service called WebXACT, which rolls their old site checking service in with their new accessibility checker. The free online version lets you check single pages for quality, accessibility, and privacy issues; the pay version lets you check your entire site and has some additional features.
When I first visited WebXACT and entered the URL of my personal home page, it was not clear from the UI that it would be recursively retrieving pages based on the links in the entered page. Based on my server access logs, the WebXACT bot appears to lie about its identity (it claims to be MSIE 6), and it does not respect robots.txt while merrily recursing through my links. This means it fell into my spambot trap and was automatically banned, which confused it very much. It always appears to come from 65.61.165.199, which I unblocked long enough to get an actual report, but which I am re-blocking now until they fix their bot.
The quality report appears to be their old version of Watchfire, which checked for dead links, missing ALT text, missing @width and @height attributes on images, and a few other things. A useful service, to be sure, although simply coding to web standards will catch some of the same errors.
The accessibility report is more problematic. Accessibility checking has always been problematic, since there are some checks that simply can not be automaed. WebXACT appears to be a spruced-up version of Bobby, which was a fine tool for what it did. However, WebXACT appears to share Bobby’s main weakness, which is to say that it is still gives scary-sounding false positives on anything it can’t verify automatically, with no way for the end user to manually override and say “OK, I checked this one, let’s move on”. Maybe their pay service has this; I can’t tell.
The privacy report checks for the presence of a P3P privacy policy. Many people believe that the P3P model has a number of fundamental flaws, see for example this critique from 2000, and this somewhat more recent critique.
But the main problem with WebXACT is that it does not live up to the rules it purports to require from others. The WebXACT service itself is not even remotely accessible, since it requires Javascript to function. This fails WCAG checkpoint 6.3 (“Ensure that pages are usable when scripts, applets, or other programmatic objects are turned off or not supported”). This is a priority 1 checkpoint. If that sounds bad, you’re right. But it gets worse.
I turned off Javascript in Mozilla and the WebXACT home page displayed this warning:
WebXACT may not fully support your browser.
At this time WebXACT requires JavaScript support to operate.
I happened to have my previous report still open in another tab, but it was now next to useless because they use Javascript for all their navigation links within the report. Now that Javascript was turned off, I could not even switch between the tabs within my report, or view linked documentation pages explaining, for example, that I should ensure that pages are usable when scripts, applets, or other programmatic objects are turned off or not supported.
I visited the WebXACT home page in Lynx and did not see the Javascript warning, but when I tried to check a page it got as far as the progress bar page and never progressed.
When WatchFire fixes the accessibility problems in their accessibility checker, I will be happy to re-review it. Until then, I can’t recommend it to anyone. I can’t put my trust in a tool that doesn’t live up to its own rules.