This is the question I’ve been asking myself for some time. Note specific use of the word measurement. I believe accessibility assessments are useful; but I’m not convinced that trying to assign a rating to a site is at all helpful and indeed I think it can actually be harmful.

I’ve been looking at the accessibility of some LA sites with other PSWMG members as part of an accessibility supplement to the Better Connected 2008 report, so I need to be clear here. I’m not speaking on behalf of PSWMG, I’m not speaking on behalf of anyone other than myself here.

These are simply my thoughts and opinions, which may or may not be shared by others, and to be honest haven’t changed much over the last year or so. This post may have been somewhat inspired by this year’s SOCITM kerfuffle, and the fact that I feel as interim chair of the PSWMG, I should explain my personal viewpoint.

Again, to clarify, the combined viewpoint of those PSWMG members tasked with producing the accessibility details for SOCITM will be covered as part of SOCITM’s accessibility supplement, so I can’t cover them here, and I can’t claim to know the viewpoint of all the PSWMG members (like any organisation, I’m sure some people would disagree with others!), so this is just my take on things. Although reading the Better Connected discussion on PSF, it is clear many have a similar viewpoint to myself…

Are we sufficiently clear on that? Good. Then lets begin.

Firstly, the current kerfuffle.

It is with the utmost regret we find ourselves having to report, yet again, Socitm Insight – in collusion with RNIB — has delivered a massive betrayal, dis-service and kick in the teeth over website accessibility to council website managers and developers the length and breadth of the land.

…an ‘important amendment’ to the original statement claiming, unbelievably, that 92% of council websites are inaccessible. This, we maintain, is a nonsense, for reasons that will be made clear and (just as we said last year) is a matter which brings the entire Report into disrepute.

The RNIB ‘rankings’ give no true picture of the current state of local authority website accessibility and should therefore be ignored completely.

Public Sector Forums

Strong stuff, no?

So why such strong language, exactly?

I think mostly because this problem cropped up last year too. Last year, the methodology was similar — testing against WCAG 1.0 to ‘measure’ the accessibility of sites. Last year it was criticised:

BC’s assessment of the accessibility of local authority websites is fundamentally flawed. Admittedly this is a reflection of the use of the Web Content Accessibility Guidelines 1.0 as the instrument of measurement…

The solution, however unpalatable it might be to the bean counters who seem to have a desperate need to rank and score us all, is to abandon the concept of ranking 468 websites for accessibility, and to stop testing them against an 8 year-old set of guidelines.

…The current state of affairs is like asking the Michelin Guide to judge restaurants on the quality of their cutlery

Dan Champion on Blether

That was from 2007. So were my comments:

So maybe it’s time to stop “ranking” websites in terms of accessibility? Some webistes which failed to achieve level AA conformace might actually have been more accessible to users with disabilities than some of the ones that did achieve AA conformance.

Let’s stop saying that WCAG is a measure of accessibility. It isn’t. WCAG shows you things you can do that will likely make your site more accessible. But throw some real people at your site and it either will or it won’t be accessible to them, irrespective of what your WCAG conformance level says.

Remember: WCAG Conformance is not the same as accessibility.

Real people. Real tests. Real progress?

Me, on Assessing Accessibility Part 1:The SOCITM Story

So both Dan and myself (and others) have made our positions clear. WCAG Conformance is not the same as accessibility. We were quite clear last year that this was our stance.

Unfortunately, despite the fact that many people seemed to agree with us, no-one seems to have listened…

And the worry I have is that by continuing to grade sites against WCAG 1.0, people will (quite reasonably, but incorrectly) assume that a site which passes WCAG 1.0 at Level A will be more accessible to users than one which doesn’t, and that a site which passes WCAG 1.0 at level AA will be more accessible than one which doesn’t, and so on. This is nonsense.

All WCAG 1.0 conformance can tell you is to what extent the site complies with WCAG. Nothing more, and it certainly can’t give you an accurate rating of the ‘accessibility’ of a site.

Now I’ve not read Better Connected 2008. I’ve only been involved in the accessibility supplement to be released later. It may be that SOCITM do not imply WCAG Conformance equates to real-world accessibility. But unless they specifically state that it doesn’t, people will assume that because that’s what they are measuring, that it’s somehow worthwhile. And it isn’t.

Here’s why.

It is not possible to mark up changes in language in an alt text.

If you don’t mark up changes in language, you can’t reach the most basic WCAG 1.0 conformance level. Therefore if you have a picture which shows some text in more than one language (e.g. something with “Welcome” and “Bonjour!” on it), you cannot conform to even the most basic level of WCAG 1.0.

That doesn’t mean your site is inaccessible. In this case — because you only need one fail, anywhere on your site to stop you reaching a conformance level — all this would mean was that screen reader users would probably have Bonjour pronounced to them incorrectly. And if their screen reader didn’t come with a French language pack in the first place, it would be pronounced incorrectly anyway.

Does that mean :

…one or more groups will find it impossible to access information in the document.WCAG 1.0

Of course it doesn’t.

And, in case you are wondering whether there are any real world situations where this sort of thing occurs, yes. Many sites frequently use images of foreign text (Urdu, Gujarati, Cantonese etc) to provide assistance to communities who have that as a first language. If the alt text is not marked in the appropriate language, this is a WCAG 1.0 fail.

So the site has two choices:

1. offer a poorer service, by not offering information in multiple languages and comply with WCAG 1.0
2. fail WCAG 1.0 by offering images of multiple languages and have a site of more benefit to its users

The theoretical ideal is for the information to be presented as text in the appropriate language, but that isn’t always possible. You would require each site, each user of the site (irrespective of whether they’re accessing from home, internet cafe etc), each resident, each council PC to be installed with all of the language packs for all of these languages or it wouldn’t work. That’s just not practical. Using images is.

That is why a “measure” against WCAG 1.0 is plainly a nonsense. The site that provides the better service is seemingly penalised for doing so, and if SOCITM continue to take a an approach to accessibility based on guidelines (particularly out-dated guidelines) rather than real user experience, then they are doing their membership a disservice: they risk pushing down real-world accessibility at the expense of conforming to 9 year old guidelines.

The RNIB have done a good job of measuring how well sites conform to WCAG 1.0 for the Better Connected report. That’s what they were asked to do, and I don’t criticise them for it (I have a great deal of respect for their team). Unfortunately, that tells you nothing useful about the real-world accessibility of the sites in question. If the RNIB had not been asked to focus on WCAG 1.0 on ‘measure’ accessibility, I think they would have come out with something more useful.

I’d like to think that SOCITM will understand this and will look to improve the way they assess the accessibility of sites next year, and instead of using a measurement which (sometimes incorrectly) ranks sites against one another, I’d rather they took a user-centric approach, looking at common tasks and what problems people with different types of disabilities would face when carrying out these tasks. And I’d rather they didn’t use this as a “ranking” either, because just because a site does well in one area doesn’t mean it will necessarily do so in another. But then again, that’s what I was hoping for last year too…

But it just might be beneficial to list the most common accessibility problems encountered, the actual impact these would have on real-world users (for example, does the use of a deprecated attribute cause any problems for any users? If not, then why the hell fail a site for it as WCAG 1.0 would do?), and common ways which developers could resolve (or avoid) these problems.

That really could be used to help us all be “Better Connected”.