Assessing Accessibility Part 2: ThePickards Audit

Tuesday, March 20, 2007 20:44 | Filed in Accessibility, Articles, Disability, Standards, The Pickards

After offering my own thoughts on the Better Connected 2007 thing, where I mentioned some of the problems in trying to assess accessibility, particularly when it comes to measuring it across different websites. I’ve since volunteered to give someone else’s site a good going-over and do an accessibility audit for them — and provide them with a score.

I am very much aware this gives people the opportunity to shoot me down in flames. Fair enough. I shouldn’t be in the kitchen if I can’t take the heat, kind of thing. However, I thought it might help if I’m open about what I’m doing, it’s limitations and so on first

Firstly, I’m not a professional accessibility auditing company. Secondly, I don’t have a disability. Thirdly, I don’t have access to the complete spectrum of assistive technologies. Finally, I don’t have access to a wide variety of test users. It’s just little old me.

Nonetheless, I’m confident that I can still do a pretty darn good job of auditing the accessibility of someone’s website. Oh, I’m not saying I’m in the same league as the Shaw Trust with their pan-disability checking, or in the same league as the RNIB with their See it Right audits, or even that I can compare to that excellent new body the Usability Exchange which connects website owners directly to disabled users for feedback. Now there’s an idea, huh?

So if I’m not this, and I’m not that, and I’m not the other, how come I think I can cut the mustard when it comes to accessibility auditing/testing?

Precisely the same way you get to Carnegie Hall. Practice.

I’ve been evaluating the accessibility and semantic construction of websites for some time now. I’m “the standards guy” at work (not to be confused with that standards guy) and have been for some time. I’ve one of the members of Accessites, and I’ve evaluated the accessibility of a good number of sites for that. I’ve also been a member of AccessifyForum for some time, where I’ve not only offered accessibility-related advice to others, but I’ve listened to what other people have had to say and have learned a lot. I’ve even tried to contribute to accessibility standards (but we’ll not talk about that).

Accessibility is something I understand, and accessibility testing is something I do, and I do well. That’s not being arrogant about it — many other people might be better — but I do have significant experience in the area.

So that’s my justification for why I feel I can do this. I understand the concepts, and I’ve had plenty of experience at applying ‘em. Hope everyone is happy with that.

What am I looking for?

I’ve already publicly — and frequently — made the point that WCAG compliance does not equate to “accessibility”, and that scoring and measuring sites according to some kind of accessibility metric is somewhere between difficult and pointless. So, purely in order to spite myself, I’m therefore going to score my accessibility auditing along some kind of metric.

Hah! That’ll teach me.

So what am I actually looking for?

Firstly, I’ll explain that I am only going to look at ten different pages of the site. From ten pages, I’m fairly confident — assuming that they are specifically chosen, and chosen carefully — that I can get a representative sample. I’m doing this because I’m going to look through the pages manually, and given that my testers compose of me and my two cats (and allowing them anywhere near anything called a “mouse” is not a good idea), I simply don’t have the resources to look at more pages than that.

WCAG Compliance

The first part of the actual audit will be to evaluate the sites against WCAG 1.0. Yes, I am aware that I have said WCAG isn’t great for measuring accessibility like this, but it is so far the only internationally recognised standard we have, and it is the one that people will normally reference when they are claiming that a site is inaccessible.

So a necessary part of the audit is to compare the site against WCAG 1.0 to see how it stands up. I’ll also offer my comments on the various checks to indicate how important I think they are, but — and this is important — the WCAG rating I give to the site will not reflect my personal weighting. It will simply reflect the WCAG conformance level that I believe the site has achieved. I wouldn’t be surprised to see a number of fairly accessible sites not even making it to single-A conformance, to be honest.

So that’s part 1 — the WCAG conformance level and breakdown of failures.

ThePickards Criteria

Part two of the audit will look more at what I believe to be important. In many ways this can be considered to be an edited version of the Accessites Criteria, but with some significant changes.

I’m then going to break these down into what I’m terming standard and important criteria. Fairly self-explanatory so far?

Important Criteria
1. Text must be resizable in modern browsers
2. Site renders logically and is usable without CSS, images or both being supported
3. Site is navigable and operable using just the keyboard
4. Links should be obvious and easy to use
5. All information available visually must be provided in the content (includes use of images and colour to convey meaning)
6. Language used is appropriate to the site’s content
7. Luminosity contrast ratio of foreground and background of at least 5:1 where information is being conveyed, or the luminosity can be altered to such a ratio by means of site customisations
8. Labels are explicitly associated with their controls
9. Site ‘works’ in the latest version on multiple modern browsers
10. New windows are not opened without somehow warning the user in advance
11. If data tables are used, the row or column headers are associated with the appropriate cells
12. Site ‘works’ whether or not javascript is enabled
Standard Criteria
1. The pages on the site validate
2. Deprecated elements and attributes are not used
3. The CSS used on the site validates
4. Tables are not used for layout
5. Links and form controls offer hover styling
6. Links and form controls offer active/focus styling for keyboard users
7. The main language used on the page is identified
8. Significant passages in alternate languages are marked up appropriately. Words or phrases commonly in use such as c’est la vie do not need to be, and neither do real names
9. Lists are used sensibly and where appropriate
10. Headers are used sensibly and where appropriate
11. Text is not right-justified
12. At least one of a sitemap, a search facility or a table of contents is provided where the site contains more than 20 pages
13. The first occurrence of an acronym or abbreviation on a given page is expanded by some means, including use of the acronym and abbr elements, expanding within the text, linking to a definition, or providing a glossary with an appropriate entry.

You’ll notice that there are a mixture of things in here. Some relate to usability; some relate to accessibility; some relate to how the site operates with different equipment and software (some people would call this ‘universality’, others make it a part of accessibility).

These criteria are what I judge to be important. If I’ve missed anything out, or you can think of some reason why I’ve prioritised something incorrectly, let me know. I’m certainly open to suggestion. On the other hand, this ain’t a democracy. It’s about how I am going to evaluate sites. I’ll certainly open to suggestions, but I reserve the right to ignore the ones where I’m not swayed by the awesome power of your arguments.

In some cases it will obviously be subjective, too. Do I feel lists have been used properly? Well what if I feel that on the whole, list markup is pretty goog but there are one or two things which maybe should have been marked up as a list but weren’t?

Well under WCAG, it’s a binary state. It either passes — you do everything right; or you make one or more mistakes anywhere and it fails. Under my marking scheme however, I am allocating points to the different critiera — up to 5 points for something I consider to be important, and up to 2 points for something I consider a “standard” criterion.

I’ll then tot these up to give an overall figure and work out the percentage of the maximum that was achieved. I’m going to express this as a percentage rather than a score, because this gives me a better comparison over time if I later revise the criteria I use.

And then we come to the third and final leg of the audit…

Simulated User Testing

As I’ve mentioned before, the user pool I have to select from is …um, me, so that does set some limits as to how useful this simulated user testing will be. Nevertheless, I’ll do my best. I plan to have a look at the site “as myself”, and decide on five realistic tasks that I’d like to complete using the site (including just finding out particular pieces of information).

I’ll then assess, on an admittedly subjective scale, how easy it was to complete each of those tasks — as well as detailing any problems I encounter — when using various tools to simulate accessing the site as though I were:

  • Myself
  • A user requiring the use of screen magnification tools
  • A user requiring the use of a screen reader
  • A user unable to use the mouse
  • A user with colour blindness (looking particularly at Deuteranopia, as the Deuteranopia/Deuteranomaly types are the most common forms)

This will obviously not cover all of the possible situations — I’ve missed out the whole category of hearing problems and of cognitive problems, for a start.

Hearing problems I have deliberately missed: I don’t find that many websites rely on auditory information. Having said that, if I do test a website where I find auditory information to be a requirement, I will note this down and comment on it appropriately in the report, as that would be something pretty major.

Cognitive problems — ranging from mild dyslexia through to severe cognitive disability — are much more difficult to mimic and so I won’t even try. I have incorporated criteria in the second stage that relate to dyslexia and will help with milder cognitive disabilities, and so this will already be picked up in the scoring to some extent.

Nonetheless, I am aware that this is maybe a weak spot within my audit procedure and is somewhere where genuine user testing with disabled users would be of benefit. Not that that’s an option for me, being the one-man band with no resources that I am, but it’s something I could recommend to anyone I do an audit for…

And that’s it.. That’s what I’m planning to do. Does anyone have any constructive advice, comments or suggestions?

You can leave a response, or trackback from your own site.

29 Comments to Assessing Accessibility Part 2: ThePickards Audit

  1. Rosie Sherry says:

    April 16th, 2007 at 11:29 pm

    One obvious thing missing under user testing is simulating so called ‘normal people’, or intended audience of the website. An argument from your side is that ‘myself’ was in reference to a normal person, if you come under this ‘category’ :)

    Compatibility is a big point that you have covered a bit by saying using the latest browser versions. In addition to this I would say that it would be worth trying to source stats (if available) for the website. For example, if it’s an internal site for employees who only use IE, there would be little justification for you to test on other browsers.

    Other issues to consider are general software testing issues. Does the basic functionality work as expected? on different browsers? Can users actually perform the tasks? Does the site perform well? Are there any broken links, and do they link to the right place?

    I’m not quite sure what you mean about right justified text, I’m presuming you mean it would be difficult to read if it was right justified? I only point this out because your ‘What am I looking for?’ subheading is right justified :)

    My background is in software testing, so when I test a website I include accessibility, usability and software testing practices/knowledge. I believe applying all three aspects gives the best chance of delivering a quality website. They all overlap anyways :)

    Overall, it’s a pretty comprehensive list. I wouldn’t put yourself down about saying you don’t compare to other services/companies. I bet you know alot more than most people do. I do accessibility testing, but don’t necessarily involve disabled users all the time. The truth is that it is usually down to budget, most of the time there isn’t the budget for testing, let alone going for the gold award.

    Oh, and if it were down to me I wouldn’t use a points system. It’s a bit like a badge on a website, it’s just looking for trouble!

  2. says:

    August 31st, 2011 at 3:06 am

    How To Change Your Business…

    [...]It’s a known truth that right skill is very useful when doing something new and especially if it is something very important..[...]…

  3. garment news daily says:

    September 9th, 2011 at 2:57 pm

    Websites we think you should visit…

    [...]although websites we backlink to below are considerably not related to ours, we feel they are actually worth a go through, so have a look[...]……

  4. phentermine says:

    September 22nd, 2011 at 6:04 pm

    Blogging About No Prescription Drugs…

    With many skills you can be good at at many more jobs and make less mistakes while doing it….

  5. hypnotist says:

    September 25th, 2011 at 10:01 am

    What Is Hypnosis…

    [...]When you have knowledge, skills and experience these are are crucial to make you happy in every area of life…[...]…

  6. cashu says:

    October 2nd, 2011 at 9:37 am

    Plug In Your Happiness…

    Thanks for writing this exact interesting article. This is really what i’ve been looking for. Keep up the fascinating content coming….

  7. oil painting photo says:

    October 3rd, 2011 at 2:53 pm

    Portraits of Famous People…

    …If you are conscious when working at your projects you will do more than if you don’t have much skills…..

  8. gafas de sol carrera says:

    October 22nd, 2011 at 2:18 pm

    Choose Quality Sunglasses Instead Of Cheap Ones…

    [...]We all know that skills come pretty handy when doing something new and even more it if is important to us.[...]…

  9. pursevalley says:

    October 25th, 2011 at 9:11 pm

    Business Purses Are More Important Than You Think…

    [...]We are absolutely sure that skills come pretty handy when having no experience with some kind of work and even more it if is important to us…[...]…

  10. plastic surgery internet marketing says:

    November 4th, 2011 at 3:31 am

    Where To Find Best Methods For Marketing…

    It’s a known truth that right knowledge can be very important when we are doing something new and even more it if is important to us….

  11. dining solutions direct says:

    November 8th, 2011 at 10:42 pm

    Eat Well Or How to Do It…

    [...]To have the right knowledge you can achieve at many more jobs and making no bad decisions..[...]…

  12. Marketing says:

    November 9th, 2011 at 8:16 pm

    You Business Is Stagnating?…

    …Having good skills you can successfully do at many more jobs while making less mistakes doing it.[...]…

  13. ipad battery charger says:

    November 24th, 2011 at 11:24 pm

    What Are Car Warig Lights…

    It’s a known truth that right skill is very useful when doing work for the first time and even more it if is important to us.[...]…

  14. Neutrik connectors says:

    December 2nd, 2011 at 11:36 am

    Disco Lamps…

    [...]To have many skills you can successfully do at a lot of jobs and doing almost no mistakes…[...]…

  15. tom ford sunglasses says:

    December 3rd, 2011 at 2:02 pm

    What Are Tom Ford Sunglasses…

    [...]To have good skills you can be good at many more things while making less mistakes doing it..[...]…

  16. west virginia mountaineers shirts says:

    December 7th, 2011 at 9:14 pm

    Book Stores Online…

    [...]This is truly cool. I got some interesting suggestions from this[...]…

  17. apartment classifieds says:

    December 11th, 2011 at 10:45 pm

    Best Info On Personal Development…

    [...]Writing about a subject like this can be rather challenging. It really is fantastic to understand some individuals nevertheless care about this[...]…

  18. billiga solglasögon says:

    December 15th, 2011 at 4:14 am

    New Website About Solglasögon…

    [...]I see you understand a great deal about what you’re writing about. Interesting read. I’ve just had a different concept pop into my mind[...]…

  19. news online says:

    December 21st, 2011 at 2:58 pm

    What Are The Best Funny Videos…

    [...]Awesome data that I’ve been seeking for some time! Hold up with the superior writing.[...]…

  20. lunettes tag heuer says:

    December 23rd, 2011 at 5:18 pm

    What Is Lunette Tag Heuer…

    [...]It’s great that individuals even now know quite a bit about thing like that. This definitely includes a large amount of people today agreeing with.[...]…

  21. Electronic Cigarettes says:

    December 28th, 2011 at 1:54 pm

    What Is Best Addiction Treatment You Should Consider…

    [...]This is truly cool. I’ve just had one more idea pop into my mind[...]…

  22. challenge coins says:

    January 2nd, 2012 at 6:12 pm

    There Are Many Hobbies You Can Enjoy…

    [...]Are you unquestionably positive this really is the suitable information. I head a thing else. You will find out extra about this topic[...]…

  23. Atlanta Roofing says:

    January 11th, 2012 at 1:04 am

    Get More Info About Atlanta Roofing…

    [...]Are you unquestionably certain that is the right facts. I head something else. I’ve just had a further concept pop into my mind[...]…

  24. women fragrance says:

    February 4th, 2012 at 1:14 am

    Perfumes For Women…

    …Having the right knowledge you can do quality work at many more things and make less mistakes while doing it……

  25. celular dual chip says:

    February 10th, 2012 at 4:18 pm

    Celular Items…

    [...]This is really cool. It is fantastic to understand a lot of people still care about this[...]…

  26. India travel says:

    February 16th, 2012 at 9:52 pm

    India Travel Online Reservations…

    [...]Writing about a subject like this can be really challenging. I’ll examine your site within the long term.[...]…

  27. spring 2012 shoes says:

    April 11th, 2012 at 6:01 pm

    Spring Has It’s Own Fashion Rules…

    [...]It’s wonderful that people nevertheless know a lot about thing like that. I’ve just had yet another strategy pop into my mind[...]…

  28. Affordable Health Insurance In Missouri says:

    June 25th, 2012 at 11:15 am

    Why to Insure Your Wehicle…

    [...]It’s great that folks nonetheless know a good deal about aspect like that. It’s very good to understand a variety of folks nonetheless care about this[...]…

  29. Affordable Health Insurance In Oklahoma says:

    June 27th, 2012 at 9:51 am

    Oklahoma Insurance Companies…

    [...]Writing about a subject like that’s definitely demanding. It genuinely is very good to understand some folks nevertheless care about this[...]…

Leave a comment