How accessible are Britain's online newspapers? Part 10 - Methodology

Over the last weeks I have been looking at the websites of the major UK newspapers, and assessing them for their accessibility features. I've worked out a way of scoring them on these features, and so tomorrow I'll be publishing a feature comparison chart, and a league table of which papers performed better in the testing.

(Of course, everybody loves publishing league tables of someone else's performance, they just don't like to themselves be rated that way - just ask MPs how they feel about the rankings on TheyWorkForYou.com.)

Before publishing the table though, I thought I should explain a bit more about the methodology behind the testing and scoring.

For each newspaper I looked at the homepage, and then the article page of the most prominent story on that page at the time. This is, of course, by no means a comprehensive test, and clearly on web properties like The Telegraph or The Guardian where there are considerable variations in page design across the piece, you'd want a larger sample size of page to test.

Text resize

In theory, the internet ought to be able to deliver textual content to absolutely any device using any kind of visual, auditory or sensual display. That is actually one of the founding principle of the thing. In practice, people still tend to specify that this particular piece of text should be rendered in light blue at 12px high using the Verdana font, and nothing else.

At the very least, users like my dad who is still keen to use the internet, but whose eyesight isn't as good as it once was, ought to be able to easily make text appear larger in their browser to help them use the net.

Having controls to adjust the size of the fonts included in the page itself earned papers two point - provided it gave sufficient contrast, or was on every page. In the end I award The Guardian one point for graudally rolling this functionality out, and The Independent half-a-point for including one, even though I felt the actual contrast offered in the font sizes was negligible.

Each paper also earned two points if the homepage and an article page rendered correctly when the font size was increased when browsing with Firefox, and it didn't turn into a messy soup like The Daily Star.

I also tested the papers in Internet Explorer 6 (Internet Explorer 7's zoom function makes this problem easier to get around, by being able to zoom whole pages regardless of how they have been put together). If the site was still laid out correctly in IE6 at the largest text setting then a newspaper earned another two points.

All together there were 6 points available in this section.

Alt images

Next I browsed the homepage and an article page with images switched off in my browser. This allowed me to establish:

• whether the site had alternative text for images
• whether that alternative text was semantic and explained what it was replacing, rather than simply saying 'image' or 'icon' or other unhelpful text.
• whether the navigation of the site still work with images turned off
• whether the functionality of the site still worked with images turned off

To test the latter I would try and do something interactive on the site like leave a comment, and see whether the lack of image rendering was an issue when trying to carry out this task. Most sites passed this test, but notably The Sun and the Daily Mail used CAPTCHA images with no alternative method of verifying that you were a human, excluding users with less than 20/20 vision form interacting with their sites.

There were two points available to each paper for the presence of alt images, with another point available for newspapers whose alt text was descriptive or semantic. Newspapers earned two points if lack of image rendering on the homepage didn't impact navigation, and two points if I didn't discover any impact on functionality.

There has been some discussion in the comments on some of my reviews as to whether newspapers should have alt text for all images, or whether they should concentrate on providing alternative text for functional graphics, and ignore alt text for 'decorative' pictures or stock photography. My own take on this is that alt text should be used to, as much as possible, replicate the user experience for a visiting browsing without images, and should also support a wide set of contexts.

JavaScript

As well as not handling images, many assistive technologies struggle with functionality that relies on JavaScript. I therefore re-browsed the homepage and an article page of a site trying to use the functionality like search, navigation drop-downs or comment submission with JavaScript switched off.

At the BBC I used to work on the rule-of-thumb that about 10% of users would not have JavaScript enabled either for accessibility, corporate IT policy, or pure paranoia reasons.

Sites got two point if I couldn't find any navigational problems and two if I couldn't find any functional problems. Only The Daily Mail and The Times scored four points on this test.

The majority of newspapers had at least one failing in their navigation or their functionality when JavaScript was turned off.

HTML navigation aids

I inspected the HTML of the newspaper pages to spot a couple of accessibility features that might not be made apparent to all users. Newspapers gained two points for including some kind of 'skip navigation' or 'go to main content' link which enables faster browsing using a screen reader, and also got a point for specifying accesskey attributes on some of their links.

Screen-reader simulation

For the final part of the testing I viewed the pages through a simulation of how they would be processed by a screen-reader like JAWS.

For this test I used a Firefox add-on called FANGS. This converts a page to a text output of what JAWS would read out. I did it this way, rather than use JAWS, because I know from experience of watching people use it, that the way sighted and non-sighted people approach the software is very, very different, and I had little hope of getting up to the speed-of-use I seen demonstrated.

I marked newspapers on how quickly a user of JAWS would get to the content on the page. For the homepage I counted this as the top headline, and for an article page I counted this as the start of the headline of the main article.

Newspapers started with 2.5 points, and lost half-a-point for every 250 words that JAWS would have to read to the user before they could reach the content.

The lowest numbers were registered by The Times, The Mirror and The Telegraph. I tried to balance the scoring here, so that papers which had a high word count before reaching the contents of a story but which also included 'skip navigation' links were not unfairly penalised.

Colour contrast issues

One test that I tried and failed to carry out was a test on any colour-blindness issues there might be on a page. I was particularly looking to see if any graphical navigation elements were unusable for people with the different types of colour-blindness.

I used an online tool called Vischeck to test this, but the tool is in a beta test state, and was unable to cope with some of the complicated coding on the newspaper homepages. It meant it couldn't process a couple of them.

As I couldn't assess them all equally, I discounted these findings - although I didn't see any major problems on the newspapers that I did manage to test.

The results

As I mentioned, tomorrow I shall be publishing an accessibility feature comparison chart for the 9 newspapers I looked at, as well as a table of how they performed using my scoring method.