Blog Usability Study Doesn't Say Whether Blogs Are 'Useful'

Last week Catalyst Group Design published a report on a usability study of one particular blog - BusinessWeek's Well Spent.

The report makes a couple of useful points, particularly around the comment functionality of blogs.

- The core purpose of submitting comments to a blog is not universally understood.
- Few, if any, blogs declare exactly what will happen when a post is submitted - though some indicate after submission that there will be a review. Doubt about whether or not an obviously non-offensive comment will get posted could have a dampening effect on a core tenet of blogging: real-time reader/author dialogue.

I think the second point is particularly valid - expect to see changes in the currybetdotnet template soon to make more explicit how I deal with comments on this site.

They also make some comments around the confusion surrounding RSS. Their test subjects were internet-savvy, but not bloggers or conscious users of blogs. The whole subscribe/XML/RSS thing was way out of their scope

Participants were unanimous in declaring that the site was unclear in explaining the purpose, value and function of RSS. All also felt that the site was aimed at an audience other than them - specifically, technically advanced people or heavy bloggers.

- [Those links and terms] mean nothing to me…There's this whole language you have to be immersed in…I don't feel like it's accessible at all.

- I think it's a free-for-all. They didn't do a good job explaining it. They aren't doing a very good job if they are targeting me.

- [It's aimed at] a person that knows blogs inside and out. That is up on blogs, blogs, blogs. Like on eBay, with the...die-hard eBay'ers who know the ins and outs.

- I would not know at all [that I could do that with RSS]. I think it's targeted at someone who knows computer programming or website design. I wouldn't see a lot of people knowing about this. It looks like a technical thing.

One of the reasons for adding RSS icons for News and Sport prominently on the BBC homepage was to move these icons and terminology into something that was hopefully a safe and comforting space. I'm sure if we user-tested the page right now people would not understand what they meant or why they were there, but I hope that having them there makes them a more familiar part of the internet landscape for our very mainstream audience.

Another point made, that may eventually have a bearing on the way this site is organised, was that most bloggers have 'I-forgot-I-was-a-superuser' syndrome:

To anyone who spends a fair amount of time reading or writing blogs, certain features are very familiar: recent posts, a category structure that organizes older posts, and archives that contain all posts filed by date. While these appear to be fairly straightforward informational motifs, they mostly proved opaque to those tested, with the exception of archives.

I can think of lots of reasons when visiting a blog to check out the category links - I never myself think, oh, I wonder what such-and-such-a-blog posted on April 3rd. Hence there are no by date archives on currybetdotnet. But in the mainstream, date is a much more understood navigational aid than the concepts bloggers regularly rely on.

The report though has some encouraging pluses at the end, one of the test subjects thinks they might now add RSS to their Yahoo!

- I will use blogs more in the future. One, I will explore RSS to bring feeds into My Yahoo!. Also, I am just realizing there is much more mainstream information in blogs [than I had thought]. Or I would use them to get the 'off the record' ramblings of favourite writers.

There are still plenty of superuser issues though:

- The amount of help provided is very limited. There are clues, but I want to know I am in the blog section. Just like [in a newspaper] I want to know I am in the editorial section vs. the opinion section.

- It's not well explained at all. This is the thing with a lot of these Internet trends: people assume you have the knowledge already, because why otherwise would you be looking at blogs? That can be alienating to prospective users.

From a BBC point of view the desire to know where you are [editorial vs opinion] within the site is really important to get right. There is a tension at the moment about whether the BBC's blogs are bloggy enough (hence Paul Mason doing Newsnig8t on Typepad), or whether the audience understands the different vocal tone required (hence the General Election blog generating comments along the lines of 'is this really news?')

Overall the findings of the testing are useful, but I still have a nagging disquiet about the way the tests were carried out. They set out with a likely premise - using Well Spent because it:

represents a likely direction for "mainstream" blogging - in that it is part of a family of blogs integrated into to a well-recognized, branded web destination. As such it is a "business" blog, rather than someone's hobby and is clearly intended for broad readership.

and

it was possible to create a realistic scenario by which mainstream users would arrive at a blog post landing page on Well Spent - specifically, by searching for 401K information on Google.

However, in some ways they ended up testing the audiences expectation of the concept of a 'blog', rather than page itself.

When asked to characterize what they were looking at, almost every person tested called the post landing page an "article," with several specifying it was clearly from BusinessWeek.

So far so good, but the reaction of the users changed when the dreaded 'b' word was wheeled out:

- I would be annoyed if I was reading this and then later realized, oh, this is a freakin' blog. Wow. Ok.

- I would have to take your word for it. I am surprised if I am on a blog.

- I would not believe that [this is a blog].

There are some findings that are quite worrying for the 'blogosphere' as a whole - and especially for big brands dabbling with integrating the blog model into their web offering:

Over half the participants were unsure of the significance of the post being part of a blog - but almost all felt that a blog was somehow less credible as a source of information.

Nearly half those tested stated that because of this perceived difference in editorial standards, the blog should be more clearly separated from the main BusinessWeek site. Of particular concern was the fact that the authors' credentials were not readily visible.

That in the end emphasizes what is for me the most frustrating element of this study. In the end Catalyst Group Design didn't draw any conclusions in the report about whether the subjects found the information on the blog useful. They may have a couple of quotes saying that people might use blogs and RSS in the future, but what they didn't address was if the mainstream internet user hits a page on a blog following a search query, regardless of whether they subsequently understand the category, navigational hierarchy, or subscribe to the RSS, was that a good internet experience?

Using this blog as example, if someone is searching for ghost stories from Enfield, and via Google hits upon my Haunted Cockfosters and Enfield post, does it matter that they are on a blog? Or that I work for the BBC and write about that lot? Or is it just (hopefully) a useful page that tells them some ghost stories and where to find the haunted places, with pictures, that leaves them thinking the internet is great and that Google (not currybetdotnet) helps them find 'just what they want'?