Putting a F__k Off Dalek on the BBC Homepage isn't big or clever - part 4
This is part 4 of a 4 part article - 1 2 3 4
Download a print version of this article
Conclusions
I think there are really two points that I'd like to draw out from this that are more generally informative than just a story about how the BBC once put a massive Dalek on the homepage.
The first is about how mainstream users deal with change. The Open Tech conference is all about innovation and change - about changing the way we do things on the web and on other devices, about changing the relationship between content providers like the BBC and developers, and about changing our interaction models.
By virtue of being there, or reading this article, you are in a small sub-section of the UK's internet super-users. It is very hard to keep that in the forefront of your mind. Change worries people. Change in a familiar setting can be very disruptive. People won't like change unless you bring them along with you.
One user mailed us to say
"where's my old BBC home page......why does this feverish need for change penetrate every damned area of everything....did anyone actually ask for change - or did someone decide we needed it?"
If any of the developments you see at Open Tech are to gain a mainstream audience, they will need to overcome the inertia against change in the mainstream internet audience.
Secondly, I'd like to point out how useful it has been to do this kind of testing on our live user-base. We got a much wider sample of views than we would have been able to afford otherwise - even the BBC's new media budget doesn't stretch to user-testing focus groups over 1,000 strong.
Nimbly testing in public isn't something that the BBC has historically been good at. We very rarely label anything a 'beta', and when we did, iCan was in a public beta stage for a couple of years before it finally blossomed into "Action Network" a few weeks back. Tim O'Reilly at EtCon talked about the advantages that services like Flickr have by being in a seemingly perpetual beta state, over the difficulties long drawn-out releases like Microsoft's Longhorn experience.
The key though isn't just to slap a beta gif underneath your logo and then hope people will be more forgiving of a service, but to genuinely attempt to learn from the feedback you get from the audience. And iterate, and iterate, and iterate again.
Since we did the Dalek page we have done two similar ones - one for the Africa Calling Live8 concert at the Eden Project in Cornwall, and one to remember the victims of the London bombings.
The Live8 page was up for three-and-a-half days, yet only generated about 50 negative comments (around 7% of the 650 the Doctor Who page received). Why? Because we learnt from the feedback and adjusted the design to eliminate the commonest complaints - the banner area was reduced, and with less space taken up by the image we could restore all of the content that we had removed, eliminating the need to provide a 'return to the normal layout' option.
So, although people are resistant to change in a familiar place, you can gain a great advantage by user-testing 'in the wild'. However, without that feedback loop in place though, and a willingness to adapt your beta again and again, you will not get the maximum value out of the test.
With thanks to the people who made the page possible - Janet, Gemma, Andrew, Robert and Nick, and especially to Alice who analysed the feedback.