"Designing for doubt" - Michael Blastland at London IA

“Public debate over stories involving numbers tends to be ‘We’re doomed! We’re saved! We’re doomed!’” - Michael Blastland at London IA

Whilst I was taking a break from actively blogging I was still taking notes at the events I was attending, so I thought this week I’d whizz through some of the things I went to in April and May. One of the speakers at April’s London IA event was Michael Blastland. It was the second opportunity I’ve had this year to see him speak, having attended a panel session he spoke at about reporting crime statistics. For London IA, his theme was “designing for doubt”.

Michael argues that politicians and journalists have generally made a very poor job of understanding and presenting numbers. Whilst we are all currently excited by the new possibilities with data that the network has enabled, he believes that the correct interpretation of numbers will still be the difference between making the world a better place or not.

He illustrated his talk with several case studies, showing how easy it was to manipulate numbers. One was the impact of an education programme on the rate of teenage pregnancies in the Orkney Islands. A selective graph seemed to show dramatic results, with the incidence of youth pregnancy slashed. A more detailed look at the numbers revealed the fundamental truth of Michael Blastland’s simple but common sense message:

“Numbers go up and down. And sometimes stay the same.”

Women are not, he pointed out, queuing up on the Orkneys to get pregnant at a nicely regular rate to please statisticians. With a low sample size there are always likely to be wide fluctuations in the numbers of pregnant teenagers from year to year.

To illustrate the principle, he got the audience to play a dice-throwing game. His point was that in circumstances where randomness and chance plays a roll - for example accident figures - we tend to take action in the areas where the numbers are high. A cursory understanding of “Play your cards right” should teach us to be aware that if you have some high points in a random selection of numbers, they are likely to be lower next time around through the law of probability alone. Regardless of whether you have installed speed cameras or hired road safety druids.

Whether it is sex education or road safety, Michael Blastland said that he sometimes feels a bit mean pointing out the flaws in the understanding of these numbers:

“You don’t want to put out anyone’s fire, they all believe they’ve found an uplifting story that is going to change the way we live”

And then there is the stubbornness of people themselves.

One of the questions after Michael’s talk wondered if part of the problem was that “people will cling to their belief in the face of incontrovertible facts”. Michael agreed, saying:

“They reject the data, they reject your authority. They don’t give a damn about the data.”

A lot of this stems, Blastland said, from a fundamental misconception of what we are working with. Data sounds hard - both in the sense of being hard to process, and being hard and fast. However, he argues that “the raw material is a lot softer than it looks”.

From a design point of view, he urged those in the room to experiment with the design paradigms that represent uncertainty - tricks like blurring and fogginess. He showed examples of GDP predictions with progressively wider margins of error as time went on, or graphs that included “error bars” or visible confidence measurements.

Michael makes a very compelling case that a lot of our policy decisions happen because people take action when numbers are peaking, and so are already likely to fall. It makes for quite a depressing message, and one that certainly causes you to think twice next time you hear a news report that starts “Government figures show that thing x has gone up/down by n%” with no accompanying information about sample sizes, error margins, or longer term trends.