When Oyster Card fails think "Thanks for the uptime"

In Friday's Evening Standard there was a small article by Local Government Correspondent Ross Lydall about the Oyster card:

"Oyster card users have been overcharged about £1 million because of delays in limiting fares, it was claimed today.

The revelation came as pressure grew for an inquiry after a computer crash caused the £50 million smartcard system to collapse.

Card readers across the 270 Tube stations and on 8,000 buses failed yesterday morning - taking five hours to repair and losing Transport for London thousands in revenue."

I noticed the failure on Thursday, as although now and again I have been on buses where individual card readers are not working, it was the first time I got waved through every ticket gate from start-to-finish on my journey to work.

I still thought it was quite odd to mix the two issues into one article though - after all one part of the story is about a system failing, and the other part is about the gradual roll-out of functionality.

Lynne Featherstone, Liberal Democrat member of the London Assembly has called for a "full and transparent investigation" into the crash, which seems like a fairly standard knee-jerk political reaction to public computer glitches. On any IT project there has to come a point where you weigh the cost against the benefit of resilience. If TfL lost "thousands" in revenue, how much more would it be spending to put complete redundancy into the entire Oyster network to save those "thousands"?

It struck a chord with me as there was a clear parallel with my work this week. In the last few days we suffered a couple of failures with our online voting application. It doesn't degrade particularly gracefully due to the standard way that the site returns error messages across all of the BBC's dynamic applications. Within seconds myself and my team had understandably irritated producers on the phone and on email wanting to know what was wrong and how soon it would be fixed.

The application has been really stable with only three significant downtime incidents in the two years I have been producer on it. It occurred to me that none of those producers had thought to phone during the good times to thank us for keeping the service running, they just take it as a given that it is going to work. More importantly, I realised I was just as guilty. During that time it had equally never occurred to me to make sure I regularly thank the tech team who have been keeping it running smoothly all that time.

I guess it is a truism in life that you only notice IT when it doesn't perform. Certainly large public IT projects only make the news when they go wrong. Even if the Oyster card network is only down across 270 stations for that five hour period in an entire calender year, you'll never see a news story with the headline "Well done TfL for 2,365,195 hours uptime"