Whatever Paul Waugh thinks, The Guardian's MPs Expenses crowd-sourcing experiment was no "total failure"

I spotted a tweet earlier by the BBC's Rory Cellan-Jones suggesting that the Evening Standard's Paul Waugh had described The Guardian's crowd-sourced MPs expenses experiment as a "total failure".

It is an interesting comment. I've got a great respect for the way that Paul has embraced Twitter and blogging in his role at the Standard, and I'm almost certain he was suggesting it had been a failure from a purely editorial point of view.

Indeed, as my friend John helpfully pointed out, the users never got through the complete stack of documents.

And, as we know, thanks to the heavy redaction, we would never have uncovered the duck house and the other juicy details that The Telegraph were able to expertly lead the news agenda with for days on end, thanks to their unfettered access to the original documents.

But it is the word "total" that I would object too in Paul Waugh's statement.

I think there are several areas where you couldn't judge the project as anything other than a success.

Firstly, on a pure implementation level, it demonstrated that as a team our tech guys could, in the space of around a week, get an application deployed into the cloud but appear integrated into our site, using a technology stack that was not our regular infrastructure.

Secondly, it showed that as a business we could bring people together from editorial, design, technology and QA to deliver a rapid turnaround project in a multi-disciplinary way, based on a topical news story.

And thirdly, we learned from and improved upon it. When a second batch of documents were released, the team working on the app broke them down into much smaller assignments. That meant it was easier for a small contribution to push the totals along, and we didn't get bogged down with the inertia of visibly seeing that there was a lot of documents still to process.

It was a much smaller dataset, but the more playful and game-like interface of MPs Expenses II got 100% of the documents processed. And it served as the principle underpinning the way The Guardian investigated Tony Blair's finances.

There is a wider point here though.

Earlier today I was reading this post by Erick Schonfeld on TechCrunch. In it he wrote:

"[Marc] Andreessen asked me if TechCrunch is working on an iPad app or planning on putting up a paywall. I gave him a blank stare. He laughed and noted that none of the newer Web publications are either. 'All the new companies are not spending a nanosecond on the iPad or thinking of ways to charge for content. The older companies, that is all they are thinking about.'"

Andreessen went on to make the point that technology companies are used to having adapt around incessant technological change.

Now, we are constantly told that to survive in this new world, where we compete for the scarce attention of a fractured audience, that us old fashioned media companies have to innovate and experiment. Yet, within the media, we all have an almost Nelson Muntz-like compulsion to stand and point at any of our rivals if we perceive them to have made the slightest falter.

For me, trying new things with journalism and technology, working out why and where they fail, and making them better, is the only possible way forward. As an industry, we will never learn to innovate with technology if we are not prepared to fail in public.

Apologies in advance - comments are currently disabled across the whole of this site, as I'm going on holiday and can't bear the thought of coming back and having to moderate the spam. If you would like to comment on this post, please drop me a mail to martin.belam@currybet.net and I'll make sure it appears. And a reminder, the views expressed on currybetdotnet are my own, and do not reflect the views of Guardian News and Media Limited, or any current or former employers or clients.