#dendatameet notes - Olga Pierce and Jeff Larson of ProPublica

Last week I attended #dendatameet in Manchester, and I've already blogged my notes from talks by Oscar Westlund, Julian Tait and Paul Bradshaw. Laura Oliver has a good round-up of the day over at journalism.co.uk. Here is my last set of notes from the day - a session over video link featuring Olga Pierce and Jeff Larson of ProPublica.

Olga Pierce and Jeff Larson of ProPublica

Olga and Jeff gave a really engaging talk despite being slightly disembodied by appearing on the screen, with the occasional manic bit of echo whenever we attempted two-way conversation across the Atlantic. Probably the key phrase to emerge was the "walled garden", and why it was a bad idea.

Olga explained that usually, when she did a bit of journalism that involved facts and numbers, she'd end up with a huge database or spreadsheet. A fraction of that would make it into a story, and the rest would sit and gather digital dust on her hard-drive. Which is why she now turns it over to Jeff who finds ways to publish the data out to the community.

It was very obvious that at ProPublica they had a good culture of collaboration, and Olga said that often a journalist and a developer would sit down and discuss how to cover a story together. In order to get good interactive news applications that tell stories they argue that developers have to go to editorial meetings.

It is something we try to do at The Guardian - but with the tech team on the 3rd floor and the news room on the 2nd floor, it can be hard to get organised in the hurly-burly of day-to-day print production. I recently spent a morning sitting alongside our technical architect on the politics desk during the frantic live blogging of the election campaign. I think it was beneficial to both sides. I was left in awe of the speed with which content was produced, subbed and made live, but was also able to help formulate some immediate (and hopefully helpful) changes to our publishing tools, which were implemented before the campaign came to an end. That wouldn't have happened if we had stuck to our "walled gardens".

The other "walled garden" that Jeff and Olga discussed was the one between 'content' and 'interactive feature'. Jeff said a good news application for the user wasn't just one where they type in their state and get some numbers back, it was one where they type in their state and get a story back.

They explained that ProPublica always try to publish their raw data in ways that are easy for other people to consume, e.g. as CSV files, and that often it gets used in ways that they didn't imagine - for example comparing levels of unemployment insurance payouts across state lines when they had built an app to measure the fiscal responsibility of each state. They added that readers appreciate "a little honesty" around margins of error, and they often have a 'nerd page' where they can really detail the minutiae of what may be right or wrong with their calculations and the limitations of source data.

Another great tip from them was using Google surveys to find sources and leads for stories. If you just put up 'email your unemployment stories to this address' you end up with hundreds of impenetrable emails to wade through. If you make a short survey, you can very quickly collect data with more of a focus, and use the power of machines to analyse it and spot trends.

Next...

...erm...unusually, I don't know what is next on currybetdotnet. I am at Future of Web Design in London today and tomorrow, and then JEEcamp on Friday, and there are also interesting things afoot at Kings Place, and so I shall almost certainly be blogging about some aspects of those either here or on Inside Guardian at some point this week or next. But quite what, or when, I'm not yet sure...