Day #2 at Guardian Hack Day #2

Last week The Guardian held a second internal hack day. I gave a lightning talk about "Graceful Hacks", and yesterday I posted some of my thoughts on how the first part of the day panned out. Hacking ran from midday on Thursday until midday on Friday, and the presentations were held on the Friday afternoon.

In total there were around 45 hacks, which was a phenomenal amount given the size of the teams, the time available, and the amount of beer and Rock Band that were available as distractions on the Thursday night. Guest hackers included people from Google, the BBC, Timetric, Omniture and Tinker It.

A few themes emerged out of the hacks, and here are some of my favourites.

It is getting more real-time all the time

Several of the hacks focussed on exposing data in real or near-real time. A 'least read' component exposed the day's hidden treasures on The Guardian site, or things it had been a complete waste of time producing, depending on your viewpoint. By contrast, a couple of non-technical hacks put together dashboards using Yahoo! Pipes and iGoogle of the 'most read' stories across a range of news sites and topics, and looked to provide alerts for when stories were getting a big push on social media.

The Nasrat-Tackers team produced a hack which used RabbitMQ to pipe lots of information onto the screen in real-time from The Guardian's logs. Whilst the standard data-warehouse approach of a heavy-duty web analytics package is essential for a newspaper to produce audited ABCe figures, this kind of real-time alert system has real potential to shape the editorial response to activity on the site. Incidentally, I love the @tackers Twitter bio - "Software developer. Non conference goer: I do real work."

'The AAARG' was perhaps a sign of performance-driven journalism to come. It lit up when a visitor landed on a page on the site, informed an author if someone had pressed the 'Yahoo! Buzz' button on their story, and then told them when the visitor departed.

It spawned apocalyptic visions of a cyber-newsroom full of journalists with scoreboards above their desks indicating how many people were currently reading their articles, with roaming data-driven robots prodding them with stun guns if their stories weren't popular enough. Or maybe that was just me thinking that...

Commercial opportunities

With a constant litany of how news is failing to pay for itself ringing in our ears, quite a few eyes in the room were on the commercial possibilities of any of the hacks to emerge. There was a new interface for the Guardian's Soulmates dating service on offer, which, after an initial set of 4 questions, started to resemble 'Am I Hot or Not?'. It looked like quite a compelling interactive proposition, if a little cruel.

Another hack used the Yahoo! Term Extraction API to match affiliate links up to Guardian content. There was a nagging thought in the back of my brain that a zillion splogs and SEO-driven sites perfected this art online years ago, and they didn't even bother with the nuisance of having to write original content. If only we simply stuck to news about v|ag3a, replica watches and real estate we'd be laughing, and the future of spammy affiliate journalism would be assured.

Things that move

Getting things to move is always extremely popular on Hack Days. One hack, inspired by David Cameron's 'too many twits might make a twat' quip, involved a moving chart of which political party was being tweeted about the most. Not only was the hack impressive, but the fact they got it all hooked up to demo was even more impressive. At one point when they were called to the stage, the message went round that "They'll be a while. They are currently on fire upstairs".

Ken Lim, meanwhile, wowed the crowd with a small robot that showed joy or distress depending on whether Ken's Twitter friends were tweeting about success or failure. And you could tickle the robot's belly.

The robot is now a fully fledged user of Twitter - @guardianrobot - and will respond to people who tweet #ineedahug or #highfive. But only if someone in the Guardian office actually gives the corresponding high five or hug to the robot.

Data, data, data, data, data, data

As the News Innovation unconference showed, whilst geeks have an almost insatiable thirst for data, the editorial purpose is not always so clear. That wasn't the case on Hack Day, where several data driven projects had clear editorial agendas behind them. A couple of hacks visualised data around the carbon economy, looking at the carbon efficiency of individual countries, or plotting changes in carbon emissions globally over time.

Other visualisations included the geographical spread of stories The Guardian was covering, changes in the level of user registration sign-up on the site over time, and David McCandless demonstrated a graph showing progressive media panic in scares from 'Y2K', 'avian flu' up to 'swine flu'.

The wisdom of crowds

If I take one thing away from the second Guardian Hack Day though, it will be a useful reminder about the value of 'the wisdom of crowds'. Having sat in a room where nearly everybody was chanting for @mibgames to beat-box again, I'm now not sure that all crowds are equally wise...