The Big Red Button - Breaking news on the bbc.co.uk homepage

The events of the last 24 hours have proved to me that even if computers don't possess a malevolent sense of humour, they certainly possess an acutely ironic sense of timing.

Last night, during the coverage of the presidential election in the US, we intended to try out a new bit of technology on the bbc.co.uk homepage, to replace the promotional area of the homepage with content sucked directly from the BBC News feeds.

As I've written before, for example after the tragic events at Beslan unfolded, there are times when major breaking news stories are happening around the globe, and it seems more appropriate for the BBC homepage to be carrying the latest news, rather than the regular feature driven content.

This has been a potential problem for us for some time - the production system behind the homepage is designed for the publication of advanced scheduled material, not the kind of rapid response these kind of events require. In fact it was this kind of difficulty with the system that lead to me and evilcoffee being stuck in Bush House at 9pm at night, hacking the homepage to keep the coverage up-to-date when the shock-and-awe campaign was launched against Baghdad.

A little while ago the Core Web Products team began to have our minds focussed on how we could finally solve this long-standing issue, and thus was born "The Big Red Button" - an admin interface that allows either the editor of the homepage or the duty editor of BBC News Online to put the bbc.co.uk homepage into "Breaking News" mode, by literally pressing a big on-screen red button.

However, last night a vital component of our content production infrastructure decided to inexcusably break. This naturally lead not to a smooth try-out of the new technology, but to frantic calls to engineers in the early hours of the morning, and an intensive effort to patch up the homepage until the normal back-end service could be restored.

Still, by virtue of the fact that this afternoon there was an admission from John Kerry that the election was lost, we were able to finally use The Big Red Button at around 5pm.

Most importantly for me on a personal note, it vindicated the fact that whilst thinking aloud in a meeting at Television Centre with the Editor of News Online little more than a month ago I inadvertently blurted out that we would deliver a technical solution for this within five weeks, and therefore in time for the election. The people around me have really done me proud today, both in coping with the fall-out of a major system failure, and in making my ill-advised assertion a reality

The day didn't pass without gallows humour though. This morning on the bus my wife asked me if I had any plans for this evening. Having had very little sleep I replied: "Yes. I intend to go to where we house the BBC servers and kick every single machine. And when I'm done kicking them, I shall go back to the start and kick each one again for good measure".

I hate computers.