"Local government and linked data" at Online Information - Part 1
"This is difficult because it is new, not because it is technically difficult." - Richard Wallis
As with the 'social media in the enterprise' session which I blogged about on Monday, one of the key learnings from the local government linked data session at Online Information last week was that many of the issues were structural and cultural, and not at all about implementing technology. Richard Wallis, Chris Taggart and Noel Hatch all featured in a 90 minute session with a focus on local government, which is now expected to be issuing a lot of spending data by central government.
Several times during the session reference was made to Tim Berners-Lee's 5 star rating for getting data published. Local authorities are being told to publish the data in any format just to get it out there, whilst being urged to aim for the most malleable and reusable ones like plain text and CSV files.
There was also some sympathy for local authorities from the panel. Data.gov.uk developed very quickly, but it was only with the arrival of the coalition government in May that the edict that all local authorities should release all spending data over £500 was issued. That would have come as quite a shock to most of them. When we criticise those that simply dump PDFs on the web, we should perhaps remember that they haven't had a huge amount of time to prepare the cultural change around releasing data, which is significantly more of a challenge than the technical one.
Nevertheless, the technical challenge is still there. I've seen a comment thread where an elected councillor was genuinely trying to find out what was so bad about PDFs, and one anecdote from the stage at Online Information mentioned a local authority sending data "in CSV format" by simply changing the names of the files from ending in .xls to .csv. But if you've not been in the habit of manipulating data on computers outside of the Microsoft Office or Adobe Acrobat environment, why would you know the best formats for open re-use?
The plan also raises some interesting moral issues. As Chris Taggart pointed out, precedent from FOI requests suggests that individual consultants should not be anonymised when data is released. But what about payments to foster parents? Clearly, it is also a statutory duty of councils to protect the vulnerable, which must over-ride the immediate transparency considerations.
Richard Wallis made the good point that interoperable open linked data is beneficial not just to programmers wanting to make services and mash-ups, but makes data visible between government departments. It reminded me of the fact that when at the Guardian we publish our 'bubble diagram' of total government spend, we always get requests from government departments for a copy, as it is the only way that they can get an overview of the figures.
The question remains though, when does this new wealth of information tip from allowing a few programmers to make pretty graphs, to actually informing local democratic choice?
Talis are hoping they can help get us there faster by offering local authorities free hosting on the Talis triple-store platform to get their data the next step along - into a proper semantic linked format.
They are also building prototypes to demonstrate the value. One, bis.clients.talis.com, took only three weeks to put together, despite there being huge problems with merging the datasets from 3 separate research councils who all compiled their statistics in different ways. "It's amazing how knowing that the Prime Minister is going to make an announcement about your work focuses timescales on a project" observed Wallis wryly.
Next...
In my next post, I'll have some more of my notes from this Online Information session looking a local government and linked data, particularly the initiatives in Kent presented by Noel Hatch.
"Richard Wallis made the good point that interoperable open linked data is beneficial not just to programmers wanting to make services and mash-ups, but makes data visible between government departments. It reminded me of the fact that when at the Guardian we publish our 'bubble diagram' of total government spend, we always get requests from government departments for a copy, as it is the only way that they can get an overview of the figures."
That is just plain SAD! I'd like to believe that at the very least people in the government were able to see what is being spent where.
I'm a little torn on this issue. One one side I am all for transparency in local and federal government information. For instance, about every few months the local papers will publish a story about how someone had to go to crazy lengths to get information that "should" be freely available through the Freedom of Information Act. Apparently you can only get the info if you can wait out the intentional/non-intentional holding actions.
I have trouble with the "line". Where do we draw the line at what should be public knowledge and what should not. How much privacy are we willing to give up for the greater good?