What open Government data gives us with one hand, closed state data takes away with another

The oddest thing to me about the release of Government department spending data last week is the way it 'feels'. It feels like it is being published not as 'a good thing to do' in itself, but as a corrective measure. As if, instead of treating the civil service as the instrument of government, the Coalition is treating them like 'the enemy within', who cannot be trusted with money unless the threat of public exposure hangs over them.

In an interesting blog post earlier this week, Simon Briscoe pointed out some of the obvious limitations with the data as it is currently being published. There is little context, some obvious duplication, and no way of knowing whether the sums involved cover projects or services of short, medium or long-term duration. And until we have month after month after month of comparable data, we cannot draw any firm trends from the numbers.

There was another issue I saw, which goes to the heart of building a linked data ecosphere on the web.

Using the Guardian's data explorer tool, you can get a comprehensive list of suppliers. Wouldn't it be wonderful if you could instantly cross-reference that with the records at Companies House?

I'd love to be able to get an instant snapshot of how many of these companies are large, medium or small enterprises. Over time you could use that to measure whether the intention to open up Government service tendering to wider competition was on track or not.

It would also be interesting to compare the sums being paid to a company's declared accounts. What sort of proportion of their turnover derives from state spending? That would help identify companies who were almost solely dependent on state subsidy for their existence.

So how would you do that with linked data?

Well, ideally you'd want the spending data dump to feature a company's registered number. Or at least a unique supplier number that could be mapped to a Companies House record.

But, here comes the barrier.

All of this data is available via WebCHeck online from Companies House, but it is behind a paywall. It is £1 to get a general record, and an additional £1 to get different types of detailed records like accounts for each company. So what open Government data gives us with one hand, closed state data takes away with another. You can do all that cross-referencing, but it is a job for some researchers with access to a budget, rather than a dataset that could be compiled pretty much automatically.

But let us not be too negative.

Whatever the ideology behind the release of the spending data, and for all the faults that data so far has, this Government does seem to be committed to the open release of data in a way that no other has been before, and they should be applauded for it.