Sniffing out bias and censorship on the BBC News site

A mail on the BBC's backstage.bbc.co.uk mailing list yesterday pointed me to News Sniffer. It is a site that monitors online news output to detect revision and bias. At the moment it has two main sections - "Watch Your Mouth" monitoring the BBC News message boards, and "Revisionista" monitoring edits to output on the BBC, Independent and Guardian website. I'm always intrigued by systems that mix automation with human input, and the News Sniffer also allows users to "recommend" revisions which display bias.

Striking a balance between one size fits all and tailoring the way the system parses the HTML is a tough one, but I think the Revisionista script could benefit from a couple of tweaks to more accurately reflect the semantics of the page being scraped.

One of the articles from the BBC News site that has had the most votes for bias is this one. The difference as it appears on News Sniffer is that in the middle of the article the phrase "Many Palestinians feel Mr Blair has ignored their suffering" has been deleted and the phrase "Mr Blair met with the families of captured Israeli soldiers" added.

Embedded in the text this looks like a clear attempt to move from a negative image of the British PM to a more positive and sympathetic to Israel image. However, if you look at the original article on the BBC News site, this isn't content from the article itself, as News Sniffer records it, but the captions for the photgraphs included in the article, which have changed. Going back through the versions you can see that the article always had one picture from each "side" of the conflict.

Likewise News Sniffer currently records as revisions when pull-quotes from the message boards are changed, or when the Have Your Say call to action is edited, or a link added. Whilst they are clearly changes to the published article, they are not in the same league as paragraph long revisions of text.

These do exist, as this article about Cuba clearly illustrates.

It is, of course, terribly hard for machines to rank the importance of a particular change, but I'd be keed to see the site experiment with some kind of "interestingness" algorithm about the edits. You can't say for certain that an edit of only two or three characters is insignificant - it will probably be a grammatical or typo correction, but clearly the addition of 'not' to a sentence would only be a small change in the character count but a big semantic change. I'd like to see News Sniffer flag up really big changes more though, where whole paragraphs have been added or taken away.

The idea of monitoring BBC News for changes is one that has been done before for the homepage alone as one of the backstage.bbc.co.uk prototypes - "BBC News front page archive" by Matthew Somerville - and I know one member of BBC staff who built a similar system for inside the BBC that showed up differences in articles as they were republished.

On the News Sniffer blog I saw that the person behind the site had recently contacted Biased BBC about it. I loved some of the comments it generated, where initial glee quickly turned to mistrust as they realised the person behind News Sniffer was "left leaning" and generally a "not-we". One commenter suggested that the News Sniffer site would turn out to be biased itself. It took me a few minutes to get my head around the concept of being instinctively biased against a site that sets out to detect bias in a news source you already believe to be biased :-)

I found the "Watch Your Mouth" section very interesting indeed. It attempts to capture all the comments that have been moderated off the BBC News "Have Your Say" section.

The "Watch Your Mouth" section has a chart of the most censored threads. I'd prefer to see this expressed as a percentage as well as just the raw numbers of comments, as you'd expect busier threads to attract more moderation. Some of the posts have clearly been pulled because they are low quality or break the house rules about flaming people - like this one:

How large is your ecological footprint?

>>>>My footprint is huge and I like it that way. How dare anyone try to control what I consume.
Angie, USA


>>Typical american attitude. Selfish, arrogant and greedy.
ali denk


Gee, Ali, will sanctimonious stereotyping save the planet? Are you living in a cave, using no energy and eating grass? If not, get over yourself.

Others seem pretty lightweight to get 'censored'. A comment on the thread "What's your favourite tranquil spot" which was pulled was 'I love the silence at Old Trafford when the visiting team scores a goal'. I laughed out loud, but it got deleted from the BBC site. Of course, out of context you can't see if this individual user had a difficult pattern of behaviour (e.g. posted anti-Manchester United comments in every single thread regardless of the topic), or whether this was one post as part of an ongoing feud in the thread. However, it does seem like a lot of pretty innocuous content is getting deleted.

I was also intrigued by the legal position here. According to the BBC's Have Your Say rules, users

agree to grant the BBC a perpetual, royalty-free, non-exclusive, sublicenseable right and license to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, perform, play, and exercise all copyright and publicity rights with respect to any such work worldwide and/or to incorporate it in other works in any media now known or later developed for the full term of any rights that may exist in such content, consistent with privacy restrictions set forth in the BBC's Privacy Policy.

Clearly the user hasn't given permission for News Sniffer to reuse the content, and I wondered where the legal responsibility lies if the BBC has pulled comments from Have Your Say for legal reasons (e.g. copyright infringement or potential libel proceedings) and it is republished with a permanent link on News Sniffer.

I think it would be interesting to also do this kind of monitoring on a range of forum's associated with news sites. They all have their standards of moderation, and it would be good to get some kind of cross-media "percentage zapped" score. Of course, this only works on post-moderated boards, and will never answer the question "I submitted loads of comments to news organisation x, and they never publish them because they are biased about y".

From the comments left by the author on Biased BBC, I understand the code producing the site is going to be released as open source, allowing people to run it themselves, and tailor it to their own media interests.