Busting the BBC's 600 Linux users myth
It seems my ex-boss Ashley Highfield dropped a bit of a clanger this week with a claim that amongst BBC.co.uk's 17 million or so users, 5% of them use Macs, but only about 400 to 600 users run Linux. The Linux community has been spluttering with outrage at the low figure given, and there is already a group on Facebook with the aim of collecting more than 600 members who run Linux and visit the BBC website to prove the figures wrong.
I've no doubt that Ashley has been badly briefed here, as I'm fairly sure it is several years since he has got down and dirty with logfiles himself. I would venture that it was probably as a result of someone in the BBC's Marketing or Comms department misunderstanding something coming out of the BBC new media stats system. Of course, it does somewhat beggar belief that sitting on the same floor as a collection of some of the brightest technical minds I've ever met, it didn't occur to ask anyone technically-minded if the figures made sense before putting them out into the public domain.
Or, it could just be a misplaced comma or something on the part of the magazines carrying the quote - perhaps the new BBC Internet Blog will be able to clear that up?
[UPDATE: It has, and the figure is revised upwards to between 36,600 and 97,600]
Anyway, I've already spotted people using my 2005 article "The software used to access the BBC homepage" to refute Ashley's figures - so I thought it was worth re-iterating them here, along with the caveats that are in that article.
In 2005 I wanted to answer the question "How many people use Firefox to access the BBC homepage", and so I went through the logfiles and examined 32 million requests to the BBC.co.uk homepage, looking at the user agents. Amongst those I found about 100,000 coming from machines claiming to run Linux, which equated to a 0.41% share of the requests.[1]
Please note though, that these are requests I'm talking about, not unique users, and from the user agent you can't always tell if a request is coming from a person, or a bot. Or from a person or bot disguising themselves as running something that they are not. Apart from the Hungarian based KummClient which is always sure to shout out 'Linux rulez' as it pays a visit.
You can read the whole 2005 article here, which also discusses the share of different flavours of Windows, and how Firefox was performing against Internet Explorer at the time.
Back to 2007, and in a sense the number Ashley mentioned isn't that important. It doesn't matter whether it is 600 users, 6,000 users or even 60,000 users. That is still a very small drop in a very big ocean. Whether it is the iPlayer, or any other service provided by the BBC, the Corporation still has to choose carefully where it spends its money. Of course the BBC should provide universal access to its services, but it also has to work out what gives value for money for the majority of Licence Fee payers.
Recently there has been controversy over the BBC's plans to make more Gaelic TV programming. It has a very small potential audience, and the cost apparently equates to £257 per potential viewer. Rent-a-quote MPs have been quick to jump on the "Well, in these times of widespread cuts, how can the BBC be spending £x on such-and-such, when I think it should be spending more on something else instead". Usually more coverage of politics, actually, if you ask MPs.
A Linux version of the iPlayer needs to be looked at as part of that equation - how much is it going to cost to make per user? Even if it was available tomorrow, it would still have to carry some form of 7-day download window and 30 day expiry DRM as required by rights-holders for programmes for which the BBC doesn't hold 100% of the rights - i.e. just about all of them.
And given that the very passionate, vocal, but tiny Linux user base also seem to be the most likely to say things like "Plucking signals straight out of the air and onto a hard drive isn't hard with 2 * 500GB drives and 6 DVB-T and one DVB-S tuners (with room for expansion)", how many of the theoretical 600/6,000/60,000 Linux users would actually use a DRM-enabled Linux BBC iPlayer?
[1] And I should add that at the time the article was vigorously rubbished on Slashdot for also under-counting Linux, because I didn't analyse the whole of the long tail, and I didn't make the obvious assumption that a large number of requests that appeared to be from Windows were probably from Linux users disguising their User Agent. [Return to article]
You make a common but likely inaccurate assertion that "Linux users" are likely to building their own PVRs etc and thus would not want the iPlayer. (Certainly this was true once upon a time)
I contend that there is a relatively new and fairly extensive breed who really are "Linux users" rather than Linux officianados ie: They are just given a Linux system to work with and that's it. They don't spend all day extolling it's virtues but the company they work for has them sit in front of it to do work.
The whole thing really harks back to the requirement for DRM, which is pretty much incompatible with at least some flavours of Linux.
If it wasn't for the DRM requirement, the BBC could concentrate on making their own Windows client (to reach the biggest audience), and then opening up to allow developers to make their own clients for all other platforms (saving them lots of money and effort). Unfortunately, that can't happen.
There is a corollary to "if you build it they will come" and that is if you dont, they wont.
I use linux, solaris and windows - with firefox on all of them. If I want to do anything more than read text on the bbc site, I walk over to my gaming system that runs windows or boot the laptop into XP instead of linux. It's an inconvenience that means unless I'm already sat at a windows box I don't bother with a lot of the BBC sites content at all.
That's more likely to be the reason that linux shows up less often in the logs than it might. Not so much that there's fewer users out there, but because it's a market they are not serving.
I agree with you there Frankie, the 'Windows Only' and the DRM issues tend to be conflated - although they are interlinked they aren't the same thing. With the BBC's podcasts, for example, because the rights-holders have allowed it in some cases, or because the BBC has much greater control on the rights in others, the BBC has been quite happy to give stuff away for free in mp3 format for anyone to keep and do what they will. TV is just much, much, much more complicated from the business side of things.
Ian Douglas at The Telegraph has published a breakdown of the OS usage they see on their site - bottom line is that there is just a fraction under 1% of Linux users in their audience. Not sure I've seen a newspaper be so up-front about that kind of usage data before - good stuff.
How do you differentiate the Linux users, from those using a plugin like User Agent Switcher? I frequently leave my user agent string set to a Windows variant, because I keep running into poorly crafted web sites that don't display properly unless they think they are communicating with some version of IE. This is the solution frequently offered to new users when they encounter this problem.
I'd note that I routinely have my Linux systems miscounted as Windows. I've run into a number of Web sites that simply refuse to work if the user-agent string says anything other than Internet Explorer. Note that it's not that they won't work in any other browser, in fact they don't use anything IE-specific on their pages and work just fine in Firefox. But on every page is a small script that checks the user agent and displays an "Upgrade to IE" page instead if it doesn't see the right user agent. The easiest way to get past this is simply to tell Firefox to use IE's user-agent string, which includes all the flags to get the system classified as Windows instead of Linux. Opera's browser-spoofing setting does the same thing. So if you give people a reason to spoof their user-agent at all, it seems very likely that they'll spoof it to something that'll also get them misclassified as running Windows.
Ashley has now responded to the point about Linux figures
I absolutely agree that there will be an undefined element of the Windows figures which are spoof user agents from other systems.
However, when people complain that the figures derived from looking at logfiles are wrong because of spoofing, it is a little like shooting the messenger. A lot of businesses had to sit up and take notice of Firefox because they began to see it being used more and more.
Clearly spoofing the UA string is important to get past IE-only sites that are not really IE-only, but spoofing the UA string on sites where it is unnecessary just statistically weakens the argument for how widespread Linux adoption is.
Sorry to hark back to the DRM thing, but I agree with Martin that TV rights are a minefield.
The danger for the BBC isn't so much that Linux users can't access the content (it really is a relative minority), but that the core mainstream Windows don't bother.
I installed the iPlayer on my Windows machine at work (Mac at home), but couldn't get the thing to work, so gave up. Coupled with all the viewing/downloading timeframe restrictions, and the inability to put content on a portable device (not that I've got one, but I see more and more commuters watching video on their iPods/devices), might end up leading to a lower than expected adoption rate. And that's not even counting the problem of needing different systems for content from Channel 4, ITV, etc. Ease of use is everything.
I think that the 600 figure is probably about right, and the other figure is probably just Windows users disguising themselves as linux clients.
I'm already fed up with the BBC bleating that it's not their fault about the rights, and they have to use DRM. There are two flaws with that argument. 1) DRM doesn't work -- the iPlayer's DRM has already been broken, and that's inevitable because the DRM vendors talk a lot of shit about how marvelous their technology is, but in fact it's all snake-oil, so why not save the fortune being wasted on DRM and spend it on usability instead. 2) The BBC has quite a lot of power here -- if they say to providers, if you won't let us broadcast your program via iPlayer, then maybe we'll have to favour other producers that do... they'd suddenly get a lot more people who didn't mind so much -- especially if they point out during the meeting that the claims made for DRM have all been proven to be false so far, so why should adding technology that just pisses people off to the programs have any relevance to the contract negotiations?
Personally, I think that if they produced an iPlayer that had no DRM, but was really easy to use, and by default deleted the programs after a week, 99.999% of people would go with the flow, and let it -- I suggest basing it on miro (a.k.a. democracy-player) which is Free Software and a sight better than iPlayer from I can gather.
It is a nice argument Philip, but not one that will wash in the current regulatory framework I'm afraid. You may recall that the BBC gave away mp3s of some classical music without DRM a couple of years back. They have been expressly forbidden by the BBC Trust and OFCOM from doing such a thing again as it is deemed to distort the commercial market for classical music. It would be impossible for the BBC to give away TV without DRM at the present time without commercial broadcasters protesting. And the rights are not just to the producers of a programme, remember, it is also the rights to any music used in it, and the residual rights of the actors / actresses / performers in the show, who expect to earn money from any subsequent overseas sales, repeats and DVD/VHS sales, and, now, paid-for downloads.
Really? I thought the problem was with rights-holders who make the material?
They manage to give away Radio, or at least allow streaming with no DRM.
I remember Ashley saying that they could give away some content--such as news or current affairs--programmes without DRM. Why don't the BBC just do that? Then when the rights-holders finally come to their senses and realise DRM stops nothing, they might consent to the BBC providing downloads.
Personally, I think it's all very fishy. Ex-Microsoft employees and a pile of rubbish excuses. We--that is GNU/Linux users--have been stitched up by Microsoft again, just the Nigeria fiasco. Just like the OOXML and ISO debacle.
There's a very applicable saying about this situation: 'don't judge how big to build the bridge based on how many people are swimming in the water.'
As I think I've said before, the thing that annoys me most about this whole debate is that I end up coming across as some kind of Microsoft/DRM apologist. currybetdotnet is a LAMP production, and I ran projects within the BBC where I insisted on using Open Source. I'd love iPlayer to be open source and DRM free, and I'd love it if Linux users accounted for more than 1% of the BBC's audience.
But, this isn't just case of the BBC saying they can't be bothered with Linux, or they just fancy trying a bit of DRM. There is a whole regulatory framework and commercial market that the BBC has to be sensitive to.
Absolutely.
As long as it is speech radio with no or limited background music.
And produced by the BBC.
And isn't an audio book.
None of those things are downloadable. Music programmes are only available on demand in a non-download format for 7 days after the event because the music labels won't agree to more.
When the rights-holders allow it. Ever tried listening to sports commentary online?
Erm, sorry to poke a whole in that but the BBC have been expressley order to make iPlayer platform neutral:
If the BBC can ignore that requirement why can they not ignore the DRM requirement?
No hole poked Andy, I don't believe that the BBC Trust have insisted that the 7 day catch-up service has to be via a download mechanism.
I believe that if the BBC make available on demand streaming of 7 days worth of catch-up TV in a format that will work within browsers on the 3 main platforms, then they will have met the Trust's requirements.
And given that the Trust have set six monthly reviews of the BBC's progress in this regard, that suggests a "reasonable" timeframe for the Trust is probably in the region of 12 to 18 months.
Which, I'm sure, means plenty more debates like this in the meantime - yay! ;-)
I think there's a possibility that might have been overlooked.
Basically, would somebody using a computer and software that he was unwilling or unable to discard make a point of visiting a news site if he already knew that large parts of it didn't download or display properly?
For various reasons we run Linux on SPARC hardware, and while standard web sites are fine anything containing dynamic content such as Flash tends to be a problem. The cause of the trouble isn't the browser plugin on the client, it's the JavaScript embedded in the HTML page that tells the server what the browser is and confirms that a suitable player is available.
Now I'm not necessarily fingering the BBC for incompetence or malevolence here- for the simple reason that I very rarely go deeper than their standard static-content news pages so have never pulled their dynamic pages to pieces.
In other words if they aren't logging "superficial" traffic at http://news.bbc.co.uk/text_only.stm then they don't count me. I wonder how many other people they miss for the same reason?
"A Linux version of the iPlayer needs to be looked at as part of that equation - how much is it going to cost to make per user?"
There are plenty of ways to build cross-platform applications. Obviously, building a new client from scratch that runs only on Linux would have a significant cost, but if the original code base were built around, say, QT and standard libraries, producing a Linux client would be a 5 minute compile away.
Obviously, the BBC chose not to do that. Maybe they couldn't figure out how to deploy cross-platform DRM, or maybe they use DRM as the excuse but the truth is that the man in charge used to be a top manager at Microsoft and has some animosity towards Linux? Which is more likely?
I know there are those who think Microsoft's tentacles are akin to Satan's in their cunning and tenacity, but Chris, are you really suggesting you believe that Erik Huggers, who joined the BBC in May 2007, influenced the design decisions on a project that was originally being trialled as the iMP in October 2005?
Nice selective quote; the missing qualifier "maybe" changes the context slightly, no?
The truth is that I don't know why the BBC chose this route. As a professional software developer, building cross-platform solutions is easier now than it's ever been, and yet the developers at the BBC deliberately chose not to do this, and instead to build an application around Windows-only technologies. In this day and age, tying yourself to a single platform, when writing software intended for millions of uses, is a very unusual thing to do. It isolates substantial markets; not just Linux users, but mobile phones (e.g. Symbian, Blackberry), Playstation 3, DVD and settop boxes, video iPods, etc. - these devices will never be able to play BBC iPlayer content.
Sorry Chris, I did not mean to distort your meaning, but I read the sentence as 'maybe x, maybe y BUT the truth is...'
Still if we've got to grammar nit-picking then a reference to the 'N' word can't be far behind
As far as cross-platform development goes, my understanding of the iPlayer development is that 90% of the effort was geared towards correctly splicing content out of the broadcast stream, matching it to the metadata from the EPG, working out if it should be going on to the iPlayer play-out, and ingesting it automatically.
The bit the public see, i.e. the current website and Kontiki based Windows XP client, are the tip of a very large iceberg. That front client bit can be changed, or a second one added to service another platform etc etc as is seen fit.
So there isn't any way of having DRM on Linux? Have RealNetworks (whose DRM-capable RealMedia format has been used by the BBC website for years) suddenly disappeared off the face of the earth?
I work at Long Beach City College in Long Beach, California. We probably have no more than 30 or 40 LINUX regular LINUX users on our campus, and probably no more than 10 to 15 of those would like to listen to the BBC on a regular basis, were we not to find listening to the BBC on our recently upgraded Fedora 9 LINUX computers so tedious (actually, impossible, so far.) That still gives us a healthy percentage of the 400 or so LINUX users world wide who would like to listen to the BBC. Congratulations, us! Relying on NPR and Pacifica for our news isn't really bad . . . it just feels incomplete.
Tom Wilson
I have been using Linux system for a long time then shifted to Windows. I found it good to use Windows system for all my gaming activity instead of using Linux.