SEO at the BBC: The early years

Yesterday I blogged some notes I’d made as our SEO Editorial Executive Chris Moran talked about SEO at The Guardian, and how he went about his job in a newsroom in 2011.

It made an amusing contrast for me to the humble way I went about SEO for the BBC when I started there in 2000. My first job was “Registration co-ordinator”, a role I shared with the wonderful Anne Scott.

Our job was to register BBC content with search engines.

Literally register it.

Usually this was done by putting new URLs into a webform on a search engine website, but we built relationships with some of the search engines that allowed us to bulk submit a text file of new BBC web addresses. The Sitemap XML format everybody uses now was not developed until June 2005.

When I started at the BBC, Google was a new kid on the block, having arrived in 1999, and so we were still concerned with how BBC content was indexed and listed on a variety of engines including Yahoo!, AltaVista, Hotbot, Lycos, Excite, Wisenut, Teoma, Looksmart, Northern Light, the Open Directory Project and a host of other niche search engines and directories.

Anne & I went on a “sub-editing for print” course to hone our skills at writing 25 word descriptions when submitting content to Yahoo! and other directories.

This was the early days of trying to make money from search, and whilst at that time Google carried no adverts and only organic results, it was clear that paid search was the way forward. Inktomi, whose index appeared on various sites, began a paid inclusion programme which would not, they claimed, affect rankings, but would guarantee inclusion. Yahoo! began charging a recurring fee for inclusion in their directory. I took part in several edgy meetings where we took a firm BBC position of “Thanks, but no thanks. We are a not-for-profit public service website, and so we’d rather not pay. If you think your search engine looks credible without our content, then please, feel free to exclude it.”

Once a month we’d put together our referral stats. This was a hideous process as the BBC did not have any sitewide web analytics in place. We had access to weekly reports generated from the server logs by Analog. About three days a month were spent doing dull manual data entry to add up the referrals to different parts of the BBC site to see whether our roles were having any impact. Since we’d both started on six month fixed term contracts, it was a case of willing the numbers to go up.

They did, not just because more people were coming onto the internet, but because we were getting out and about around the BBC doing presentations on how search engines worked, and what could be done to optimise content. A lot of BBC Online in 2001 and 2002 was still being built in Dreamweaver or Homesite rather than a CMS, and so editorial staff could make significant improvements to the way that title tags and metadata reflected the content.

Anne and I drafted a “Metadata standard” for BBC Online which set out how staff were expected to populate title tags, and meta description and keyword tags. We stressed that the keyword tag was virtually worthless, but at that time the majority of search engines displayed the contents of the description tag in search results pages rather than a contextual snippet. The description tag was an opportunity to sell content to the user. This example, from May 2003, is typical of the slides that I used to present.

We split the whole of BBC Online’s content into two sections, covering one each. Mine included News and Sport, which meant several trips to Television Centre or White City to attend meetings where, as you do when you are young and keen to make an impression, I’d brashly assert that BBC News should do x or do y with their code or their headlines, oblivious to the combined expertise and experience in a room full of the likes of Mike Smartt, Pete Clifton and Nic Newman.

Some things haven’t changed much though. Matt Cutts was the public face of Google to webmasters back then, and Danny Sullivan was the world’s expert on the search engine business. Some have changed. I once had to phone up Marissa Mayer to apologise about an unpaid BBC invoice to Google. I expect she has people who deal with things like that for her now...