Newspaper "Site Search Smackdown": Round 1 - The Daily Mail vs The Sun

The other week I wrote about the potential threat to newspaper revenue from Google's new 'Search in search' feature. Links to the article appeared on a few blogs, and Kevin Anderson made the point that Google was doing search better than most newspapers:

"Where I might disagree is Martin's argument that it negatively impacts user experience. He says that Google's position is that they can provide search better than the news sites. Well, the sad truth is that whether it's information architecture or search, most news organisations have been very slow to improve these parts of their services. Some news and media organisations have forced their users to use Google because their own search is unusable."

One of the interesting issues is that a newspaper site search ought to be able to index content directly from a CMS faster than Google can crawl a site, and so I thought I'd do a test of how fresh newspaper site search indexes were. Back when I was involved full-time with search at the BBC, the Corporation was deeply paranoid about this, and the performance target it aimed to meet was getting a BBC News story indexed and displayed in the number one slot for relevant keywords within 15 minutes of it being published.

I thought it might be fun, rather than just to do a straight set of reviews, to pit the newspapers against each other in a series of winner-takes-all smackdowns, starting with The Daily Mail versus The Sun.

The rules

The rules of the contest are quite simple. Go to a newspaper homepage at around 9am UK time. This is after the print edition has hit the streets, and when the online version has had overnight to publish the paper's main news. Take a note of the ten most prominent online headlines. Then use the newspaper's own search engine, and type in each of those ten headlines exactly as they appeared. Newspapers score a point if the story comes up in the #1 slot for that search query. Doesn't sound too exacting a test? You'd be surprised how often newspaper search engines fail to deliver...

The Daily Mail

I tested these newspapers on Saturday April 5th, and the Mail's lead story was about 'family meltdown in the UK'. Searching for the headline brought the story straight up as the number one result.

The Mail's picture feature that day was some footballer's wives and girlfriends going to watch some horses racing. Who said it was a slow news day? The story had been published on the previous Friday afternoon, and so again a simple search for the headline bought it back at #1 in the results.

And so it continued, and in fact the Mail set a cracking early pace - the first nine headlines I searched for all returned the corresponding story from that day in the top slot.

However, with the tenth story, the Mail blotted their otherwise perfect copybook.

For some reason typing in the search terms "Cleared after two years, the teacher falsely accused of punching tearaway" caused the Daily Mail's search engine to hang every time I tried it - I've no idea why. Anyway, I couldn't get any results for that query, and so it was a very valiant 9 out of 10 for the Mail.

The Sun

With the Mail performing so strongly, The Sun was up against the wall from the off. The paper is always at a disadvantage with search engines anyway, because the style of their pun and nickname laden headlines makes them memorable for humans, but very difficult for a search engine to understand conceptually.

With the ongoing election saga in Zimbabwe, the protest-blighted Olympic Torch arriving in London the next day, and a major terrorist trial going on, it was obvious that the Saturday's main story for The Sun would be 'actress from soap opera is pregnant'.

Nevertheless, with such unpromising keyword material as 'Corrie Tina to be a mum', and second story, 'Street urchin to Bond girl', The Sun's search engine held it own, bringing both those stories back as the number one result.

However, it all fell apart for the paper after that. A search for "Harry's Royal Flush" pulled up an old story about Prince Harry gambling, not the day's story about him looking flushed after having been to a club with a stripper. "Choc row killers caged" brought up nothing. "Hay good looking", "Abbey getting Weddy" and "Tot's Primark plunge" all similarly drew a blank from The Sun's search engine.

In the end, of the 10 stories I searched for, 5 appeared as the top result from The Sun's search engine, and five were either not retreived or were not indexed. That meant a straightforward score of 5 out of 10. Up against the impressive performance of the Mail, the red-top paper had no chance.

The verdict

A thumping victory for the Daily Mail's search technology. They've set the early pace with a score of 9 out of 10. Can any newspaper better that?

In Round 2

In the next bout, I'll be pitting the search results of The Telegraph against those of The Independent.