Friday, January 16, 2009

Analyzing Deck-Chair Patterns

It happens a lot this time of year: various news outlets try to run the numbers and figure out what were the bestselling books of the previous year. And, usually, they do so by using the entirely wrong numbers, and making so many hidden assumptions that the exercise is ludicrous.

Take, for example, this Bookseller article, which purports to determine the "most popular writers globally" during 2008, by counting the number of bestseller slots (on unspecified charts, prepared by unspecified organizations, using unspecified methodologies) in nine countries.

The countries are: France (population: 65 million); Germany (population: 82 million); Italy (population: 60 million); Netherlands (population: 16 million); China (population: 1.335 billion); Spain (population: 45 million); Sweden (population: 9 million); the UK (population: 61 million);and the USA (population: 306 million).

From the commentary, the "rankings" appear to have been determined by the number of times an author's books appeared on bestseller lists -- with no notice taken of placement on those lists or of the actual quantity of books sold.

And, as you can see, the countries are not the same at all -- even leaving aside book-buying habits and patterns, and the average sales level for a "bestseller" in each country. The largest country, China, is 148 times as large as the smallest, Sweden, which one might think would make a big difference in their book-buying habits.

One might think.

But, on the Bookseller's chart, presumably, ranking #10 in Sweden in a particular week is approximately as good as ranking #1 in China. This is some of the worst methodology I have ever seen: it's no better than a cruise-ship purser analyzing the patterns of the deck-chairs that he's just finished arranging.

Any analysis of book sales, and author popularity, must be based on sales figures. Bestseller list placement is only ever at best a proxy for sales, and cannot then be analyzed at a further remove.

Slightly more excusable is the annual Publishers Weekly reading of the bestseller entrails, which also showed up this week. It's primarily devoted to ranking the big US publishers according to their books' positions on PW's own weekly bestseller lists, which may make some suspicious persons frown slightly. Since it's purely about one country, and mostly about books in aggregate rather than discretely, it almost, but not quite, reaches the level of actual facts.

But, all in all, I'm coming to think that there are a whole hell of a lot of reporters who need to be press-ganged into an elementary statistics course.
Your Hornswoggler is Andrew Wheeler Released into the wild 1/16/2009 12:33:00 PM
Recurring Motifs: ,

1 comment:

Anonymous said...

Nowhere in The Bookseller's article that you refer to do we claim that the bestseller lists are based entirely on an individual author's global sales. Rather, we are very careful to point out ("according to analysis of the 2008 international fiction bestsellers published by book trade magazines including The Bookseller, Publishers Weekly and France's Livres Hebdo") that it is based on an author's appearance in the monthly Top 10 bestseller lists available in European publications. Unfortunately, these charts (which are compiled by an external consultant) do not contain volume sales data, but it is currently the best way of analysing book sales on the continent, hence The Bookseller's reason for publishing such lists.

But you are right in that it is an inaccurate art, which is why at The Bookseller, we are very careful never to suggest in any of our monthly European charts that any one author is "the bestseller" in Europe.

However, as you are no doubt aware, we also publish numerous UK charts on a weekly basis (containing sales volumes) with incredibly accurate figures/book sales data supplied by Nielsen BookScan. We also publish BookScan-measured data on a regular basis for territories including New Zealand, Ireland, Australia, USA, Spain, Italy, South Africa and Denmark.

Until other national book organisations in other countries sign up to BookScan data, or an alternative, The Bookseller's hands are somewhat tied. But we will continue to publish the European charts on a monthly basis—charts which are published in numerous respected international newspapers and magazines on the continent.

Therefore, we respectfully decline your offer to be "press-ganged into an elementary statistics course".

Mon Jan 19, 05:26:00 AM 2009

Post a Comment