On petunias and whales: Part 1

Several posts in and I’ve finally got to the blog comment that was getting way, way to long to be a blog comment.

The story so far: The Hugo Awards have been dominated by Sad/Rabid Puppies who have claimed left-wing cliques of ruining the awards. A rag tag team and/or an unruly mob and/or revolutionary cadres (delete according to preferred narrative) of anti-puppies have reacted in various ways and Puppies have reacted back etc. My post is dealing with a relatively early pre-2015 nomination argument from puppy supporter Dave Freer. Dave Freer is an author based in Australia who posts on the Mad Genius Club blog among other places. The Mad genius Club is one of three main pro-puppy blogs (the other two being Brad Torgersen and Monster Hunter Nation )

Dave takes an approach I like – which isn’t to say his argument holds water (it doesn’t) but it is an argument that attempts something other than vague accusations. To borrow from the language of science methodology, Dave attempts a falsifiable argument. He sets up an argument which we can check and evaluate. Put another way: he is trying to show his working.

He sets up his argument as follows: http://madgeniusclub.com/2014/08/25/a-very-surprised-looking-sperm-whale-and-a-bowl-of-petunias/

Try, if you can, to realize this is not an attack on any one individual or work, I’m not saying so-and-so should have won. Nor do I think no outspoken left-winger should ever win. If that was happening, I’d be here fighting to see they got a chance. Nor is this an attack on the Hugo committee – their sins seems to be largely of omission, not commission. I would love to see the award become – as it was in my youth, a commendation, above ideology. People claim this is the case. If that is true, it is easy to test.

So let us take the hypothesis ‘There is no ideological bias in Hugo awards,’ which we’d love to prove true and test it mathematically to do so.

This is ambitious. Basically take a hypothesis “There is no ideological bias in Hugo awards” model it statistically, compare the actual outcomes with the model based on the hypothesis. If the two do not match (within some tolerance) then the hypothesis has to be rejected.

It isn’t much of a spoiler to say that Dave’s post then does what he sets out to do. He consider what an unbiased (ideologically) Hugo award would be like, looks at the outcome and identifies how much the actual outcome is different and hence comes to the conclusion that the Hugo’s are biased. Damn! Case closed!

Having mathematically demonstrated that the Hugo’s really are biased, it becomes very difficult to see the puppy reaction as extreme.

Yeah, but no.

An argument like Dave Freer’s can go wrong in multiple ways and it pretty much does go wrong in each of them.

1. Demonstrating a statistical bias is not the same as identifying the source of a bias. A bias can have many sources and Dave Freer’s argument consider only that the nomination/voting process must be awry. In reality statistical bias can come from many sources – in this case wider society. Which takes us to…
2. Rejecting the original hypothesis “There is no ideological bias in the Hugos” is actual rejecting more than that sentence. What is being rejected is all the assumptions used to construct the model of that hypothesis. Any dodgy assumptions maybe the cause. Which takes us to..
3. Assumptions about the population you are looking at may be flawed. Dave makes assumptions about authors being like the general population. That is a necessary step because we don’t have great data on genre authors as a distinct population. However it is an untested assumption and as most people in the general population do not write science fiction and fantasy books, it is a step that may be flawed.
4. The base figures being used may be incorrect. Dave acknowledges this by trying out different scenarios but without knowing what the most appropriate figures are, the whole chain of reasoning can come unstuck.
5. To get those figures requires some counting of categories. In this case Dave has to consider the proportions of people in some notional political categories – and that isn’t easy.

Critics of Dave’s argument needn’t go in to much depth on any of those points (or combinations thereof) to feel unconvinced or dismissive. I think that is a bit unfair. I think it really is worth examining because it raises an interesting question and an interesting way of answering it.

But I’ll pause for breath here before going into details.