A bit more on Dragons and probabilities etc

I had some weird conversations yesterday about Dragon Award stats. One was a brilliant take down of my figure that 10 men out of 10 had won Dragon Awards from 2016 in the two headline categories. Aha! Four years and two categories is only EIGHT! Yeah but it really is ten men. James S A Corey is actually two people and, even harder to believe, apparently John Ringo and Larry Correia are different. Mind you…if I only count Larry Correia once (because he is the same person whichever year he’s in) then it is back to 8 again…You’ll note that however we count it the answer comes out the same: 100% have gone to men in the two headline categories.

The discussion does raise a relevant point about why statistics is hard. Even a basic stat like a count of how many out of how many requires engaging your brain and thinking carefully about what you are counting. It was suggested that I should have said 10 men out of 8 awards…which I guess makes it clearer what was being counted but is horrible arithmetically. It looks like “10 out of 8” i.e. 125% which is nonsense because we are diving two different things and creating a derived unit of men per awards.

I’ll point people back to this post https://camestrosfelapton.wordpress.com/2019/08/10/dragon-award-by-gender/ and this post https://camestrosfelapton.wordpress.com/2019/08/11/more-dragon-stats/ where I talked in more detail about what I counted and how.

To round off that previous gender post here is an equivalent graph of winners by gender in the book category:

Like the graph in the previous post of finalists, I’m using counts by gender which reduces the gender disparity by only counting two joint authors of the same gender as 1 but two joint authors of different genders as 1 each per gender. Same caveats about gender as a binary classification apply as with the earlier post.

Worst year was 2017 which was also peak Rabid Puppy influence.

A couple of conceptual questions have come up that are related. I was asked elsewhere what the chance was of so many authors on Brad’s list winning. A different question with the same kind of issue was asked by James Pyles – basically what was the chance of N.K.Jemisin winning a Hugo three times in a row.

Both questions aren’t something that can easily be answered and they sort of miss the point of the kind of comparisons against chance you might do with gender. With the Brad list these were people who were plausible winners, the outcome wasn’t surprising. There’s no expectation that the result of an award is a random event when looking at individuals – the same is true with Jemisin. We could say, well there’s 7 billion people on earth and one winner so the chance is 1/7 billion and the chance of winning three times is (1/7 billion)^3 and then concluding that everything is impossible but the comparison is silly.

Comparing with chance is there to test a kind of hypothesis: specifically whether the result is plausibly the result of chance. If the probability is tiny then we can reject that it happened by chance. We already know that somebody winning a Dragon or a Hugo isn’t by chance because names aren’t picked out of a hat.

So why compare gender of winners to chance events if we know winning isn’t a chance event? Good question. Because, we are testing another level of hypothesis. With gender, the hypothesis could be stated as ‘gender is an irrelevant variable with regard to winning award X’.

Consider this. Imagine if all Dragon (or Hugo) winners were born on a Tuesday. That would be remarkable. Day of the week surely isn’t connected to whether you win an award or not! We might reasonably expect only one-seventh of winners to be born on a Tuesday. We might do extra research to see if across all people if day-of-the-week is evenly distributed. We might fine tune that further and consider only English speakers or only Americans etc. The point being that if day-of-the-week departed from chance then we would reject that day-of-the-week is irrelevant.

If we did find that, it wouldn’t tell us why or how day-of-the-week was relevant. One response I’ve seen to producing gender stats is people saying that they don’t pay attention to author’s gender when voting. Even if we ignore subconscious influences and take that at face value, all that does is remove one possible cause of a gender disparity, it doesn’t make the gender disparity go away.

Another response is that looking at gender stats is ‘politics’. Well, yes, it is but it is relevant even if we otherwise lived in a gender neutral utopia. Again, imagine if Tuesday-born people won far more sci-fi awards than other people — that would be fascinating even though we don’t live in a world of Tuesday-privilege.


16 thoughts on “A bit more on Dragons and probabilities etc

  1. I may be about to offend the mathematician in you, but I’m not sure Dragon Awards has enough data yet for quantitative analysis to be useful in making a convincing argument about gender disparity. Lets give it ten years.

    Then again if you decided you wanted this blog to become an exhausting labour of love (got a spare few thousand hours?) you could attempt some qualitative research on why people vote for who they do in the dragons and whether gender is a factor. Maybe bribe them with a Sunday beer seeing as you don’t need to get it past an ethics committee. (Note: I haven’t voted in the dragons but would be willing to in exchange for the aforementioned bribe)

    Liked by 2 people

      1. Yeah you can only work with the data available. It’ll be interesting to see where these awards go a few years down the track. My suspicion is good old fatigue will lead to awards going the way of “Let’s give it to this book because it was good” rather than being politically motivated. So eventually it will likely end up reflecting popular opinion rather than any specific group. Outrage is a finite fuel resource and even members of groups trying to game awards eventually get bored.

        Liked by 1 person

  2. This is the first time that I have ever considered it a good thing that I haven’t yet been able to pursue a Masters in Statistics to expand on my Bachelors in Mathematics… how dare you, Cam! 😛

    Liked by 2 people

  3. As far as the counting goes, I suggest the event is “Award X was won by Gender Y.” Therefore if the same author wins twice, you count that twice. I suggest discarding any results where coauthors were not of the same gender.

    If the Dragon subset you’re looking at is really just 8 awards, and they’re all male, then that’s 1 chance in 256, assuming an unbiased result would be 50/50. That’s probably significant.

    The trouble with looking at different subsets of the awards is that each time you take another bite of the apple you need to tighten your requirement for statistical significance. For example, if you use a 5% threshold but then try 20 different combinations, odds are one of them will get flagged. That’s why some researchers are using a one-in-a-million threshold these days. Note that the gender bias in the Hugos in the post-puppy era is at the one-in-one-trillion level, so it’s not actually that hard to exceed the one-in-a-million level if you have a real signal.

    Liked by 1 person

  4. The admins are conservative and they pick the winners, so yes, there will be a preference for men winners, though some women are far right members of the Puppies like Sarah Hoyt. The Puppies are also far right and so will lean towards picking men in their voting. Women won in the Horror and YA/MG categories where the Puppies were not very focused on getting the vote out and so the admins probably just went with the actual popular vote. But that’s a pattern that’s going to spread over the years.

    Statistically, there’s no real clear way to measure it because we simply don’t know how much the popular vote affects the nominations and the winners and for each category of award. We can only guess by looking at the pattern of who the nominees and winners are, but since the admins are the determiners, it really is about the views and considerations of the award admins, under the current rules of the Dragon Awards.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.