The other week I posted some comments at Mad Genius about this post http://madgeniusclub.com/2016/02/15/on-teachers-kids-and-hugos/ – unfortunately most of the responses were not exactly shining examples of how to have a rational discussion. However, one chap did decide to engage on a level above name-calling, which was nice.
The seed of the sub-sub-discussion was this comment I made:
February 15, 2016 at 2:56 pm
Arrgh hit ‘send’ accidentally
Yes an argument could be made but
1. Sad Puppies 3 was called a slate by Brad T
2. Previous Sad Puppy campaigns had been called stacking campaigns
3. The Rabid Puppy campaign was clearly functioning as a slate (yes, yes I know that the Sads don’t control what Vox does etc)
4 . The quality of the works nominated were highly variable – strongly indicating that some peoples were voting along a slate rather than judging works by their quality (e.g MZW’s excellent ‘Soft Casualty didn’t get nominated but his execrable Wisdom From… did)
And there was an OKish discussion about 4 despite various people trying to derail it. Further on Dave Freer joined in:
February 15, 2016 at 5:54 pm
Only if backed up by independent empirical data. Your taste is not everyone’s. This is a fan award – a measure of popularity, which can be measured independently by sales. So one can fairly say Dune was a quality nomination. A novel which after a couple of years has gone out of print is not.
To which I replied:
February 15, 2016 at 6:24 pm
Ok but neither past Hugos nor any of the Sad Puppy picks have been simply the best selling SF/F. So while we would expect some relation with popularity there us at least some other latent trait in play.
And to which Dave appears to have lost his train of thought and responded to something else altogether:
February 15, 2016 at 11:20 pm
Jim Butcher, Larry Correia Aren’t ‘bestselling’!? That’s probably the most bizarre thing you’ve yet managed to say, and you’ve come up with some doozies.
Well I pointed out his error but Dave had started deleting my posts. I guess he didn’t like somebody making the obvious correction. Never mind. Maybe the ‘simply’ bit confused him i.e. they may have been best-selling but that wasn’t all that they were. Did I pitch that too subtle? Maybe.
Another commentor joined in to say:
February 16, 2016 at 12:07 am
I’ll look into any sort of correlation there over the next couple of days. I don’t think I’ll get it all together tonight. Maybe by Saturday?
And eventually they did: https://westfargomusings.wordpress.com/2016/02/21/preliminary-analysis-part-1/
Good for them. Turning to the data is always a smart move.
He starts with:
I indicated to Camestrosfelapton that I would look into his claims that recent Hugo Nominations by the Sad Puppies were not up to snuff
Hmmm. OK not quite what I said. I don’t know if he is trying to address the garbled version from Dave Freer or if he had read what I’d said and misunderstood. To clarify: my point was that the stuff nominated by the Sad Puppy/Rabid Puppy slates was highly VARIABLE in quality. That was one of the pieces of evidence that points to slate voting i.e. some of it was OK to good (Totaled) and some of it was execrable sh!T (Wisdom from I really Can’t be bothered typing its name anymore)
So kamas716 has collated Goodreads rankings for Hugo Best Novel winners and nominees going back in time – which is an interesting data set. From that he looked at the mean ranking and found that The Dark Between the Stars was OK and that Skin Game was: “actually the highest rated novel ever nominated for a Hugo”.
Now what question are we trying to answer with this data? The questions is one about the quality of the books and we are using popularity as a PROXY for quality because we can’t quantify popularity directly. How good a match is it? Well, as his data shows it looks OK but consider that Skin Game is the highest rated novel EVER nominated for a Hugo (he went back to 1953). That is a good sanity check on the data – is Skin Game really the best SFF novel written since 1952? Better than Dune, The Man in the High Castle, the Lefthand of Darkness etc? Does that sound plausible? Yeah, probably not. That doesn’t mean a popularity rank is useless as a proxy but it does suggest it has limits e.g. it might correlate with quality but not perfectly and not to the point that we can compare whether Skin Game is actually better than The Dark Between the Stars just with a Goodreads rank (as it happens I think Skin Game is a LOT better than DBTS but that is beside the point.
Unfortunately even if it was a good proxy Kamas176 didn’t apply it to the point I was making: the Puppy nominees were highly VARIABLE in quality. For that he needs to compare across categories and consider the range or another measure of spread. Still all interesting stuff.