Camestros Felapton

FELAPTONL

Crunching reform or rollback

There is an on-going discussion at File770 on the 5/6 Hugo nomination rule:

While the Sad and Rabid Puppies slates were filling up most of the slots on the 2015 and 2016 Hugo ballots, majorities at the Worldcon business meetings passed and ratified several rules changes that made it much more difficult for that to keep on happening. The success of these majorities has tended to overshadow how many fans did not want any changes made – no matter how often Vox Day dictated what made the ballot – or else did not want these particularchanges made. And there are business meeting regulars who evidently feel now is the time to start turning back the clock.  
Here’s a matched set of proposals to end the “5 and 6” part of the Hugo nomination reforms. If you are going to the Dublin 2019 business meeting, you will have to decide whether the claims made about convenience and efficiency warrant undoing the protective rules put on the books just a few years ago.

http://file770.com/reform-or-rollback/

The proposal states that:

“The losers will be those who had placed sixth in recent years. There is only one case of a sixth-placed finalist at nominations stage going on to win the Hugo in the last three years (the rather odd situation of Best Fan Artist in 2017, where two finalists were disqualified). On the other hand, a reduced pool of finalists increases the cachet of being among that number.”

I have some doubts about this point. Firstly, 2017 and 2018 isn’t a lot to go on and 2017 still had some residual Rabid Puppy action and hence isn’t a great example for 6th places. We really only have 2018 as ‘regular’ year of the two big voting reforms EPH and 5/6.

I won’t rehash all the arguments from the File770 discussion (at least not yet) but I did want to look at the specific issue of how likely is it that a 6th place nominee might win the Hugo in their category.

Obviously, there are zero examples of this from 2018 but it would be wrong to infer that the answer is therefore zero chance. Instead, I decided to look at how ranks change between the EPH nomination stage and the instant run-off voting (IRV) final stage.

To do that I looked at the nomination rank (EPH) and final rank (IRV) of Hugo and Campbell nominees from 2018. I discarded categories which had declined nominations because I felt they might have weird impacts. Here’s an example of the Novel data:

IRV EPH Dif Mag Finalist Category
1 1 0 0 The Stone Sky Novel
5 2 -3 3 Raven Stratagem Novel
4 3 -1 1 Six Wakes Novel
3 4 1 1 Provenance Novel
2 5 3 3 The Collapsing Empire Novel
6 6 0 0 New York 2140 Novel

In the example: IRV column shows the rank of the work through the elimination process; EPH shows the nomination rank; Dif is EPH minus IRV (negative means the work was less popular in the 2nd stage); Mag is the magnitude of the change regardless of direction.

The average difference has to come to zero (everything balances out) but the (mean) average of the magnitude comes to 1.27 i.e. on average finalists shift about one place from first round to second round. Of the 90 finalists listed 25 had no change, 34 changed by 1 (i.e. the modal change), 19 by 2, 7 by 3, 4 by 4 and only 1 by 5. That last change was a drop from 1 to 6 rather than a rise but does demonstrate the scale of possible change.

How about 6th placers in general? The magnitude of the shift for those ranked 6th in nominations was 1.2 but that was also the average of the difference (i.e. with direction). Of course, if you are in 6th place you can’t get a negative change in your rank in the second stage because you can’t get lower (assuming you don’t get No Award of course and I didn’t model that).

Of the 15 6th placers I looked at, 5 didn’t shift at all, 4 shifted up by 1, 5 shifted by 2 and 1 shifted by 4 (Sheila Williams in Best Editor Short).

I’ll put all the numbers after the fold but I think the figures point to it being unlikely in general that a 6th placer will go on to win in the second round but not so unlikely that we won’t see it every so often.

Data in EPH rank descending order after the fold.

Corrections welcome: I had to hand type in the ranks and the IRV stage in particular may have errors.

IRV EPH Dif Mag Finalist Category
6 6 0 0 New York 2140 Novel
6 6 0 0 Clearly Lettered in a Mostly Steady Hand Short
6 6 0 0 My Favorite Thing is Monsters Graphic
6 6 0 0 Blade Runner 2049 DPL
6 6 0 0 Sarah Kuhn Campbell
5 6 1 1 Children of Thorns, Children of Water Novelette
5 6 1 1 Sleeping with Monsters BRW
5 6 1 1 Star Trek Discovery: Magic to Make the Sanest Man Go Mad DPS
5 6 1 1 Galactic Journey FZ
4 6 2 2 Binti: Home Novella
4 6 2 2 Escape Pod Semi
4 6 2 2 Bogi Takács FanW
4 6 2 2 Maya Hahto FanArt
4 6 2 2 A Skinful of Shadows YA
2 6 4 4 Sheila Williams EdShort
5 5 0 0 The Black Tides of Heaven Novella
5 5 0 0 Paper Girls, Vol. 3 Graphic
5 5 0 0 SpringSchoenhuth FanArt
4 5 1 1 Sun, Moon, Dust Short
6 5 -1 1 A Lit Fuse BRW
6 5 -1 1 The Deep DPS
4 5 1 1 John Joseph Adams EdShort
6 5 -1 1 The Book Smugglers Semi
6 5 -1 1 Rocket Stack Rank FZ
2 5 3 3 The Collapsing Empire Novel
2 5 3 3 Wind Will Rove Novelette
2 5 3 3 Get Out DPL
2 5 3 3 Foz Meadows FanW
2 5 3 3 Summer in Orcus YA
1 5 4 4 Rebecca Roanhorse Campbell
4 4 0 0 Dr Who: Twice Upon a Time DPS
3 4 1 1 Provenance Novel
3 4 1 1 A Series of Steaks Novelette
3 4 1 1 The Martian Obelisk Short
3 4 1 1 Luminescent Threads BRW
3 4 1 1 Bitch Planet,Vol. 2: President Bitch Graphic
3 4 1 1 Thor: Ragnarok DPL
3 4 1 1 Neil Clarke EdShort
5 4 -1 1 Fireside Magazine Semi
3 4 1 1 nerds of a feather, flock together FZ
3 4 1 1 Mike Glyer FanW
5 4 -1 1 The Art of Starving YA
3 4 1 1 Jeannette Ng Campbell
6 4 -2 2 River of Teeth Novella
6 4 -2 2 Steve Stiles FanArt
3 3 0 0 Down Among the Sticks and Bones Novella
3 3 0 0 Beneath Ceaseless Skies Semi
3 3 0 0 GraceP.Fong FanArt
4 3 -1 1 Six Wakes Novel
4 3 -1 1 Extracurricular Activities Novelette
4 3 -1 1 Black Bolt, Vol. 1: Hard TIme6 Graphic
2 3 1 1 SF Bluestocking FZ
5 3 -2 2 Carnival Nine Short
1 3 2 2 No Time to Spare BRW
5 3 -2 2 The Shape of Water DPL
1 3 2 2 The Good Place: The Trolley Problem DPS
5 3 -2 2 Jonathan Strahan EdShort
5 3 -2 2 Camestros Felapton FanW
5 3 -2 2 Katherine Arden Campbell
6 3 -3 3 La Belle Sauvage YA
2 2 0 0 And Then There Were (N-One) Novella
2 2 0 0 Saga Volume 7 Graphic
2 2 0 0 Black Mirror: USS Callister DPS
2 2 0 0 Strange Horizons Semi
1 2 1 1 Welcome to Your Authentic Indian Experience Short
1 2 1 1 Sarah Gailey FanW
1 2 1 1 Geneva Benton FanArt
3 2 -1 1 In Other Lands YA
4 2 -2 2 Iain M. Banks (ModernMasters of Science Fiction) BRW
4 2 -2 2 Star Wars: The Last Jedi DPL
4 2 -2 2 Journey Planet FZ
4 2 -2 2 Rivers Solomon Campbell
5 2 -3 3 Raven Stratagem Novel
6 2 -4 4 Small Changes Over Long Periods of Time Novelette
6 2 -4 4 Lee Harris EdShort
1 1 0 0 The Stone Sky Novel
1 1 0 0 All Systems Red Novella
1 1 0 0 The Secret Life of Bots Novelette
1 1 0 0 Monstress, Vol. 2: The Blood Graphic
1 1 0 0 Wonder Woman DPL
1 1 0 0 Lynne M. Thomas and Michael Damian Thomas EdShort
1 1 0 0 Uncanny Magazine Semi
1 1 0 0 File 770 FZ
1 1 0 0 Akata Warrior YA
2 1 -1 1 Fandom for Robots Short
2 1 -1 1 Crash Override BRW
2 1 -1 1 Likhain FanArt
2 1 -1 1 Vina Jie-Min Prasad Campbell
3 1 -2 2 The Good Place: Michael’s Gambit DPS
6 1 -5 5 Charles Payseur FanW

camestrosfelapton

, , ,

12 responses to “Crunching reform or rollback”

  1. You know, for all the questions about slating and whatnot, I kind of care most about the fact that adding a sixth nominee increased the # of works that could be nominated that might’ve deserved it – and which I might’ve missed otherwise – without diluting the award at all really. So I’d more be in favor of extending this than ending it early or letting it sunset.

    Whole thing just strikes me as a motion by lazy – or well, tired – administrators

    Liked by 1 person

    • I seeing it as 5 finalist plus one for good luck. I doubt we’ll have a sweep again thanks to EPH etc but I think we will see some attempts to game a nomination onto the ballot by some of the aggressively self-promoting authors out in the wilds. 5 is enough normally so if a spot was lost to somebody attempting to spam the awards, the damage would be less. Mind you this works for 6/6 also.

      Liked by 1 person

  5. 5/6 does prevent sweeps: in ensures that even if a slate gets a million votes it can’t get a sweep. (A conspiracy of slaters, as opposed to a single slate, could still get a sweep by doing tricks with divided votes, but that would both require extra work, and expose their duplicity, removing the ‘I was just making recommendations’ excuse.)

    5/6 can also offset the occasional damaging effect of EPH, where a work with a large fan club pushes past a work with broader appeal, because its votes are more focused (e.g. Schlock Mercenary/Sex Criminals). The displaced work has a chance of getting back in in sixth place, as it would have done in that case.

    Now in an ideal world, where slates remained undreamed of, it would be better to have fewer finalists: it’s not only administrators who are overburdened by the number of works we have to deal with. But as it is, I think we have to keep the current system.

    Liked by 1 person

    • A slater whose only goal is to “burn down” the awards by forcing No Award in most/all categories can easily get around 5/6 simply by preparing six slates of 5 each and telling followers to use the last two digits of their telephone numbers to select the one to use. Vox Day’s slate was so disciplined that when they learned a particular nominee wasn’t eligible, nearly all of them recast their votes on just 24 hours notice, so the “extra work” of a split slate really wouldn’t be a big deal for them. And as far as excuses go, they really don’t care about appearances.

      In a world with EPH, there’s actually no reason to restrict the number of nominations. It exactly cancels out the benefit of making more nominations. EPH fights slates by making it easier for organic votes to overlap, so there’s a benefit to allowing the people who do want to make ten nominations to do so since it increases the chances they’ll overlap with each other. Of course that doesn’t count the impact on Admins of having to cope with extra nominations or of someone trying to break the system by making a thousand nominations.

      As far as the ideal number of finalists goes, it really comes down to answering two rather difficult questions: 1) how many worthy works are there in any given year and 2) how many unworthy works end up gamed into the finalist list? EPH only protects against slates; it actually makes it slightly easier for an author who’s just trying to promote one single work.

      My own impression is that, in any given year, there are just two or three dubious finalists across all four of the traditional fiction categories. What worries me more is that I’m seeing a pattern where the very best stories don’t get nominated at all (not even in the long list) because they aren’t available for free online.

      Liked by 2 people

      • At the moment, we have more than five or six excellent stories, novels, comics, dramatic presentations, etc… every year. And yes, occasionally nominators miss a really great work due to limited availability. And sometimes, we also have things on the ballot where I’m completely baffled how they managed to get nominated (The Good Place, cough), but someone seems to like them.

        Liked by 1 person

        • Well, I did run the numbers. Vox Day, at his peak, had 400 people voting in lock step. If he’d had that many in 2015 and deployed them optimally, he would have swept all but one category, assuming 5/6 was all that stood in his way. Against EPH and 5/6, he’d have swept only two, although he’d have secured 70% of the nominations. I could rerun it for EPH without 5/6, but the effect of 5/6 is so small, it wouldn’t make much difference.

          With the 200 people he actually had in 2015, he could have swept all but three categories against 5/6. Against EPH and 5/6, he couldn’t have swept any, although he’d have grabbed half the nominations over all.

          Even with EPH, a determined slater could seriously pollute the lists of finalists, but he/she would be hard-pressed to “burn the awards down.”

          Liked by 1 person

      • It’s true that if a slater’s aim is to burn the awards down they won’t be deterred from conspiratorial tactics by a sense of shame, but that may not be the aim in every case. If we base our calculations specifically on VD’s motivations, we won’t be prepared for other possible attacks. What if a group of dinosaur-lovers put together a dinosaur-based slate? What if some fans of TOR, annoyed at the domination of the awards by Orbit, put together a TOR slate? Etc.

        On the larger issue, I’m not sure there is a clear line between worthy and unworthy, or even between gamed and ungamed. A lot of things are there, I think, because a particular author or series has a lot of fans, rather than because people surveying the field have picked out those works as the best. In a way that’s totally fair; you can’t say these nominations don’t represent a real preference, as slated ones don’t; people are voting for what they like. At what point does this turn into gaming? I agree it does at some point – when people say ‘I have no interest in the Hugos as such, I’m just a fan of Joe Smith and want him to get an award’, that’s annoying (and I have seen people saying things like that). But I don’t think you can draw a line.

        I think the size of the final list has to be based on what one can reasonably be expected to read. If there were a hundred things on the list one could obviously not read them all, so it would become an exercise in ‘vote for what you are already a fan of’ – which is fine, it’s what the Dragons would be if they worked properly, it’s to a large extent what Goodreads Choice is, but it really isn’t in the spirit of the Hugos.

        Like

Blog at WordPress.com.