EPH numbers crunched

Bruce Schneier and Jameson Quinn have crunched the numbers on EPH using the 2015 nomination data and simulated data. There is an explanatory post at File770 here and also the paper is available here https://www.schneier.com/academic/paperfiles/Proportional_Voting_System.pdf

ephtable

In the table: AV stands for Approval Voting (the system used at the nomination stage); SDV is Single Divisible Vote; SDV-LPE is the proposed E Pluribus Hugo proprosal; SDV-LPE-SL is a tweaked version of EPH that gives it a bit more welly.

Basic summary is that EPH does what it claims but the impact is less than people might hope. For Best Novel it does very well and in Short Story it does much less well.

However, I think it is good enough looking at those numbers. There are two prongs to responding to slates – a social response and a structural response. The structural response helps the social response. For Short Story there remains an issue of a very broad field in which people’s choices tend not to coalesce round a small number of nominees. That makes it very prone to slates.

What EPH does overall is ensure that there is at least one non-slate contender which is less than ideal but does ensure that:

  1. People have something good to vote for untainted by slates
  2. A non-slate winner is possible which eliminates the positive reward for slating
  3. There is a choice above No Award which means the category gets a winner and hence the negative reward for slating (i.e. griefing the awards by forcing a No Award victory) is eliminated
Advertisements

12 comments

  1. David Brain

    Yeah, in the end, it’s categories like Short Story that are always going to be “broken”, regardless of what system is used, given the sheer quantity of potential nominees and the unlikelihood of consensus. I mean, it’s true that in general I read the works that make the short list in that category and normally consider them all to be very good, but I am always acutely aware that I probably didn’t even read *any* of them before, whereas I have normally read probably 3 of the finalists in, say, the Novel category. I don’t think that there’s much that can be done about that.

    Like

  2. Mark

    Any thoughts on how EPH might have operated on this years noms? We seem to now know that in 2015 there were c150 rabid/straight-slate voters and 150 less-slatey/sad voters to produce the 2015 results above. For 2016 we probably have more rabids (Greg says 200, Chaos Horizon says more, VD says even morerer) but we also have something like double the non-slate voters.
    The diversity in short story etc probably defeats the benefits of extra voters, but I wonder if the lower spread in e.g novella has a chance of it benefiting at the same levels as novel?

    Like

    • camestrosfelapton

      I suspect so. One thing I’m curious about is whether Larry Correia wad nominated but quitely declined – if so the ‘rump’ puppy vote was bigger and that throws more randomness into the Rabid vote.
      My early guess was 200 Rabids but with very tight discipline. The noise etc coming as a side effect of the hostage nominees.

      Like

      • Mark

        I think Larry was saying “keep me out of it!” loudly enough that they got the message. If SP4 could get Larry on to the list, shouldn’t they have been able to boost JCW into contention?

        Like

      • camestrosfelapton

        I don’t think of it as SP4 so much as the general set of members who joined because of SP3/2 – many of those would be more followers of Larry rather than committed pups. In other words Larry possibly had enough ‘organic’ votes to get on the ballot as a consequence of past actipn plus some non pups just liking his recent book plus some Butcher fans joining because the RP slate meant he was in with a chance. But maybe I just want the irony of Larry getting nominated in a legit way but him turning it down because he’s hugoed out.

        Like

      • Mark

        That’s true, there will have been some bog-standard Larry fans in the mix, and some bog-standard Butcher fans too.
        I wonder if Larry today shows what Brad will get to in a year’s time – the stages of Slating Failure are Anger, Defiance, Resignation, and Shut Up I Don’t Want To Hear Any More,

        Like

      • Lurkertype

        I rather like the idea that Larry (who started the whole thing because he coveted a Hugo) is now forced to face his own toxicity and turn down possibly-legit nominations. There’s probably a word for how I feel in German. 😉

        Camestros is right, though — the whole debacle probably hasn’t hurt Larry much, sales-wise. The GunNutz still love his stuff. Any evil SJW readers who’ve gone off him thanks to this probably weren’t reading him much anyway. He’s got a series that consistently sells well, as he never tires of telling us.

        Brad, though… doesn’t. He wasn’t a bunch of books into a rollicking successful series when he became notorious for toxic Puppy waste. For a supposedly macho Army guy, he sure gets hurt fee-fees and goes into histrionics a lot. He also publicly said a lot more evil shit than Larry did, and was the Sad figurehead for the big No Award year. That’s got to hurt sales. Hope he’s with Baen, cuz Tor ain’t gonna touch him now.

        They imagined a cabal and one came into existence — but not the one they thought. Larry and Brad obviously never heard those phrases of Nietzsche’s about fighting monsters and looking into the abyss.

        Anyway, EPH is just a beginning. As long as we can get one legit nominee per category, it’s cool. Further years can take up 4/6 or additional rounds of voting, or whatever.

        Like

  3. supplanter

    So, guys. All honor and respect to the people who developed EPH last year. They poured enormous effort and time into coming up with a system to address the problem of Sad-style slating without completely disenfranchising the slaters. They worked in fact to make sure that even slate supporters had a voice. It was an exciting effort for the people involved, and they did it with the best will in the world. That said.

    A core design goal of EPH was that slates should get finalist slots in proportion to their proportion of the nominating membership. e.g. if slate supporters constitute 20% of the nominating members, they should be able to place about 20% of the nominees on the ballot. At this, EPH stands revealed as a failure, and not a narrow one. Nor does EPH+ achieve this goal or even come close.

    By the table Camestros reproduces above, the ~300 RP/SP supporters constituted 15% of the nominators for Best Novel, but under EPH and even EPH+ would control 40% of the slots. For novella, novelette, short story and Related Work the numbers are on the order of 30% of nominators, 80% (EPH) and 60% (EPH+) of slots respectively.

    At a very rough level, the base magnification power of a slate looks like about 5x. EPH reduces that to around 3x and EPH+ to 2.5x-ish. Even when it comes to self-dealing as opposed to malicious slating, EPH and EPH+ are bandaids.

    Like

    • camestrosfelapton

      One of the key missing pieces of information is how slate-like non-slate voters are. The more slateish without being lockstep they are the better they get represented in EPH.
      So the ‘ideal’ problem EPH solves is one with a bunch of biggish factions (I wrote a post imagining it being used for a giant US Presidential primary in which both parties where in the same process).
      Now how close is that to the actual voter behaviour? I suspect with Best Novel it is a lot closer than with Best Fan writer or Short Story. So with Best Novel there will be networks/families of connected books (the sort of stuff Amazon exploits for recommendations). But with Fan Writer past data suggests people often just voted for one person. With Short Story the field might be so big as those network effects don’t apply.
      Put another way, that magnification effect is different per category. That is really shitty news when it comes to Best Short Story because it is (IMHO) a really important Hugo category that fosters the genre [Not that I’m OK with VD wrecking Best Related Work but it is one I’m less not OK with being wrecked]

      Like

      • supplanter

        I accidentally unsubscribed.

        Yeah, I consider the Short Story Hugo to be central to the genre. It has historically been a place where maturing writers can gain real recognition.

        Like

  4. Pingback: The EPH Analysis | Camestros Felapton