Three Stage Voting and the Hugos

The Back Story

(see also Nicholas Whyte’s post here http://nwhyte.livejournal.com/2855654.html )

The procedural response to the Rabid Puppy and Sad Puppy campaigns has consisted of use of existing rules and the proposal of new rules. I’ll pick on three to start with.

  • To vote you need to be a supporting member of Worldcon and to have paid the associated cost.
  • No Award – the existing procedure that allows voters to pick ‘none of the above’ from a set of finalists. This was used by voters to deliver a very forthright rejection of the slates in 2015.
  • 5-6 a rule change as a consequence of the Puppy campaigns – people nominate five works but six works become finalists. This change was voted on in 2015, ratified and came into force for 2017. It limits the impact of slate voting on the list of finalists. If it had been in place in 2015 and everything else had been the same, then in more categories there would have been at least one more non-slate finalists.

The first process limits the extent to which the Hugos are vulnerable to click-farms or other coordinated attacks by large numbers of spurious voters.

Together, those second two processes amount to damage control. No Award stops unpopular finalists winning by being the only option because of a slate. 5-6 makes it likely that there is at least one non-slate finalist that could win, instead of the whole category going to No Award. Neither, in themselves, would stop a repeat of the Sad Puppies 3 / Rabid Puppies 1 combo but they strongly disincentivise the use of slates as a way of trying to control the award.

The next two are more complex:

  • EPH – a voting system for the nominations that limits the impact that one disciplined minority can have on the nomination process. By ‘disciplined minority’ I mean a large group of voters who vote in a similar way. That could be people voting on a slate or it could just be an natural grouping of people with very similar tastes. It doesn’t stop a slate from getting works onto the ballot but it helps other works make it as well.
  • EPH+ – a tweeking of EPH to improve the extent to which slate voting has limited impact.

My feelings about EPH have been that it is a good thing in itself, regardless of the slates. More generally it further disincentivises slates. Its down sides include some mathematical complexity which leads to some lack of transparency with the counting process.

The question in 2016 was whether together these would be enough. With the Rabid Puppies still impacting the 2016 ballot and analysis suggesting EPH would not be completely sufficient to prevent impact from a well disciplined slate, there was concern that other measures were needed. More pressing was the tactical change from the Rabids – their nominees came in three kinds:

  • Hostages – plausible, popular works nominated to prevent a blanket ‘No Award’ vote against slates hard.
  • Promotional – Castalia House works nominated as a publicity stunt by the publisher and chief Rabid Puppy.
  • Trolling/griefing works – works intended to mock the Hugos or defame individuals or otherwise nominated in the hope of causing consternations.

Those three categories were overlapping.

The voters rose to the occasion again. The hostage plan didn’t work and at least one of the trolling attempts backfired when the inimitable Chuck Tingle was adopted as popular hero. However, despite the failure of Rabid Puppies 2 to make a lasting impact, it still suceeded in being a pain in the arse for everybody. In other words the griefing element of the campaign was a reward for the Rabids.

What to do? There was much discussion but a consensus arose around a proposal called Three Stage Voting or 3SV. In short, adding an additional round of voting after the initial nomination period. This round would allow votes to remove works not regarded by voters as legitimate possible finalists.

The Current State of Affairs

I believe it is fair to say that on the whole Sad Puppies were motivated by a genuine desire to gain some award recognition. Vox Day and the Rabids were more motivated by general mischief. However, both were motivated by the promotional/publicity-stunt elements of getting nominated for the Hugos.

Larry Correia’s capacity to motivate a significant number of fans to buy supporting memberships and nominate led to there being a significant Puppy voting block. Correia’s withdrawal from the Hugos has led to a decline in that block. More overtly Sad Puppies such as Sarah Hoyt have overtly stated that spending money on such memberships is a waste and essentially giving money to people the pups dislike.

In short, the immediate cause of the Puppy Kerfuffle has gone. The Sads have ceased to be a factor. However, that doesn’t mean others may not try similar antics in the future.

Of the rules above, I think the top two effectively stopped the Sad Pups – it’s just the effect was not immediate nor was it obvious to the organisers of Sad Puppies 1,2 & 3 what would occur. The effort of a slate, combined with membership costs, combined with the likelihood of a humiliating loss to No Award was enough to make the whole Sad Puppies campaign as a slate an unattractive option. 5-6 and EPH sealed the deal.

Longer term 5-6 and EPH(+) also mean that if another naive slate campaign arises from some other quarter, the damage done while that campaign follows the same cycle will be much less.

That’s all great as far as it goes. The 2017 finalists were rich and varied and good in all categories but…

The griefers are still there and their motives are not the same. The Rabid Puppies managed to get some griefing-style and promotional style nominees on the ballot. That impact was diluted by quality works but it was still present. The griefers don’t need to win to feel that their behaviour has been rewarded. Their goal is not a Hugo nomination but to create ill-will and to make people deal with their crap.

So 3SV is the Answer?

As things stand the only option on offer to deal with the griefing element is 3SV. There is no viable alternative that I’m aware of. A strong admin role during the nomination phase could theoretically remove griefing nominees but that idea is a non-starter: Worldcon/Hugo admins do not want that power and the consensus I’ve seen from Hugo voters is that such a proposal would never pass.

3SV is more acceptable because it passes that strong-admin style decision to the voters. In doing so, it potentially deals with other issues such as eligibility questions.

But, does it actually do the job?

It certainly would be another check against slate voting but I really think that is a solved problem. The issue is does it deal with griefing?

Nominally it does, in so far as it can prevent offensive works becoming finalists. However, does that solve the problem? Put another way which of these is the actual problem with the griefers:

  1. Works nominated by griefers becoming finalists?
  2. Hugo voters having to deal with works nominated by griefers?

3SV is a barrier to 1. but by its nature it makes 2. more feasible.

3SV will give voters an opportunity to reject specific works/nominees from the top 15. They won’t know how many nominations those nominees will have got or their ordering.

Now to get in top 15 for Best Novel in 2017 took about 200+ votes. A not insubstantial barrier but obviously significantly less than it takes to get an actual nomination. In less popular categories the number needed to get in the top 15 is substantially less – in many it is less than 100 and in some around 30 to 40.

So what, you might think, these nominees won’t be finalists and get no bragging rights or special status. I think that is missing the point. Works with abusive or insulting titles (for example) could be more easily gamed into the top 15 and by doing so griefers will get their psychological reward of Hugo voters acting to vote those works down. This may seem like an extraordinarily petty motive but the existing motives for the Rabid to spend more effort getting finalists on to the ballot are no less petty.

Appearing on the initial list will get more attention than appearing on the post-ceremony top 15 list. Although the two lists will contain the same works, the initial list is a list of potential winners in a way that the post-ceremony list isn’t. For the purpose of Castalia House’s publicity stunt motives, it may be more than sufficient to encourage nominations. For the same cost as getting one nominee as a finalist, the griefers can get multiple nominees on the initial list.

But ‘can’ is not the same as ‘will’ and its heart this is a psychological game rather than a procedural exercise. Are people like Vox Day or other people who might act in bad faith more motivated by the thought of gaining a single finalist and less motivated by a system that would deny them that? In which case 3SV could be a success. However, I think that they are primarily motivated by a desire to cause upset and dissension – in which case I fear that 3SV would simply a way of lowering the bar for their antics at precisely the time when they may have fewer resources to engage in them (where ‘resources’ are voting members of Worldcon).

Yet maybe that whole line of reasoning is a mistake. Acting in any diretion solely on the basis of the likes and dislikes and strange motivations of trollish-ideologues is, arguably, an error itself. A better approach is to place your own interests first and make those paramount. I think EPH is an example of that because I think it is a good thing in itself – yes, the Puppy antics gave the impetous to introduce it but it is defensible as a thing in its own right. For those who DON’T think EPH is defensible in its own right, the argument still holds – you’d be right to oppose it regardless of whether it was an effective vaccine against hydrophobia.

So is 3SV a good thing, in a circumstance in which there are no griefers? It does possibly pass questions of eligibility of borderline SFF works to the membership but, I’m not sure it will be an effective tool in that regard. Otherwise – does it do anything useful? I’m genuinely open to suggestions there – I’m not sure it does but that maybe just that I haven’t thought it through entirely.

Advertisements

22 comments

  1. Nicholas Whyte

    Works with abusive or insulting titles (for example) could be more easily gamed into the top 15 and by doing so griefers will get their psychological reward of Hugo voters acting to vote those works down.

    Exactly.

    Liked by 2 people

  2. Standback

    I think it’s a procedure that makes sense for the problem it’s trying to solve, but will have some very serious effects on the award as a whole, and might present some pretty disastrous side-effects and vulnerabilities.

    We get screening for griefers. That’s good and may be a necessary defense mechanism.
    But the price we’re paying is an additional step in an already-complicated process, which adds both difficulty for the voters, and potentially a critical, possibly-toxic feel to the awards themselves.

    Difficulty for the voters, because they have less time to nominate and/or consider the shortlist (OMG, guys, voting on the shortlist is SO. MUCH. READING, it is insane), and because you’re now asking them to make an informed decision about a new element (“So, this piece discussing pedophilia in the SF community; this piece discussing harassment by Requires Hate — do you think these are valid submissions for the shortlist? Take your time; go research; I’ll wait right here”). And a shift in tone, because suddenly you’ve got a “nix something you don’t like” option, and you literally cannot activate that option — or even try to activate that option — without that correlating to public campaigns, which correlate to outrage and toxicity.

    (Just as an example, I’ve seen many people discussing the problematic treatment of gender in Too Like The Lightning, and stating they would be placing it under No Award for that reason. I’m all for those discussions; they’re important. But can you imagine if there was a brief 3SV window and people — even just a small group, with the best of intentions — tried to get it stricken off the shortlist then? Held a public campaign trying to get TLTL 3SV’d, whether or not they were successful? It would be awful.)

    So: 3SV is a huge, award-changing modification. I’m very apprehensive about it, and I don’t think that being singed by Beale — and having weathered that, strengthened the system, and are now pretty much past it — is justification for something so drastic.

    The other thing is: 3SV opens up a huge new vulnerable flank. Yes, it protects the shortlist, which until now is all the flank we had. But it does that by creating a new flank, a pseudo-longlist, which is bigger, much easier to break into, and therefore much more vulnerable.

    If griefers can muster the votes for an entry or two on the shortlist, they’ll be able to capture lots of slots on the longlist. In fact, griefers who can’t muster the votes to hit the shortlist, griefers which we otherwise might be blissfully oblivious too, will be able to capture lots of slots on the longlist.
    If divisive items can promote bitterness and toxicity, imagine how many more divisive items we’ll potentially have with a longlist.

    It’s a lose-lose situation.
    Either 3SV usually goes smoothly and without rancor — and then we have, de facto, a longlist, which is more vulnerable than our shortlist ever was. We’re also wasting a few extra weeks every year on something that isn’t helping that year.
    Or it doesn’t, and then all we’ve done is move the battleground. If we assume a 3SV battle is swift and decisive, that might be an acceptable fix — but dedicated griefers are going to adapt, and I don’t think we want to see how that pans out.

    In fact, I’ve previously discussed a “Hornet’s Nest” approach to attacking 3SV. It goes like this:
    – Prod some hopefuls from various communities, in SF or even outside of it, to take aim at the Hugo spotlight.
    Some of those will be successful enough to be listed in the 3SV phase.
    – When they are not shortlisted, portray this as spiteful action of Hugo voters (whether it is or not)
    – Outrage ensues; each participant goes back to their home community to rally some allies against the awful, exclusionary Hugo fanbase.

    This isn’t hard. This isn’t hard. It’s just looking at the new system 3SV creates, and seeing where it’s vulnerable.

    That’s how I see it. I’m worried about this one, and unless somebody has some very persuasive reassurances for me, I’m gonna stay worried.

    Liked by 3 people

    • Cora

      “Too Like the Lightning” is another good example, since it is very much a love it or hate it book, so it could easily be knocked off the ballot, if 3SV were implemented. And if we ever get a critical mass of Sharkes at the Hugos (who seem to ignore them up to know), “A Closed and Common Orbit” might be another casualty of 3SV. And I’d say that both would be a loss to the Hugo ballot.

      Liked by 1 person

      • Andrew M

        I’d take it that the criterion for voting down would generally be agreed to be stronger than just ‘does not deserve an award’, and as I understand it the Sharkes just feel ACACO does not deserve an award; they aren’t offended by its existence.

        But I can think of lots of things there would have been campaigns to vote down (probably not successful, but they would have stirred up conflict anyway). Blackout/All Clear (because it isn’t a book, and because people were clearly voting for it on name recognition alone, since it so obviously had no merits of its own. Yes, people said that). The Wheel of Time (not a book, again). ‘Shadow War of the Night Dragons’. The Garcia thing. ‘F*** me, Ray Bradbury’. Laura Mixon (condemned as being as bad as the slates by a prominent commentator). Etc.

        Liked by 2 people

  3. Cora

    I see similar problems with 3-stage voting.

    Hugo reading and voting already is a lot of work (and even more this year because of more nominees, most of which were actually good, and the new series category) and this adds even more semi-finalists to evaluate. And not everybody is as plugged into the genre conversation as I am. Evaluating the nominees was a lot more difficult for my Mom (though she instinctively recognised the puppy poo for what it was, even though she’s only vaguely aware of what’s been going on). 3SV would have been difficult for me and impossibe for her.

    And while no one will mourn Stix Hiscock, JCW and Safe Space as a Rape Room, if they get kicked off the shortlist, there are less clearcut cases. If 3SV had been in effect in 2016, we would have lost Chuck Tingle for example and that would have been a pity. Also what about edge cases like Jim Butcher or Sebastien de Castell, who are not puppy poo, but wouldn’t have made the ballot without slates? There’s also the problem of works which are legitimately on the ballot, but which you nonetheless dislike and no award during the final voting. For example, I hated Seveneves and Neptune’s Children and no awarded both, but they were legitimate nominees. Should I have downvoted them anyway? What about works that you think are miscategorised, e.g. Andy Weir for the Campbell, when the self-published version of The Martian came out in 2012 or the Writing Excuses podcast and the filk CD in best related work?

    3-stage voting is a huge can of worms and no one I want to open.

    Liked by 2 people

    • camestrosfelapton

      Moira Greyland’s piece is another case – I placed it below No Award and the Rabid motives for nominating it were obvious & malicious but how would it fare with 3SV?
      Not so sure I’d have felt as OK voting it off at that stage.

      Liked by 1 person

      • Cora

        Yes, the Moira Greyland piece is another edge case. Clearly nominated by the rabids for trolling purposes and a hugely problematic piece, but Moira Greyland is also a survivor of sexual child abuse.

        Like

  4. Matt Y

    I can see the reason why people want it and understand the arguments or it but I just don’t see that it’s needed. Those that want to claim their work was Hugo nominated still would be able to, and it’ll still probably show up in a wiki page that it was nominated and then removed. It just seems like engaging those looking to troll the awards rather than establish reasonable protection against slate voting (like EPH). Plus it already seems like the Hugo voters have a way of expressing that a work or editor was unworthy, by voting them below No Award. I just don’t see the added value of 3SV.

    Like

  5. Andrew M

    This is a side-issue, but I think it’s worth mentioning; I feel your analysis of the kind of things the RP slate nominated is too simple. There is at least other thing:
    – Things that fit the original Puppy manifesto of ‘popular but neglected’. (Butcher, Anderson, King, Brown, de Castell. Weisskopf, indeed, whom Day continued to nominate despite apparently not really liking Baen very much.)

    There is possibly also a fifth:
    – Things the slate compiler happened to like. (Supernatural, perhaps.)

    Nothing really turns on that, but something does turn on the fourth. People say ‘if they nominate obvious rubbish, self-promotion or abuse, we can vote it down immediately. And if they nominate popular stuff that might have been shortlisted anyway, what’s the problem?’. But they might nominate stuff that’s OK, that you wouldn’t want to vote down sight unseen, and yet is not outstanding, not the sort of thing that really deserves an award. And a lot of such things getting shortlisted still skews the ballot and shuts more worthy things out.

    Liked by 2 people

  6. stevejwright

    Well, here’s how I see it:-

    What I want to see on the Hugo final ballot is the “best” six (or five, or whatever) works of SF in a given category, for the values of “best” defined by the (admittedly amorphous) choices of the nominating Worldcon members.

    What I’m actually getting is the “best” five (or four, or whatever-1), plus one slot marked “Reserved for trolling and rampant self-promotion by an objectionable griefer”.

    Do I want this? No. Is it going to get better by itself? Almost certainly not. (Bear in mind, this is a guy who is keeping up a running feud with John Scalzi over an incident which is old enough to start worrying about its GCSEs; he does not let things go. Besides, the longer he gets away with it, the likelier it gets that some other troll will get a gang together and start reserving a finalist slot for their own use.)

    So. Something needs to be done. What? EPH+ and 5/6 don’t address this particular issue – they have mitigated the initial slating problem, true (I think we can regard that as a solved problem, at least for the moment), but they don’t work on this. If I’m to get what I want (“best” six works on the final ballot), somehow or other the spoiler candidates must be weeded out beforehand.

    How is this to be done? Direct action by the administrators has been suggested, and generally rejected, because a) it changes the character of the Hugos from a straightforward popular choice to a (partially) juried award, b) it opens up the administrators to charges of partiality and lack of transparency, and c) the people who actually do the grunt work of administering the Hugos have largely said they don’t want it, because of a) and b). So, if we can’t have the Hugo administrators taking out the trash, someone else has to do it. Who? The best bet seems to be… the membership as a whole. Hence, three step voting, where we get to look at the possible finalists and are asked “are you sure about all of these?”

    And, of course, three step voting is problematic, in all the ways that have been discussed – though maybe not quite so problematic as people think, provided the bar for rejection is set high enough. There’s going to be a lot more diversity of opinion over Too Like the Lightning than there is over, say, “Alien Stripper”, and it’s unlikely (though possible, I grant you) that TLtL would garner enough downvotes to get rejected outright. But, yes, it’s unwieldy, it introduces complications, it has potential for abuse – all the other stuff people have said about it is probably true.

    But.

    The alternatives seem to be “do nothing”, with guaranteed bad results, or “kick the problem over to the admins”, which won’t work because the admins won’t play ball. So, of the two available options, I will take the one that’s potentially bad over the one that’s definitely bad. The alternative, it seems to me, is for someone to come up with a system that will weed out the spoiler material without having the disadvantages of 3SV. I’d definitely support that, if someone can come up with one. But until that happens, in this imperfect world, I’m for the least imperfect available option.

    Liked by 1 person

    • Andrew M

      Is it going to get better by itself? Almost certainly not.

      I’m not sure of that. Remember, VD did not urge his minions to sign up this year. I mean, he explicitly said ‘I’m not urging you to sing up this year’. His partial success in nominations was possible because he had a lot of members left over from last year. I think it’s perfectly possible that this is the last year he will have any impact. (He can go on announcing a list, if he likes. But only three people will vote for it.)

      If the abuse continues and this is the price of stopping it, well, perhaps. But if the abuse stops, but this disruptive procedure is now on the books and we’re stuck with it, that seems to me unreasonable. (Yes, I know that strictly speaking we aren’t stuck with it; but changing the rules is always a laborious thing, and will we want to do it again?)

      Like

      • camestrosfelapton

        Yes.
        IF the rabids continue to enroll as members in the same numbers as 2016, then 3SV serves a role to stop Griefing-Works getting into the top 6. BUT
        IF Rabids stop enrolling entirely then 3SV will largely be unnecessary but maybe not itself problematic. HOWEVER.
        IF Rabid numbers REDUCE but aren’t zero then 3SV is an opportunity for them to continue with different but related tactics. It is this last option that concerns me i.e. 3SV inadvertently sustaining what it aims to deal with.

        Like

      • Mark

        Valid point – his minions appear to believe that success is whatever he says it is, so if he can point to getting #14 on the list as a “win” then it doesn’t matter that no-one else sees it that way, they’ll keep on thinking they’re winning.

        Liked by 1 person

  7. Lurkertype

    The problem I have with 3SV, without even considering griefing, gaming, etc. is timing. Holy crap, ANOTHER stage to submit things in the already short window we have to ponder the short list? AAAGGGHH!

    And the last thing we need is more campaigning, arguing, etc.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s