The Three Laws of Reviewbotics?

I’m circling a subject and theseposts are all thinking out loud. I’ve defined reviews and criticism very broadly because while there are different functions, intents and processes there are no hard boundaries between the roles I’ve been discussing.

Pondering

I can see that collation (such as editing a magazine or an anthology) and aggregation (such as reader ratings, aggregated reviews, Amazon rankings) are not what we normally think of when we say ‘reviews’ or ‘literary criticism’ but they play an important part in the discourse. Goodreads and Amazon, in particular, show the interplay with many of the issues with reviews as an aide to consumers, as a normative pressure on genres, and as a minefield where objective data about subjective evaluation can be mistaken for deep ‘objective’ truth about intrinsic qualities of books*.

I did briefly touch on algorithms in the post on objectivity. As a commenter suggested (sorry – I can’t find the comment) not all machine learning algorithms have that quality of repeatability. However, let’s assume I’m focusing on the ones that do and which are relatively reliable** Algorithms used by booksellers will become more sophisticated based on buyer’s past purchases and represent another function that aides consumers but also provides normative pressure on people’s reading. Likewise, they share that same element of trying to bottle objective data from subjective choice (with varying degrees of success) but success & apparent objectivity may hide deep biases.

Put all these things into a big cauldron and call it a discourse – a roving discussion between readers and writers and writer/readers and editors and everybody.

What expectations should we make of the elements that contribute to that discourse? Well, I’ve loaded that question already by putting an emphasis on the collective discussion.

1. We should look at reviews (and related review-like things) in how they contribute to the broader discourse – including variations on that discourse within particular communities.
2. We should also consider the more immediate ethical consideration of whether the review (or related review-like things) harms a person in some tangible way.

Those two things are not unrelated – a review that used ethnic slurs at a writer undermines the wider discourse and harms an individual.

There are limitations to the second category. While ‘do no harm’ sounds ethically straightforward, in reality, reviewers can’t know the individual circumstance of every writer. Even a positive review may turn off a potential reader and hence lose a writer a customer. General statements that are intended to be helpful may lead to a work being perceived as sitting in one subgenre or even the writer being typecast as writing only within that subgenre.

If reviews are read and people pay attention to them then the reviewer wields a kind of power by helping shape the discourse. That power might be tiny but it still exists and the use of the power entails the use of influence over other people. The only ethical purity at an individual level is not to write a review (or compile an anthology or edit a magazine or give a book a star rating on Amazon…) But that purity fails at a collective level because it is an abrogation of our capacity to do good.

The point being, that by thinking only of individual harm we can become impotent in our capacity to do good. As social beings, our exchange of ideas is a good in itself and further our enjoyment and edification through reading and exchanging stories is a good in itself and further the diversity, freshness, variety and multiplicity of perspectives in that exchange are good in themselves.***

A sketch of some “rules”

So here’s maybe a start for the hyper-critic oath (‘hyper’ because I’m overthinking this and ‘critic’ because ‘reviewer’ doesn’t work for the pun).

• First, do no obvious harm. Don’t ever slander a writer. Avoid attacking them personally, even indirectly [that’s not always possible because writing is to varying degrees an extension of the self. In addition, some texts themselves are INTENDED to be harmful to others (I’ve reviewed many here over the years) BUT while we can all think of exceptions the norm should be to review texts, not people.] This does not mean treating all people the same – if you knew that somebody was currently in a vulnerable emotional state, then maybe reviewing their book isn’t a great idea. The flip side of that is you can’t reasonably tailor reviews around what a writer you don’t know might be feeling. And obviously don’t use slurs, stereotypes or language which we know to be harmful – such as overt racism, sexism etc. In an inequitable society, some people are more vulnerable to others and if we KNOW that we have to be mindful of that while bearing in mind the points below as well.
• Second, add meaningfully and positively to the discourse. By ‘positively’ I mean improve the discourse and I don’t necessarily mean ‘only say positive things’. To some extent that’s a request to write interesting things! But it is also a request to reduce the impact of systemic bias. Every reviewer has a degree of bias in their reviewers which I’ll talk about more below – some of that we can reflect upon and remedy (e.g. subconscious or less overt racism in reviews or what works are chosen for review) but a lot of it we can’t see precisely because we have those biases. Where such biases are particularly problematic is when many people have the same biases and they act together to create a broader social bias. Reviews that highlight writers from marginalised groups disrupt those collective biases but a variety of perspectives in general also helps.****
• Third, be honest but with the limitations of the points above. Being honestly racist is no virtue and being dishonestly nice about a work you hate is no virtue either (it may even be corrupt). A reviewer has an ethical relationship with the reader of the review as well as the writer of the text being reviewed. Because the reader is more faceless, anonymous and impersonal, it is easy to discount the ethical connection with them when compared to the writer, who is easier to identify as an individual. Honesty is a way of balancing those ethical tensions. Honesty here included being honest about your potential flaws and biases, as well as the limitations of whatever approach you are using (in the case of aggregating data that’s rather like a section on an academic paper identifying the limitations of a methodology.

Gosh, the joy of not knowing what you are going to write before you start writing is tempered by the weight of wondering whether I believe any of the above! Like I said at the start, this is thinking out loud – Future Camestros reserves the right to call Past Camestros a moralising nincompoop.

Bias and gatekeeping

Every approach has a bias. Every approach involves gatekeeping. Variety and diversity in the overall discourse limit the impact of both. Self-reflection and being aware of how an individual reviewer or process fits within a bigger picture can reduce the harm done by individual biases.

We shouldn’t despair that bias is unavoidable and we shouldn’t give up on reducing the harm of our biases just because the task is fractally complex.

Every editor and every review gatekeeps marginally by picking what to review. One way of eliminating that is by attempting to review most things in a particular area – that avoids some aspects of gatekeeping and introduces others.

Actively seeking out texts from groups who are underrepresented in the wider discourse undermines the gatekeeping of others BUT it makes the individual reviewer/editor a potentially more powerful gatekeeper over the targetted group. Personal integrity and morality should limit the abuses of the power gained (the first rule above) and have multiple voices limits the power by opening many gates (the second rule).

Bias can be personal or systemic. Here I’m not thinking so much of overt prejudice, as those things that impact what we do in ways, that if asked, we would say we did not intend. That first rule I added can itself add biases – for example, I’ve seen an organisation avoiding stereotypes of indigenous people in a specific way that led them to essentially erasing indigenous people completely from materials they were producing.

Self-reflection and criticism at an individual level, pluralism overall reduces bias. People taking different approaches to reviews can temper the biases those approaches may have. Positive reviews add to the mix but have potential biases in what does not get reviewed. Negative reviews may often misrepresent works because it is neccesarily hard to appreciate a work you didn’t enjoy but they still play a role in informing the reader – who has their own capacity for critical judgement. Multiplicity of approaches tempers the potential biases.

Aggregation of data can amplify biases but aggregation of data can also reveal biases that people were not aware of. Again, honesty and self-reflection and criticism or criticism all have a role to play.

An artfully crafted conclusion

…is not what I have here. I just ran out of things to say and I want to eat some toast. There’s strawberry jam in the cupboard and my mind has started wandering.

🙂

---

*[Put another way – there may well be deeper truths hidden in that data but the mere aggregation of the data doesn’t mean that such truths are there. They need to be found and established and they might not be what people expect.]

**[‘reliable’ here doesn’t mean ‘correct’ it just means past behaviour will predict future behaviour. A stopped clock is reliable, as is one which is 5 minutes off the actual time.]

***[Obviously with a big “in my opinion” here]

****[Looking at this point now, it sounds like a dig at people who write reviews that just say what everybody else says. I’m not really saying that – adding volume is still adding something (aggregation matters). However, there is still value in fresh takes and interesting insights. Also, we should be mindful that there’s no need for everybody to pile on with bad reviews of appalling books.]