Category: Statistics
Reviews and discourse
This is thinking out loud and a bit long and waffly in a field beyond my expertise. Approach with skepticism 🙂
Reviews and literary criticism are not one and the same thing. Criticism (in its literary sense) is a tool of the reviewer and at the same time, reviews can be seen as a subset of criticism. In its wider sense criticism is intended to examine text and shed light on what those texts do. Reviews are more overtly functional and are often characterised as being there to inform potential consumers.
Of course we should be first of all suspicious of this distinction and also the description of both reviews and criticism. I often write things entitled as ‘reviews’ not to inform potential readers/viewers but to share my feeling and experiences as a consumer of that media. In particular, reviews with spoilers or which really require the reader to have experienced the book/movies/tv-show are not written with potential consumers in mind but rather people who have already consumed the media. Much of reviewing that is available online is better described as ‘commentary’ – more akin to post-games discussion of sport matches or analysis of news stories. It may be less high-brow than what would be recognised as literary criticism (and less informed by models of literary criticism) but it is closer in kind to it than ‘reviews’ in the sense of information for potential consumers.
Yet another role for reviews and criticism is improvement, change or the establishment of norms. Editorial reviews (at a high-level – I don’t mean proofreading) are one example but it is something that can be seen in literary criticism and in more general reviewing. Identifying problems in texts or discussing whether a text fits within a genre form parts of wider discussion about what counts as being a problem in a text or what defines a genre.
To clarify for the purpose of discussion I’ll split things into various roles:
- Reviews for the purpose of informing potential consumers (not unlike product reviews of goods).
- Post-consumption sharing of experiences.
- Criticism for the purpose of understanding a text.
- Reviews/criticism for identifying problems or potential improvements in a text.
Each of these form part of the wider discourse within a community that has a shared engagement with texts. As this is a broad, multi-faceted and diverse discourse that ranges across multiple venues, there are no hard borders between those roles. A review ostensibly for helping people what to read next may incorporate particular norms about the genre because having norms provide a way of judging and reporting on texts. Likewise analysis of what is going on in a story for its own sake can encourage somebody to read a story (or ensure they stay well away from it forever!)
Identifying problems with how race, ethnicity, nationality, religious belief, gender, sexuality or disability are represented in a text would fall into the fourth point on my list but also fits with the first point on my list and will often be derived from the second point on my list. Lastly, a discussion of such issues may revolve around the third point on my list I.e. what is actually going on in a text and what kind of representation is being used.
I’m well out of my philosophical comfort zone by this point having strayed out of the analytical and insular and into the phenomenological and continental but let’s persist.
Common to all is the sharing of personal experiences with others. Put another way, reviews and criticism bridge subjective experiences to intersubjective community understanding.
How we experience a text (story, film etc) is something we can examine and discuss and it is something that we can analyse, something we can find patterns with and it is also something where we can aggregate data. Aggregation and quantification of subjective data does provide a way of looking at subjective experiences using tools designed for “objective” data (and see the previous essay for what I mean about “objective”). It is another way of looking at shared experience.
Related to that is the role of anthologies and magazines. Both are traditionally and important part of this wider discourse about the nature of science fiction and fantasy. By collecting stories together and present a set of stories as examples of the genre and as examples that are of at least some minimum quality, anthologies are also a form of both review and criticism that wittingly or not describes possible boundaries of the genre. To call an anthology a work of ‘criticism’ may sound odd but it covers many aspects of the points above (e.g. a ‘best of the year’ style anthology is normative by presenting examples of some standard of ‘best’ and also a way of guiding consumers to stories they may like and also represent some of the subjective reaction of the editors/compilers).
Similar points can be made about awards and competitions and here the overt nature of a discourse becomes clearer. With fan awards discussion and shared experience is an important part of the process. Juried awards can also engender debate and discussion and I’d argue the most interesting ones are the ones that evolved this aspect (for example the Clarkes by virtue of the Shadow Clarkes have become a more interesting award).
Where are you going with this Camestros! I hear you shout (assuming you’ve read this far). OK, OK, I’ll stop waffling.
My point is, if we are to discuss what reviewing should be like and what kinds of reviews and reviewing activity people should be doing, we have to consider it against this wider discourse. It is the big broad discussion that is the important thing – along side the health and welfare of individuals.
Objectivity and stuff
I wanted to write about some of the interesting things people have been saying about reviewing but part of my brain obviously wants to talk about reason and evidence and those sorts of things. I guess I haven’t done much of that this year in attempt to look less like a philosophy professor.
Anyway – objectivity! The thing with objectivity as a word is that we (including myself) use it in a way that implies various things which maybe aren’t really part of what it means. Objectivity carries positive connotations and connotations of authority in contrast to subjectivity. Those connotations suggest impartial judgement and a lack of bias. That’s all well and good – words can mean whatever a community of users want them to mean but I think it creates confusion.
Here is a different sense of ‘objective’ – to say something is objective is to say that two people can follow the same steps/process and come up with the same answer reliably. Maybe we should use a different word for that but such processes are often described as ‘objective’ because they clearly contrast with subjective judgement.
The thing is that meaning does not in ANYWAY imply a lack of bias. Lots of systematic or automated processes can contain bias. Indeed we expect there to be biases in, for example, processes for collecting data. More extreme examples include machine learning algorithms which are inherently repeatable and ‘objective’ in that sense (and the sense that they operate post-human judgement) that nonetheless repeat human prejudices because those prejudices exist in the data they were trained on.
Other examples include the data on gender disparity in compensation for Uber drivers – the algorithm was not derived from human prejudices but there was still a pay disparity that arose from different working patterns that arose from deep-seated social disparities.
However, there is still an advantage here in terms of making information and data gathered more objective. Biases may not be eliminated but they are easier to see, identify and quantify.
Flipping back to ‘subjective’, I have discussed before both the concept of intersubjectivity (shared consensus opinions and beliefs that are not easily changed) as well as the possibility of their being objective facts about subjective opinions (e.g. my opinion that Star Trek: Discovery was flawed is subjective but it is an objective fact about the universe that I held that opinion).
Lastly the objective aspect of data can be mistaken for the more subjective interpretation of the data. In particular the wider meaning or significance of a data set is not established simply by the fact that the data is collected reliably or repeatedly.
Consider another topic: IQ. I’ve discussed before aspect of IQ and IQ testing and much of the pseudoscientific nonsense talked about it. Look at these two claims between Roberta and Bob:
- Roberta: My IQ is higher than Bob’s.
- Roberta: I am more intelligent than Bob.
The first statement may be an objective fact – it is certainly a claim that can be tested and evaluated by prescribed methods. The second statement is more problematic: it relies on opinions about IQ and the nature of intelligence that are not well established. The objectivity of the first statement does not establish the objectivity of the second. Nor does the apparent objectivity of the first imply that it does not have biases that may also impact wider claims based upon it.
Reading Peterson 11 – Notes & Facts & Hypothesis
Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7, Part 8, Part 9, Part 10, Part 11, Part 12,…
There’s no shortage of notes in Jordan B Peterson’s book 12 Rules for Life but that doesn’t mean every assertion related to facts is referenced. Also, when references are used they aren’t always tightly associated with the argument. Take this for example from chapter 2:
“This is perhaps because the primary hierarchical structure of human society is masculine, as it is among most animals, including the chimpanzees who are our closest genetic and, arguably, behavioural match. It is because men are and throughout history have been the builders of towns and cities, the engineers, stonemasons, bricklayers, and lumberjacks, the operators of heavy machinery.” – Peterson, Jordan B.. 12 Rules for Life: An Antidote to Chaos (p. 40). Penguin Books Ltd. Kindle Edition.
Now there is a lot wrong with that statement factually but the right reference here, if this was an academic essay, would be to a source discussing historical patterns of employment. Peterson instead links to some modern labour statistics here https://www.dol.gov/wb/stats/occ_gender_share_em_1020_txt.htm The tables do use the term ‘traditional occupations’ and ‘non-traditional’ based on proportions of women involves but this is ‘traditional’ in a very loose sense and includes “Meeting, convention, and event planners”. My point here isn’t that the table is wrong of even questioning gendered-roles in employment – just that a lot of references are weak in this fashion. It is vaguely related but not neatly tied to Peterson’s argument.
(This is quite long – so more after the fold)
Looking at some crowdfunding data
I’m mainly just curious how such things work but I picked on data from a Go Fund Me campaign that I know people might be morbidly curious about.
The site gives a list of donations made with the amount and how many days ago the donation was made. Doing some minor spreadsheet wrangling, it is fairly easy to turn this into graphable data. The only departure from literal truth is I used the order in which the donations are listed to spread out the data points more evenly across each day of the campaign – so the smooth growth within each day is just to make the graph easier on the eye (the raw data would just give a big vertical chunk of points).
Compared with the fundraising goal the graph looks like this:
If we assume a growth rate of $20 every three days than this campaign should reach its target in about 1317 days or about three and a half years. Of course, events may change that.
Felapton Towers Scoop – How Numbers are Disappearing
Look, there’s just some breaking stories that you can only read here thanks to the deep investigative journalism that my crack team of journalists do. In this case – plugging two digit numbers into Google n-gram.
I don’t know what I expected the graph to look like but apparently peak numbers-in-books was sometime in the late 1980s after which the bubble burst plunging books into a deepening two-digit-numbers-as-words recession.
The decline is present in both the US and the UK:
And is present for three digit numbers also:
OK but what about single digits? I hear you ask. Surely the blue-chip of the numerical world are still going strong? Nope. Number 1 obviously has been number 1 for sometime but at the turn of the millenium, even it felt the decline.
Four digit numbers? That’s a whole other ball game. The four digit market is dominated by YEARS. So peak 2000 was shortly after the year 2000 (this impacts two digit numbers a bit as well – 90 gets a bit of a boost in the 1990s).
Here is what I think is going on. The internet and the proliferation of software for sharing numerical data has created other avenues for publishing numerical data. Consequently printed documents with really large amounts of numerical data have become a smaller proportion of books published. Data sets are more likely to be made availbleas downloadable files (text, CSV etc) rather than as printed volumes.
A way to test that hypothesis would be to look at a corpus that was only FICTION. Changes in how data is published shouldn’t impact fiction! However, style habits may impact numbers written as digits – the normal prescription is that smaller numbers should be written in words. So, I’ll look at the number one hundred and twenty three as a test case – it should be written as digits normally.
Unfortunately…the results were inconclusive. When I clicked on the examples the “English Fiction” corpus was drawing from they were all NON-FICTION. Grrrrr Google giving me free tools to explore data to my hearts content and you make them not entirely perfect!
So, I can’t definitively tell you were the numbers have gone. Sorry.
You say ‘a-loomin-um’, I say ‘al-you-min-ee-um’, we both say ‘bunkum’
I resolved to not bother talking about Vox Day for awhile but circumstances compel me. The synergies of nonsense bind extreme nationalism, Trumpism, misogyny, creationism and antivaxxerism. It is always remarkable to see what apparently scientific studies the Alt-Right will quote as if gospel and which they will turn their selective scepticism too.
What is all this about? It is the old and thoroughly debunked canard that vaccines cause autism. The idea is rooted in two coincidences: an increase in the numbers of people diagnosed with autism (primarily due to better clinical descriptions of autism spectrum and increased awareness among doctors and the public) and the timing of when autisim symptoms are often identified at an age close to when early childhood vaccinations occur. Campaigners against vaccinations have been looking for a more substantial way of linking the two and one generic culprit has been ‘toxins’ in vaccines – i.e. various additives used in the manufacture of vaccines. For a long time the supposed guilty party was mercury, particularly in the form of thiomersal – a preservative used in some vaccines. However, studies linking the two were famously debunked and many vaccines didn’t use thiomersal or other mercury compounds anyway.
Of later the antivaxxers have been pointing their fingers at a different metal: aluminium – which is just like the metal aluminum but more British. ‘Aluminium adjuvants’Â are an additive to vaccine that use aluminium. Adjuvants are any substances added to vaccines whose role is to provoke an immune response (see here for a better explanation https://www.cdc.gov/vaccinesafety/concerns/adjuvants.html ). Tiny amounts of aluminium are added intentionally because the body’s immune system will react to the aluminium and it is that principle (which is central to the whole idea of vaccines) that has vaccination critics concerned.
Back to the study quoted. Vox Day is quoting from The Daily Mail:
http://www.dailymail.co.uk/health/article-5133049/Aluminium-vaccines-cause-autism.html
BUT….the Mail article is little more than a cut and paste from here:
https://www.hippocraticpost.com/infection-disease/aluminium-and-autism/
Which is an article by a “Chris Exley” who mainly writes alarming articles about the terrible things aluminium might do to you. Exley is quoting a study from Keele University which is available here:
http://www.sciencedirect.com/science/article/pii/S0946672X17308763
And that study was conducted by three people including…Professor Chris Exley. Who, conincidentally enough is on the editorial board of the journal the study is published in:
https://www.journals.elsevier.com/journal-of-trace-elements-in-medicine-and-biology/editorial-board
It is a long chain and yet oddly this is a rare case where the populist half-baked version of the study is alomost directly from the scientist involved.
Now I don’t know much about Professor Exley’s field, so I can’t really comment on the validity of the methods used. The study involved detecting aluminium in a very small number of samples of brain tissue from dead people who at some point in their lives had been disagnosed with an Autism Spectrum Disorder. There’s not much in the way of comparisons in the paper and I get the (perhaps mistaken) impression that the method is relatively new. The paper correctly concedes that “A limitation of our study is the small number of cases that were available to study and the limited availability of tissue.”
But take a critical look at the next step in the reasoning. Exley hedges what he says but Vox follows the dog whistle:
“So, the obvious question this raises is: how did so much aluminum get into the brain tissue in the first place? And the obvious answer is: from being injected with vaccines containing aluminum.” (Vox Day)
Of course a moments thought reveals that cannot be the answer. Most people do not have a diagnosed Austism Spectrum Disorder but most people are vaccinated. For Exley’s hypothesis to be correct there would need to be some additional factor, which Exley does describe in his media article:
“Perhaps there is something within the genetic make-up of specific individuals which predisposes them to accumulate and retain aluminium in their brain, as is similarly suggested for individuals with genetically passed-on Alzheimer’s disease.”
Well perhaps there is but Exley’s study doesn’t show that. More to the point, if this IS true then vaccines and aluminium adjuvants are irrelevant – we are encounter far more aluminium in our diets than we do from the tiny amounts we might get from vaccinations. Exley has zero reason to point at vaccines, indeed his speculation would imply that vaccines CANNOT be the main reason for larger amounts of aluminium in his samples because neccesarily bigger sources are more likely.
Exley appears to be trying to join two different healthscare bandwagons together: general concerns about aluminium in stuff (see his other posts) and antivaxxerism.
Is the study itself flawed? As I said, I don’t know but the connection the paper makes to vaccines has zero substance and no evidence from the study itself. That in itself should have raised red flags with reviewers.
…
In the past, I’d have gone to Science Blogs for some extra background on something like this but that venerable home of blogs has been wound down.
Luckily ‘Orac’ of Respectful Insolence has set up their own blog here https://respectfulinsolence.com/ and has a deep dive into Exley’s paper here:
Yup, it is as dodgy as somebody dodging things in a dodgy dodge. Orac points out the dubious funding source:
“The second time, I noted that he’s one of a group of scientists funded by the Child Medical Safety Research Institute (CMSRI), which is a group funded by Claire and Al Dwoskin, who are as rabidly antivaccine as anyone I’ve seen, including even Mike Adams. Among that group of antivaccine “scientists” funded by CMSRI? Anthony Mawson, Christopher Shaw, Lucija Tomljenovic, and Yehuda Shoenfeld, antivaccine crank “scientists” all. And guess what? This study was funded by CMSRI, too. Fair’s fair. If antivaxers can go wild when a study is funded by a pharmaceutical company and reject it out of hand, I can point out that a study funded by an antivaccine “foundation” is deserving of more scrutiny and skepticism.”
And it just gets worse from there. No controls, some tiny sample jiggery-pokery with the numbers and so on. Best read directly.
Blogstrology
It is only a tiny step from pointless science to pseudoscience and I’m thinking…it’s a rainy Sunday and my head hurts…
After my previous post on this topic, it occurred to me that I should check the profile of some other websites. I’d already identified that Vox Day’s blog was disproportionately Goat-Wolf-Rabbit. What about Monster Hunter Nation?
A clear Tiger-Goat-Cow blog. Cats do quite well at MHI in terms of raw numbers but not when compared against their general frequency.
Moving away from the right, how about File770?
Mike is running a Cat-Tiger-Goat blog it seems. Now note that the search method includes comments, so it may be the readers that have a thing about cats (this has been independently confirmed).
What do all three blogs have in common? GOATS.
[ETA – Rocket Stack Rank www.rocketstackrank.com is interesting because the animals mentioned would be more determined by their incidence in short fiction. Overall low frequencies and RSR has no presence on the otter or goose dimensions. Wolf-Rabbit-Cat blog – “Cat” strongly assisted by reviews of the works of Cat Rambo 🙂
Goat has a presence but is just shy of the top 3.
