A bad survey about the ‘Intellectual Dark Web’

This is an edited version of three Twitter rants from yesterday. It started as an off-cuff reaction but I was too far into it before I thought that it should be a blog-post rather than Tweets.

Stephen Pinker tweeted out a very weird bit of science theatre created by Michael Shermer.

Pinker has enough critical thinking skills that he should look at it with hefty scepticism…but obviously isn’t. It’s pretend science, using play-acting at science to refute what is obvious and ignores the core issues.

The “survey” by Michael Shermer (which should be a red flag in itself) was sent to 34 notable people associated with the label “Intellectual Dark Web” and asked where they stand on a number of issues. The survey was anonymous, so the views identified in the survey can’t be matched to the individuals asked. https://www.skeptic.com/reading_room/preliminary-empirical-study-shedding-light-on-intellectual-dark-web/

Each and every one of the people surveyed is a public figure who have made multiple public statements about politics and social issues. I don’t need an anonymous survey to find out what Andy Ngo or Sam Harris thinks, I can go and read what they say. And it is what they SAY that matters and what defines the IDW term not what they might privately think. If Sam Harris thinks he has warm & fuzzy liberal beliefs that’s nice but the whole point of the “dark web” label was the contrarian issues he promotes. Maybe Ben Shapiro secretly believes Global Warming is real and climate change is caused by humans. I don’t know but what matters is he propagandises the opposite. If an anonymous survey of the 34 “Intelectual Dark” Webbers reveals that their underlying views are more centrist and mainstream then that is not evidence that the public perception of their public positions is wrong. Rather it confirms a key point about the IDW.

The fundamental issues with the disparate group lumped together as the Intellectual Dark Web is that they are DISINGENUOUS about their politics. It’s not news that Jordan Peterson thinks of himself as moderate and reasonable. We knew that already. It doesn’t change that he (and Harris & Shapiro & Ngo & Quillette) frame and enable a perspective that bolsters the far right. The whole “we are the reasonable ones” is part of the schtick of the IDW. That they’ll boost that in an anonymous survey is, frankly, wank.

Let’s be sceptical as I’m sure Dr Pinker and Shermer would want us to be. Let’s take one conclusion Pinker raises from the survey: The members of the IDW are “concerned w climate” Let’s look at the survey: The survey agrees: “67% strongly agreed that global warming is caused by human actions (no one strongly disagreed)” So their you go! Hoorah! No, no let us be sceptical first. If this was GENUINELY true would it not be easily observed?

To the empircism-mobile! Here’s the output of the Quillete Climate tag https://quillette.com/tag/climate/ zoiks! A hefty TWO article, one concern trolling Greta Thunberg and the other saying people shouldn’t be mean to capitalism. Yes, Quillette is just one source but it is one that connects Steven Pinker on the one hand (who we can observe genuinely does advocate for action on Global Warming) with Andy Ngo on the other hand (who genuinely does have connections with the alt-right and violent far right groups) via Claire Lehmann (Quillette’s founder, fan of Pinker and one time boss of Ngo).

Yes, Steven Pinker himself has a better record on the of global warming but the issue he raised was to look collectively at the IDW and their media-organs. Broadly this is not a group trying to do very much about helping with the issue. And wow, think of the actually good the IDW could achieve given their actual audience. Whatever they may think of themselves, collectively they do have the ear of many on the right – exactly where climate change denial and bad science on the topic is endemic. You’d think these out spoken people might be busy being outspoken on a potential planet wide disaster.

It gets worse. The actual sample was only 18 not 34 people. Nearly half of the 34 didn’t answer. So when the survey says “67%” (the percentage favouring gun control and which believes global warming is real) actually means “12 people” That’s actually both more plausible and more wretched. Even if we accept that 12 of those IDWs think climate change is real, it says almost nothing about the group. Any one member of the original 34 people is a hefty 3% of the population being sampled and hence missing any one of them can have a large impact on the results. This is particularly true given that we already know that the label of “Intellectual Dark Web” is being attached to a group with a very broad range of views on many topics.

Shermer is assuming non-response to the survey is random across the traits being surveyed (i.e the 18 is a random sample of the 34). There is no reason to believe that and really anybody who is wants to seriously call themselves a sceptic should dismiss any general conclusion from the survey without substantial additional supporting evidence.

Indeed there’s good reason to assume that the 18 who responded is not a good random sample of the 34, just on the nature of the numbers. It is very hard with small numbers in a survey for the sample to be representative because one person makes a big difference. Shermer hides that by quoting percentages rather than raw totals but with small number percentages hide how few people he’s talking about. It’s not invalid to look at proportions with small sample sizes, sometimes that is all you have but there’s a point where 12 out of 18 is more informative than 67%.

We can illustrate the issue with the women who were surveyed. Of the 34 named people in survey associated with the “Intellectual Dark Web” 8 (24%) are women. In the survey 3 (17%) are women. So are the IDW 17% women (generalising from survey) or 24%? Obviously 24% is the correct figure but 17% is the equivalent of the the kind of survey conclusions Shermer presents. In fact any one woman listed is 13% of the IDW women, so one more woman answering makes a huge different to sub-sample of women. Any one person is 6% of the whole sample of 18 people!

Circling back to 67% claim. Again assuming everybody who responded is being honest (which I doubt) the survey actually found that 12 people of the 34 who were asked believed in gun control and the same number believed that global warming was real (which I’ll add isn’t saying much, some prominent sceptics will say global warming is real, just as many anti-vaccination campaigners will say they support vaccinations – it is the ‘but’ that follows where the issues lie). That might mean 67% or there about of 34 believe in gun control but a safer conclusion is no less than 35% do (12/34) and no more than 82% (28/34). Given how granular this data is, hoping the estimate is in the middle isn’t supported.

This is why I call it theatre. It is the wrong methodology applied badly. It illustrates methodological snobbery. Synthesising the complex views of a small group of people is exactly where qualitative methods work better. It is a domain where you need to put on your humanities hat and apply those humanities skills. Shermer is using sciencey film-flam by presenting a pointlessly anonymous survey and presenting the results as percentages as if there were proportions of the whole group.

Don’t get me wrong I absolutely LOVE applying basic quantitive methods to things and place where they don’t always make sense. It’s very much my hobby but even on this less than 100% serious blog I’d throw more caveats at better numbers than Shermer is using.

Cat psychology

A reader asks me:

“Hi Cam,
You are lucky that your cat can talk and is so ready to share his views. I never know what my cat is thinking. Can you share some of your experience with Timothy and give us all some insights into the inner lives of our cats?”

I’m always happy to help and I’ve compiled this chart to help you match your cat’s facial expression to their thought process. Obviously this is based on Timothy and your own cat maybe different.

The Right would rather men died than admit any flaws in masculinity

I shouldn’t read Quillette. For those unfamiliar with the Australian/International online magazine, it is part of that genre of modern political thought that could be called anti-left contrarianism, that covers various soughs from Steven Pinker to Jordan Peterson. Its stock style of article is shallowness dressed up as depth, utilizing the same style of misrepresentation of issues as the tabloid press but with longer sentences and a broader vocabulary.

Over the past few days it has published a couple of pieces on the American Psychological Associations Guidelines for Psychological Practice for Men and Boys. Now you would think that the stalwart defenders of innate gender differences would be happy that an influential body like the APA would be overtly recognising that men and boys have distinct psychological needs that require special advice for practitioners. After all, is this not the ‘moderate’ criticism of the rise of feminism? That somehow, men’s needs and men’s issues have been sidelined? Ha, ha, who am I kidding 🙂 The APA guidelines were characterised by MRAs, conservatives and the so-called “Intellectual dark web” as a direct attack on masculinity.

Here is one particularly stupid piece at Quillette that reflects the harrumphing style of response: https://quillette.com/2019/01/23/thank-you-apa/ The writer (a professor of psychology at North Dakota State University) either haven’t read the guidelines or is actively misrepresenting them.

However, a second piece is what actually caught my attention. It’s better written but also is attacking a strawman version of the guidelines: https://quillette.com/2019/01/23/how-my-toxic-stoicism-helped-me-cope-with-brain-cancer/

The writer describes how his stocial attitude helped him through a diagnosis & treatment for brain cancer and uses that to lambast the APA’s (apparent) criticism of stoicism in its guidelines. I, perhaps foolishly, left a comment on the piece. What follows is an edited version of my comment.

The piece is basically a strawman argument. It misrepresents what the APA guidelines say to imply that the guidelines have blanket disapproval for people acting stoically. e.g. Take the APA’s own article on the guidelines:

“It’s also important to encourage pro-social aspects of masculinity, says McDermott. In certain circumstances, traits like stoicism and self-sacrifice can be absolutely crucial, he says”

https://www.apa.org/monitor/2019/01/ce-corner.aspx

In the guidelines themselves, the word “stoicism” appears only twice and in neither case is a blanket condemnation of it. Once is in relation to difficulties SOME men have forming emotional bonds with other men:

“Psychologists can discuss with boys and men the messages they have received about withholding affection from other males to help them understand how components of traditional masculinity such as emotional stoicism, homophobia, not showing vulnerability, self-reliance, and competitiveness might deter them from forming close relationships with male peers”

American Psychological Association, Boys and Men Guidelines Group. (2018).
APA guidelines for psychological practice with boys and men

And the other connects with a broader health issue of men not seeking care that they may need:

“Psychologists also strive to reduce mental health stigma for men by acknowledging and challenging socialized messages related to men’s mental health stigma (e.g., male stoicism, self-reliance). “

American Psychological Association, Boys and Men Guidelines Group. (2018).
APA guidelines for psychological practice with boys and men

Neither example relates to be being stoical in the face of medical diagnosis but rather social pressures that mean some men (no, not ALL men) don’t seek care that they need (including for physical ailments) because of a misguided belief that they have to battle through by themselves.

The writer’s example is NOT an example of the case the APA guidelines were addressing. The writer sought out medical care, received a diagnosis and stuck with treatment. The writer self-described actions are the OPPOSITE of what the guidelines are discussing — they show a man taking their health seriously and SEEKING HELP. That’s good and healthy but many men aren’t doing that and as a consequence are dying of treatable diseases

As guideline 8 points out:

“For most leading causes of death in the United States and in every age group, males have higher death rates than females”

American Psychological Association, Boys and Men Guidelines Group. (2018).
APA guidelines for psychological practice with boys and men

At least some of this is due men not seeking out healthcare they need:

“Between 2011 and 2013, men’s mortality rates for colorectal cancer, a generally preventable disease with regular screenings, were significantly higher than women’s, suggesting that many men do not engage in preventative care (American Cancer Society, 2015).”

American Psychological Association, Boys and Men Guidelines Group. (2018).
APA guidelines for psychological practice with boys and men

A stoical attitude need not be toxic but when misapplied/misunderstood or adopted out of a feeling of social obligation, it can take on a harmful form of thinking that you shouldn’t seek out help. I’m glad the writer’s stoicism was of the positive kind but the writer should perhaps also take greater care in researching what the APA guidelines had actually said.


To put not too fine a point on it: toxic aspects of masculinity kills men. There is nothing pro-man about it. Nobody is actually sticking up for men by pushing back against the APA guidelines.