Part 2 Is focused more on the fallibility of science. Like Part 1 it lacks focus or connection with a unifying argument. In some ways it acts more like an appendix to Part 1, with a look at various different issues in depth.
Part 2 is split into several major sections:
- Scientists are human, too – which looks at human failings and spits in science but primarily concentrates on the pressure on academic scientists to ‘Publish or Perish’
- Lies, Damned Lies, and Statistics! – which looks at statistical analysis and failures by scientists when conducting statistical tests.
- The vaccine controversy – a case study on the Andrew Wakefield affair (which I discussed in my first review)
- The problems of peer-review – a look at issues with peer review with links to some notable cases.
- It’s a process, not a conclusion – which ironically acts as a sort of conclusion to the whole essay but oddly isn’t the last section.
- Internet memes and the love of science. – which is basically just some complaints about the Facebook site https://www.facebook.com/IFeakingLoveScience (the title of the site isn’t “feaking”). You can safely skip that bit.
I’ll go through the sections in turn to varying degrees of detail.
Scientists are humans too.
This starts so very, very badly but it gets better.
The bad bit is this:
While popular culture prefers to paint Galileo as persecuted by the Church for his science — indeed, consequently founding a counter-religious illuminati of scientists — careful study of history reveals that Galileo was not “persecuted” for his beliefs, but rather he was sanctioned by Rome for his personal actions in defiance of a church order of which he was a member. We certainly have plenty of parallels today in which it is easy to point to scientists whose behavior casts a shadow on their own work.
There was some reference to this in Part 1 as well, but the conclusion about Galileo was less underlined.
Thanks to the wonderful research done by Baen authors Eric Flint and Andrew Dennis for 1634: The Galileo Affair, we also know that the popular view that Galileo’s 17th century “apostasy” consisted of defying the Catholic Church over heliocentrism, was in fact over other violations of church doctrine, rather than heliocentrism, which had been gaining Church acceptance for more than a century.
This is a less objectionable version, although it is odd that Roberts draws his conclusions from “Baen authors Eric Flint and Andrew Dennis” as if there has been no prior (and extensive) scholarship of Galileo’s dispute with the church.
Roberts take in Part 2 on Galileo is frankly bizarre – as if Galileo was living in the twentieth century and the Catholic Church was his local golf club. In reality a sanction from the church in renaissance Italy was a sanction from a powerful organisation with powers akin to a state. Whatever its motives might be when the church made demands of a person the potential for torture and death where not impossible outcomes. Essentially Roberts fumbles a better point about how the Catholic Church was not as deeply opposed to the notion of heliocentricism as people might think. That the church was trying to control how scientific ideas were disseminated isn’t in doubt nor that they were willing to use extreme methods to do so. As an example of a scientist whose “behavior cast a shadow” on their work it is a very poor one.
After that the sections settles down a bit and discusses issues around publishing papers including funding and impact factors. It a reasonable overview.
Lies, Damned Lies, and Statistics!
The next section called “Lies, Damned Lies, and Statistics!” is one of the strongest sections in the essay. It has a reasonable explanation of the use of statistics by treating “roses are red, violets are blue” as a statistical thought experiment. It then goes onto discuss a specific paper: “Power failure: why small sample size undermines the reliability of neuroscience,” by Katherine S. Button et al. April 10, 2013 issue of Nature Reviews, Neuroscience .
This paper looked at problems in multiple studies in the field of neuroscience due to small sample sizes. It raises some interesting issues and I assume Roberts picked it out because it relates to his own field (notably some of his comments sound a bit defensive).
The vaccine controversy
The section on statistics finishes with a segue into the issue of fraud. That leads into the section on Andrew Wakefield. Again this is a largely adequate recount of the affair but finishes with a strange conclusion that I discussed in my review of Part 1.
The problems of peer-review
The next section attempts to tackle the issue of peer review. Much of it is a good(ish) opinion piece in which Roberts describes his own views. It is preceded by a set of links intended to give an overview of the issues of peer review. These links are a mixed bag of journalism, research, opinion and out right cranks.
The crank bit is once again on global warming. Roberts makes mention of the Climategate controversy. As a case study Climategate is certainly of relevance but Roberts for some reason picked one of the least credible people to cite: Lord Monckton: http://wattsupwiththat.com/2009/12/01/lord-moncktons-summary-of-climategate-and-its-issues/
To discuss Climategate in detail would derail the rest of this review but it would be fair to say that in so far as issues around peer review are concerned what was primarily revealed was people saying less diplomatically in private what they had previously said a tad more politely in public. In particular the Von Storch affair is a relevant example of issues around peer-review on politically charged topics http://www.realclimate.org/index.php/archives/2006/04/a-correction-with-repercussions/
It’s a process, not a conclusion
Which brings us to a kind of conclusion to the essay. Yes there is one more section to go but that is best treated as an unnecessary appendix.
The last paragraph of the penultimate section (which I’ll call a conclusion for the sake of coherence) is as follows:
Unfortunately, the gatekeeper position can all too easily be corrupted as we have seen. Any scientific conclusion which agrees with the gatekeepers is too easily labeled as a “consensus,” while dissenting opinions are labeled as “fringe,” “deniers,” or even “fraud.” The section on scientific blunders in the beginning of this essay certainly highlights the error inherent when new evidence and scientific results comes along and relegate the former consensus position to the same historic scrap heap and geocentrism. I will sometimes state that any two scientists will produce three different scientific opinions. In even the narrowest aspects of my research field –- with possibly a total of only 200 labs in the world which study the same aspect of Neuroscience –- it is difficult to get even half of them to agree on any one theory. A true consensus in the sense of agreement of >90% of scientists in that field would require so many coincidences, that are mathematically extremely rare.
This is very much where Roberts needs to start if he ever intends to write a similar essay again. If he wishes to attempt to show that things are too easily labelled as a consensus and dissenters too easily labelled as frauds then he needs to build a case. Instead he seems to have worked against himself.
The consensus on vaccines was a section he treated in depth (good for him) but in that case the consensus was confirmed and the dissenter really was a fraud and the anti-vaccination lobby really is a fringe (albeit an influential one). On the other hand with heliocentricism he attempted to downplay the Catholic Church’s attempt to marginalize Galileo’s views.
Some more clear modern examples of a consensus that wasn’t correct, dissenters who were actually marginalized, called frauds or treated as a fringe but who were later vindicated would have been good. A discussion of plate tectonic might have better fit the bill.
Of course with that a discussion of actual cases of fraud (as he did include) would be illuminating as well, so reader could compare and contrast.
The last sentence is worth considering as well. In it he states that a sense of agreement of greater than 90% (as has been claimed over anthropogenic global warming) would be extremely rare in the field of neuroscience (possibly any specific field? the text is a tad ambiguous). This point sort of makes sense and sort of doesn’t. There is not going to be consensus in the areas that people are most actively researching. Why? Because it stands to reason that these are the areas that are most open to question – the areas in which people are testing hypotheses most actively. It would be odd to find consensus in these areas. However, stepping back from the areas of most activity, consensus is neither rare nor remarkable. Researchers act within frameworks based on established knowledge. These aren’t areas that are discussed much in normal science amongst colleagues precisely because they are taken for granted. The appeal to ‘consensus’ only arises when people within a field are challenged on those largely unspoken issues – either by students studying the topic or from scientists challenging these assumptions (perhaps from related fields) or when the topic has political or policy implications.
To conclude, Part 2 has some good bits but it suffers the same problem as Part 1.
- It lacks focus and structure.
- The examples do not illustrate the more general thesis of the essay
- Some of the historical and contemporary references are dodgy and reflect some confused right wing talking points
- The author is unclear as to the purpose of his essay but tends to talk as if he has shown things that he hasn’t provided examples of or arguments for.
Like many of the other puppy nominated works, I am struck by how half-baked this whole essay is and this is a point I will return too later.