Why Science is Never Settled – a review of part one of the essay

Reviewing two (here and here) of the Best Related Work Hugo nominees made me realize I had to do at least one more. Why Science is Never Settled by Ted Roberts is an essay on the scientific method. It isn’t science fiction and it isn’t appalling but it isn’t good. Unlike The Hot Equations it isn’t trying to apply science to science-fiction but unlike Wisdom from My Internet it is not just awful rubbish whose only resemblance to a book is pagination. Roberts has written about his views on the Puppy kerfuffle here.

The essay is in two parts. Part 1 discusses his general view of the scientific method and Part 2 discusses more particular issues. This review covers Part 1 only – partly because it became quite long and unwieldy and partly because the character of the piece changes. A review of Part 2 is here. In places I will refer to sections from Part 2 in this review.

Overall it is a weak essay but with some good to fair parts. The writer is a working scientist with obvious experience with statistical analysis, experimental method and peer review. He clearly is giving an informed insider’s view of science that gives an overview of the processes involved. It does give insights into the writer’s own thinking and it may have been better presented as a set of ruminations on the topic of science.

If we review it as an example of Best Related Work it is a definite technical fail. It’s connection with science fiction is that the author writes some fiction and has had non-fiction published by Baen books. Been is a publisher of SF/F and is spoken of more favorably by the puppy campaign than Tor books.

It is more fair to review at as an essay on science without reference to the Hugo Awards. I’d don’t have strong feelings about strongly policing award categories for taxonomic exactitude and so I’m putting aside the question for the moment of whether it counts. Instead I’ll consider it in terms of its content.

I’ve seen reviews elsewhere that have treated this essay kindly – giving it a passable rating as something you might give to somebody as an introduction to the methodology of science. I would suggest that would be unwise as its faults are many.

Overall it lacks focus: there is not a clear view point that the author tries to establish. For much of it he seems to be dancing around various issues. There are coy references to some topic which suggest that the intended audience is a right wing one (e.g. the title echoes claims by political supporters of action on climate change that the science is settled i.e. that the debate should now be about policy rather than whether anthropogenic global warming is occurring). Having a right-of-center viewpoint is not in itself a problem but it not a viewpoint that the author actually develops or discusses but rather vaguely eludes to.

At this point it is best if I work through the essay in stages. I will use indented italics for quotes from the text because the ‘blockquote’ style provided is a bit hard to read for lengthy chunks of text:

We live in an age filled with the wonders of scientific advancement –- from medical/health care, to computers, to self-driving cars –- yet we also have groups that loudly proclaim their distrust of anything “technological” or “scientific” and turn toward mystical and superstitious explanations instead. But this is not intended as a political rant, and I am not necessarily referring to the groups and actions that you might infer from the title or previous statements. Read on, and let’s look at what science is, who scientists are, and examine the ways in which Science, as a field, makes mistakes, changes its mind, and arrives at its findings. We will then compare and contrast those observations with the quasi-religious approach which declares that the scientific evidence for or against a particular subject is “settled”, that there is “consensus” among all right-thinking scientists who support that view, and that the opposition are “not real scientists” at all.

This paragraph is the best attempt the essay gives of presenting us with a thesis for the essay. Unfortunately, while discussing unfortunate stereotypes about science and scientists  it presents new ones. This portrait of a way of discussing an area of science as two camps with one camp being a favored one and the other acting as an opposition is itself a distorted view of scientific discussion. Some public discussions about science may indeed be public conflicts between opposing views of credible scientists but in other cases the ‘opposition’ (i.e. the talking head provided by news outlets to put an opposing view) may well not be a real scientist at all.

Later in Part 2 of the essay Roberts looks at the case of Andrew Wakefield’s vaccine study that claimed that vaccinations were implicated in autism. Roberts correctly describes how Wakefield’s research was initially published in the prestigious medical journal The Lancet but later retracted by the Lancet when research proved to be dubious. Wakefield was discredited but is still active in the anti-vaccine movement. Roberts appends this comment:

In a strange turnabout to the notion of settled science, Wakefield’s supporters accuse the medical authorities of U.S. and U.K. of the dogmatic approach and failing to acknowledge a link between vaccines and various diseases and disorders.

What is strange about it? The consensus has been that the recent increase in autism diagnoses was not due to vaccinations. Wakefield challenged that consensus (fair enough) but it turned out his research was very dodgy (not fair enough). Partisans for the anti-vaccination position (the ‘opposition’) still back Wakefield. This is not a strange turn about it is simply an example that sometimes 1. there is a consensus and 2. the opposition are cranks and not actually reputable scientists.

Science is in part a method of self-correction by comparison with known facts. You can use it informally as well as formally and in this case Roberts really should have returned to his original thesis at the start of the essay and considered: is this actually the right model to use? Is this really a good way of thinking about science and science policy?

Skipping back to the early stages of Part 1 Roberts says this:

ONE OF THE MOST IMPORTANT CONSIDERATIONS IN JUDGING THE PRONOUNCEMENTS AND PROCLAMATIONS OF “SCIENCE” IS TO UNDERSTAND THAT SCIENCE IS A PROCESS OF EXAMINATION AND EXPLORATION, not a “fact”, pronouncement, or conclusion. In essence, science consists of formulating a hypothesis from observed facts, creating experiments to prove or disprove the hypothesis, observing the results of the experiments, then making a conclusion to accept or reject the original hypothesis on the basis of those observations. This is commonly known as The Scientific Method and derives from Aristotle’s original definitions of deductive reasoning.

As with much of the essay it is wrong by virtue of being half right. Science certainly is a process but it is also a body of knowledge that is both produced by the process of science and informs it. I think Roberts has some sense of this by referring to Aristotle and deceptive reasoning. Aristotelian reasoning works most effectively by reasoning from the general to the particular – and there is an element of science that id deductive from established principles and theories. Of course those theories also part of what science produces.

Within that Roberts also describes an outline of what he sees as the scientific method but it is a very narrow view. Essentially it is a stereotype of what we could call laboratory based experimental science. However that model is a poor fit to a wide range of scientific endeavors. However his more general statement that science is a process of ‘examination and exploration’ does imply a broader perspective.

Roberts then uses a neat example with a question about adding a powdered ‘creamer’ to his coffee. I like this example as it neatly illustrates some of the logic and reasoning of an empirical method – for example setting up a hypothesis and considering conditions under which it fails, considering controls and tests and an overall methodology to considering the issue. It is one of the better parts of the essay.

However he ruins a bit by saying ‘Please note that nowhere in this explanation did I mention “facts,” “conclusions,” “consensus,” or “settled.”’ as if this had been the main point of the example. Naturally a single experiment with a single experimenter is not going to include much reference to consensus. As for conclusion – well simply not using the word doesn’t mean his example didn’t have a conclusion. Here it is:

7. Evaluate the hypothesis — we observed that creamer does indeed increase the volume of the coffee. Thus we reject the Null Hypothesis and instead accept the Alternative Hypothesis.

Roberts then undermines the good work and runs into a section he calls “Famous Scientific Blunders”. Naturally having said that science is a process (according to Roberts) a scientific blunder should be a blunder in the process.

The Science is Settled (Blunder # 1) -– The Sun revolves around the Earth
The Science is Settled (Blunder # 2) -– The human body is regulated by four “humours” which control health, emotion and mental state
The Science is Settled (Blunder # 3) _– Dinosaurs were cold-blooded, dumb lizards (with a brain the size of a walnut)

Of these three examples two are examples of conclusions drawn without regard to science. They relate to topics covered by science but they weren’t scientific blunders except in the sense of people using other forms of reasoning and drawing on alternate understandings other naturalistic ones.

The dinosaur example is a better one which demonstrates how, as more evidence has been gathered, that our understanding of dinosaurs has improved. That does demonstrate that science an scientific knowledge is evolving and growing but it doesn’t particularly help his initial claim of combating a “quasi-religious” approach to science.

The next section is another mixed bag called ‘Scientific Theories Change All the Time’.

•    In the 1970’s, scientists predicted global cooling. In the 90’s and 2000’s it was global warming, in the 10’s there’s again talk of cooling.

This is a whole pile of confused wrongness. In the 1970s *some* (indeed very few) climate scientists predicted global cooling may occur. That belief was based on assumptions that temperature sensitivity to human generated greenhouse gases was fairly low. At the same time many other climate scientists were finding that CO2 and other greenhouse gases lead to much more significant warming than had previously been believed. Of course the potential for global warming from CO2 emissions had been floated as an idea decades before and was becoming an increasingly concerning topic among climate science even before the 1960s. During the 1970s despite some odd concerns about cooling (which gained some press attention) the evidence was pushing people towards global warming being a potential problem. In the 1980s more evidence from temperature records and on temperature sensitivity was found.

The claims from anti-warmist activists of a potential ‘global cooling’ predate the 10’s and rest on the so called ‘pause’ – a period of slower increase in the temperature records in this century as well as other possible causes of cooling such as changes in solar activity. However this so called ‘pause’ is a decreased positive trend rather than cooling. The extent of reduced trend is disputed and total heat content in the oceans has continued to increase.

•    For centuries, gastric (stomach) ulcers were thought to be due to stress, spicy foods and excess stomach acid, until 1984, when Australian physician Barry Marshall drank a culture of Helicobacter pylorii and developed stomach ulcers from the bacteria alone.
This is a much better example of science changing as it is based on direct facts.

The basic synopsis is correct but on the whole the work was warmly received. An OK example of scientific consensus changing but not a good example of entrenched beliefs battling an alternate view.

•    Newton, Einstein and Hawking have each had the final word in the field of Physics — at least until the next run of the Large Hadron Collider.

While at times Newton may have been thought to have had the last word it certainly isn’t true of Einstein. Einstein himself did not feel he had satisfactorily answered many key aspects raised by his work and of course, by his death the conflict between quantum mechanics and general relativity were substantial. Hawking has made great strides to resolve the two models of physics but he doesn’t think he has had the last word either. This point is another straw man.

•    No human could survive transonic speeds… until Chuck Yeager broke the sound barrier; no human could survive the radiation of space… until the Soviet and American astronauts spent days in orbit in the 1960’s; no human could survive the passage through or space outside the Van Allen radiation belts… until the Apollo moon missions.

I know much less about these examples but it is easy to see that there was not a broad consensus that each of these things were true because the US government did not slow huge quantities of money into what the would have assumed are suicide missions. The point being that prior to Chuck Yeager breaking the sound barrier, many people must have assumed that it was relatively safe to do so (indeed a little introspection about the physics would point to acceleration being far more worrisome that velocity). A similar issue arises with the other examples – there was not a broad scientific consensus on those issues prior to astronauts demonstrating that it was relatively safe to do those things. Are we supposed to imagine that the initial Apollo moon missions were developed with the default assumption that all the astronauts would die as they hit the Van Allen belt? Note that it is certainly possible that some people may have thought these things or thought them at some point prior but the topic is not ‘what some people may have thought once’.

•    The pesticide DDT was used to kill mosquitoes, particularly in regions of the world prone to malaria. However, it was proven that DDT to cause the shells of bird eggs to become thinner, thus endangering many species if DDT use continued… until it was determined that the report amount of thinning was actually smaller than the margin of error for the measuring instruments of the time.

DDT is one of the key areas of modern science that has been disputed by a section of the US right. There are various claims made based in some cases on fact and on other cases on gross distortion. A notable critic of the regulation of DDT as a pesticide has been Steven Milloy who runs JunkScience.com and has, in the past, been a commentator on Fox News. Here is his section on DDT egg-thinning claims http://junkscience.com/1999/07/26/100-things-you-should-know-about-ddt/#ref6
You will note that most of Milloy’s references come from the 1970s, which was the height of the scientific ‘battle’ over DDT. After that peak the cause of DDT became a sort of ideological pseudo-scientific guerrilla war – oddly connected to lobby groups for the smoking industry. Various claims are made by pro-DDT websites that exaggerate the extent to which it is regulated (often making false claims about worldwide bans) and claiming that malaria deaths since regulation have been caused by a decline in use of DDT as a means of mosquito control in malaria-ridden areas.

Long story-short: DDT in some bird species in sufficient concentration does cause thinning of egg shells which causes population declines. http://npic.orst.edu/factsheets/ddttech.pdf  – there really isn’t any serious (apolitical) doubt about that.

•    Human memory is mystical and metaphysical, with no connection between the mind and the brain… until it was demonstrated that memory is associated with specific anatomical areas of the brain… and then it was discovered that trained flatworms could be ground up, and molecules extracted and fed to other flatworms who then had the “memory” without the need to learn… except that memory requires specific patterns of anatomical connections, and chemical and electrical signals… until it was found last year that some memories may actually consist of molecules that can be inherited! [But that’s a topic worthy of its own article!]

Yes, a topic worthy of its own article because that paragraph ends up not really being an example of anything. Certainly “human memory is mystical” is not a proposition around which there was some kind of scientific consensus. It is an area in which there are still many unknowns. As an example of an area in which we used to know very little and know some more, it is a good example and so in a sense an area in which science has changed but did anybody ever thing that the body of knowledge that is scientific was fixed an immutable? Again this is an odd straw man to knock down. Science adds to our knowledge, that is why we do it.

That gets us close to the end of part one. Just a conclusion for Part 1 to go.

THE HALLMARK OF SCIENCE IS THAT IT IS ALWAYS HYPOTHESIZING, ALWAYS COLLECTING DATA, ALWAYS TESTING, AND ALWAYS REFINING OR LOOKING FOR NEW THEORIES. In fact, the only indication of a good theory is whether one can make valid predictions with the theory. One successful prediction, however, is not enough — after all, just one failed prediction negates a theory, one correct prediction simply means the theory works for now, or until a failed prediction or a more comprehensive theory comes along.

One of the better paragraphs, except it is an odd conclusion to draw from the examples given. It is a sound point but having given three examples of blunders from the past and six examples of changing theories, this statement seems oddly disconnected. It strongly suggests that Roberts needed to find better examples. A strong theory shown to be wrong by an experimental design would have been good. Instead Roberts picked on examples that either couldn’t be adequately described in a paragraph or were actually examples of confused quasi-scientific claims in politically partisan debates.

Roberts then goes on to say:

In fact, I am often asked how a scientist with a personal religious faith can reconcile evolutionary theory with that faith. My response is that a scientist uses theories all the time. Their job is not to judge the truth of a theory, but rather its utility. As long as a theory is consistent with observations, can be used to test and/or predict the data one has in hand, and successfully predicts results, it is a useful theory. The concepts of truth or belief are irrelevant to the Scientific Method, and should never contaminate the science.

The first sentence sounds like a really great topic for an essay and I really would like to read more on that. However the response that follows is disappointing – essentially he makes it sound like science is little more than a game. No truth? No belief? Really or is that just a veneer of a kind of low-rent postmodernism?

Consider Roberts’s earlier example of the experiment with the powdered hazelnut flavored coffee creamer. The experiment was designed to see if the coffee creamer added volume to his coffee and he expressed his null hypothesis as:  powder dissolved in a liquid does not add any volume. At the end of the experiment the results showed that he should reject the null hypothesis.

What would Roberts have us take away from that:

  • Is it not a fact that the powder adds volume? According to Roberts we need to avoid terms like ‘fact’
  • Is the claim that adding powders to the liquid (specifically creamer to coffee) increases volume a claim that is not-true (but not false either)? Does he regard this claim as sitting still in doubt?
  • Does the experiment seriously not affect his beliefs about the consequence of adding creamer to his coffee?

Actually I call BS on that. The experiment does tells us about the truth of claims in a practical sense (sure a kind of practical, provisional truth but that is the kind of truth that we use in most cases). The experiment does (or should) influence our beliefs – having gone to all that trouble does Roberts really want to remain undecided on the topic of creamer in coffee?

Now maybe Roberts has a seriously great theory of epistemology in which he has well worked out notions of truth, knowledge and belief AND when explained we might all understand the context in which he says these odd things. Another great topic for an essay. However he has not introduced that epistemology here and as a consequence this notion he includes of truth and belief never contaminating science is simply stated by fiat.

We can see why he might say that: for example if his particular religious faith has an issue with the theory of evolution by natural selection then he can adopt the theory purely from a utility point of view. However that creates a dilemma. He either has to find a non-religious basis for saying natural selection is epistemologically different (i.e. it is one of a subset of theories in which truth and belief should be put aside) or apply to all of science. He decides to claim the latter.

Can he manage that constantly in reality? No, and even in this essay he doesn’t manage to. For example earlier in the essay he talks about how science is a process: From those observations, a hypothesis is formed and experiments are proposed which will test – under controlled conditions – whether that hypothesis is true or false.

So in describing the scientific method he puts deciding whether the hypothesis is true or false as a central step BUT by the end of section 1 he is saying truth is irrelevant to the scientific method?

At this point it is becoming clear this is less of an essay and more somebody thinking out loud. As it happens I like the idea of thinking out loud and committing your thoughts to a more concrete medium and even letting people point out the contradictions. It is great for your head. I don’t fault Roberts for this – it is just not how the essay is presented. If he had put the bit about reconciling faith (or perhaps politics) first then we would have the beginnings of an interesting personal essay – warts and all. However, we have instead an essay that appears to claim to argue for a proposition ‘Science is Never Settled’. Can I suggest a different title? ‘My Thoughts on Science are Never Settled’ – I think the bones of that essay is here and it could be a brilliant one from Roberts.

Unfortunately we can’t judge this work on the basis of a better essay that he might have written. While this essay has many good qualities it is not a good one.

5 thoughts on “Why Science is Never Settled – a review of part one of the essay

  1. Regarding some of his claims:

    No human could survive transonic speeds… until Chuck Yeager broke the sound barrier;

    I believe the primary issue was aerodynamic stability of an airplane when going transonic, which required new airfoil designs. Old-style wings had a habit of ripping off. I don’t think there was serious doubt getting someone to supersonic velocity if you used a rocket and capsule approach (wings aren’t needed there, of course). It was a matter of building a plane so that the wings wouldn’t come off.

    Sure, this was a big deal with news reporters and other non-experts, but I think subject matter experts had a much more nuanced view of this.

    no human could survive the radiation of space… until the Soviet and American astronauts spent days in orbit in the 1960’s; no human could survive the passage through or space outside the Van Allen radiation belts… until the Apollo moon missions.

    Ugh. I’m somewhat familiar with this as I’ve been involved in a number of Moon landing hoax discussions. Some people get it in their head that crewed spaceflight (at least in or above the belts) would be

    The Van Allen belts got their name in 1958 when the Explorer 1 flew through them and detected the radiation on its instruments. We had almost no information it it before that. James Van Allen was a researcher involved with that. The space radiation environment was of serious interest both for manned and unmanned (including commercial and military) spacecraft, as you had to plan on hardening the electronics sufficiently to last the specified time for the mission. So there were further unmanned flights to map out the radiation environment, and near-earth radiation models were developed. Before crew ever flew, there was an excellent understanding of the radiation environment.

    Radiation was a major design concern with Apollo, and they carefully worked out how much would be attenuated by structure. Also, they developed high-inclination flight trajectories that skirted the majority of the belts, and the travel time through the edge of the belt was quite short, so between the shielding, the path, and time, they didn’t pick up much radiation.

    A large solar flare coming at Earth would have been their biggest radiation problem, if it happened. But they’re pretty rare, and they had some options and a plan if it ever happened. The key thing is that the flights were short, so though they did pick up a bit more radiation than someone on the ground, the total increase in dose was quite small. People on the ISS get higher doses the Apollo crews because they’re in space longer.


  2. *sigh* a missed word. New word in square brackets: [ ]

    Some people get it in their head that crewed spaceflight (at least in or above the belts) would be [deadly].


  3. Thanks Stobor,
    I wasn’t as familiar with those points – I think with some re-wording Roberts probably could have made use of those examples in a useful way. Again though it shows how slapdash a lot of the nominated work is. If you re writing an essay on the process of science you really need to have examples that are technically correct. There is little sign of Roberts doing his homework.

    I had thought this essay was just some blog post that had been elevated by the Puppy nomination – but Baen had published it in its free 2014 non-fiction eBook.


Comments are closed.