Ask Ethan: How Can A Non-Expert Evaluate Conflicting Claims By Actual Experts?

Find out the truth for yourself.

If this past year has shown us anything, it’s how thoroughly we rely on high-quality expertise. As the COVID-19 pandemic swept the globe, it wasn’t the voice of science that stood out above the fracas, but rather the machinations of politicians, only a small fraction of whom followed the best scientific recommendation available at the time. It put into context a number of related issues — vaccines, climate change, and fluoridated drinking water among them — where society’s policies don’t align with what science indisputably indicates. Even worse, you can often find people with expert credentials who advocate for either sides of an issue, further muddying the waters. If you yourself aren’t an expert not only in a particular field but in the relevant sub-fields to the issue at play, how can you know whom to trust? That’s what Dr. Larry Moran wants to know, writing in to ask:

“I assume that you are not an expert on fluoridated drinking water, climate change, or COVID-19 and yet you feel confident that you can identify the correct scientific position on each of these topics. How do you do that and is it something that the average intelligent person can do as well? Isn’t that the real question?”

The answer to the second question is yes, but it takes a lot of hard work for each and every issue you want to evaluate. Here’s how we do it to the best of our ability.

There’s some “pre-work” that goes into any sort of investigation like this, where you have to first identify what you think you know about the issue at play. What are the different things that you’ve heard various people say, and how credible do you think each of those statements are? Before you even begin to evaluate the credibility of the claims at play, you have to identify what the various claims are, as well as your preconceptions about them.

Are vaccines safe and effective? How safe, quantitatively, are they, and how effective are they? What are the side effects and who is at risk of them? Will they alter your DNA, and if so, in what way?

What about the science of wearing masks? How effective are they, and what is the role that surfaces, droplets, and aerosols play in the transmission of the novel coronavirus and its variants?

What about fluoridated drinking water? Or GMO crops? Or climate change?

Identifying the relevant issues — the ones that need to be addressed in order to reach an informed conclusion — has to be your starting point.

Once you’ve identified the various facets of a complex but important issue, you’ll want to identify all the areas of universal agreement. Vaccines have, quite obviously, been a fantastic public health advance. Diseases that once ravaged populations across the world — measles, polio, diphtheria, pertussis, etc. — have been brought to the brink of eradication, largely due to vaccination efforts.

Vaccine side effects are rare, but they do exist, largely in the form of allergic reactions. However, a number of vaccine trials fail because of potentially deadly side effects; the blood clots associated with the Johnson & Johnson and AstraZeneca are prominent examples, while the CDC has catalogued a number of well-documented historical incidents.

If practically all legitimate professionals on all sides of an issue can agree on a set of facts, you can safely take those as a starting point where you’re unlikely to get anything wrong.

Then you come to the next step: identifying false claims that are either completely unsupported or directly contradicted by the science. This is a step that you must take, or you’ll forever be bogged down by disingenuous arguments that are designed to distract you from the actual hard work of figuring out what is true and what is false.

However, not every issue is as easy to resolve as “person #1 says the sky is green and the grass is blue, while person #2 says the sky is blue and the grass is green.” Some issues take a lot of research to unpack, particularly when there are a number of voices — especially if at least one of those voices comes from a source you generally respect and trust — that are actively working to spread misinformation.

And yet, a good non-expert can cut through a large amount of that misinformation simply by either talking to or listening to the conversations that a number of credible experts have had in various public forums.

For example, consider the following questions:

• Do vaccines cause autism?
• Is the Earth’s temperature warming?
• Is fluoride a waste by-product of the aluminum industry?
• Are organic, non-GMO crops healthier for you than non-organic, GMO crops?
• Will an mRNA vaccine alter your DNA?

Of course, there are many other examples, but these are ones where the information should be particularly easy to obtain simply by sifting through the internet. Vaccines definitively do not cause autism, and there are no large-scale, non-fraudulent studies that show that. Earth’s temperature is not only warming, but the warming is accelerating and is now at the 5-sigma level: where there’s less than a 1-in-7,000,000 chance that this is a fluke of the data.

And we can go on.

Fluoride is used in the aluminum smelting process as a catalyst, and it is true that the aluminum industry is the largest consumer of fluoride compounds. Once upon a time, America’s largest aluminum company (ALCOA) did acquire a sodium fluoride production plant, and sold sodium fluoride. However, their last sale occurred back in 1952, because it became cheaper for ALCOA to buy fluoride compounds elsewhere than it was to manufacture it themselves. None of the fluoride in municipal water supplies has ever participated in the aluminum manufacturing process for the past 59 years.

While there may be environmental differences in the impacts between organic and non-organic farming, there is no nutritional difference, and that has been studied at length and in-depth. Fascinatingly, GMO crops are often far more nutritious than their non-GMO counterparts, with golden rice and vitamin A (and its ability to prevent hundreds of thousands of cases of blindness annually) a stunning counterexample to the non-factual claim.

And when it comes to mRNA vaccines, they only instruct your cells to produce a protein, which your immune system then attacks; your DNA is never altered. In an ironic twist, contracting COVID-19 and suffering its ill effects can, and perhaps often does, alter your DNA.

But that’s simply the low-hanging fruit. There are plenty of issues where you yourself lack the necessary expertise to discern between various viewpoints, even after doing your homework and fact-finding to the best of your abilities.

In those instances — and this is where, in my personal experience, many science writers go wrong — it’s vital to find the correct, relevant experts. It’s very easy to find a number of scientists who support positions that, even if they haven’t quite reached full-blown crackpot territory, are certainly contrarian positions that are well out of the scientific mainstream.

Could the dinosaurs have been wiped out by a comet, instead of an asteroid?

Could today’s temperature increase on Earth be a temporary effect, to be followed by a cooling period?

Could masks actually increase the transmission of certain viruses, rather than reducing the spread?

Even though studies supporting each of these positions have been published by prominent scientists, passing peer review in the process, the answer is a resounding “no” in every one of these instances. The reason isn’t because I trust “expert B” and not “expert A,” nor is it because I myself know enough to evaluate the claims on their own merits for myself, even though I do, in fact, earnestly attempt to understand these issues as deeply as I can.

Instead, it’s because I know — and contact — a number of experts that I trust in a variety of general fields: physics, astronomy, biology, chemistry, geology, climate science, etc. When there’s a contentious issue I don’t fully understand, or a novel claim that appears to conflict with what I know, I:

• ask them about it,
• listen to what they say,
• ask follow-up questions to better understand the claims and the different lines of evidence pointing towards the conclusions,
• ask about errors, uncertainties, and conflicting interpretations of the data,
• ask about different methodologies and what the drawback or omissions are in each case,
• and to learn about where the boundary is between what’s established, what’s suggestive, what’s speculative, and what is untenable nonsense.

In all of these three aforementioned cases, the claim in question, despite a copious amount of reporting suggesting that it’s plausible, is robustly ruled out.

The combination of Chicxulub crater’s magnitude in size, the energy required to create it, and the high abundance of elements like iridium found in the ash layer that blankets the Earth from 65 million years ago thoroughly rule out a cometary nature for the ancient impactor.

The stadium wave hypothesis, which noted oscillations (rises and falls) in the temperature at various places, like in the Atlantic Ocean, hypothesized that these oscillations were driving the apparent temperature rises seen around the globe. By hypothesizing that temperatures would fall and sea ice would regrow throughout the 2010s, and with the data clearly showing that not only was the hypothesis itself ill-founded but that both global and oceanic temperatures rose and accelerated during the past decade, the claim has been definitively falsified.

And the idea that cloth masks could increase virus transmission was from a 2015 study that compared cloth masks with medical grade masks, not with an unmasked population. That claim never had any teeth to it, but you wouldn’t know it unless you had dug deep enough into the literature (and understood the context of the study) to pick that information out.

The unfortunate fact of the matter is this: there are a lot of people — scientists, science writers, reporters, and laypersons alike — who think they’re doing a good job of separating fact from fiction, proving “the experts” wrong when, in fact, they are not. There are a number of reasons for this, and they include:

• hubris, where they think they know more than they do,
• succumbing to shortcuts, where they don’t do a sufficient amount of the necessary background research to put these new claims in their proper context,
• falling for the fallacy of “telling both sides” to a story, even when the scientific weight of the evidence is firmly on one side and not the other,
• or, to promote a favored conclusion, they simply lie.

There are red flags that you can look out for, of course. Someone who tells you to “do your own research” is almost always deliberately attempting to undermine established science. A claim that “evidence for [this new position] is increasing all the time” is almost always advocating for something that the weight of the evidence doesn’t, in fact, support. People who use a single, personal experience as a counterexample to a scientific consensus are following anecdotes, not data. And if someone has to appeal to a conspiracy theory, that “the truth is being suppressed,” you can fairly safely conclude that no amount of scientific data will convince them.

You may not ever be able to reason someone else out of a position they didn’t reason themselves into, but you can reason yourself into a more accurate position anytime you choose simply by doing the necessary work. Hopefully, if you’re so inclined, you now have the necessary tools — and humility — to go and do so for yourself.

Send in your Ask Ethan questions to startswithabang at gmail dot com!

Starts With A Bang is written by Ethan Siegel, Ph.D., author of Beyond The Galaxy, and Treknology: The Science of Star Trek from Tricorders to Warp Drive.