How To Overthrow A Scientific Theory In Three Easy Steps

The hallmark of a good scientist is changing your mind when new evidence arises. Here’s what that looks like.

Science, like many things in life, is always a work-in-progress. While a successful scientific theory has questions it can answer, natural phenomena it can accurately describe, and robust predictions it can make, it’s also fundamentally limited at any point in time. Any theory, no matter how successful, has a finite range of validity. Stay within that range and your theory works very well to describe reality; go outside of it, and its predictions no longer match observations or experiments. This is true for any theory you pick. Newtonian mechanics breaks down at small (quantum) scales and high (relativistic) speeds; Einstein’s General Relativity breaks down at a singularity; Darwin’s evolution breaks down at the origin of life.

Even our best theories of today may be superseded with tomorrow’s science. Here’s how it happens.

Step 0: recognizing successes and failures of the leading theory. The proverbial holy grail of scientific theories is what’s called a final theory of everything. This was Einstein’s ultimate dream, and remains the dream of many other scientists across a variety of fields. Such a theory would predict all natural phenomena in the Universe given any initial setup and conditions. You could calculate the outcome of any experimental setup in advance; you could predict how any system would evolve arbitrarily far into the future. The only limitation you’d face would be from not having an arbitrary amount of computational power, rather than any theoretical limitations.

But we are not there yet. We do not have a working theory of everything; we have a slew of very successful theories that are fundamentally limited in scope. In every field, we have phenomena we can observe or experiments we can design where the predictions of our best theory either contradict the data or yield nonsense. In addition, there are often problems or puzzles that cannot be explained with the theories we have.

Why do neutrinos have mass? Why does the Universe consist of large amounts of matter but not antimatter? What happens to the gravitational field of an electron as it passes through a double slit? And why do the fundamental constants have the values that they have? An unexplained phenomenon that is observed, but doesn’t have a theory to predict it, is often the impetus for a scientific revolution. This is our starting point.

Step 1: reproducing all the successes of the leading theory. So, you’ve got a new theory that you hope will supplant the presently leading one? Great! Your first order of business is to demonstrate that your new theory doesn’t fail where the old one succeeded. The more successful the prevailing theory, the taller an order it is to meet this goal. For example:

• Want to supersede General Relativity? You’ve got to explain gravitational lensing, the precession of Mercury’s orbit, the Lense-Thirring effect, gravitational redshift, the Shapiro time delay, and — most recently — gravitational waves from merging black holes and neutron stars.

• Want to go beyond Darwin’s evolution? You still need to explain the emergence of biological diversity, the response to selection pressures, and how inheritance works, among others.
• Want to improve on the Bohr atom? You’ll have to, at minimum, reproduce the successes of explaining the various energy levels in an atom and the scattering experiments of Rutherford and others off of the atomic nucleus.

This also means your new theory cannot make new predictions that contradict observations that have already been made or experiments that have already been performed. It isn’t enough to get a selection of these predictions right; you have to reproduce every single success of the prior theory. If you can’t equal what you’re trying to replace, you won’t surpass it.

Step 2: succeeding where the prior theory did not. We only conceived of a better theory because there was some motivation or impetus driving us to create one. (Remember, we had a “Step 0” here!) Something wasn’t right with the old theory; there was something it couldn’t explain. Newtonian physics couldn’t explain the mechanics of fast moving particles; the ray theory of light couldn’t explain interference patterns; the universal law of gravitation couldn’t account for Mercury’s orbit.

All of these puzzles led to many new ideas which would explain these phenomena, but not every idea could also reproduce the pre-existing successes. For example, a hypothetical planet interior to Mercury — dubbed Vulcan — was proposed by Urbain Le Verrier to explain its anomalous orbit. Other scientists proposed that the Sun’s corona was massive. Another team, Simon Newcomb and Asaph Hall, determined that if you replaced Newton’s inverse square law, which says that gravity falls off as one over the distance to the power of 2, with a law that says gravity falls off as one over the distance to the power of 2.0000001612, you could explain Mercury’s motion. Finally, Einstein did away with Newton altogether, replacing his gravitational “action-at-a-distance” with curved spacetime.

All of these ideas were seriously considered for many years; all but one fell by the wayside when they were confronted with the all-important third step.

Step 3: you must make new, testable predictions that differ from the original theory’s. If there were a new planet interior to Mercury, it should have been detectable with a telescope. If the corona were massive, we should detect a greater particle/matter density than what we observe. If Newcomb & Hall’s theory of gravity were correct, it would affect the observed orbits of the Moon, Venus, and Earth in ways that do not match with observations. And if Einstein were correct, it would have meant that, with space being curved by mass, that a background light source should follow a curved, rather than a straight, path. It would curve according to the amount General Relativity predicted, not by a null amount nor by the amount you’d get in Newton’s gravity by assigning a photon a mass given by its energy (through E = mc²). In 1919, during a total solar eclipse, this prediction of Einstein’s was put to the critical test.

Lo and behold, light bent according to Einstein’s predictions! In a tremendous revolution, we had a new theory of gravity, put to the test many times over the past 99 years, and passing that test wherever the observations or experiments were of high-enough quality. It took similar theoretical developments and experimental/observational confirmation to arrive at all of our leading scientific theories, from genetics and DNA to the Big Bang, cosmological inflation and dark matter. These aren’t our greatest theories because the math is so pretty or they match our intuition so well, but because they describe the natural phenomena in this Universe so successfully.

As science becomes a more developed, evidence-rich enterprise, it becomes a more herculean task to create a single theory that explains the full suite of data. Yet that’s exactly what the most successful theories do. No matter how successful an idea has been in the past, all it takes is one inconsistent observation to throw the whole thing into doubt. Our greatest scientific theories of today will most likely all fall in the future as new and superior evidence is gathered.

Massive neutrinos are a hint of physics beyond the Standard Model; the black hole information paradox is a hint of gravity beyond General Relativity; the fact that sexual reproduction exists is undeniable, but how it arose is still unknown. These puzzles, and many others, may serve as harbingers of a monumental scientific advance. Until then, we can only speculate at the frontiers of science, in our attempts to take these three massive steps towards a better understanding of the Universe.