The "Eureka myth": Why big ideas don't form in a single moment

A weekly newsletter featuring the biggest ideas from the smartest people

For the most part, we recognize big breakthrough ideas. They cause crises in normal science; they punctuate disciplinary equilibria; they are disruptive innovations, new genres of cultural production, moments of outstanding genius. Not only do they allow us to calculate volume; as we will see, they confirm the germ theory of disease, enable heavier-than air flight and redefine the nature of music. They are landmarks of enquiry: Newton’s Principia, Harvey’s Motion, Lavoisier’s Chemistry, Lyell’s Geology, Smith’s Wealth of Nations, Darwin’s Origin of Species, Freud’s Interpretation of Dreams.

But they are also the businesses that altered history with a new practice: think of the scale of the Dutch and English East India Companies, industrial techniques in Arkwright’s cotton mills or Wedgwood’s potteries, major conglomerates like AEG, process and product innovators like Ford, new service models like McKinsey, the mathematical finance of a hedge fund like D. E. Shaw, the digital sprawl of companies like Google and Facebook. Each book or business is not only sharply distinct from what went before, but opens a new space of possibility that defines the subsequent landscape. There are limits to incrementalism: you can improve the horse or cart or embellish Newtonian mechanics all you like, but doing so doesn’t deliver the automobile or special relativity.

Big ideas can be found in all areas of human endeavor. You find the above pattern with linguistic philosophy; the Internet; human rights; the concept of zero; the steam engine; the iPhone; utilitarianism; calculus; the periodic table; helicopters; entropy; double-entry bookkeeping; written constitutions; writing itself; deep machine learning techniques; Jacobean tragedy; Spacewar! and Grand Theft Auto; information theory, quantum theory and game theory; Cartesian grids, rationality and ego. This is an ecumenical approach to ideas, but only by taking such an approach can we see the overarching picture of change, or its absence.

There is something undeniably romantic about the notion of ideas as heroically catalytic moments. It’s also misleading. It suggests that breakthroughs conform to the Eureka myth; that our most significant instances of thought or endeavor are not rooted in material conditions, or constrained by their histories. Like the original Eureka moment, breakthroughs aren’t ex nihilo miracles. Like all ideas they are composed of pre-existing ideas recombined. It is not then that break-throughs are radically new ideas born whole; rather they are especially significant combinations, formed by slow processes of accretion at the bottom of an S-curve as much as bolts from the blue.

Although we might associate big ideas with outsized impact, that doesn’t imply a speedy or completionist account of their production. When we think of natural selection and evolution, we think of 1859, On the Origin of Species and Charles Darwin. In some ways it is the archetypal big idea – a huge, world-changing scientific notion associated with a single author, year and book. But of course, Darwin didn’t ‘invent’ evolution. He built on theories from the likes of Anaximander and Lucretius, Erasmus Darwin (his grandfather) and Jean-Baptiste Lamarck. Darwin read Adam Smith, and was thus familiar with the idea that an undirected process with numberless small instances of local competition could have extraordinary results: in Smith’s case this was economic growth. Darwin was familiar with Thomas Malthus and his studies of population. Charles Lyell’s key work on geology had radically shifted the perception of time. Darwin specifically acknowledged that ‘descent with modification’ had been recognized by thirty-four predecessors. Indeed, he was a generalist relying on an extensive communication with specialized experts – he maintained a continuous discussion with hundreds of correspondents (at least 231). His research was the work of a lifetime – a gradual realization over years on the Beagle and decades of patient study.

Natural selection is a classic big idea. But its long, complex gestation, its prolific mixture of existing theories and ideas – these are typical as well. Big ideas don’t spring into being fully formed, even though it can look like that. Rather, ideas themselves form and grow in an evolutionary manner. Discrete leaps are, under the hood, often the results of gradual cumulative processes and unlikely admixtures. Big ideas ‘broker’ other ideas in interesting ways – whether that’s Elvis Presley brokering gospel and the blues or Gutenberg’s printing press coupling the wine press with the idea of casting a seal. Johannes Kepler united the previously disparate fields of physics and astronomy, using new data uncovered by Tycho Brahe to prove the elliptical orbits of the planets. Marx’s theory of capitalism combined Hegel’s philosophy with classical political economy and an emergent socialist tradition. Likewise Freud didn’t ‘invent’ the unconscious, an idea that had pedigree everywhere from the Upanishads to Thomas Aquinas, Montaigne and Romantic artists. Picasso exploded Western traditions of art by bringing in new, supposedly ‘primitive’ forms from Africa and elsewhere. The Wright brothers combined the aerodynamics of bird flight with bicycle technology. And on and on.

Zoom into any given area and there is what the writer Arthur Koestler called ‘bisociation’; collisions between previously unconnected ideas. Ideas are fusions, productive confrontations of older ideas. Every ‘new’ is a new synthesis. This means that the prior nature, structure, communicative architecture and social context of ideas powerfully conditions what comes next. To understand the future of ideas means to understand clearly their past and present.

This aggregative sense takes us away from Eureka, but we need to go further. In that story the critical moment happens in the bath as a sudden, singular moment of realization. It forgets that Archimedes would still have needed to conduct experiments, double check results. It omits that he would have to think carefully about presenting his findings. After all, he dealt with an all-too-human court riddled with power politics and petty jealousy. And it doesn’t say how Archimedes’ insight became canon, part of general human knowledge, a staple of the global curriculum. Someone codified it, someone taught it. Ideas don’t spread or enact themselves.

Hence breakthrough ideas actually involve several discrete stages. Every idea must go through:

Conception: Archimedes in the bath. Newton watching the apple tree. The spark, the initial cross-fertilisation. Conceptual ignition. Often this is surprisingly slow; it took Darwin decades to formulate his theory of natural selection after an initial suspicion. Epiphanies can come in dribs and drabs as well as in sudden spurts.

Execution: Darwin not only had to think through his ideas, eventually he had to publish them. The execution is the initial paper or book, the proof of concept, the prototype, the unveiling. It is how an idea is enacted, demonstrated, shown to the world. If a breakthrough stays in your head, it’s not a breakthrough.

Purchase: On the Origin of Species didn’t win everyone over immediately. (Indeed, it still hasn’t.) It needed debate and scrutiny, elaboration, further proof, serious discussion and active persuasion. But eventually everyone working in biology, arguably everyone with an exposure to science, was influenced: it had gained purchase. Purchase describes the point of widespread diffusion, adoption or acceptance wherein a breakthrough can fulfil my conditions of impact.

It may be that in some eras one stage of the process becomes easier, while blockages accumulate in others. Different ideas might struggle at different points. Some papers are written in a blizzard from concept to publication, but then languish for decades or even centuries before being uncovered and finding purchase. Grasping the future of big ideas means finding the contemporary pinch points on this spectrum.

Big ideas are fragile, imbricated with forces far beyond the control of any individual or even any society. Two of those forces are particularly indicative of how ideas function. The first is luck. In the annals of invention, discovery and creation, the role played by serendipity is dizzying. Robert Koch created bacterial cultures after accidentally leaving a potato out to go mouldy, while a few years later Alexander Fleming stumbled on penicillin by accidentally leaving such a culture in his laboratory sink during a spell of freak weather. Radiation and X-rays were both uncovered during the search for other things. Columbus found the ‘New World’ by mistake. The pacemaker was meant to record the human heartbeat, not control it. Happy accidents are behind inventions from Newcomen’s steam engine to the spinning jenny to vulcanized rubber. Just as every idea is formed of other ideas, so it also involves an element of chance – a random meeting of minds, a lucky experiment, a date missed, an accidental find, a serendipitous connection. Misreadings, faulty copies and fluky mistakes are legion, as powerful, if not more so, than directed efforts or ‘heroic genius.’

At the same time ideas repeatedly exhibit multiple discovery, wherein many researchers stumble on the same breakthrough at once. Examples include calculus, oxygen, logarithms, evolutionary theory, photography, the conservation of energy, the telephone and the polio vaccine. No fewer than twenty-three people can claim credit for the invention of the lightbulb. In the early 1920s two researchers, William Ogburn and Dorothy Thomas, found 148 examples of multiple discovery in science alone – and this barely scratches the surface. Artistic movements sometimes seem to spring onto the public stage with styles and coteries fully formed. Polygenesis, in seeming contradiction to luck, suggests that individual endeavors and chance events don’t matter that much.

Chance plays a huge part in the specifics of any given breakthrough, but they also have their time. Arthur Koestler called this phenomenon ‘ripeness’, suggesting that a given society must be ready for an idea. He cites the steam engine, invented as a mechanical toy by Hero of Alexandria in Egypt during the first century CE but only fully realized in eighteenth-century Britain.

Look closely at any breakthrough, and you find it rests on a mind-boggling series of contingencies; zoom out and they look something like an historical inevitability. In fact, the latter creates conditions for the former; the wide context creates the inflammable matter for quasi-random sparks. Both tendencies indicate that big ideas are acutely sensitive to contexts and trajectories, both on the micro level of encounters, conversations, the weather, and at the macro level of economies, cultures, intellectual history. This means that we, just as much as Archimedes and Hero of Alexandria, are constrained by the soil in which we work. Some ideas are likely to be made ‘ripe’ by our society, others blocked or ignited by random chance.

What, then, of the ideas of the twenty-first century and beyond? We know an exciting set of transformational ideas are being conceived, executed, finding purchase. We know they will be composed of prior ideas, but will nonetheless stand out. We know too that they will be grounded in societal context, but that luck will play a role.

The question becomes whether we are creating the right conditions for their continued evolution today.