Swarm intelligence: Bee-inspired AI can help improve decisions

The most counterintuitive, surprising, and impactful new stories delivered to your inbox every Thursday.

Let’s face it, we humans make a lot of bad decisions. And even when we are deeply aware that our decisions are hurting ourselves — like destroying our environment or propagating inequality — we seem collectively helpless to correct course. It is exasperating, like watching a car heading for a brick wall with a driver that seems unwilling or unable to turn the wheel.

Ironically, as individuals, we are not nearly as dysfunctional, most of us turning the wheel as needed to navigate our daily lives. But when groups are involved, with many people grabbing the wheel at once, we often find ourselves in a fruitless stalemate headed for disaster, or worse, lurching off the road and into a ditch, seemingly just to spite ourselves.

The fact is that when groups, particularly large ones, make decisions that impact our collective future, we frequently struggle to find the best path forward. It wasn’t like this for most of human history, for societal decisions were made in intimate groups — think a handful of tribal elders. But these days, it is a major problem, as our future is guided by large and complex organizations, from massive corporations to huge governments.

Swarm intelligence: how honeybees find a new home

It turns out that Mother Nature has been working on this problem for hundreds of millions of years, evolving countless species that make effective decisions in large groups. From flocks of birds and schools of fish to bee swarms and ant colonies, these groups do not reach societal decisions the way humans do — by taking votes or polls or surveys — and they certainly don’t pass “opinion data” up a hierarchy to a handful of deciders who claim to represent the group.

So, how does nature do it?

The answer is by thinking together in real-time systems, deliberating efficiently until they converge on optimized solutions. Biologists call this “swarm intelligence,” and it enables groups to make significantly smarter decisions than the individual members could achieve on their own.

Consider honeybees. They live in colonies that can exceed 10,000 members. And like us, they are faced with critical decisions that impact the collective future of their society. For example, when they outgrow a hive, they need to find a new home to move into. It might be a hollow log, a deep cavity in the ground, or a crawlspace in your roof.

It sounds simple, but this is a life-or-death decision that will impact their survival for generations. To find the best home that they can, the colony sends out hundreds of “scout bees” that search a 30-square-mile area and identify dozens of candidate sites. That is the easy part. The hard part is selecting the best possible solution from all the options they discovered.

As it turns out, honeybees are discriminating house hunters. They need to pick a home that is large enough to store the honey they need for winter, insulated well enough to stay warm on cold nights, ventilated well enough to stay cool in the summer, while also being protected from the rain, safe from predators, and close to fresh water. And of course, it needs to be near good sources of pollen.

This is a complex, multivariable problem. To maximize survival, the group needs to choose the best option across many competing constraints. And remarkably, they do it extremely well. Biologists have shown that honeybees pick the best solution over 80 percent of the time. A human business team trying to select the ideal location for a new factory would face a similarly complex problem and find it very difficult to choose optimally, and yet simple honeybees achieve this.

The “hive mind”

They do so by forming real-time systems that efficiently combine the diverse perspectives of the hundreds of scout bees that explored the available options, enabling group deliberation that considers their differing levels of conviction until they converge on a single unified decision.

But wait. How can bees express their diverse perspectives with varying levels of conviction? Remarkably, they do it by vibrating their bodies. Biologists call this a “waggle dance” because it looks like the bees are dancing, but really, they are generating complex signals that represent their support for the various home sites under consideration. By combining these signals, the bees engage in a multi-directional tug-of-war, pushing and pulling on the problem until they converge on a solution they can agree upon. And it is usually an optimal solution.

And unlike us humans, bees don’t entrench into gridlock or settle on poor solutions that nobody is happy with. And they certainly don’t split up and go off in different directions. They reach decisions that are best for the group as a whole. The phrase “hive mind” often gets a bad rap, implying mindless drones, but that is not true — a hive mind is just nature’s way of combining a group’s diverse perspectives with the aim of maximizing their collective wisdom.

It’s not just bees. Fish schools with thousands of members skillfully navigate the ocean by thinking together efficiently, smoothly handling the challenges they face each day. And unlike us humans, they don’t get stuck swimming toward disaster, unable to agree which way to go. This begs the question: if birds and bees and fish can make effective decisions by deliberating in real-time systems, why can’t people do it?

How humans can harness swarm intelligence

That’s what I wanted to know, so seven years ago, I founded Unanimous AI with the aim of exploring this idea. Unlike most AI researchers that aim to replace people with algorithms, our goal has been to connect people together with AI, enabling networked human groups to form “artificial swarms” that can efficiently converge on optimized decisions. And it works, enabling teams of all sizes to make significantly more accurate decisions and predictions.

To enable swarming, our first challenge was fundamental — people cannot “waggle dance.” This meant we needed a new method for groups to express their opinion, enabling all members to push and pull on the problem together, while modulating their individual levels of conviction. We came up with a solution that reminds some people of a Ouija board; but of course, there are no spirits involved, just AI algorithms based on the biological principles of swarm intelligence.

The technology is called Artificial Swarm Intelligence, or as we usually call it, Swarm AI. It enables groups of all sizes to connect over the internet and deliberate as a unified system, pushing and pulling on decisions while swarming algorithms monitor their actions and reactions. The algorithms are trained on human behaviors, determining each person’s level of conviction so it can guide the swarm toward solutions that best reflect their collective sentiments.

Figure 1 below shows a “human swarm” in the act of deliberating. The size is about 100 people, all working together to make decisions by collectively moving a glass puck. Each of the gold magnets you see is controlled by a person using their mouse or touch screen, each logged in from anywhere in the world. By continuously moving their magnets, they are expressing their sentiments and conviction in real-time, generating signals similar to waggle-dancing bees.

As shown in the time series (Figure 2), the swarm quickly converges on a solution with the glass puck moving to an answer in under 60 seconds. This happens through a combination of human input and AI analysis: The swarming algorithms assess each person’s contribution every 250 milliseconds and adjust as the participants react to the changing motion of the swarm.

While the process looks clean and simple to the participants, the swarming algorithms see a complex cloud of behavioral data which it uses to guide the puck. This creates a feedback loop, for as soon as the AI guides the swarm in a particular direction, the participants react, thus generating an updated cloud of behavioral data for the algorithms to process. This repeats in real-time until an answer is converged upon, usually within 60 seconds.

Swarm AI produces far better group decisions

The big question is whether Swarm AI accomplishes its goal of producing better group decisions. To answer this, we worked with university researchers to perform rigorous studies across many disciplines. In an NSF-funded study performed at Stanford, radiologists were tasked with making pneumonia diagnoses using Swarm AI technology. Their decisions were generated in small groups, either by traditional vote or real-time swarm. When using Swarm AI technology, diagnostic errors were reduced by over 30 percent.

In a recent study performed in collaboration with MIT, groups of financial traders were tasked with predicting the weekly change in the price of gold, oil, and the S&P 500 for a period of 20 consecutive weeks. The groups made these predictions either by vote or by swarm. When using Swarm AI technology, the group showed a 36 percent increase in forecasting accuracy.

In a study performed at California State University (Cal Poly), 60 business teams were tasked with taking a standardized subjective judgement test, either as individuals, by group vote, or by swarm. The study showed that when teams deliberated as a swarm, they significantly outperformed individuals working alone or teams working by majority vote.

In an effort conducted by the United Nations, Swarm AI technology has been used to forecast famines in hotspots around the world. The results showed that swarming makes the consensus building process more efficient, saving time in reaching critical decisions and helping generate buy-in among stakeholders.

In a NESTA-funded study performed at Imperial College London, groups of voters in the UK were asked to prioritize solutions to the controversial Brexit conundrum. Prioritizations were generated either by traditional polling or real-time swarming. The results showed that when the prioritizations were generated through swarming, the top priorities were viewed far more favorably by the general public than the top priorities generated by polling.

Swarm AI could reduce political polarization

This last result highlights an important fact: polls are polarizing, highlighting the differences within a population while doing little to help groups find common ground. In fact, polls often drive groups to entrench in extreme positions, making it harder to reach good decisions. This problem has been amplified by social media, where each vote in the form of a like or share or upvote influences the next, causing extreme positions to rapidly snowball into entrenched polarization. Nature’s method of swarming takes the opposite approach, highlighting common ground and helping groups find solutions they can best agree upon, which are often the smartest solutions.

We humans need to make better decisions. Fortunately, the problem may simply be the methods that we have been using to harness our collective wisdom. For most of human history, groups were small, and decisions only had local impact. But that has dramatically changed in recent years, so our methods of decision-making may need to change as well. I believe the biological principle of swarm intelligence can point us in the right direction, enabling us to make group decisions, big and small, that more accurately reflect our collective insights and aspirations.