Trust Data, Not Instinct

You know that great idea you have for improving your business? Ronny Kohavi, an architect at Microsoft’s Online Services division, says there’s good reason to suspect it’s actually lousy. Software industry studies show that when ideas people thought would succeed are evaluated through controlled experiments, less than 50 percent actually work out. At Microsoft, Kohavi devoted himself to winning others to his cause as he built his experimentation platform. He staged events — such as offering a shirt for correctly guessing the outcome of experiments. He didn’t have to give out a single shirt. Intuition never matched data.