. . . Kevin Dunbar is a researcher who studies how scientists study things how they fail and succeed. In the early 1990s, he began an unprecedented research project: observing four biochemistry labs at Stanford University. Philosophers have long theorized about how science happens, but Dunbar wanted to get beyond theory. He wasnt satisfied with abstract models of the scientific method that seven-step process we teach schoolkids before the science fair or the dogmatic faith scientists place in logic and objectivity. Dunbar knew that scientists often dont think the way the textbooks say they are supposed to. He suspected that all those philosophers of science from Aristotle to Karl Popper had missed something important about what goes on in the lab. . . . So Dunbar decided to launch an in vivo investigation, attempting to learn from the messiness of real experiments. . . .
Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit. Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.) The scientists had these elaborate theories about what was supposed to happen, Dunbar says. But the results kept contradicting their theories. It wasnt uncommon for someone to spend a month on a project and then just discard all their data because the data didnt make sense. Perhaps they hoped to see a specific protein but it wasnt there. Or maybe their DNA sample showed the presence of an aberrant gene. The details always changed, but the story remained the same: The scientists were looking for X, but they found Y.
Dunbar was fascinated by these statistics. The scientific process, after all, is supposed to be an orderly pursuit of the truth, full of elegant hypotheses and control variables. (Twentieth-century science philosopher Thomas Kuhn, for instance, defined normal science as the kind of research in which everything but the most esoteric detail of the result is known in advance.) However, when experiments were observed up close and Dunbar interviewed the scientists about even the most trifling details this idealized version of the lab fell apart, replaced by an endless supply of disappointing surprises. There were models that didnt work and data that couldnt be replicated and simple studies riddled with anomalies. These werent sloppy people, Dunbar says. They were working in some of the finest labs in the world. But experiments rarely tell us what we think theyre going to tell us. Thats the dirty secret of science.. . .
According to Dunbar, even after scientists had generated their error multiple times it was a consistent inconsistency they might fail to follow it up. Given the amount of unexpected data in science, its just not feasible to pursue everything, Dunbar says. People have to pick and choose whats interesting and whats not, but they often choose badly. And so the result was tossed aside, filed in a quickly forgotten notebook. The scientists had discovered a new fact, but they called it a failure.
The reason were so resistant to anomalous information the real reason researchers automatically assume that every unexpected result is a stupid mistake is rooted in the way the human brain works. Over the past few decades, psychologists have dismantled the myth of objectivity. The fact is, we carefully edit our reality, searching for evidence that confirms what we already believe. Although we pretend were empiricists our views dictated by nothing but the facts were actually blinkered, especially when it comes to information that contradicts our theories. The problem with science, then, isnt that most experiments fail its that most failures are ignored. . . .
But this research raises an obvious question: If humans scientists included are apt to cling to their beliefs, why is science so successful? How do our theories ever change? How do we learn to reinterpret a failure so we can see the answer? . . .
While the scientific process is typically seen as a lonely pursuit researchers solve problems by themselves Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Interestingly, the most important element of the lab meeting wasnt the presentation it was the debate that followed. Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data theyd previously ignored. The new theory was a product of spontaneous conversation, not solitude; a single bracing query was enough to turn scientists into temporary outsiders, able to look anew at their own work.
But not every lab meeting was equally effective. Dunbar tells the story of two labs that both ran into the same experimental problem: The proteins they were trying to measure were sticking to a filter, making it impossible to analyze the data. One of the labs was full of people from different backgrounds, Dunbar says. They had biochemists and molecular biologists and geneticists and students in medical school. The other lab, in contrast, was made up of E. coli experts. They knew more about E. coli than anyone else, but that was what they knew, he says. Dunbar watched how each of these labs dealt with their protein problem. The E. coli group took a brute-force approach, spending several weeks methodically testing various fixes. It was extremely inefficient, Dunbar says. They eventually solved it, but they wasted a lot of valuable time.
The diverse lab, in contrast, mulled the problem at a group meeting. None of the scientists were protein experts, so they began a wide-ranging discussion of possible solutions. At first, the conversation seemed rather useless. But then, as the chemists traded ideas with the biologists and the biologists bounced ideas off the med students, potential answers began to emerge. After another 10 minutes of talking, the protein problem was solved, Dunbar says. They made it look easy.
When Dunbar reviewed the transcripts of the meeting, he found that the intellectual mix generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analogies to express themselves. (Thats because, unlike the E. coli group, the second lab lacked a specialized language that everyone could understand.) These abstractions proved essential for problem-solving, as they encouraged the scientists to reconsider their assumptions. Having to explain the problem to someone else forced them to think, if only for a moment, like an intellectual on the margins, filled with self-skepticism.
This is why other people are so helpful: They shock us out of our cognitive box. I saw this happen all the time, Dunbar says. A scientist would be trying to describe their approach, and theyd be getting a little defensive, and then theyd get this quizzical look on their face. It was like theyd finally understood what was important.
What turned out to be so important, of course, was the unexpected result, the experimental error that felt like a failure. The answer had been there all along it was just obscured by the imperfect theory, rendered invisible by our small-minded brain. Its not until we talk to a colleague or translate our idea into an analogy that we glimpse the meaning in our mistake. Bob Dylan, in other words, was right: Theres no success quite like failure.
