The way we do science is broken: frequently inefficient, unreliable and even totally invalid, the road to consensus is fraught with hostility and academic pettiness. Cory Clark and Philip Tetlock argue for an alternative to the critique-reply-rejoinder format with the radical potential to transform the pursuit of knowledge.

If we were naïve observers, we might think of scientists as earnest detectives—carefully sifting through the evidence, pursuing all reasonable leads, and updating their beliefs as needed. We might imagine that scientists get together, exchange notes, form brilliant and empirically accurate beliefs, and then share these state-of-the-science ideas with the public. These ideas usually would be true and thus form a reliable basis for designing effective interventions and policies.

To be sure, science has accomplished remarkable feats, from vaccines to spacecrafts. But science is far from the idealistic version portrayed above. Science is the single most effective mode of knowledge formation to date. But, it can also be inefficient, hostile, petty, unreliable, and invalid. An average disagreement in science looks a bit like this: Scholar A forwards an idea. Scholar B contradicts it. Scholar A mocks Scholar B in a commentary, collects more data, and finds herself correct. Scholar B mocks Scholar A in a commentary, collects more data, and finds himself correct. Over time, they come to hate one another, and through the collection of more and more data, become only more convinced of their own initial beliefs. In good cases, third-party observers eventually make sense of the mess of seemingly contradictory evidence, and a “scientific consensus” is born.