RISKY RESEARCH

HOW MUCH risk should a journal assume when publishing new—and potentially contentious—findings? This may sound purely academic, but there are implications in how the rest of us accept things that go against our intuition.

In a recent issue of Science, 12 July 2013, published by the American Association for the Advancement of Science, its Editor-in-Chief Marcia McNutt offers thoughts on this matter that I summarize here.

Marsha McNutt, Editor-in-Chief, Science, is also a Ph.D. geophysicist. Image from Science, 12 July 2013.

McNutt is hardly an ordinary Editor-in-Chief. Previously, she taught at MIT, Stanford and the University of California, Santa Cruz; served as president and CEO of the Monterey Bay Aquarium; director of the U.S. Geological Survey; and science advisor to the U.S. Secretary of the Interior. A scuba diver, she has trained with the U.S. Navy Underwater Demolition Team and Seal Team.

What, she posits, if a finding might offer profound good—provided it’s correct—but could mislead us if it’s incorrect? Should the finding be ignored? Should it be accepted? Some risk seems inevitable.

McNutt observes, “Science moves forward by communicating findings that challenge old ideas and force us to test new theories against the evidence. The key is to contain that risk.”

Risk. Image from Science, 12 July 2013.

Notice, she’s not talking about outright fabrication, which is utterly unacceptable in science—and in life in general. McNutt contrasts what she calls observational fields versus experimental ones. The latter take advantage of controls, multiple trials, varying some initial conditions and holding others constant.

“Experimental results should be reproducible to within known uncertainties,” McNutt notes. Such a paper “may still be risky, but the risk is generally quantifiable.” And, thus, the degree of risk is understood.

The challenge is with advances that depend on new observations—particularly on theoretical musings.

“We have only one Earth to study,” McNutt says, “and no ‘control planet’ from which to gain statistical reliability. The initial conditions are lost in time and cannot be determined with any certainty.”

In topics such as astrophysics and ecology, the object being studied is “larger than any laboratory and the time scales exceed human lifetimes.”

Yet McNutt urges, “The questions that need to be answered are too important to be ignored merely because the definitive experiment cannot be designed.” What’s required is assessment of results based on the newest and most complete information, not just prior knowledge.

She closes with a geologic parable: Years ago, there was a decades-long debate on a seismic discontinuity in the Earth’s mantle, on how heat was, or wasn’t, transferred by convection from one layer to another. A mentor of hers said he was one of the few in the debate who was intellectually honest—because he didn’t object to changing his mind when new observations came to light.

Some colleagues, McNutt’s mentor observed, were practicing religion, not science. ds

© Dennis Simanaitis, SimanitisSays.com, 2013