to see if their benefits seem greater for patients in established
market economies or in developing countries. If there is a dif-
ference, are there clues in the data as to why?
In another project, he is analyzing meta-analyses of expensive antibiotics. “The agenda of those funding the study can be
biased, and so you may have a reliable large study or many large
studies, but the framing of the question is too narrow and
restricted. They compare an expensive antibiotic to a slightly less
expensive antibiotic from the same company and find that both
work and one works better. But they never compare the expensive antibiotic to penicillin or another inexpensive antibiotic.
The studies are well done, but they ask an irrelevant question.”
Studies underwritten by drug or medical device companies aren’t the only research that risks being biased by financial incentives. Competition for funding from any source can
influence researchers to focus on designing a study that is more
likely to produce a positive outcome. What does “more likely”
mean? Allowing too much leeway in determining which side
effects are reported or finding reasons to toss out data that does
not support a hypothesis can create skewed, unreliable results.
There are also competitive professional pressures to get high-impact results published in top-tier journals, which means important confirmation studies for new findings can have a harder time
getting published.
Biases are often unconscious, Ioannidis acknowledges. “Sci-
ence itself is an unconscious bias. We want to discover things
and make a difference. That doesn’t mean that will happen. I
can waste my whole career on something that doesn’t matter.”
That pressure can inspire even a very good researcher to lose
objectivity in ways that impact results. There was nothing
nefarious about doctors doing observational studies about
women taking hormones and extrapolating that they had less
heart disease. The problem, Ioannidis says, is that it’s likely that
the women who sought and took the hormones during that
period tended to be more health-conscious in general and prob-
ably made healthier lifestyle choices that led to their better
heart health. It wasn’t until a large randomized sample was
scrutinized that the risk from the hormones themselves
became clearer. The accumulation of problems like this led
Ioannidis to conclude in the 2005 PLoS paper: “For most
designs and settings, it is more likely for a research claim to be
false than true.”
The PLoS paper is the most viewed article in the history of
Public Library of Science. Robert Tibshirani, a professor of
health research and policy, and statistics at Stanford, agrees it
was a watershed. “That paper really questioned the paradigm
and showed that something is broken,” he says. “People were
suspecting a lot of positive results were borderline or not even
true. John was so thorough in analyzing this. It’s not like he
took anecdotal stories and made big claims. His team worked
for years and did the statistics and groundwork.”
Dean of Medicine Philip Pizzo is a big proponent of Ioanni-
dis’s work and recruited him in 2010 from his post as chair of
hygiene and epidemiology at the University of Ioannina School
of Medicine in Greece to run the Stanford Prevention Research
Center. “Because of the ever escalating cost of health care, medicine must become more evidence-based, and the foundations
of care must be clear and defined criteria. Dr. Ioannidis’s work
compels the medical community to look more carefully and
critically at medical recommendations that have ‘assumed
truth’ but not critically and scientifically defensible foundations,” Pizzo says.
Part of the appeal of coming to Stanford, Ioannidis says, was
the emphasis on interdisciplinary collaboration. (Another was
living in California; his wife Despina Contopoulos-Ioannidis is
a pediatric infectious disease specialist and clinical associate
professor at Stanford. They have one teenage daughter.) Both at
SPRC and in Greece, where he maintains a research team at the
University of Ioannina, and with scores of collaborators around
the world, Ioannidis is training a new generation of research
sleuths. In a 2011 study about so-called medical reversals, Ioannidis’s colleague Vinay Prasad of Northwestern University analyzed 35 trials published in 2009 in a major medical journal
that tested a variety of established clinical practices including
prenatal care, cancer screening and surgical interventions. The
researchers found that 46 percent of those trials reported
results that contradicted current practice, and 3 percent were
