“Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true.” This quote is part of an article by Jonah Lehrer appearing in the Dec. 13, 2010, New Yorker Annals of Science column, in which Lehrer questions the validity of the scientific method.

He uses several examples of studies conducted over the past few decades in which scientists are unable to repeat their already proven (and in many cases, widely acknowledged and accepted) results. For instance, the symmetry theory of the 1990s, which proposed that women (animals and humans) preferred males with better symmetry (and thus, better genes), slowly fell apart after more research. Another example mentioned in Lehrer’s article is that of Jonathan Schooler’s “verbal overshadowing” theory which had promising results that soon began to decline upon experimental repetition.

The article also comments on the pharmaceutical industry, specifically in the area of clinical trials, and drug versus placebo effectiveness. Lehrer writes about the declining effect of antipsychotic drugs and how many scientists are realizing that “well-established, multiply confirmed findings have started to look increasingly uncertain.”

The problem, reports Lehrer, may have to do with selective reporting—a concept that is not new to the drug industry. Many analysts have found that research containing positive results is reported and published far more often than research with negative results. In addition, things like cultural bias can change the ultimate findings of similar experiments. For example, Lehrer’s article notes that during a set time period, 100% of 47 studies on the effectiveness of acupuncture reported positive results across China, Taiwan, and Japan, while only 56% of 94 studies conducted in the US, Sweden, and UK ended up finding therapeutic benefits to acupuncture. In many instances, researchers find ways to interpret data so that they pass significance tests, according to John Ioannidis, an epidemiologist at Stanford University, who is interviewed in Lehrer’s article.

Overall, the article has many points that the pharma R&D industry may want to consider. Although these ideas may not be new or earth-shattering, it may be worth calling into question new findings—even if they are repeatable or “proven.” In some cases, new efforts toward comparative effectiveness research may be able to determine the true effectiveness of certain treatments and ultimately ensure that the patient is getting the best possible option in the short-term, and in the long term, should that option change years down the road when updated information becomes available.