|
|
|
Name of Submitter:
|
Corinna Loeckenhoff
|
|
Title of proposed idea:
|
Incentivizing sound research practices and addressing the file drawer effect
|
|
What is the major obstacle/challenge in the biomedical research field? What is needed to overcome this obstacle/challenge?
|
|
Across biomedical, behavioral, and social sciences, current editorial practices reward research that shows statistically significant results that are novel, counterintuitive, and likely to attract media coverage. Although this promotes discovery and divergent thinking, it discourages the publication of null results and replications - even though the availability of such data is critical for testing the robustness of findings, identifying false-positives, and evaluating practical implications (Schooler, 2011 – Nature). At the same time there are rising concerns about undisclosed flexibility in the reporting of research results. A surprising percentage of researchers admit that they selectively focus on dependent measures, experimental conditions, and subsamples that support their claims while neglecting to report on aspects of their studies that “didn’t work” (John, Loewenstein, & Prelec, 2012 – Psychological Science). In combination, these contingencies skew the research record, promote the “file drawer effect”, and disadvantage researchers who refrain from selective publishing and other questionable research practices. This situation wastes researchers’ time, squanders money, limits scientific progress, and may ultimately lead to the implementation of faulty policies. |
|
What emerging scientific opportunity is ripe for investment by the Common Fund?
|
|
Over the last couple of years, prominent scholars across a range of disciplines have raised awareness about such issues (e.g., Crocker & Cooper, 2011 – Science; Schooler, 2011 – Nature; John, Loewenstein, & Prelec, 2012 – Psychological Science; Simmons, Nelson, & Simonsohn, 2011 – Psychological Science; Gupta & Stopfer, 2011 – Nature; Munafo & Flint, 2010 – British Journal of Psychiatry). The proposed solutions range from discipline-specific journals of null results, to stringent reporting guidelines for authors, and the establishment of comprehensive data repositories. Although the scientific community appears to be ready for change, the proposed strategies are likely to fail unless the necessary infrastructure is made available and powerful incentives encourage researchers to alter their approach to doing research. |
|
What are the potential Common Fund investments that could accelerate scientific progress in this field?
|
|
A Common Fund initiative could begin by assembling a task force to develop discipline-specific strategies to address publication and reporting biases. In a next step, the Common Fund might support the establishment of data repositories or novel publication venues to address the file drawer effect. In addition, RFAs and other grant mechanisms could be tailored to incentivize systematic replications and other aspects of sound research practice. Ultimately, it may be possible to adapt the registration and results reporting mechanisms that are already in place for clinical trials to a wider range of NIH funded research. |
|
If a Common Fund program on this topic achieved its objectives, what would be the impact?
|
|
If successful, the proposed initiative has the potential to shift current research practices towards greater transparency and reproducibility, improve safeguards against scientific fraud, prevent costly duplication of work, and allow for a more objective look at the actual research record (as opposed to the published research record). |
|