Worsening file-drawer problem in the abstracts of natural, medical and social science databases / Pautasso (2010)
Citation - Pautasso, M. (2010). Worsening file-drawer problem in the abstracts of natural, medical and social science databases. Scientometrics, 85(1), 193-202.
- The file-drawer problem is the tendency of journals to preferentially publish studies with statistically significant results. The problem is an old one and has been documented in various fields, but to my best knowledge there has not been attention to how the issue is developing in a quantitative way through time.
- In the abstracts of various major scholarly databases (Science and Social Science Citation Index (1991–2008), CAB Abstracts and Medline (1970s–2008), the file drawer problem is gradually getting worse, in spite of an increase in (1) the total number of publications and (2) the proportion of publications reporting both the presence and the absence of significant differences.
- The trend is confirmed for particular natural science topics such as biology, energy and environment but not for papers retrieved with the keywords biodiversity, chemistry, computer, engineering, genetics, psychology and quantum (physics). A worsening file-drawer problem can be detected in various medical fields (infection, immunology, malaria, obesity, oncology and pharmacology), but not for papers indexed with strings such as AIDS/HIV, epidemiology, health and neurology.
- An increase in the selective publication of some results against some others is worrying because it can lead to enhanced bias in metaanalysis and hence to a distorted picture of the evidence for or against a certain hypothesis.
- Long-term monitoring of the file-drawer problem is needed to ensure a sustainable and reliable production of (peer-reviewed) scientific knowledge.
Rosenthal famously argued in 1979 that journals are filled with the 5% of studies with significant differences, whilst the drawers (or computer folders, now) are full of the other 95% of studies.
- medical research (Begg and Berlin 1988)
- biology (Csada et al. 1996)
- psychiatry (Gilbody et al. 2000)
- oncology (Krzyzanowska et al. 2003)
- sociology (Gerber and Malhotra 2008)
- systematic reviews in medicine (Tricco et al. 2009)
- research on drug addiction (Vecchi et al. 2009)
- orthodontics (Koletsi et al. 2009)