Opening the File Drawer: Assessing and Understanding Publication Bias in the Social, Economic, and Behavioral Sciences by Utilizing two German Probabilistic Panels (“PubBias”)
What is publication bias?
Publication bias refers to the prioritized and selective reporting of statistically significant scientific results. It is based on the assumption that statistically significant findings are generally regarded as “better,” “more valuable,” or “worthy of publication” compared to non-significant results. This premise may lead researchers to either a) publish only their significant results without mentioning other effects below standard statistical thresholds, b) change their hypotheses (or research questions) post hoc, or c) not publish their results at all. This can be problematic, as time and resources may repeatedly be invested in conducting research that has already been conducted but not published. Moreover, the insights from non-significant results (by assuming the null hypothesis) are lost for science and society. In psychology, the effects of this selection pressure became evident during the “replication crisis” of the past decade.
Research focus
The research project “PubBias” (DFG 512014619) aims to assess the prevalence and determinants of publication bias in the social, economic, and behavioral sciences in Germany on the basis of two probabilistic, academic, longitudinal surveys in order to shed light on the decision-making process of researchers when publishing their scientific results. For this purpose, we compare successful research proposals (2012–2021) from the GESIS Panel and the Innovation Sample of the Socio-Economic Panel (SOEP-IS) with their subsequent publications and conduct an author survey that addresses the decision-making process regarding these publications and hypothetical works.
Figure: Schematic representation of the various stages on the path from study submission (left) to publication (right). The steps shown in the central box (data analysis, writing, review process) will be investigated in greater detail as part of our author survey.
Who we are – The team behind “PubBias”
“PubBias” originated from the cooperation between the GESIS Panel and the SOEP-IS and has been funded by the German Research Foundation (DFG) since 2023.
Our research team consists of six researchers from various disciplines (including psychology, sociology, and survey methodology) based in Berlin (SHARE BERLIN Institute), Mannheim (GESIS – Leibniz Institute for the Social Sciences) and Ann Arbor (University of Michigan).
Team members
(Director SHARE Infrastructure, SHARE BERLIN Institute; Prof. Survey Research, FU Berlin)
(Associate Scientist, Socio-Economic Panel, DIW and SHARE BERLIN Institute; LIFE IMPRS Fellow)
Dr. Jessica Daikeler
(Team Leader Assessing, Survey Data Quality, Survey Design and Methodology, GESIS – Leibniz Institute for the Social Sciences, Mannheim)
Dr. Désirée Nießen
(Postdoctoral Researcher, Survey Design and Methodology, GESIS – Leibniz Institute for the Social Sciences, Mannheim)
Dr. Bernd Weiß
(Team Leader GESIS Panel, Deputy Head of Survey Design and Methodology, GESIS – Leibniz Institute for the Social Sciences, Mannheim)
Asst. Prof. Dr. Henning Silber
(Research Assistant Professor, Institute for Social Research, University of Michigan, Ann Arbor, USA)