Refine
Has Fulltext
- no (2)
Document Type
- Article (1)
- Working Paper (1)
Language
- English (2)
Is part of the Bibliography
- yes (2)
Keywords
- biodiversity and ecosystem services (1)
- conomics (1)
- inter- and transdisciplinarity (1)
- knowledge integration (1)
- open science (1)
- political science (1)
- project coordination (1)
- replication (1)
- reproduction (1)
- research transparency (1)
- stakeholder participation (1)
This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators' experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.
The management of biodiversity represents a research topic that needs to involve not only several (sub-) disciplines from the natural sciences but, in particular, also the social sciences and humanities. Furthermore, over the last couple of years, the need for the integration of other kinds of knowledge (experience based or indigenous knowledge) is increasingly acknowledged. For instance, the incorporation of such knowledge is indispensable for place-based approaches to sustainable land management, which require that the specific ecological and social context is addressed. However, desirable as it may be, such an engagement of the holders of tacit knowledge is not easy to achieve. It demands reconciling well-established scientific procedural standards with the implicit or explicit criteria of relevance that apply in civil society a process that typically causes severe tensions and comes up against both habitual as well as institutional constraints. The difficulty of managing such tensions is amplified particularly in large integrated projects and represents a major challenge to project management. At the Helmholtz Centre for Environmental Research - UFZ, several integrated research projects have been conducted over the past years in which experience has been gained with these specific challenges. This paper presents some of the lessons learned from large integrated projects, with an emphasis on project design and management structure. At the centre of the present contribution are experiences gained in the coordination and management of LEGATO (LEGATO stands for Land-use intensity and Ecological EnGineering - Assessment Tools for risks and Opportunities in irrigated rice based production systems, see www.legato-project.net), an ongoing, large-scale, inter- and transdisciplinary research project dealing with the management of irrigated rice landscapes in Southeast Asia. In this project, local expertise on traditional production systems is absolutely crucial but needs to be integrated with natural and social science research to identify future-proof land management systems.