welcome to the nonlinear library, where we use text-to-speech software to convert the best writing from the rationalist and ea communities into audio.
this is: Why scientific research is less effective in producing value than it could be: a mapping, published by C Tilli on the effective altruism forum.
Contributors: David Janků, David Reinstein, Edo Arad, Georgios Kaklamanos, Sophie Schauman
Introduction
Research has produced, and is producing, a lot of value for society. Research also takes up a lot of resources: Global spending on R&D is almost US$ 1.7 trillion annually (approx 2 % of global GDP). This is an attempt to map challenges and inefficiencies in the research system. If we addressed these challenges and inefficiencies, research could produce a lot more value for the same amount of resources.
The main takeaways of this post are:
There are a lot of different issues that cause waste of resources in the research system
In this post, these issues are categorized as related to (1) choice of research questions, (2) the quality of research and (3) the use of the results produced
Though solutions and reform initiatives are outside the scope of this post, there does seem to be a lot of room for improvement
The terms “value” and “impact” are used very broadly here: it could be lives saved, technological progress, or a better understanding of the universe. The question of what types of value or impact we should expect research to produce is a big one, and not the one I want to focus on here. Instead, I will just assume that when we dedicate resources to research we are expecting some form of valuable outcome or impact. I will attempt to map up inefficiencies in how that is produced.
This post is a result of a collaboration that came out of the EA Global Reconnect conference. After the conference, a group of EA’s interested in improving science started recurring coworking sessions to discuss ideas for related projects and forum posts. I have received invaluable support and feedback from many people in this group, particularly those mentioned as contributors above.
The plan is that contributors to this post and perhaps others in the group will follow up with additional forum posts on metascience that focus more on specific issues, initiatives or solutions. David Reinstein is working on a post titled “Slaying the journals”, a proposal for peer review/rating, archiving, and open science aimed at avoiding rent-extracting publishers, reducing careerist gamesmanship, and making research more effective. There are many previous initiatives that I am not aiming to cover here: just as examples, Open Philanthropy has previously published several pieces related to metascience, and Center for Open Science is working to improve scientific research with a focus on improving transparency and reproducibility.
My hope is that this mapping could be of use to people who want to get an overview of issues in the research system, and to initiate a discussion about potential valuable projects or interventions.
1. Overview
The causes of inefficiencies in the research system can be roughly categorized in three areas: (1) the choice and design of research questions can be flawed, (2) the research that is carried out can suffer from poor methodological quality and/or low reproducibility, and (3) even when research successfully leads to valuable results, they are not adequately incorporated into real-world solutions and decision-making. Each of these three areas can be broken up into underlying drivers, many of which fit into the broad categories of i. publishing, ii. funding and iii. culture.
Figure 1 shows the three main problem categories described in this post in the structure that builds on the idea of a problem tree. The main problem that is in focus (“Research is less effective in producing value than it could be”) is placed at the top and “root causes” of that problem are written out below, with arrows indicating causal relationships.
Figure 1. I...
view more