Ingrid Harvold Kvangraven
This week it was announced that Abhijit Banerjee, Esther Duflo and Michael Kremer won the Economics Nobel Prize (or more accurately: the ‘Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel’). The trio of economists were awarded the prize for “their experimental approach to alleviating global poverty”.
On social media and in mainstream newspapers, there was an exceptional level of praise for the laureates, reflecting their existing rockstar status within development economics. The Financial Times even claimed that the Nobel “will help restore [the] profession’s relevance”. However, the widespread calls for celebration need to be considered with a cautionary counterweight.
The experimental approach to poverty alleviation relies on so-called Randomised Control Trials (RCTs). Inspired by studies in medicine, the approach targets specific interventions to a randomly selected group (schools, classes, mothers, etc), and then compares how specific outcomes change in the recipient group versus those who did not receive the treatment. As the groups are assumed to be otherwise similar, the difference in outcomes can be causally attributed to the intervention.
While the laureates first pioneered this work in the 1990s in Kenyan schools, the approach is now widely considered the new “gold standard” in development economics, also sometimes simply called “New Economics”. The approach has become enormously influential among governments, international agencies and NGOs. The body of work pioneered by the laureates, or the randomistas as they are sometimes called, is meant to alleviate poverty through simple interventions such as combating teacher absenteeism, through cash transfers, and through stimulating positive thinking among the people living in poverty. Sounds good so far?
While the laureates’ approach to poverty research and policy may seem harmless, if not laudable, there are many reasons for concern. Both heterodox and mainstream economists as well as other social scientists have long provided a thorough critique of the turn towards RCTs in economics, on philosophical, epistemological, political and methodological grounds. The concerns with the approach can be roughly grouped into questions of focus, theory and methodology.
Focus: tackling symptoms and thinking small
The approach that is being promoted is concerned with poverty, not development, and is thus a part of the larger trend in development economics that is moving away from development as structural transformation to development as poverty alleviation. This movement towards “thinking small” is a part of a broader trend, which has squeezed out questions related to global economic institutions, trade, agricultural, industrial and fiscal policy, and the role of political dynamics, in favour of the best ways to make smaller technical interventions.
The interventions considered by the Nobel laureates tend to be removed from analyses of power and wider social change. In fact, the Nobel committee specifically gave the Nobel to Banerjee, Duflo and Kremer for addressing “smaller, more manageable questions,” rather than big ideas. While such small interventions might generate positive results at the micro-level, they do little to challenge the systems that produce the problems.
For example, rather than challenging the cuts to the school systems that are forced by austerity, the focus of the randomistas directs our attention to absenteeism of teachers and the effects of school meals and the number of teachers in the classroom on learning. Meanwhile, their lack of challenge to the existing economic order is perhaps also precisely one of the secrets to media and donor appeal, and ultimately also their success.
The lack of engagement with the conditions that create poverty has led many critics to question as to what extent RCTs will actually be able to significantly reduce global poverty.
Theory: methodological individualism lives on
In a 2017 speech, Duflo famously likened economists to plumbers. In her view the role of an economist is to solve real world problems in specific situations. This is a dangerous assertion, as it suggests that the “plumbing” the randomistas are doing is purely technical, and not guided by theory or values. However, the randomistas’ approach to economics is neither objective and value-neutral, nor pragmatic, but rather, rooted in a particular theoretical framework and world view—neoclassical microeconomic theory and methodological individualism.
The experiments’ grounding has implications for how experiments are designed and the underlying assumptions about individual and collective behavior that are made. Perhaps the most obvious example of this is that the laureates often argue that specific aspects of poverty can be solved by correcting cognitive biases. Unsurprisingly, there is much overlap between the work of randomistas and the mainstream behavioral economists, including a focus on nudges that may facilitate better choices on the part of people living in poverty.
Another example is Duflo’s analysis of women empowerment. Naila Kabeer argues that it employs an understanding of human behavior “uncritically informed by neoclassical microeconomic theory.” Since all behavior can allegedly be explained as manifestations of individual maximising behavior, alternative explanations are dispensed with. Because of this, Duflo fails to understand a series of other important factors related to women’s empowerment, such as the role of sustained struggle by women’s organisations for rights or the need to address unfair distribution of unpaid work that limits women’s ability to participate in the community.
Note that there is nothing embedded in RCTs that forces randomistas to assume individuals are rational optimising agents. These assumptions come from the economics tradition. This is therefore not a critique of RCTs per se, but of the way RCTs are employed in the laureates’ work and in most of mainstream economics.
Method: If you didn’t randomise it, is it really knowledge?
While understanding causal processes is important in development economics, as in other social science disciplines, RCTs do so in a very limited way. The causal model underlying RCTs focuses on causal effects rather than causal mechanisms. Not only do RCTs not tell us exactly what mechanisms are involved when something works, they also do not tell us whether the policy in question can be reliably implemented elsewhere. In order to make such a judgement, a broader assessment of economic and social realities is unavoidable.
Assuming that interventions are valid across geographies and scale suggests that micro results are independent of their macroeconomic environment. However, while “effects” on individuals and households are not separate from the societies in which they exist, randomistas give little acknowledgement to other ways of knowing about the world that might help us better understand individual motivations and socio-economic situations. As it is difficult to achieve truly random sampling in human communities, it is perhaps not surprising that when RCTs are replicated, they may come to substantially different results than the original.
Not only do RCTs rarely have external validity, but the specific circumstances needed to understand the extent to which the experiments may have external validity are usually inadequately reported. This has led even critics within the mainstream to argue that there are misunderstandings about what RCTs are capable of accomplishing. A deeper epistemological critique involves the problematic underlying assumption that there is one specific true impact that can be uncovered through experiments.
Recent research has found that alternative attempts to assess the success of programs transferring assets to women in extreme poverty in West Bengal and Sindh have been far superior to RCTs, which provide very limited explanations for the patterns of outcomes observed. The research concludes that it is unlikely that RCTs will be able to acknowledge the central role of human agency in project success if they confine themselves to quantitative methods alone.
There are also serious ethical problems at stake. Among these are issues such as lying, instrumentalising people, the role of consent, accountability and foreign intervention, in addition to the choice of who gets treatment. While ethical concerns regarding potential harm to groups is discussed extensively in the medical literature, it receives less attention in economics, despite the many ethically dubious experimental studies (e.g. allowing bribes for people to get their drivers’ licences in India or incentivising Hong Kong university students into participation in an anti-authoritarian protest). Finally, the colonial dimensions of US-based researchers intervening to estimate what is best for people in the Global South cannot be ignored.
Why it matters: limits to knowledge and policy-making
There will always be research that is more or less relevant for development, so why does it matter what the randomistas do? Well, as the Nobel Committee stated, their “experimental research methods now entirely dominate development economics”. A serious epistemological problem arises when the definition of what rigour and evidence means gets narrowed down to one single approach that has so many limitations. This shift has taken place over the past couple of decades in development economics, and is now strengthened by the 2019 Nobel Prize. As both Banerjee and Duflo acknowledged in interviews after the prize was announced, this is not just a prize for them, but a prize for the entire movement.
The discipline has not always been this way. The history of thought on development economics is rich with debates about how capital accumulation differs across space, the role of institutions in shaping behavior and economic development, the legacies of colonialism and imperialism, unequal exchange, the global governance of technology, the role of fiscal policy, and the relationship between agriculture and industry. The larger questions have since been pushed out of the discipline, in favour of debates about smaller interventions.
The rise of the randomistas also matters because the randomistas are committed to provoke results, not just provide an understanding of the situations in which people living in poverty find themselves. In fact, it is one of their stated goals to produce a “better integration between theory and empirical practice”. A key argument by the randomistas is that “all too often development policy is based on fads, and randomised evaluations could allow it to be based on evidence”.
However, the narrowness of the randomised trials is impractical for most forms of policies. While RCTs tend to test at most a couple of variations of a policy, in the real world of development, interventions are overlapping and synergistic. This reality recently led 15 leading economists to call to “evaluate whole public policies” rather than assess “short-term impacts of micro-projects,” given that what is needed is systems-level thinking to tackle the scale of overlapping crises. Furthermore, the value of experimentation in policy-making, rather than promoting pre-prescribed policies, should not be neglected.
The concept of “evidence-based policy” associated with the randomistas needs some unpacking. It is important to note that policies are informed by reflections on values and objectives, which economists are not necessarily well-suited to intervene in. Of course, evidence should be a part of a policy-making process, but the pursuit of ineffective policies is often driven by political priorities rather than lack of evidence.
While randomistas might respond to this by arguing that their trials are precisely meant to de-politicise public policy, this is not necessarily a desirable step. Policy decisions are political in nature, and shielding these value judgements from public scrutiny and debate does little to strengthen democratic decision making. Suggesting that policy-making can be depoliticised is dangerous and it belittles the agency and participation of people in policy-making. After all, why should a policy that has been proven effective through an RCT carry more weight than, for example, policies driven by people’s demands and political and social mobilisation?
While the Nobel Prize does leave those of us concerned with broader political economy challenges in the world anxious, not everything is doom and gloom. Firstly, the Nobel directs attention to the persistence of poverty in the world and the need to do something about it. What we as critical development economists now need to do is to challenge the fact that the Prize also legitimises a prescriptive view of how to find solutions to global problems.
Secondly, the fact that a woman and a person of colour were awarded a prize that is usually reserved for white men is a step forward for a more open and inclusive field. Duflo herself recognises that the gender imbalance among Nobel Prize winners reflects a “structural” problem in the economics profession and that her profession lacks ethnic diversity.
However, it is obvious that to challenge racism, sexism and Eurocentrism in economics, it is not enough to simply be more inclusive of women and people of colour among those who are firmly placed at the top of the narrow, Eurocentric mainstream. To truly achieve a more open and democratic science it is necessary to push for a field that is welcoming of a plurality of viewpoints, methodologies, theoretical frameworks, forms of knowledge and perspectives.
This is a massive challenge, but the systemic, global crises we face require broad, interdisciplinary engagement in debates about possible solutions.
(Ingrid Harvold Kvangraven is a Lecturer in International Development at the University of York.)
