Findings from the first DFID development impact bonds evaluation report

The UK's Department for International Development (DFID) launched in 2017 a development impact bonds (DIBs) pilot programme that aims to generate understanding of how and when DIBs can enable efficient and effective delivery of programmes in DFID priority results areas. Ecorys, a research and consulting organisation, was commissioned to undertake an evaluation study of the DIBs pilot programme, including an in-depth review of four case studies: International Committee of the Red Cross Humanitarian Impact Bond, Quality Education India DIB, Village Enterprise DIB, and Cameroon Cataract Bond. In this blog James Ronicle, Associate Director at Ecorys and former GO Lab Fellow of Practice, reflects on some of the most interesting and surprising findings from the evaluation report.

Ecorys has been evaluating impact bonds for almost six years, with multiple projects spanning over 40 impact bonds. We have developed a strong understanding of how impact bonds affect the design, delivery and performance of services. You would therefore think that we would have a reasonably good expectation of how the DIB mechanism would affect the set up and design of development interventions. Whilst some of the findings from the first wave of our evaluation met these expectations, we were also surprised by some of the findings, and I have described some of the most surprising findings below. 

A development impact bond (DIB) is an outcomes-based funding structure for the delivery of public services in low- and middle-income countries. DIBs leverage government aid agency or philanthropic funding, non-profit service provision, and private capital. DIBs are an adaptation of the social impact bond, pay for success, and social benefit bond models used in high income countries such as the UK, US, and Australia. For more information, read GO Lab’s introduction to impact bonds.

Variation across DIBs 

One thing that we found particularly surprising was the degree of variation within these four DIBs. Some things were reasonably consistent and elements you would expect to see in any impact bond  – payments attached to outcomes and upfront financing being provided by organisations that were not delivering the intervention. However, beyond that the DIBs are different in almost all aspects, such as their size (from $2m to the equivalent of $25.7m) and levels of capital guarantees, that is, the proportion of investor capital that is guaranteed to be returned to investors (from 0% to 100%). Even the types of stakeholders vary, with outcomes funders ranging from bilateral donors to foundations, and investors ranging from commercial to philanthropic organisations. DIBs therefore need to be seen more as a loose group of outcomes-based funding structures, rather than as a specific mechanism. 

Yet despite the variation across the DIBs, what was equally surprising was there were some consistent effects on the projects as a result of using the DIB mechanism - the so called ‘DIB effect’. The main effect of the DIB was in ‘pushing the boundaries’ of where it might be possible to introduce a results-based financing (RBF) approach. By sharing the financial risk of non-performance between service providers, investors and outcomes funders, this enabled projects to be funded that, without such risk sharing, could not have been launched through a RBF contract because the service provider would have struggled to bear all the risk themselves.

Further to this, the DIB fostered new working relationships between stakeholders and led to greater levels of collaboration than is normally seen, primarily because the DIB aligned all stakeholders’ interests but also because the intensive design stage forced closer partnership working.

What was the 'DIB effect'? 

However, we also need to ask ourselves the extent to which the effects we saw were indeed ‘DIB effects’, or whether they were in fact ‘novelty effects’; the effect whereby individuals may perceive and respond differently in a situation that is novel compared with how they would in a normal situation*. There were a number of DIB effects that may have materialised, or strengthened, because these are some of the first DIBs to be launched, and these effects may diminish over time. This ‘novelty effect’ appeared to have both positive effects on the projects (in terms of fostering a sense of commitment to ensure challenges were overcome and the projects launched), and negative effects (in terms of creating nervousness at the notion of the first DIBs ‘failing’, which led to some risk aversion around the selection of service providers and levels of risk taken on by investors; this may lead some to question the added value of the DIB).  

This report sheds some light on how the DIB mechanism affects the set up and design of development projects. Future reports will examine how the DIB mechanism affects project delivery (in 2020) and performance and sustainability (in 2022). We expect these reports to carry on surprising both us, and others, as they continue to develop our understanding of the purpose and value of impact bonds. 

Read the report  

Read the Independent Evaluation of the UK DFID DIBs Pilot Programme. You can also read a summary of the report. 

*Gravetter and Forzano, 2018. Research Methods for the behavioural Sciences. Cengage learning.