How to program for uncertain results? The innovation journey of a 'slightly unsusual' programme in UNDP
Innovations are driven by risk-takers. Part of UNDP’s role in innovation is to provide the space for risk-takers to develop and test their ideas. And it turns out that sometimes these are not individuals, but entire programmes! The Pacific Risk Resilience Programme (PRRP) covers the Pacific countries of Fiji, Solomon Islands, Tonga and Vanuatu, and takes an unusual approach within UNDP’s programme portfolio. Now in its fourth year, it didn’t exactly follow the standard programming approach in which a challenge and a development model is identified in the beginning, and a set of interventions is designed that would then be rolled out over the course of the next years, with clear activities and results prescribed for each year. Instead, PRRP didn’t actually describe the model or the interventions themselves at all. Instead, they let the model emerge over time by running sprints of interventions and evaluating them frequently, something that is known in the information technology world as ‘agile development’. I've talked to the programme manager, Moortaza Jiwanji, about their approach, what they learned from doing things differently, and the implications of their experience for UNDP programming.
Q: Why did you feel the need to do things differently than in ‘traditional’ programming?
The main reason was that we that we had to develop something for which there was no precedent. We were venturing into unknown territory, and it made little sense for us to prescribe what results would look like 4-years in advance with a results framework that pretends to know exactly what activity would be best to deliver by year 4. We simply couldn’t see how that would work.
Climate change and disasters have a real impact on people in the Pacific. Despite the unprecedented levels of funding and programming in the region, it is disheartening to see how communities are still experiencing the same types of impacts of climate change and disasters and in some cases, these are becoming worse! This is particularly concerning given that the symptoms of climate change such as cyclones, flooding and droughts are likely to increase in intensity and frequency in the future.
Much of this programming in the Pacific has led to some concrete results on the ground, but we felt that there was not enough thought being put into addressing the root causes of these vulnerabilities. It is less obvious to see how development itself is being adjusted to address these risks. For instance, why is it that schools and houses are still built in flood prone areas without the appropriate materials and design codes? It is also becoming increasingly clear that development itself is a primary cause of this vulnerability to climate change. This could perhaps explain the cyclical nature of these impacts.
We realized that something needed to change within development itself and not just in climate change programming. Most programming is focused on technical solutions such as building sea-walls rather than dealing with the root causes. It seemed at the time that there was not much programming experience in dealing with climate change from a ‘development’ perspective. We knew what needed to change but there was limited experience in the region and globally to show us how.
That’s why we decided to develop a model for risk-informed development without any preconceived ideas about what it would look like and how it would work. Our starting point was to address deep-seated governance issues, not for climate change, but for development. We also felt that this would be an opportunity for UNDP to build a niche for itself, particularly given that we are a ‘development’ agency and also deal with governance reform. We were able to do this through the Pacific Risk Resilience Programme (PRRP) which started in 2013. PRRP was funded by the Australian government as they also felt willing to try something different given that they were also not seeing the aggregate results in the region.
Q: What exactly was ‘different’ about your approach?
We knew we had to do two things differently: First, in order to tackle the root causes of climate change and disaster risk, we had to work deep within development itself. And second, because at the time back in 2013 there was not much experience of dealing with climate and disaster risk from a truly ‘development’ perspective, we had to follow an approach that was largely experimental at the time and depart from more traditional approaches to programme design and implementation.
So what was different about our approach? First, unlike most development partners in this area we did not work as an outside partner with climate change and disaster management functions in government. Instead we programmed ‘from within’ governance systems where our government partners owned the development interventions from community to national level. We also used a human-centered design approach that really focused on developing individual mechanisms with the same people that were going to apply them in their government ministries and agencies. Both these aspects allowed our country partners to help design and fully lead the initiatives themselves rather than UNDP leading the way. This admittedly raised some eyebrows at the time as there was an expectation for climate change to work with (and provide funding for) the ‘usual suspects’.
Second, unlike standard programme design approaches we did not predetermine our activities and outputs well in advance for the next four years. Instead we built smaller and targeted experimental interventions where we saw some prospects of success (see adjacent diagram), e.g. where we found receptive partners and conducive political environments. This allowed us to understand which activities did yield the best results as we implemented them. We then spent a great deal of time and energy in measuring any apparent successes and even more importantly failures. Learning from these experiences was the most important ingredient and we spent a great deal of time and energy on doing this collectively with our partners. So whether a pilot leads to measurable successes or failures, the real success of this approach comes in how well you learn and subsequently redesign and modify approaches based on these learnings. Based on this iterative process we would then develop the overarching model as it emergedfrom those experiences. The interesting thing about this experience is that we developed this modus operandi ourselves, completely unaware that UNDP was promoting through its global Innovation Facility exactly such innovation and design approaches that encourage this type of experimentation. At the time of designing this programme back in 2013 we decided to call this approach ‘emergent design’, but it aligns very much with the innovation principles of agile development and problem-driven adaptive iteration.
Based on the learning from these experiments, we have now developed an overarching model around the concept of ‘risk governance’, designed and tested to risk-inform development ‘from within’ and at all levels of governance. For more information you can read our recently launched policy brief and you can also see practical examples of how this is benefiting countries in the Pacific on our website www.pacific-prrp.org.
Q: What were the challenges you encountered?
Risky business. Developing a programme based on emergent design or agile development principles is extremely exciting. However, it can also be quite stressful because in essence you are taking a significant risk in programming something that has not been tested successfully yet. This is particularly challenging when it comes to convincing your programme stakeholders, such as your donor, country partners and even internal management.
Raising eyebrows. In the early days, we seemed to develop somewhat of a reputation as being the slightly unusual programme within UNDP. This was not always cast in a positive light and this is partly because we did not have a fixed and clearly defined results and resources framework over a four-year period.
Buy-in from stakeholders. There are three types of stakeholders that we dealt with through this experience: the country partners (or beneficiaries); our donor partner; and UNDP itself. The approach of working from within and building governance systems to risk-inform development was most positively received by our government and donor partners and then eventually with our managers within UNDP. This took a little while perhaps largely because we were venturing into the unknown and did not have a clear narrative to describe and justify our approach, particularly in the early days.
It can take some time to show predictable and regular results. Agile development or emergent design approaches can take some time to achieve tangible results. It is almost by definition impossible to predict when and how results are going to be achieved. This was particularly challenging when working in an environment where programmes are expected to report on results against clearly defined outputs and targets at least every quarter.
Q: What were the benefits of taking this approach compared to more traditional approaches?
In essence, agile development allowed us to get results that otherwise would have never emerged had we prescribed our specific outputs the traditional way several years in advance. And on top of it, the solutions that we did get through the agile development approach now address the actual problem we’re trying to tackle much better.
Over time we saw that taking this approach was extremely beneficial, particularly to our government partners, in offering more sustainable and realistic solutions to the complexities of climate change and disasters in the Pacific. You can see this by the fact that our country partners are now advocating for this approach within their own countries.
Unexpected solutions. What’s really interesting is that taking this approach has led to solutions that we would have never designed up front. For instance, we now have Ministries of Women leading on climate-informing community development initiatives. Private sector networks are now being formed to not only work better together in times of disasters but also to provide a more effective link with government and partners. Local governments are leading the way in risk-informing infrastructure projects. You can see these examples and others on our website under ‘Results’ on www.pacific-prrp.org.
Ability to adapt. Secondly, most of our country partners have really appreciated the ability of UNDP to adapt to a constantly changing environment. They often feel that projects that have fixed activities and outputs for a four-year time horizon are unrealistic and can compromise their own ability to initiate real change on the ground. What we see now is that our partners are leading the way and collectively we continue to discover new innovations.
Finally, taking this approach to development programming is immensely rewarding both on a professional and personal level. It almost feels as if there is no other way to deal with the complexities of development in the Pacific and even beyond.
Q: What would you recommend to others who want to take this approach?
I would recommend four key things. First, don’t be afraid to fail and be completely open about this to your partners. This is critical in finding innovative solutions to complex development challenges. Secondly, invest in smaller and manageable initiatives through prototypes. This will help minimize your risks and allow for real creativity. Third, you will have to tailor your results framework in a way that you frame your described activities and outputs as e.g. number of experiments run and evaluated, number of experiments identified for scaling up, etc., rather that describing up front what these experimental interventions will specifically look like. This will give you the leeway to explore uncommon and innovative solutions, while at the same time hold yourself accountable to measurable milestones within this agile development journey. Finally, taking a leap into the unknown can be risky and cause negative perceptions of your work around you. Develop a small group of like-minded colleagues from within and outside the organization who are genuinely willing to try this out and support you. At the same time, it is imperative to engage management early on in an open but confident way about what you are doing and why.
Q: What could all this mean for the future of UNDP’s programming?
We had very interesting conversations with counterparts within institutional donor organizations who frankly told us that refining this agile development approach further could be very rewarding for UNDP. It would allow the organization to position itself as a unique implementing partner that can offer a different way of programming than most other implementation contractors, especially in programmes that try to tackle government reform issues. I feel that the future for UNDP and similar organisations working in this space lies in innovating its programming itself through such agile development, or ‘emergent design’ principles. Not exclusively, but at least as part of its portfolio. Not only is there a lack of this approach in the development space, but more country partners will want this because it is particularly suited to addresses complex development challenges for which no clear solutions exist yet. This needs to go beyond mimicry though and requires fundamental behavioral shifts in terms of how we design, execute and evaluate our work. But the outcomes are worth it. As I said, this has been the most rewarding professional and personal experience for me so far.
Q: Why did you feel the need to do things differently than in ‘traditional’ programming?
The main reason was that we that we had to develop something for which there was no precedent. We were venturing into unknown territory, and it made little sense for us to prescribe what results would look like 4-years in advance with a results framework that pretends to know exactly what activity would be best to deliver by year 4. We simply couldn’t see how that would work.
Climate change and disasters have a real impact on people in the Pacific. Despite the unprecedented levels of funding and programming in the region, it is disheartening to see how communities are still experiencing the same types of impacts of climate change and disasters and in some cases, these are becoming worse! This is particularly concerning given that the symptoms of climate change such as cyclones, flooding and droughts are likely to increase in intensity and frequency in the future.
Much of this programming in the Pacific has led to some concrete results on the ground, but we felt that there was not enough thought being put into addressing the root causes of these vulnerabilities. It is less obvious to see how development itself is being adjusted to address these risks. For instance, why is it that schools and houses are still built in flood prone areas without the appropriate materials and design codes? It is also becoming increasingly clear that development itself is a primary cause of this vulnerability to climate change. This could perhaps explain the cyclical nature of these impacts.
We realized that something needed to change within development itself and not just in climate change programming. Most programming is focused on technical solutions such as building sea-walls rather than dealing with the root causes. It seemed at the time that there was not much programming experience in dealing with climate change from a ‘development’ perspective. We knew what needed to change but there was limited experience in the region and globally to show us how.
That’s why we decided to develop a model for risk-informed development without any preconceived ideas about what it would look like and how it would work. Our starting point was to address deep-seated governance issues, not for climate change, but for development. We also felt that this would be an opportunity for UNDP to build a niche for itself, particularly given that we are a ‘development’ agency and also deal with governance reform. We were able to do this through the Pacific Risk Resilience Programme (PRRP) which started in 2013. PRRP was funded by the Australian government as they also felt willing to try something different given that they were also not seeing the aggregate results in the region.
Q: What exactly was ‘different’ about your approach?
We knew we had to do two things differently: First, in order to tackle the root causes of climate change and disaster risk, we had to work deep within development itself. And second, because at the time back in 2013 there was not much experience of dealing with climate and disaster risk from a truly ‘development’ perspective, we had to follow an approach that was largely experimental at the time and depart from more traditional approaches to programme design and implementation.
So what was different about our approach? First, unlike most development partners in this area we did not work as an outside partner with climate change and disaster management functions in government. Instead we programmed ‘from within’ governance systems where our government partners owned the development interventions from community to national level. We also used a human-centered design approach that really focused on developing individual mechanisms with the same people that were going to apply them in their government ministries and agencies. Both these aspects allowed our country partners to help design and fully lead the initiatives themselves rather than UNDP leading the way. This admittedly raised some eyebrows at the time as there was an expectation for climate change to work with (and provide funding for) the ‘usual suspects’.
Figure 1: The Innovation Feedback Loop
Second, unlike standard programme design approaches we did not predetermine our activities and outputs well in advance for the next four years. Instead we built smaller and targeted experimental interventions where we saw some prospects of success (see adjacent diagram), e.g. where we found receptive partners and conducive political environments. This allowed us to understand which activities did yield the best results as we implemented them. We then spent a great deal of time and energy in measuring any apparent successes and even more importantly failures. Learning from these experiences was the most important ingredient and we spent a great deal of time and energy on doing this collectively with our partners. So whether a pilot leads to measurable successes or failures, the real success of this approach comes in how well you learn and subsequently redesign and modify approaches based on these learnings. Based on this iterative process we would then develop the overarching model as it emergedfrom those experiences. The interesting thing about this experience is that we developed this modus operandi ourselves, completely unaware that UNDP was promoting through its global Innovation Facility exactly such innovation and design approaches that encourage this type of experimentation. At the time of designing this programme back in 2013 we decided to call this approach ‘emergent design’, but it aligns very much with the innovation principles of agile development and problem-driven adaptive iteration.
Based on the learning from these experiments, we have now developed an overarching model around the concept of ‘risk governance’, designed and tested to risk-inform development ‘from within’ and at all levels of governance. For more information you can read our recently launched policy brief and you can also see practical examples of how this is benefiting countries in the Pacific on our website www.pacific-prrp.org.
Q: What were the challenges you encountered?
Risky business. Developing a programme based on emergent design or agile development principles is extremely exciting. However, it can also be quite stressful because in essence you are taking a significant risk in programming something that has not been tested successfully yet. This is particularly challenging when it comes to convincing your programme stakeholders, such as your donor, country partners and even internal management.
Raising eyebrows. In the early days, we seemed to develop somewhat of a reputation as being the slightly unusual programme within UNDP. This was not always cast in a positive light and this is partly because we did not have a fixed and clearly defined results and resources framework over a four-year period.
Buy-in from stakeholders. There are three types of stakeholders that we dealt with through this experience: the country partners (or beneficiaries); our donor partner; and UNDP itself. The approach of working from within and building governance systems to risk-inform development was most positively received by our government and donor partners and then eventually with our managers within UNDP. This took a little while perhaps largely because we were venturing into the unknown and did not have a clear narrative to describe and justify our approach, particularly in the early days.
It can take some time to show predictable and regular results. Agile development or emergent design approaches can take some time to achieve tangible results. It is almost by definition impossible to predict when and how results are going to be achieved. This was particularly challenging when working in an environment where programmes are expected to report on results against clearly defined outputs and targets at least every quarter.
Q: What were the benefits of taking this approach compared to more traditional approaches?
In essence, agile development allowed us to get results that otherwise would have never emerged had we prescribed our specific outputs the traditional way several years in advance. And on top of it, the solutions that we did get through the agile development approach now address the actual problem we’re trying to tackle much better.
Over time we saw that taking this approach was extremely beneficial, particularly to our government partners, in offering more sustainable and realistic solutions to the complexities of climate change and disasters in the Pacific. You can see this by the fact that our country partners are now advocating for this approach within their own countries.
Unexpected solutions. What’s really interesting is that taking this approach has led to solutions that we would have never designed up front. For instance, we now have Ministries of Women leading on climate-informing community development initiatives. Private sector networks are now being formed to not only work better together in times of disasters but also to provide a more effective link with government and partners. Local governments are leading the way in risk-informing infrastructure projects. You can see these examples and others on our website under ‘Results’ on www.pacific-prrp.org.
Ability to adapt. Secondly, most of our country partners have really appreciated the ability of UNDP to adapt to a constantly changing environment. They often feel that projects that have fixed activities and outputs for a four-year time horizon are unrealistic and can compromise their own ability to initiate real change on the ground. What we see now is that our partners are leading the way and collectively we continue to discover new innovations.
Finally, taking this approach to development programming is immensely rewarding both on a professional and personal level. It almost feels as if there is no other way to deal with the complexities of development in the Pacific and even beyond.
Q: What would you recommend to others who want to take this approach?
I would recommend four key things. First, don’t be afraid to fail and be completely open about this to your partners. This is critical in finding innovative solutions to complex development challenges. Secondly, invest in smaller and manageable initiatives through prototypes. This will help minimize your risks and allow for real creativity. Third, you will have to tailor your results framework in a way that you frame your described activities and outputs as e.g. number of experiments run and evaluated, number of experiments identified for scaling up, etc., rather that describing up front what these experimental interventions will specifically look like. This will give you the leeway to explore uncommon and innovative solutions, while at the same time hold yourself accountable to measurable milestones within this agile development journey. Finally, taking a leap into the unknown can be risky and cause negative perceptions of your work around you. Develop a small group of like-minded colleagues from within and outside the organization who are genuinely willing to try this out and support you. At the same time, it is imperative to engage management early on in an open but confident way about what you are doing and why.
Q: What could all this mean for the future of UNDP’s programming?
We had very interesting conversations with counterparts within institutional donor organizations who frankly told us that refining this agile development approach further could be very rewarding for UNDP. It would allow the organization to position itself as a unique implementing partner that can offer a different way of programming than most other implementation contractors, especially in programmes that try to tackle government reform issues. I feel that the future for UNDP and similar organisations working in this space lies in innovating its programming itself through such agile development, or ‘emergent design’ principles. Not exclusively, but at least as part of its portfolio. Not only is there a lack of this approach in the development space, but more country partners will want this because it is particularly suited to addresses complex development challenges for which no clear solutions exist yet. This needs to go beyond mimicry though and requires fundamental behavioral shifts in terms of how we design, execute and evaluate our work. But the outcomes are worth it. As I said, this has been the most rewarding professional and personal experience for me so far.
Comments