Originally published by Anne Hastings on the CGAP blog
From the inception of Chemen Lavi Miyo (CLM), Fonkoze’s graduation program for the ultra poor, we were leaning towards conducting a randomized control trial to understand the impact of the program. There were various reasons as to why this was not the ideal solution for us. In the end, we decided upon an evaluation that would combine quantitative and qualitative methods, recognizing that this would be an incredible learning opportunity for us. It was essential that we invest our limited resources in an evaluation that was well designed and that enabled us learn and improve our program.
Karishma Huda and Anton Simanowitz’s evaluation of our program accomplished various things. Firstly, they fed results back to our management team immediately and continuously. The richness and timeliness of their findings enabled us to make the small adjustments that our program needed. Given our limited funding, we were only able to evaluate treatment households. But Karishma and Anton’s methodology made these households come to life and their research afforded us a more nuanced understanding of their conditions – they were not simply numbers and figures, but real households.
At Fonkoze, we knew that our strict targeting criteria had enabled us to target the poorest people. The early research from the evaluation reassured us of this, but also allowed us to develop a richer understanding of the differences between households. The authors categorized participants into ‘fast’ and ‘slow’ climbers, helping us appreciate the concrete differences in their abilities to succeed. This categorization enabled us to recognize the hurdles that slow climbers experienced early on, and how to remove these hurdles to ensure that the slow climbers graduated.
The Village Assistance Committees (VACs), an effort to enhance CLM members’ social networks by connecting them to ‘local elites’ in the village, was of particular interest to us. We took this component from BRAC’s Targeting the Ultra Poor model, and feel that it was integral to CLM members’ success, community ownership of the program, and sustainability of the intervention. Although the field staff were reporting back on the usefulness of the intervention, we needed more evidence. Capturing the dynamics and effectiveness of the VACs quantitatively would have been too complicated, expensive, and would not give us the detail that we were searching for. But through Karishma and Anton’s evaluation, we were able to realize the aspects of the initiative that were effective, and what required further tweaking.
The qualitative methodology in its own right was rich, but mixing quantitative and qualitative methodologies was the evaluation’s key strength. We hired a public health intern to do anthropometric measurements of CLM members’ children to understand their malnutrition levels. Anton and Karishma triangulated this data with our poverty scorecard, the Progress out of Poverty Index, and their qualitative interviews in order to understand the overall trends, but also to capture the detail behind these trends. Each tool provided a different perspective, and bringing it all together in the research helped us to make the programmatic decisions necessary to improve the CLM pilot.
Now that we have this foundational understanding of the program, we feel we are now ready to do an Randomized Control Trial impact assessment in order to gauge exactly which components have the most impact. Now that we are scaling up, this will help us ascertain how to minimize the cost of this program without compromising on its quality.