Evaluating Microfinance: A Guest Post

In recent years, the randomized program evaluation has become the gold standard for evaluating development programs – and the bread and butter of many development economists. The evaluations often uncover valuable new information, but are controversial, and can also be prohibitively expensive to implement for small NGO’s.

Michael Frank, the Director of Finance for the Mali Health Organizing Project, is using an alternative evaluation method, one that he hopes will allow him to evaluate a microfinance program without breaking the bank.

Evaluating Microfinance When Randomization Isn’t an Option
By Michael Frank

Recently, I set out to design a program evaluation that measured the impact of our microfinance program on the community’s wealth, as opposed to just the personal wealth of the loan recipient. My organization is focused on facilitating the development of the slum community in which we work, so if individual loan recipients are increasing their wealth, but doing it at the expense of their competitors next door, we are not achieving our goals. To study community wealth, I decided to track both loan recipients’ businesses and “other affected parties” (competitors, suppliers, distributors, and other nearby businesses).

In a perfect world, I would have done a randomized program evaluation, but that wasn’t an option for my organization due to budgetary and logistical constraints. So I had to come up with an alternative methodology, one that eliminated, or at least minimized and accounted for the selection bias stemming from the fact that people selected to receive funds are likely to be more capable than the average similar-sized business owner. After consulting with a number of advisers and colleagues, I decided to use a variation on the “waiting list-control group” method regularly used in medical studies.

Photo: Mali Health Organizing Project Mali Health Organizing Project loan recipients.

My evaluation design requires a call for loan applicants in the most similar nearby community that does not have a similar microfinance program already present. Using the same criteria as our program uses to select applicants, we will choose 120 (the same number of loans that we give out) loan recipients to be on a waiting list for the loans. We will track these entrepreneurs, and their affected parties, effectively using their community as a control group. After the study is over, we will either give the waiting list loans or compensate them in some other way, so that my tactic of using them as a control group is ethically acceptable.

This is the most efficient bottom-up method of evaluating the impact of our program on the community’s wealth as a whole that I have come up with. I would welcome other suggestions about how to improve the study, or my organization in general. You can either leave them below or email me.