Minga Blog: Evaluation—How we know when we’re contributing to social change

The world of development is lush with evaluation, a jungle of metrics and indicators meant to show how well the work is going and yet how much more we have to do. From dusk to dawn, the development worker must navigate this jungle, crafting monitoring and evaluation plans in project proposals, holding themselves accountable in weekly team meetings, tallying progress in interim reports, proudly reporting (or massaging) the numbers in final project documents, and then spinning it all into the perfect pitch in advertisements to potential donors.

This is a paradise for the bean-counting technocrats who thrive in this rich ecosystem of numbers, from basic inputs (the number of bed nets distributed, the number of nutritionally balanced meals donated, the number of refugees housed, the number of workshops run and the total participants) to complex outcomes (microbusinesses started and maintained for five years, families “lifted” out of food insecurity, increases in GDP or HDI).

And yet the rest of us might be forgiven if, tripping through this gnarly thicket of simple, observable, and measurable “indicators,” we ask, “but how are you actually making peoples lives better?” Or we say, “I know you’ve done a lot. And I know that x action led to y and z outcomes. But what really changed?” Because at the end of the day we do not aim to create a world full of bed nets or workshops or microbusinesses or high GDPs. We aim for a world that is changed, a world that is so radically changed that poverty, ill health, food insecurity, and exploitation cannot be created in the first place.

At Minga, we express this vision quite clearly in our mission: to build capacity for health and social change. But how do you evaluate that?

 Measuring whether we improve health is pretty straightforward. Tally the problems before and after, adjust for other influences, and there you go. So in our early work in Ecuador, for example, we can look at the extraordinarily high rates of malaria infection in the region of El Páramo before we began, the near-zero rates of malaria today, and we can be fairly confident that our efforts to distribute bed nets, provide local lab testing, secure access to treatment, and conduct culturally appropriate1 health education had a significant impact.

To some extent, it’s not much harder to evaluate whether we are building capacity for health. Just look at the infrastructure that we’ve put in place: health centers, laboratories, and medical staff; trained community health workers; boreholes and piping to ensure access to clean water; community gardens to enhance food security; etc. But capacity for health entails more than just the existence of health infrastructure. It also includes the ability of individuals and communities to identify risks, develop solutions, mobilize resources, secure assistance, etc. These more subtle impacts require much more nuanced evaluation. One way that we assess these more subtle aspects of community capacity for health is by examining changes in the ways that our partners describe their work, challenges, and goals. Did their analysis of community assets, problems, and responses change from the initial proposal to their final report? Did their experience with a Minga-funded project provide them with clear lessons? And are they putting these lessons into practice via new funding proposals, changes to their projects, revisions to their mission, etc.?

And that brings us to the most complex, but arguably the most important, aspect of our work—to build capacity for social change. Here we are talking not just measurable project outcomes, nor even lessons for health programming, but rather about developing forms of community power that actually challenge local patterns of exploitation and marginalization. To evaluate these impacts requires that we first understand what historical, cultural, political, and economic factors actually contribute to people’s marginalization. And then we have to look for changes.

Our work with BUVAD in Uganda provides a great example. As in many places, the small farmers of the Kayunga District are multiply marginalized. Their wellbeing and stability is affected by economic factors (small land holdings, limited ownership of agricultural technologies, and a low place on the commodity chain) and biological factors (limited access to clean water, in part because of colonial and post-colonial land use policies; thus long trips to wells and high burdens of water-borne illnesses). Addressing these issues—as most development work attempts to do—matters a great deal, but that alone is not sufficient to facilitate social change. We therefore also try to attend to socio-political factors (low levels of political influence, little attention from public servants, poor representation in policy realms). Thus, in our work with BUVAD and its fantastic executive director, Stephen, we not only provided funding for new wells but also set in motion a range of initiatives to change Kayunga’s political marginalization.

How do we know? These are some of the indicators we see:

  1. Increases in solidarity and a collective voice: Minga made a modest contribution to building national solidarity by bringing BUVAD into UWASNET, a network of NGOs working on water and sanitation issues. BUVAD is also building solidarity and self-governance at the local level via well governance committees.
  2. Increased influence over local policy: Experience with our project helped Stephen get onto the Kayunga District Water and Sanitation Committee, bolstering grassroots voices in local policy making.
  3. Access to data and education to support community demands and reduce stereotyping and the dismissal of “poor” rural people: Monitoring and evaluation work for our project equipped BUVAD with hard data that they can now bring to government officials to strengthen their demands for the fulfillment of basic infrastructure rights.
  4. Increasing autonomy so communities can not only affect the powerful but also thrive independently of them: Since working with us, Stephen has written successful grants to simultaneously boost local autonomy from those government officials and other outsiders via training for a local well mechanic and the installation of passive rainwater harvesting infrastructure.

These are all small changes, but collectively they begin to shift the dynamics that lead to people’s disempowerment in the first place. At Minga, we believe that the most significant and positive transformations will come when standard development is accompanied by this deeper socio-political work. That’s why, in each of our projects, we’re trying to figure out which factors really do build community capacity for social change—not to prove we’re great or insist that we’re still needed, but to figure out how to do this work better. We’ll continue to keep you posted!

1 – In this case, “culturally appropriate” is not just a “buzzword or fuzzword,” to quote Andrea Cornwall. Members of Minga (then Foundation Human Nature) and their international counterparts learned that people in El Páramo were not seeking biomedical attention for malaria because evil eye manifested with similar symptoms and, it was believed, consulting a medical doctor for evil eye could kill you. By carefully studying the differences in symptoms, our team was able to discern a difference in the timing of a malarial fever compared to an evil eye fever, and by teaching people to see the difference we helped get over this barrier to medical attention.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s