Minga Blog: Evaluation—How we know when we’re contributing to social change

The world of development is lush with evaluation, a jungle of metrics and indicators meant to show how well the work is going and yet how much more we have to do. From dusk to dawn, the development worker must navigate this jungle, crafting monitoring and evaluation plans in project proposals, holding themselves accountable in weekly team meetings, tallying progress in interim reports, proudly reporting (or massaging) the numbers in final project documents, and then spinning it all into the perfect pitch in advertisements to potential donors.

This is a paradise for the bean-counting technocrats who thrive in this rich ecosystem of numbers, from basic inputs (the number of bed nets distributed, the number of nutritionally balanced meals donated, the number of refugees housed, the number of workshops run and the total participants) to complex outcomes (microbusinesses started and maintained for five years, families “lifted” out of food insecurity, increases in GDP or HDI).

And yet the rest of us might be forgiven if, tripping through this gnarly thicket of simple, observable, and measurable “indicators,” we ask, “but how are you actually making peoples lives better?” Or we say, “I know you’ve done a lot. And I know that x action led to y and z outcomes. But what really changed?” Because at the end of the day we do not aim to create a world full of bed nets or workshops or microbusinesses or high GDPs. We aim for a world that is changed, a world that is so radically changed that poverty, ill health, food insecurity, and exploitation cannot be created in the first place.

At Minga, we express this vision quite clearly in our mission: to build capacity for health and social change. But how do you evaluate that?

 Measuring whether we improve health is pretty straightforward. Tally the problems before and after, adjust for other influences, and there you go. So in our early work in Ecuador, for example, we can look at the extraordinarily high rates of malaria infection in the region of El Páramo before we began, the near-zero rates of malaria today, and we can be fairly confident that our efforts to distribute bed nets, provide local lab testing, secure access to treatment, and conduct culturally appropriate1 health education had a significant impact.

To some extent, it’s not much harder to evaluate whether we are building capacity for health. Just look at the infrastructure that we’ve put in place: health centers, laboratories, and medical staff; trained community health workers; boreholes and piping to ensure access to clean water; community gardens to enhance food security; etc. But capacity for health entails more than just the existence of health infrastructure. It also includes the ability of individuals and communities to identify risks, develop solutions, mobilize resources, secure assistance, etc. These more subtle impacts require much more nuanced evaluation. One way that we assess these more subtle aspects of community capacity for health is by examining changes in the ways that our partners describe their work, challenges, and goals. Did their analysis of community assets, problems, and responses change from the initial proposal to their final report? Did their experience with a Minga-funded project provide them with clear lessons? And are they putting these lessons into practice via new funding proposals, changes to their projects, revisions to their mission, etc.?

And that brings us to the most complex, but arguably the most important, aspect of our work—to build capacity for social change. Here we are talking not just measurable project outcomes, nor even lessons for health programming, but rather about developing forms of community power that actually challenge local patterns of exploitation and marginalization. To evaluate these impacts requires that we first understand what historical, cultural, political, and economic factors actually contribute to people’s marginalization. And then we have to look for changes.

Our work with BUVAD in Uganda provides a great example. As in many places, the small farmers of the Kayunga District are multiply marginalized. Their wellbeing and stability is affected by economic factors (small land holdings, limited ownership of agricultural technologies, and a low place on the commodity chain) and biological factors (limited access to clean water, in part because of colonial and post-colonial land use policies; thus long trips to wells and high burdens of water-borne illnesses). Addressing these issues—as most development work attempts to do—matters a great deal, but that alone is not sufficient to facilitate social change. We therefore also try to attend to socio-political factors (low levels of political influence, little attention from public servants, poor representation in policy realms). Thus, in our work with BUVAD and its fantastic executive director, Stephen, we not only provided funding for new wells but also set in motion a range of initiatives to change Kayunga’s political marginalization.

How do we know? These are some of the indicators we see:

  1. Increases in solidarity and a collective voice: Minga made a modest contribution to building national solidarity by bringing BUVAD into UWASNET, a network of NGOs working on water and sanitation issues. BUVAD is also building solidarity and self-governance at the local level via well governance committees.
  2. Increased influence over local policy: Experience with our project helped Stephen get onto the Kayunga District Water and Sanitation Committee, bolstering grassroots voices in local policy making.
  3. Access to data and education to support community demands and reduce stereotyping and the dismissal of “poor” rural people: Monitoring and evaluation work for our project equipped BUVAD with hard data that they can now bring to government officials to strengthen their demands for the fulfillment of basic infrastructure rights.
  4. Increasing autonomy so communities can not only affect the powerful but also thrive independently of them: Since working with us, Stephen has written successful grants to simultaneously boost local autonomy from those government officials and other outsiders via training for a local well mechanic and the installation of passive rainwater harvesting infrastructure.

These are all small changes, but collectively they begin to shift the dynamics that lead to people’s disempowerment in the first place. At Minga, we believe that the most significant and positive transformations will come when standard development is accompanied by this deeper socio-political work. That’s why, in each of our projects, we’re trying to figure out which factors really do build community capacity for social change—not to prove we’re great or insist that we’re still needed, but to figure out how to do this work better. We’ll continue to keep you posted!

1 – In this case, “culturally appropriate” is not just a “buzzword or fuzzword,” to quote Andrea Cornwall. Members of Minga (then Foundation Human Nature) and their international counterparts learned that people in El Páramo were not seeking biomedical attention for malaria because evil eye manifested with similar symptoms and, it was believed, consulting a medical doctor for evil eye could kill you. By carefully studying the differences in symptoms, our team was able to discern a difference in the timing of a malarial fever compared to an evil eye fever, and by teaching people to see the difference we helped get over this barrier to medical attention.

The Minga Bump

In politics, a ‘bump’ is a sudden increase in popularity for a candidate; due, for example, to receiving a key endorsement or attending a key event. The most commonly noted ‘bump’ in American politics is the ‘Convention Bump’: presidential candidates almost always receive a boost in popular support in the week(s) following their party’s national convention. More recently, talk show host Stephen Colbert coined the term ‘Colbert Bump’, a jump in support that candidates supposedly received for being guests on his program.

For over a decade, we at the Minga Foundation have been fine-tuning our approach to development work. At the same time, we’ve sought simple and transparent ways to communicate our basic vision and strategy. In this spirit, I would like to introduce the ‘Minga Bump’.

At our August, 2016 Board of Directors’ Meeting, Board Member and Minga co-founder Jessica Levy coined, extemporaneously, the expression ‘Minga Bump’. For a moment I thought she might be referring to an interpretive dance, which would somehow capture our essence as an organization (those of you who know Jessica will understand my brief hunch). Though to this day I haven’t given up on the dance idea, I quickly realized that Jessica was suggesting something else: that we appropriate the political concept of a ‘bump’, and use it to describe our model of development work.

The loss of a single individual, a single instance of human potential, to problems of under-development is tragic. A common refrain tells us that every individual child who dies before the age of 5 ‘could have been the one that cured cancer,’ and education policies are given titles such as ‘No Child Left Behind.’ We at Minga believe the same type of thinking should apply to community organizations.

Small, community-mobilizing organizations appear and disappear with frequency around the world. Indeed, a core subgroup of Minga Foundation Board members got to know one another as activists for Durham Congregations, Associations, and Neighborhoods (CAN), a local organizing group in Durham, North Carolina.

Unfortunately many such organizations, whether in the United States or sub-Saharan Africa, face problems of sustainability. After coming into existence based on the initial energy of an individual or a campaign, will these organizations have staying power? Will they be able to consolidate their achievements, and set their sights on new projects and goals? Or will they whither for lack of capacity and resources?

As with individuals, the loss of potential community- and world-changing organizations is tragic. As many of you know, ‘Minga’ is the Quechua word for ‘Community Action for a Common Cause’. Community mobilization is at the heart of our vision at the Minga Foundation. We believe that genuinely transformative development occurs through strong communities and advocacy networks; indeed, that the notion of development as an ‘external intervention’ is nearly a contradiction in terms. In turn, for us, the loss of a single community organization with strong mobilizing potential has a direct impact on community members’ well-being.

This problem is all the more pressing for the following reason: most of the most promising and newly forming community organizations around the world are missed by the larger donor agencies and foundations. They do not yet have the capacity to secure the support they need from domestic and international donors; and without that support they cannot develop the very capacity they lack. A catch-22 if I’ve ever seen one….

Enter the Minga Foundation. All of Minga’s projects are conducted in partnership with local organizations in our partner communities, from the Butakoola Village Association for Development; to the Nancholi Youth Organization; to the Lubengoa Women’s Development Association. We chose these organizations, and these projects, as part of a detailed application review process. And we chose them too because we found them to be promising, and at a stage in their development in which the ‘Minga Bump’ could be crucial.

Each of our projects involves the creation of a resource, whether a health clinic, a borehole well, or HIV/AIDS testing and education; but perhaps more importantly, this resource comes with a strategy for capacity-building. We seek to locate small but promising organizations around the world that the big players miss, and to provide their community members not only with an immediate good, but with an advocacy structure that is built to last. We help organizations which might otherwise perish not only survive, but grow, evolve, and consolidate. We give community organizations the ‘Minga Bump’, and our projects thus promote well-being in the immediate and medium-term.

Much more could be said, and problematized. Many ‘bumps’ in politics are ephemeral…the 5-point jump in the polls a candidate receives after his or her convention often quickly recedes, and the race normalizes. Obviously, this temporary and short-lived jolt is not what we’re after at Minga. We seek a truly meaningful boost, which puts in place organizations whose work and community imprint will still be recognizable in 10, 20, even 50 years. This is an immense challenge; and one that we accept and successfully face with enthusiasm and passion.

It’s an honor to be a part of this amazing group and this amazing work. And now, Jessica, how about that dance…?

Choosing Wisely: Selecting Medical Missions With Lasting Impact

IMG_2337

PINCC Trainees in Pearl Lagoon, Nicaragua review flashcards with images of cervical pathology with PINCC volunteers during an afternoon teaching session.

As a family physician with a strong interest in international work and travel, I often contemplate how to best balance my personal goals and needs with those of the communities I serve.  I have a strong desire to serve internationally but with a full time job here in the U.S., I can only get away for 1-2 weeks at a time and must use my vacation time for any volunteer work.  Most medical professionals in the U.S. are in a similar situation to me and thus can only consider short volunteer stints if at all.  As a board member of The Minga Foundation, I am always searching for sustainable projects to work on.  However many medical projects that accept short-term volunteers are anything but sustainable.

There are numerous medical non-profit organizations that specialize in what many consider “medical tourism.”  These organizations send teams of volunteer physicians, nurses and support staff to an “underserved” community for several days or a few weeks, provide donated materials, see hundreds of patients, and then leave.  The volunteers have a great time during their travel, enjoy the experience of “roughing it” and leave feeling as though they have really helped people in need.  But what impact do these projects really have on the long-term health of a community?  It turns out that it depends greatly on the overall design of the project and the behind-the-scenes work between on-site visits.

Short term medical missions not only have the potential of leading to unsustainable health impacts, but can actually undermine the existing health systems of the communities they are meant to serve.  Most medical mission projects provide free care which may discourage patients from seeking care from existing health providers in the intervals between mission visits. Frequently there is a duplication of services provided on mission trips without coordination with the local health system.  The most irresponsible organizations may also bring donations of donated medications that may be expired, unavailable for patients to continue in their home-country or with instructions printed in a language they cannot read.

Projects that focus on improving the knowledge, skills or capacity of medical providers in the community being served are much more likely to lead to lasting health improvements than projects where care is provided by foreign volunteers only.  I recently had the pleasure of volunteering with one of many medical non-profit organizations that truly “got it right.”  Prevention International: No Cervical Cancer (PINCC) is a non-profit organization with a mission to “create sustainable programs that prevent cervical cancer by educating women, training medical personnel, and equipping facilities in developing countries, utilizing proven, low cost, accessible technology methods.”  Though their volunteer model relies on 1-3 week international trips by American or Canadian providers, their work leads to completely sustainable change and has truly saved the lives of hundreds of women.  Knowing this makes the experience of volunteering with them even more gratifying.

PINCC was founded in 2005 by an OB/GYN in the San Francisco Bay Area, Dr Kay Taylor, whose vision was to help eliminate cervical cancer in developing countries where it is a leading cause of death for reproductive aged women.  Thanks to cervical cancer screening programs that emerged in the past century (PAP smears), cervical cancer is now incredibly rare in the United States and other developed countries. PAP smears allow providers to diagnose pre-cancerous changes in the cervix that can be easily treated years before they become cancer.  Cervical cancer is one of the most easily prevented diseases in modern medicine, yet requires a universally available screening program and access to treatment for women whose screening tests are abnormal.  Poorly organized and underfunded health systems, poor health care access for women in rural areas, and cultural stigma are just a few reasons why cervical cancer screening programs have not been as successful in many developing countries.  HIV infection increases a women’s risk of getting cervical cancer and so countries with high HIV infection rates have also seen huge increases in deaths from cervical cancer.

IMG_3994

Nurses in Muhoroni, Kenya reviewing their training manual in preparation for the day’s screening exams.

I recently volunteered with PINCC for one week in Muhoroni, a remote village in Western Kenya, and then another two weeks along the remote Caribbean Coast of Nicaragua.  During my time with PINCC I was constantly impressed with their sustainable approach to providing medical aid. PINCC’s entire model relies on training nurses, clinical officers and physicians in underserved clinics the necessary components for establishing their own cervical cancer screening programs.  To date PINCC has trained over 400 providers at 27 different health centers!  The exponential impact of this work is so much greater than just the 15,000 women who have been screened during PINCC training trips.  Thousands more women have been screened and treated since PINCC ended their involvement with the clinic sites.

Part of the success of PINCC comes from the huge amount of care taken in selecting project sites ahead of time.  Before PINCC ever agrees to visit a community, they require a signed contract from the hosting hospital or clinic confirming their commitment to establishing a cervical cancer screening program for their patients.  PINCC will then provide between 3-4 site visits over a 1-2 year period as well as remote support between visits as needed.  PINCC will also provide donated equipment to help the programs get up and running, but they also make it clear that local health providers will have to determine their own way to fund the screening program in the long run (be it through governmental support, charging small fees for services, acquiring donations, etc). PINCC provides the necessary training and certification for providers and also helps link providers with their in-country medical resources for more challenging cases that may arise.  Very importantly, PINCC makes sure not to duplicate services by only visiting communities where there are not cervical cancer screening programs already in place, and gets permission from the national Ministry of Health to make sure they support PINCC’s work in the proposed communities.

IMG_4124

PINCC Trainees in Muhoroni, Kenya with examination headlamps after completion of first week of training.

As a volunteer, the experience of working with PINCC is incredibly rewarding. One week is spent at each clinic or hospital site with a focus on training providers in the skills needed to perform screening exams for cervical cancer. During our visits to Kenya and Nicaragua, we saw 100-200 women each week and supervised local nurses or physicians as they performed exams.  Any of the women who had abnormal exams needing treatment were treated that day and each site had several providers who were learning to perform the simple and low-cost cryotherapy procedure with the goal of certifying them on future visits.

It was rewarding to see a large number of women receiving cervical cancer screening often for the first time in their lives.  During our week in Kenya we treated over 25 women with pre-cancerous lesions and during our two weeks in Nicaragua we treated 35 women, essentially preventing each of them from developing cancer in the near future.  We also sadly saw 5 women in Kenya who likely already had advanced cervical cancer.  There may not be much to do to save their lives though they were referred on to the national hospital system for possible treatment.  In my entire 9 year medical career in the U.S. I have only cared for 2 women with cervical cancer.  Thus, I also learned from seeing these more advanced cases and the importance of the training we were providing became all the more apparent.

I cringe whenever I hear colleagues talk about volunteer stints in other countries that are clearly not providing sustainable care.  I feel that more of our volunteer aid should be focusing on improving the existing systems of care in other countries rather than swooping in and providing care that will not be sustained when we leave.  Programs that are designed to teach new skills for local providers, increase access for patients to health services (funding clinics, etc. so long as those clinics can be staffed by local providers) or provide educational and outreach services for patients (e.g. community health worker programs) will ultimately be the most impactful in the long run.  As medical volunteers we have an obligation to make sure we are truly providing meaningful help and not actually undermining the health systems of the countries we are intending to help.  I urge anyone considering a medical trip of their own to look closely at the model of care that the organization follows and ask questions if it isn’t clear how they integrate with local health systems and ensure the long term sustainability of their work.  There are plenty of good organizations out there like PINCC who need our help, but it is important to choose wisely if you want to truly make a difference.

For More Information about PINCC, visit www.pincc.org

IMG_2660

PINCC Volunteer Providers during week in Kukra Hill, Nicaragua Nov 2014.