Documents - Workshop

Morning presentations

• CORE/CSTS+ Sustainability Initiative: A Historical Perspective—Leo Ryan, CSTS+
• Sustainability Alive and Well—Sheila Lutjens, Child Survival and Health Grants Program
• Role of PVOs/NGOs to Enhance Maternal Child Health Impact—Karen LeBan, CORE
• Situation of evaluation questions—Eric Sarriot
• Measurement for Social and Behavioral Change—Eric Swedberg

CORE/CSTS+ Sustainability Initiative—A Historical Perspective
Leo Ryan, Child Survival Technical Support Project

Download the PowerPoint

The sustainability concept we adopt has consequences: our interpretation of the concept
directs our focus to certain indicators at the neglect of others; (Bossel)

The development of models and indicators, which will achieve reliability and validity,
requires qualitative research (McKinlay)

CSTS+ Project Director Leo Ryan introduced the meeting with the above two quotes, emphasizing the linkage between the desire for achieving sustainable sustainable community-based health programs and need to have useful measures for demonstrating such achievements. “We recognize that the issue of sustainability has been discussed, written about, and debated for many years," he explained, "and today’s meeting marks the next step in a specific CORE/CSTS+ initiative on the issue that began in March 2000 with a one-day Sustainability Dialogue."

“Based on recommendations from that meeting, we began a two-year process of reviewing literature on sustainability in health programs, consulting with field-based program managers and PVO backstops, and developing a framework for thinking about the critical dimensions that influence the sustainability of our programs. As many of you know, a two volume report on the research and resultant framework was recently produced and disseminated.” (Sustaining Child Survival: Many roads to choose, but do we have a map?; and The Child Survival Sustainability Assessment (CSSA)).

Before introducing the lead presenters for the morning session, he reviewed the objectives of the meeting:

• To gain perspective on the current environment regarding sustainability and child health programs.
• To provide feedback to CSTS+, CORE, and USAID on the appropriateness and feasibility of the proposed sustainability tool;
• To offer specific recommendations on how the PVO community can advance the present body of evidence on progress toward sustainability in community-based PHC programs.

The issue of sustainability is alive and well under the CSHGP
Sheila Lutjens, Child Survival and Health Grants Program

Download the PowerPoint

“This is an exciting process for the sustainability assessment and study,” stated Sheila Lutjens, director of the Child Survival and Health Grants Program. She emphasized that sustainability is still a very important part of the Grants Program and this year’s Request for Applications (RFA). Although there is not a separate section called “sustainability,” in the RFA, she reminded participants that a discussion of sustainability is scattered throughout the document and is an integral part of program considerations.

“We’re in Global now. How can we bring the PVOs/NGOs out to the forefront?” she asked.
She sees important opportunities available to PVOs/NGOs, particularly in technical leadership within Global. One of the opportunities she mentioned is the possibility of leveraging the large reach the PVOs/NGOs have in-country with their non-child-survival work.

One of the constant challenges she pointed out is how PVOs/NGOs can better tell others about what they’ve accomplished. “We have an annual report to Congress in which we can talk about the accomplishments of the PVOs/NGOs. We have to give a bit of a different slant now, in Global Health. We have to talk about data. Next year’s report will talk about what’s happening right now. How do we show that we’ve made progress?”

“We have to document and get the message out!” she stated. She also emphasized that documentation is not just writing a report. “What is the analysis that went behind the intervention?” she asked. Why did it work? Why did it not work? “That will be critical this next year,” she summarized. “We want to be able to talk about the PVOs/NGOs and what they’ve accomplished, but we need real hard data when we present our case.”

Role of PVOs/NGOs to Enhance Maternal Child Health Impact
Karen LeBan, CORE

Download the PowerPoint

“We all work within the PVO community because we believe we’re doing something unique. We work with integrated approaches, believing them to be more sustainable,” stated Karen LeBan, executive director of the CORE Group. “We work on horizontal relationship building, rather than the hierarchical relationship from the MOH to the community . . . . We use local information and believe it causes less distortion . . . . Yet we constantly have to defend ourselves. What’s the cost/effectiveness? Where’s the data? We’re weak on that.”

“We still don’t work together as well as we can to demonstrate the overall impact, rather than just the [impact of the] organization,” she continued “We have to demonstrate that what we’re doing really works--that our approaches can be brought to and maintained at scale, they reach to most disadvantaged, and have more lasting benefits than the alternatives--that’s the documentation we have to be able to show.”

“One way of showing this is through the Experiential Learning Cycle,” she stated. Step 1 is where we gain experience in implementing programs. In Step 2 we document and analyze experience of “promising practices.” “But when we go to Step 3, ‘Formulate conclusions and generalizations about promising practices,” she added, “we fall to pieces. We have very little reflective documents within and across PVOs/NGOs. We have to concentrate on this.”

One area in which she sees the possibility for improvement is in the learning cycle itself. “We’re caught in a closed loop learning cycle, as opposed to a spiraling learning cycle,” she explained. “Learning is the key to improving sustainability. We need to break the thinking that we’re all so unique and find the underlying patterns which are key to learning. What are the patterns that cut across the different PVOs/NGOs?” she asked.

“We need to learn better how to distill lessons learned,” she continued. “We need data, and we really need to be able to share these practices more with each other. This is especially true with ‘underdeveloped practices’how do we make something that appears to be sustainable even more effective?” Each one of these practices require data, analysis and reflection across the PVO community.

“Take, for example, the IMCI work plan,” she suggested. In 2003 CORE plans to gather, analyze and post progress indicators/benchmarks for each of the elements of the multi-sectoral platform. “What are the learnings in what we do (not looking at a high level, but on the process) that really make a difference?”

“There’s an urgency in doing this,” she emphasized. There is a major increase in investment to the NGO sector, and more than US$7 billion in aid to developing countries via international NGOs (Human Development Report, 2002). There is an opportunity in that all UN member states have committed to placing children’s health at the top of the international agenda, through the Millennium Development Goals. We need to decide on our role. “However,” she stated, “we can only accomplish this if we have an aggregated view to be able to guide us and advocate for what’s needed. This is what I hope we can start moving toward with the Child Survival Sustainability assessment tool.”

Situation of Evaluation Questions
Eric Sarriot, CSTS+

Download the PowerPoint

“The challenge and gold standard in health and development is child health impact,” stated Eric

Eric Sarriot--CSTS+ Capacity Building Specialist

Sarriot, Capacity Development Specialist at the Child Survival Technical Support Project. “Our efforts are not more for equity on the one side, or for child health or sustainability on the other, but it’s all for the same end.”

One of the larger questions is, “Can PVOs/NGOs deliver results?” he asked. “And even bigger than the first question is, ‘Do we have the data to show those results?’” Questions to PVOs/NGOs and criticism undermining their role will continue to haunt us until we communicate with data that donors and Congress understand, he explained. We need to show our comparative advantage in building sustainability, and that it makes investments for health impact worthwhile.

Another question is related to scale. “Can PVOs/NGOs go to greater scale?” He pointed out that we now have the new “Expanded Impact Grants” in this year’s RFA. “We’re trying to answer that question as well.” Sustainability plans are not always put into the evaluation plan, he explained. There is limited accountability. Sustainability at this point “seems to depend on the prevailing winds.” It’s not always clear.

“However,” he stated, “there are a lot of opportunities, because of all the learning and mass of information that is potentially there because you share projects in this grants program.” The amount of data that can be gained through this program and can feed into the Global Health agenda is really quite a lot, he explained. There are missed opportunities that need to be picked up on.

Some of these opportunities involve the evaluation loop. We can try to insert things in this loop, he suggested. “For example, you value equity, so it’s something you try to put on the radar screen . . . . There is an advantage to being proactive--you can influence the global agenda, rather than always waiting for the global agenda to influence you.”

Measurement for Social and Behavioral Change
Eric Swedberg, Save the Children

Download the PowerPoint

“There are two challenges in measuring progress towards behavior change and sustainability,”

Eric Swedberg--Chair of the CORE Social and Behavior Change working group.

stated Eric Swedberg, chair of CORE’s Social and Behavior Change Working Group:
1) lack of tools for measuring determinants leading to behavioral outcomes
2) indicators going beyond the individual (family and household, service systems and environment).

“What we don’t have are indicators that look at the factors that we’re trying to address through the program activities. We measure the results, but not necessarily the determinants of those results,” he stated.

“Why measure the determinants?” he asked. “What’s the advantage of measuring progress? If it’s something we’re focusing on with our activities, it’s something we should be looking at more closely,” he explained. Because behaviors can take a long time to change, it might be better to measure the determinants to show impact, rather than waiting for a significant change in the health outcomes. “We want to know which determinants have changed, so we can have some idea of which activities might have succeeded.”

We want to be accountable for our choice of strategies, he explained. “We work at various levels in order to be more sustainable. If you work just at the individual level you might have limited health improvement. If you work only at the social systems level, you have increased potential for health improvement, but when you combine the two is where you have self-sustained health improvement. By combining them, you have the ultimate impact on health.”

Back to main page