Summary and Recommendations

From September 9-11 2002, a group of 70 people (including representatives from 22 PVOs, six cooperating agencies, and five independent consultants) met in Silver Spring, Maryland to learn from each other on using data to improve child health.

The meeting provided a forum to highlight 16 projects from 14 organizations. Each case study revolved around data and how it was used in decisionmaking, including the specific actions or decisions that were taken, the outcome of those decisions, which data were used to inform these decisions, and by whom. The case studies were organized around the levels at which data was used to inform specific decisions—either community, partner, project, PVO levels.

Each day the participants met together to discuss what recommendations they would make regarding the use of data at the various levels. These recommendations were refined each day, with participants prioritizing them on the last day. The top five recommendations (including suggested responsible organizations and completion dates) were the following:

1.      Clearinghouse of tools and training manuals/Database of tools and experiences

Create a clearinghouse of tools and training manuals including reviews (peer and expert) for information systems, social autopsies. Create a database of tools and experiences in community-based health information systems (CB-HIS). Responsible organizations: CSTS, CORE MEWG, users. Tools should be sent to CSTS. Tools presented during the Data for Action workshop should be available by January 2003. The database should be useable by September 2003.

2.      Toolkit/resource/compendium of qualitative research

Develop a toolkit/resource/compendium of qualitative research including guidelines for standardized reporting of qualitative data. Include:

Responsible organizations: CSTS, and the CORE Group’s Social and Behavior Change Working Group. No date was set.

3.      Toolkit/resources on cost effectiveness and applied analysis

Develop a toolkit/resources (including economists) on determining cost effectiveness and applied analysis which would go beyond facility-based actions. Conduct a survey of PVO tools and activities. Responsible organizations: CSTS working with the CORE Group’s Monitoring and Evaluation Working Group, which could develop a Taskforce on Cost/Analysis Effectiveness with responsibility to define the product(s).Also consider Abt, URC, and QAP. No date was set.

4.      Collection of best-practices

Develop guidelines and disseminate best practices on CB-HIS including how the community defines priorities and parameters. Include mechanisms to collect information on community feedback on program strategies. Responsible organization: The CORE Group. Completion by September 2003.

5.      Inventory of CB-HIS

Hold country-wide meetings in the spring/summer of 2003, for preparing an inventory of CB-HIS in 2-3 countries, and make recommendations. In preparation, PVOs active in those countries should complete an inventory during the winter of 2002-2003. As a suggestion, they could use the inventory found on p. 23 of the draft CB-HIS CSTS/JHU SOTA papers. Responsible organizations: The CORE Group PVOs and their NGOs/partners; CORE/CORE-lettes; with the CORE Group Monitoring and Evaluation Working Group as facilitator.

Participants also requested a centralized, easy-to-access point of contact or resource with expertise in operations research (OR), who can share information and answer questions ranging from estimating costs of studies to issues relating to the implementation of studies, to tools, etc. This would also serve as a valuable mechanism to disseminate results and lessons learned from OR. As there is a level of effort of writing and costs involved, participants described a non-competitive small grants mechanism be developed, with a special emphasis on building OR capacity at the field level. To see the full list of recommendations, click on the recommendations page.

Data for Action: A Historical Perspective

Within the child survival context, the period 1985-1998 was an era of data proliferation. The standardization and evolution of the knowledge, practice, coverage (KPC) methodology provided useful data on maternal knowledge and practice to guide project planning and setting quantitative population-based objectives. Follow-up KPCs assessed achievement of population-based objectives; progress was sometimes assessed through a mid-term KPC as well. Nevertheless, outside of USAID guidelines there appeared to be limited program focus on systematic methods to monitor and improve project performance between the baseline and final KPC surveys. Further, there was growing concern that data were not used on any type of routine basis to improve implementation.

Beginning in a concerted way in 1998, PVOs, USAID, and others in the child survival community engaged in a process of extended dialogue on designing and implementing child survival activities with the greatest long-term impact. This has corresponded to a shift away from direct service delivery to partnership models, which has precipitated a focus on improving partner capacity to manage for results, and the facilitation of community use of data to manage their own health and nutrition. PVOs are now often assessing their partner(s) capacity at baseline/end of project, and using data to inform partner relationships. At the same time PVOs have begun routinely using available data including Demographic Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) for situational analyses. More recently, PVOs have begun to estimate and track coverage or behaviors for sub-project areas.

In June 1998, the CORE Group and BASICS held an “Expert Consultation on High Impact PVO Child Survival Programs.” Technical experts and field practitioners sought to identify the essential elements of successful child survival projects and to recommend how those lessons could be used in the design and implementation of child survival activities in the future. These addressed the effectiveness, sustainability, and spread (coverage) of PVO child survival programs. Presented programs used a variety of strategies including integration of services, partnering with government, participatory community approaches, operations research, management by information, and a perinatal focus.

Two strategies of project success related to the use of data emerged: (1) using information and management to detect and diagnose problems, provide feedback, and conduct situational analysis; and (2) problem-focused operations research[1]. The Expert Consultation concluded that future child survival efforts should have a strong emphasis on assessment, monitoring, and evaluation, including built-in operations research. It was recommended that PVOs develop a research agenda, test various program models in different settings, and document failures as well as successes.

Several related follow-up meetings ensued. The October 2000 CORE Group/NGO Networks/CSTS “Managing for Results” Monitoring and Evaluation Update served as an especially important milestone in furthering the impact dialogue by focusing on the purpose of monitoring and evaluation as part of the recommendation of the 1998 Expert Consultation, and providing practical monitoring and evaluation resources. As part of the structure for the Update, participants first identified then discussed three organizational levels at which data are most often collected and used, decisions made, and actions taken, e.g. the PVO itself, the local partner(s), and the community/household/individual.

These three organizational levels served as useful discussion platforms for the Data for Action workshop. Again following a worldwide call for abstracts, 16 papers were selected. Looking at the level of either their own PVO, local partner(s), or communities, presenters highlighted the types of data they collected periodically or routinely. Examples included a range of qualitative and quantitative information including results of operational research, lessons learned, study of better practices, and applications of new or established approaches and tools. Presenters then discussed their organizational use of data to improve child survival project design (for example, setting health priorities and policies, and planning for sustainability and scale-up); effectively implement/manage activities (such as cost-effective allocation of resources); and to maximize outcomes in terms of target children.

Peter Winch of the Johns Hopkins University Bloomberg School of Public Health identified several continuing challenges in the ongoing shift from service delivery to partnership. These include

Each of the presentations made at the workshop addressed questions of data use, and how data were used in decision-making at the various levels of involvement. The presentations are included on this CD in various forms: All of the papers are available in Microsoft Word, presentations are available in PowerPoint, and, if the presentations were recorded, links to streaming audio are included as well.

If you have any questions regarding this workshop or the outcomes, please send your questions to the Child Survival Technical Support Project, 11785 Beltsville Drive, Suite 300, Calverton, MD 20866 <csts@macrointernational.com>