Comprehensive review of the evidence regarding the effectiveness of community-based primary health care in improving maternal, neonatal and child health: 1. rationale, methods and database description

Background Community-based primary health care (CBPHC) is an approach used by health programs to extend preventive and curative health services beyond health facilities into communities and even down to households. Evidence of the effectiveness of CBPHC in improving maternal, neonatal and child health (MNCH) has been summarized by others, but our review gives gives particular attention to not only the effectiveness of specific interventions but also their delivery strategies at the community level along with their equity effects. This is the first article in a series that summarizes and analyzes the assessments of programs, projects, and research studies (referred to collectively as projects) that used CBPHC to improve MNCH in low- and middle-income countries. The review addresses the following questions: (1) What kinds of projects were implemented? (2) What were the outcomes of these projects? (3) What kinds of implementation strategies were used? (4) What are the implications of these findings? Methods 12 166 reports were identified through a search of articles in the National Library of Medicine database (PubMed). In addition, reports in the gray literature (available online but not published in a peer-reviewed journal) were also reviewed. Reports that describe the implementation of one or more community-based interventions or an integrated project in which an assessment of the effectiveness of the project was carried out qualified for inclusion in the review. Outcome measures that qualified for inclusion in the review were population-based indicators that defined some aspect of health status: changes in population coverage of evidence-based interventions or changes in serious morbidity, in nutritional status, or in mortality. Results 700 assessments qualified for inclusion in the review. Two independent reviewers completed a data extraction form for each assessment. A third reviewer compared the two data extraction forms and resolved any differences. The maternal interventions assessed concerned education about warning signs of pregnancy and safe delivery; promotion and/or provision of antenatal care; promotion and/or provision of safe delivery by a trained birth attendant, screening and treatment for HIV infection and other maternal infections; family planning, and HIV prevention and treatment. The neonatal and child health interventions that were assessed concerned promotion or provision of good nutrition and immunizations; promotion of healthy household behaviors and appropriate utilization of health services, diagnosis and treatment of acute neonatal and child illness; and provision and/or promotion of safe water, sanitation and hygiene. Two-thirds of assessments (63.0%) were for projects implementing three or fewer interventions in relatively small populations for relatively brief periods; half of the assessments involved fewer than 5000 women or children, and 62.9% of the assessments were for projects lasting less than 3 years. One-quarter (26.6%) of the projects were from three countries in South Asia: India, Bangladesh and Nepal. The number of reports has grown markedly during the past decade. A small number of funders supported most of the assessments, led by the United States Agency for International Development. The reviewers judged the methodology for 90% of the assessments to be adequate. Conclusions The evidence regarding the effectiveness of community-based interventions to improve the health of mothers, neonates, and children younger than 5 years of age is growing rapidly. The database created for this review serves as the basis for a series of articles that follow this one on the effectiveness of CBPHC in improving MNCH published in the Journal of Global Health. These findings, guide this review, that are included as the last paper in this series, will help to provide the rationale for building stronger community- based platforms for delivering evidence-based interventions in high-mortality, resource- constrained settings. The Task Force and the Expert Panel agreed on the following definition of CBPHC: CBPHC is a process through which health programs and communities work together to improve health and control disease. CBPHC includes the promotion of key behaviors at the household level as well as the provision of health care and health services outside of health facilities at the community level. CBPHC can (and of course should) connect to existing health services, health programs, and health care provided at static facilities (including health centers and hospitals) and be closely integrated with them. CBPHC involves improving the health of a geographically defined population through outreach outside of health facilities. CBPHC does not include health care provided at a health facility unless there is community involvement and associated services beyond the facility. CBPHC also includes multi–sectoral approaches to health improvement beyond the provision of health services per se, including programs that seek to improve (directly or indirectly) education, income, nutrition, living standards, and empowerment. CBPHC programs may or may not collaborate with governmental or private health care programs; they may be comprehensive in scope, highly selective, or somewhere in between; and they may or may not be part of a program which includes the provision of services at health facilities. CBPHC includes the following three different types of interventions: The Task Force sought documents that described community–based programs, projects and research studies that carried out assessments of changes in MNCH indicators in such a way that any changes observed could reasonably be attributed to CBPHC program interventions. At least one of the following outcome indicators was required to be present in order for the assessment to be included in the review. In addition, the review included an analysis of available documentation concerning the degree to which improvements in child health obtained by CBPHC approaches were equitable. The principal inclusion criteria for the literature review were: (1) a report describing the CBPHC program for a defined geographic population and (2) a description of the findings of an assessment of the project’s effect on maternal, neonatal or child health as defined above. The focus was on the effectiveness of program interventions on the health of all mothers and/or children in a geographically defined area, although in some cases (eg, in studies of maternal–to–child HIV transmission), the focus was on a subset of mothers and their children in a geographically defined area. Key terms for “maternal health,” “child health,” “community health,” and “developing countries” and related terms were identified to create a search query (see Tables S1 and S2 in Online Supplementary Document(Online Supplementary Document)). The United States National Library of Medicine’s PubMed database was searched periodically up until 31 December 2015 using these two queries, yielding 7890 articles on maternal health and 4276 articles on neonatal or child health (Figure 1). The articles were screened separately by two members of the study team. Assessments of the effectiveness of CBPHC in which the outcomes were improvements in neurological, emotional or psychological development of children were not included unless the reports also included one or more of the other neonatal or child health outcome measures mentioned above. Selection process of assessments of the effectiveness of community-based primary health care (CBPHC). In addition to the PubMed search, broadcasts were sent out on widely used global health listservs, including those of the Global Health Council, the American Public Health Association, the Collaboration and Resources Group for Child Health (the CORE Group), the World Federation of Public Health Associations, and the Association of Schools of Public Health asking for information about documents, reports, and published articles which might qualify for the review. Finally, the Task Force contacted knowledgeable persons in the field for their suggestions for documents to be included, including members of the Expert Panel. Documents not published in peer–reviewed scientific journals were included if they met the criteria for review, if they provided an adequate description of the intervention, and if they had a satisfactory form of evaluation. A total of 152 assessments met the criteria for the maternal health review and 548 for the neonatal/child health review (Figure 1). Table S3 in Online Supplementary Document(Online Supplementary Document) contains a bibliography with the references associated with these 700 assessments. The bibliography also indicates which references were in the maternal health review, in the child health review (and which of these were included in the analyses for neonatal health and child health), and the equity review. There are a number of cases in which a single assessment in our database is derived from more than one document. All of these references are included in the bibliography. Thus, when in Figure 1 above we refer to the number of articles/reports, there are a small number of cases in which we have combined the various articles/reports associated with a single assessment and counted this as only one assessment. Of the 33 maternal health assessments and the 115 neonatal/child health assessments included in the review that were not identified through PubMed, most (16 and 80, respectively) were project evaluations of child survival projects funded by the USAID Child Survival and Health Grants Program and implemented by US–based non–governmental organizations. These are listed separately in Table S4 in Online Supplementary Document(Online Supplementary Document). Other assessments that were not identified through PubMed were evaluations from other sources, books, or book chapters. Two data extraction forms were prepared through an iterative process. The extraction form to be used for child health assessments and the form for maternal health assessments were identical except for the interventions carried out. These forms are contained in Appendices S5 and S6 in Online Supplementary Document(Online Supplementary Document). Both forms were developed with the purpose of extracting all possible information available regarding how the interventions were implemented at the community level and what the role of the community was in implementation. Two independent reviewers each completed a Data Extraction Form for each assessment that qualified for the review. A third reviewer provided quality control and resolved any difference observed in the two reviews, and the final summative review was transferred to an EPI INFO database (version 3.5.4) (Epi Info, US Centers for Disease Control and Prevention, Atlanta, Georgia, USA). The names of the reviewers, many of whom worked on a volunteer basis, are shown in the acknowledgment section; their names and professional titles are contained in Table S7 in Online Supplementary Document(Online Supplementary Document). The assessments included in our review were carried out for field studies, projects, and programs that employed one or more CBPHC interventions for improving maternal, neonatal and/or child health. This is a heterogeneous group of assessments in the sense that they range from (1) research reports describing the efficacy of single interventions over a short period of time in a highly supervised and well–supported field setting to (2) assessments of programs which provided a comprehensive array of health and development programs over a long period of time in more typical field setting. When referring to this group of community–level activities as a whole, they should properly be referred to as “research studies/field projects/programs” but for practicality’s sake we will refer to them throughout this series simply as “projects,” and the evaluations of their effectiveness as “assessments.” An electronic database describing 700 assessments of the effectiveness of CBPHC in improving MNCH was queried using EPI INFO version 3.5.4 and STATA version 14 (StatCorp LLC, College Station, Texas, USA). For the purpose of this review, the 39 assessments with both maternal and child health outcomes have been counted as separate assessments in our analysis. Overall, 78.8% of assessments are scientific articles published in peer–reviewed journals, 4.0% are some other type of publication (mostly books or reports not available on the internet), and 12.7% are either from the gray literature (available on the internet) or unpublished project evaluations. Over three–fourths (78.4%) of the assessments included in our review were carried out in rural settings at least in part, while 16.9% and 11.1% were carried out exclusively in an urban or peri–urban setting, respectively. Among the 700 assessments in our data set, a small proportion contained data from more than one country. Thus, altogether, 786 country–specific assessments were identified. India, Bangladesh, and Nepal had the largest number of assessments (86, 77, and 47, respectively). 49.0% of the country–specific assessments came from Africa WHO Region, 28.5% from the South–East Asia Region, and 9.7% from the Americas (Table 2 and Table S8 in Online Supplementary Document(Online Supplementary Document)). 8.6% of reports assessed interventions in a single community, 38.1% in a set of communities not encompassing an entire health district (or sub–province), 37.5% at the district (or sub–province) level, 7.5% at the provincial/state level, 3.7% at a national level, and 3.2% at a multinational level. Number of assessments of the effectiveness of community–based primary health care in improving maternal, neonatal and child health by region and the countries with the greatest number of assessments *The total number of countries listed here exceeds the number of assessments because some assessments were conducted in multiple countries. The implementing and facilitating organizations for these projects were primarily private entities (NGOs, universities and research organizations), often working with governments at the national, provincial, or local level (Table 3). While communities were — by definition — involved in all of these projects, in only 4.3% of assessments were local communities the only identified implementers. Those who actually implemented projects at the local level were community health workers (CHWs), local community members, research workers, and government health staff. Implementers of projects for improving MNCH *Percentages add up to more than 100% because projects often utilized more than one Implementer. Half (49.3%) of the assessments are of projects serving 5000 or fewer women and children. 18.2% of the assessments are based on data derived from projects reaching more than 25 000 women and children. 61.9% of the projects had begun since 2000. Almost half (46.3%) of projects were less than 2 years in duration and almost two–thirds (62.9%) were implemented for less than 3 years. Among the neonatal and child health assessments, 51.6% were of only one intervention, and 87.4% were of four or fewer interventions. On the other hand, among the maternal health assessments three–quarters (75.7%) included five or more interventions. Our review includes 16 assessments of projects that were completed before 1980. The earliest report describes the health impact of an integrated primary health care project in South Africa led by Sidney Kark in the 1940s and published in 1952 [33]. The next earliest report concerns the effectiveness of tetanus toxoid immunization in Columbia, South America, published in 1966 [34]. There has been a rapid growth in the number of assessments published between 1980 and 2015, but particularly in the period 2001–2011, the decade following the establishment of the Millennium Development Goals (MDGs) (Figure 2). The surge in publications is present both for maternal and for child/neonatal health studies (data not shown). In the 5 years from 2011 until the end of 2015 when the assessment retrieval ended, there was a slight decline in the number of publications. Number of assessments in data set by year of publication (in 5-year intervals). We identified a total of 239 outcomes measured in the 700 assessments included in the review: 56 maternal outcomes and 183 neonatal/child outcomes (see Tables S7 and S8 in Online Supplementary Document(Online Supplementary Document)). Common maternal health outcomes were changes in: mortality, receipt of antenatal care, attendance at delivery by a skilled provider, facility delivery, care for obstetric emergencies, receipt of nutritional supplements, receipt of tetanus toxoid vaccination, receipt of post–partum family planning, knowledge of safe birth practices, and screening for HIV and other sexually transmitted infections during pregnancy. Common neonatal and child health outcomes were: changes in mortality, serious morbidity, nutritional status, population coverage of healthy behaviors, and changes in the appropriate utilization of health services. In addition, some assessments contained outcome measures that did not qualify for the review but were included with other indicators that did qualify for the review. These include progress in psychomotor development, changes in health–related knowledge among parents and caretakers, quality of community case management of acute childhood illness provided by CHWs, and measures of improvements in health system capacity. In the majority (61.0%) of the assessments, a control or comparison group was present. In almost three–fourths (72.5%), pre– and post–intervention data were collected. In 44.6% of the assessments, both data from a comparison group as well as pre– and post–intervention data were present. Randomized controlled assessment designs were present in 33.7% of the assessments. 27.4% of the assessments were uncontrolled before–after assessment designs. Reviewers considered the methodology to be adequate in 89.8% of the assessments, and they considered the assessment quality to be good, high, or exceptional for 88.4% of the assessments. The United States Agency for International Development (USAID) was far and away the largest source of financial support for the assessments included in our review, contributing to the financial support of one–third (33.4%) of the assessments included in the review. UNICEF supported the next largest number of assessments (15.8%), followed by the World Health Organization (14.2%), the Gates Foundation (10.7%), other UN agencies (7.7%), and the World Bank (6.2%) (Table 4). There were numerous other donors that funded a smaller number of assessments. In most (but not all) cases, the donor funded the project as well as the assessment. Leading sources of financial support for projects whose assessments were included in the database *Multiple funders may have supported a single project/assessment. We are not aware of any other similar database in existence. It serves as the basis for the subsequent articles in this series [32,35–40]. However, there is an opportunity for more analyses of the database than is reported in this series. Any of the project assessments included in this review are available to be shared with anyone who is interested (contact Henry Perry at ude.uhj@2yrreph). The potential exists for maintaining this as a dynamic database that is regularly updated and publicly available. And, the potential also exists for expanding this database beyond MNCH to include community–based approaches to other global health priorities such as HIV, tuberculosis, malaria, and chronic diseases. Our review is a comprehensive one, but we make no claim that it is a complete or systematic review. Resources and time constraints prevented screening other electronic databases beyond PubMed for reports that met the inclusion criteria. In addition, the USAID Child Survival and Health Grants program has an archive of more than 400 unpublished child survival project evaluations that meet the criteria for inclusion and are publicly available, but resource and time constraints were such that only one–fifth (80) of these could be included in our review. Since the data analysis and write up portion of this study began, we have identified several additional articles that would have qualified for the review. However, none of these would have changed the overall findings of our review. This review is limited to documents that describe the impact of project interventions. As is well–known, program failures and serious challenges encountered in program implementation are rarely described in open–access documents or in the scientific literature. This means that a serious publication bias is present and should be recognized. Nonetheless, the inability to document these experiences does not detract from the value of the numerous assessments that have been included in our review that demonstrate effectiveness of CBPHC in improving MNCH. The degree to which the assessments included in our review represent efficacy assessments as compared to effectiveness assessments is an important issue which we are not able to adequately explore. Efficacy assessments, of course, are carried out for projects that have been implemented under ideal circumstances, when field staff members have optimal training, supervision, resources, and logistical support, and when optimal community engagement has been established. These are conditions that often do not occur in routine settings. Effectiveness assessments, in contrast to efficacy assessments, are carried out under “real world” conditions. Our data extraction form did not collect information on this issue and, in fact, it is often difficult to determine exactly where a project might lie on a continuum between these end points. But it is the case that very few of the assessments in our database were of projects that were implemented without some type of international donor support or technical assistance. Thus, the database is not representative of the effectiveness of current day–to–day practice of CBPHC but rather of what has been achieved in special circumstances in which documentation of effectiveness was undertaken and in which presumably extra efforts had been made to assure the highest quality of implementation possible under the circumstances. The degree to which these projects improved MNCH depended on many factors: the type(s) and number of interventions implemented, the quality of implementation, and myriad contextual factors. And, of course, the type of outcome indicator(s) employed is important as well. Given the heterogeneity of (1) the types of interventions implemented, (2) the manner in which they were implemented, and (3) the outcome measures used to assess outcomes, it is difficult to make definitive statements about the strength of the evidence, about the magnitude of effect for any specific intervention, or about the effectiveness of one specific approach to implementation compared to another. Rather, the aim of our study is to review the broad scope of evidence related to the effectiveness of CBPHC in improving MNCH and to draw conclusions about the overall effectiveness of CBPHC, the most common strategies used in implementation, and the potential for further strengthening of CBPHC to improve MNCH globally. It is well–known that the use of family planning, birth spacing, and the reduction of unmet need for family planning all have favorable benefits for MNCH. Furthermore, the evidence on the effectiveness of CBPHC in increasing the coverage of family planning services is extensive. Thus, inclusion of this literature would have made our review more complete, but time and resources were not sufficient to carry this out. Finally, our review has not included the effectiveness of CBPHC in reducing miscarriages and stillbirths. This topic is an important one but time and resources were not sufficient to carry this out either. Seven subsequent articles are being published in this series that answer the questions posed by the review. These include: (i) an analysis of the effectiveness of CBPHC in improving maternal health [35], (ii) an analysis of the effectiveness of CBPHC in improving neonatal health [36], (iii) an analysis of the effectiveness of CBPHC in improving child health [37], (iv) an analysis of the effectiveness of CBPHC in promoting equitable improvements in child health [40], (v) the strategies employed by effective CBPHC programs for achieving improvements in MNCH [38], (vi) an analysis of the common characteristics of integrated projects with long–term evidence of effectiveness in improving MNCH [39], and (vii) summary and recommendations of the Expert Panel [32].