Effects of a community-based data for decision-making intervention on maternal and newborn health care practices in Ethiopia: A dose-response study

Background: Community participation and community health volunteer programs are an essential part of the health system so that health services are responsive and accountable to community needs. Information systems are necessary for community health volunteer programs to be effective, yet effectiveness evaluations of such information systems implemented at scale are rare. In October 2010, a network of female volunteers with little or no literacy, the Women’s Development Army (WDA), was added to extend Ethiopia’s Health Extension Program services to every household in the community. Between July 2013 and January 2015, a health management information system for the WDA’s Community-Based Data for Decision-Making (CBDDM) strategy was implemented in 115 rural districts to improve the demand for and utilization of maternal and newborn health services. Using the CBDDM strategy, Health Extension Workers (HEWs) fostered the WDA and community leaders to inform, lead, own, plan, and monitor the maternal and newborn health interventions in their kebeles (communities). This paper examines the effectiveness of the CBDDM strategy. Methods: Using data from cross-sectional surveys in 2010-11 and 2014-15 from 177 kebeles, we estimated self-reported maternal and newborn care practices from women with children aged 0 to 11 months (2124 at baseline and 2113 at follow-up), and a CBDDM implementation strength score in each kebele. Using kebele-level random-effects models, we assessed dose-response relationships between changes over time in implementation strength score and changes in maternal and newborn care practices between the two surveys. Results: Kebeles with relatively high increases in CBDDM implementation strength score had larger improvements in the coverage of neonatal tetanus-protected childbirths, institutional deliveries, clean cord care for newborns, thermal care for newborns, and immediate initiation of breastfeeding. However, there was no evidence of any effect of the intervention on postnatal care within 2 days of childbirth. Conclusions: This study shows the extent to which an information system for community health volunteers with low literacy was implemented at scale, and evidence of effectiveness at scale in improving maternal and newborn health care behaviors and practices. In Ethiopia, the rural district health system includes one primary hospital per district with 4–5 primary health care units—each primary health care unit comprises one health center with five satellite health posts to serve about 25,000 people. Health centers, staffed with health officers, nurses, midwives and laboratory technicians, provide preventive and clinical services. Health posts, each serving a kebele (community), are staffed by two female Health Extension Workers (HEWs) supported by a network of WDA volunteers in the community with little or no literacy, provide 18 packages of mainly promotive, preventive and selected curative services as part of the country’s flagship Health Extension Program. Since 2004, the Health Extension Program has established about 16,000 health posts and deployed about 38,000 HEWs [30–32]. The WDA is also known as the Health Development Army. One of the initial tasks of the HEWs was to train and graduate “model households”, households that practiced proper household hygiene and sanitation and utilized basic service provisions (such as childhood vaccination and family planning) [33]. Building on the graduated model households, the Federal Ministry of Health introduced the concept of the WDA in 2010. The WDA strategy fostered community engagement to ensure that health care services were responsive to individual and community needs by engaging women volunteers from model families to disseminate health information and to facilitate uptake of high-impact interventions. Under the guidance of HEWs, the WDA network members were organized into groups, with one WDA team leader for every six WDA volunteers, each of whom was responsible for five or six households. Thus, each WDA team leader was responsible for linking 30 households in her neighborhood to use the services provided by the Health Extension Program. To date, the Health Extension Program has established a network of about 3 million WDA members [32, 34], which has led to improved maternal and newborn health care behaviors and practices [27]. To support the Health Extension Program to implement high-impact maternal, newborn, and child health services, the Last 10 Kilometers (L10K) Project has implemented innovative, community-based strategies (see Additional files 1–3 of the first paper in this supplement) [27]. In partnership with 12 local civil society organizations, L10K has enhanced the interactions between frontline health workers and communities to achieve more accessible, efficient, and equitable maternal, newborn, and child health services [35]. Between 2013 and 2015, the L10K Platform strategy was implemented in 115 districts, covering about 3070 kebeles. The platform strategy included CBDDM; Family Conversation, a forum conducted at the household of the pregnant women with her family members during the antenatal period to reinforce birth preparedness; and Birth Notification to ensure early postnatal care. The current study was limited to the 2150 kebeles from the 83 districts in which L10K implemented only its platform strategy. CBDDM fostered the kebeles to generate and use data to improve maternal and newborn health practices. The strategy identified underserved households and linked them with HEWs and kebele command post leaders (kebele administrators) to address the barriers in access to maternal and newborn health services. Accordingly, HEWs were trained to support WDA team leaders to map the 30 households in their catchment areas, to keep each under surveillance, and to ensure maternal and newborn health services along the continuum of care: through pre-pregnancy, pregnancy, childbirth, postnatal care — including newborn care, to childhood immunization (Fig. 1). The surveillance system used images so that it could be maintained and updated by individuals with little or no education. HEWs collected data from WDA team leaders’ surveillance and drew on the kebele leaders to identify and address barriers to access maternal and newborn health services. Activities involved in Community-Based Data for Decision-Making HEWs were required to visit households routinely to update the information in the Family Folders, the central piece of the Community Health Information System of the Health Extension Program [14]. Each household within a kebele has a Family Folder, which is kept at the health post. It holds information on the family members, water supply and sanitation characteristics, and maternal and child care management records [14]. The CBDDM strategy gave an effective tool to HEWs to organize their household visits and to interact with the WDA to update the Family Folders. Although the policy to implement the Community Health Information System began in 2008, the actual implementation was initiated in October 2010 [14]; thus, the Community Health Information System was being rolled out at the same time as the CBDDM strategy was beginning to be scaled-up in the L10K areas. In the L10K areas the CBDDM was an important supplement of the Community Health Information System, which helped HEWs to personify and visualize the locations of their clients, whose records they maintained in the Family Folders. Before the CBDDM strategy was implemented, it was prototyped in selected districts, among which were 14 of the 83 districts included in this study. Nonetheless, during the study period there were no additional inputs in the prototyping areas. We anticipated that there would be natural variability in the implementation intensity of the CBDDM strategy across the 2150 kebeles in the 83 study districts. Using before-and-after cross-sectional household and health post surveys conducted in 2010–11 and 2014–15, we applied an internal comparison group design to examine dose-response relationships between improvements in CBDDM implementation strength in 177 kebeles during the observation period and improvements in maternal and newborn health care behaviors and practices during the same period. We measured the CBDDM implementation strength through interviews with HEWs and CBDDM activity records we obtained during the health post survey. Women with children aged 0 to 11 months in the kebeles reported care practices for their most recent pregnancy and childbirth. The hypothesis was that kebeles with a higher CBDDM implementation strength would have better improvements in maternal and newborn health care practices. We used data from the before and after household and health post surveys conducted in December 2010–January 2011 and December 2014–February 2015 to evaluate the larger L10K program. The survey design for the evaluation of the broader L10K program including sample size estimation parameters for the surveys are provided in Additional files 1–3 of the first paper in this supplement [27]. For the current study, the data were first restricted to the 195 kebeles included in the surveys where only L10K Platform activities were taking place, and which were visited for data collection during both the surveys. However, data from 18 kebeles were dropped, due to missing values of some of the kebele-level variables, leaving 177 kebeles in the study. The final sample size of women with children aged 0 to 11 months was 2124 in 2010–11 and 2113 in 2014–15. The household survey applied a two-stage cluster sampling method: at the first stage, kebeles were selected as primary sampling units with the probability of selection being proportionate to the population. At the second stage, the sampling strategy described by Lemeshow and Robinson (1985) was used to select the household with the target respondents [36]. The first household was selected randomly from the middle of the kebele, and from there every fifth household was visited, moving away from the middle. If the household visited had women with children aged 0 to 11 months, the women were interviewed, after seeking their consent. If a respondent was under 18 years old, then consent was sought from her husband, parents, or guardian. A quota of 12 women was interviewed from each kebele to obtain information on their socio-demographic background and the maternal and newborn health care behavior and practices associated with their most recent pregnancy and childbirth. The health posts within the sampled kebeles were visited, the HEWs based there were interviewed and the health post records were reviewed to obtain information on CBDDM implementation strength. The data collection from households was carried out by health professionals working for regional health bureaus at zonal or woreda levels. They were recruited in consultation with the regional health bureaus. The supervisors were mostly from L10K’s implementing partners. To avoid bias, supervisors and interviewers did not conduct survey work in their own areas. The supervisors were responsible for data collection from the health posts. Ethical clearance was obtained from the Institutional Review Boards of the respective Regional Health Bureaus and that of the JSI Research & Training Institute, Inc. The maternal and newborn health care practices that were expected to improve due to the intervention were the coverage of four or more antenatal care visits to a health facility (ANC 4+), neonatal tetanus-protected childbirth (defined below), delivery at health facilities, receiving postnatal care within 48 h of childbirth (PNC in 48 h), clean umbilical cord care for home births, thermal care of the newborn, and initiation of breastfeeding within 30 min of childbirth. If the mother reported receiving at least two tetanus toxoid injections during her lifetime, the last of which had occurred less than 3 years previously; if she received at least three tetanus toxoid injections during her lifetime, the last of which had occurred in the previous 5 years; if she received at least four tetanus toxoid injections during her lifetime, the last of which occurred in the previous 10 years; or if she had received at least five tetanus toxoid injections during her lifetime, her last birth was considered to be protected from neonatal tetanus. Clean cord care was defined as the umbilical cord being cut with a sterile instrument, the cord tied with sterile thread, and either nothing or only chlorhexidine applied to the cut end of the umbilical cord. Thermal care was defined as the newborn being dried and wrapped immediately after birth, delayed bathing of the newborn by 6 hours or more, and maintaining skin-to-skin contact with the baby. The analysis of clean cord care and thermal care was restricted among home deliveries. The independent variable of interest for this study was the CBDDM implementation strength. During the follow-up survey, each kebele was assigned a CBDDM implementation strength score based on four items obtained during the health post survey: (a) the proportion of WDA team leaders in the kebele who had a surveillance map of their neighborhood; (b) the proportion of WDA team leaders who had either reported surveillance data to the HEW, or from whom the HEW had collected it, during the last 3 months; (c) whether or not the HEW had updated health post records of surveillance data; and (d) whether the kebele leaders had used surveillance data to monitor the utilization of maternal and newborn health services during the last 3 months. The first three items were measured from WDA activity records maintained by HEWs, while for the fourth item, HEWs were asked if there had been any meeting of kebele leaders (kebele command post meeting) at which maternal and newborn health issues were discussed. If the response was yes, then they were asked whether CBDDM data were used during that meeting. After verifying the responses from meeting minutes, they were recorded. The first two item scores were probabilities ranging between 0 and 1, while the last two items score were binary responses, where kebeles received a score of 1 for yes and null for no. Each of these was standardized (with mean 0 and variance 1), and then all four were added together to obtain a scale. Cronbach’s alpha, which reflects the internal reliability of the four items in constructing the scale, was 0.75. The kebeles were ranked according to the score of the scale from the follow-up survey and then divided into three terciles. Due to ties, the three terciles were not equal. There were 68 kebeles in the lowest and middle terciles, and 41 kebeles in the highest tercile. The lowest, middle and highest CBDDM implementation strength score terciles were scored 1, 2, and 3 and named low, medium and high CBDDM implementation strength, respectively. At baseline, the CBDDM implementation strength score was assumed to be zero. Higher scores indicated better CBDDM implementation strength. The independent variables that were considered as controls for the multivariate analysis were the individual, household, and contextual characteristics of the sample. The individual-level characteristics were age, education, marital status, parity and religion of the respondents; the household-level characteristics were household wealth and distance of the respondents’ household from the nearest health facility; and the contextual characteristics were the HEW-to-population ratio of the kebele, administrative regions, and program stratum. The program stratum was an indicator variable indicating areas in which CBDDM was prototyped prior to the study period. A single wealth index score was constructed for each household for both survey periods using principal component analysis of household possessions (electricity, watch, radio, television, mobile phone, telephone, refrigerator, table, chair, bed, electric stove, and kerosene lamp) and household characteristics (type of latrine and water source). Households from both the surveys were ranked according to wealth score and then divided into five quintiles [37]. First, we assessed the difference in the characteristics of the respondents between the two survey periods using Pearson’s chi-square statistics adjusted for cluster survey design effects. Then we analyzed the distribution properties of the four items of the CBDDM implementation strength score using box plots. We assessed whether the background characteristics of the respondents were systematically associated with the CBDDM implementation strength score during the follow-up survey using Pearson’s chi-squared statistics adjusted for cluster survey design effects. We estimated the changes in the outcomes of interest between the two surveys and their 95% confidence intervals using post-estimation procedures following logistic regression models. The survey period was the only predictor in the logit models, and the models accounted for cluster survey design effects, using the Taylor series linearization technique for estimating variances [38]. We estimated the dose-response associations between changes in the CBDDM implementation strength score and the changes in maternal and newborn health care practices using kebele-level random effect models. We used Stata’s ‘xtlogit’ command for this purpose [38]. The models accounted for cluster survey design [39], secular change over time, and the individual, household and contextual characteristics of the respondents. We accounted for the secular change of the maternal and newborn health care indicator between the two survey periods by including the survey period as an indicator variable. The CBDDM implementation strength score entered the models as a linear term to identify dose-response association or the trend effect. To assess the contribution of the independent variables to the model, we grouped them into individual, household, and contextual factors. First, we estimated the model with all three groups of independent variables. Then we assessed the statistical significance of each of the three groups one by one using a likelihood ratio test. If a particular group was not statistically significant at p < 0.05, we dropped it [40]. We assessed the goodness of fit of the models using global Wald’s statistic and the likelihood ratio test of the kebele-level random effects. If the linear term for the CBDDM implementation strength score was statistically significant (p < 0.05) in the final model, then we tested it for departure from linearity (i.e. not a trend effect). We used the likelihood ratio test to compare the final model with the score as a linear term (restricted model) with the unrestricted model where the score was a categorical variable. If the two models were not statistically significantly different (p > 0.05) then we considered the linear term for CBDDM implementation score as adequate, and we concluded that there was a dose-response relationship, or trend effect [41]. Lastly, if we found a trend effect in the CBDDM implementation score, we did a counterfactual analysis to quantify the average treatment effects (ATEs) of CBDDM for maternal and newborn health care practices. We used Stata’s post-estimation ‘margins’ command for the purpose [38]. The regression models were simulated to predict two values for each maternal and newborn health care practice. The first value was the predicted (adjusted) maternal and newborn health care practice when the CBDDM implementation strength score did not change from the baseline value (zero); and the second value was the adjusted maternal and newborn health care practice holding the CBDDM implementation strength score at its mean value during the follow-up survey. The difference between the second and first values estimated the effect of CBDDM on the maternal and newborn health care practice – the ATE.