Effects of a participatory community quality improvement strategy on improving household and provider health care behaviors and practices: A propensity score analysis

Background: Maternal and newborn health care intervention coverage has increased in many low-income countries over the last decade, yet poor quality of care remains a challenge, limiting health gains. The World Health Organization envisions community engagement as a critical component of health care delivery systems to ensure quality services, responsive to community needs. Aligned with this, a Participatory Community Quality Improvement (PCQI) strategy was introduced in Ethiopia, in 14 of 91 rural woredas (districts) where the Last Ten Kilometers Project (L10 K) Platform activities were supporting national Basic Emergency Obstetric and Newborn Care (BEmONC) strengthening strategies. This paper examines the effects of the PCQI strategy in improving maternal and newborn care behaviors, and providers’ and households’ practices. Methods: PCQI engages communities in identifying barriers to access and quality of services, and developing, implementing and monitoring solutions. Thirty-four intervention kebeles (communities), which included the L10 K Platform, BEmONC, and PCQI, and 82 comparison kebeles, which included the L10 K Platform and BEmONC, were visited in December 2010-January 2011 and again 48 months later. Twelve women with children aged 0 to 11 months were interviewed in each kebele. Propensity score matching was used to estimate the program’s average treatment effects (ATEs) on women’s care seeking behavior, providers’ service provision behavior and households’ newborn care practices. Results: The ATEs of PCQI were statistically significant (p < 0.05) for two care seeking behaviors - four or more antenatal care (ANC) visits and institutional deliveries at 14% (95% CI: 6, 21) and 11% (95% CI: 4, 17), respectively - and one service provision behavior - complete ANC at 17% (95% CI: 11, 24). We found no evidence of an effect on remaining outcomes relating to household newborn care practices, and postnatal care performed by the provider. Conclusions: National BEmONC strengthening and government initiatives to improve access and quality of maternal and newborn health services, together with L10 K Platform activities, appeared to work better for some care practices where communities were engaged in the PCQI strategy. Additional research with more robust measure of impact and cost-effectiveness analysis would be useful to establish effectiveness for a wider set of outcomes. Ethiopia’s maternal mortality ratio and newborn mortality rate are among the highest in the world at 421 per 100,000 live births and 29 per 1,000 live births, respectively, according to the 2016 Ethiopia Demographic and Health Survey estimate [29]. Although access to maternal and newborn health services has improved, the quality of care remains an immense challenge and often the services do not respond fully to community needs [30, 31]. In 2015, the Government of Ethiopia made a commitment towards achieving the health-related Sustainable Development Goals by launching its Health Sector Transformation Plan, under which it set ambitious targets to reduce the maternal mortality ratio to 199 per 100,000 live births and the neonatal mortality rate to 10 per 1,000 live births by 2020 [7]. Moreover, the Health Sector Transformation Plan cited universal coverage of high quality maternal and newborn health services which respond to the community’s needs and are respectful to clients, among its top priorities. The Government of Ethiopia launched a number of programs to increase access to quality maternal and child health care. For example, it introduced its flagship Health Extension Program in 2004 to improve primary health care at community level and transfer ownership of and responsibility for improving health to communities and individual households through a package of promotive, preventive and basic curative services aimed at women and children [31]. The primary level of healthcare, as articulated in the Health Sector Transformation Plan, is the primary health care unit which comprises four or five health posts and one health center, which together with three or four other primary health care units is served by a primary hospital. The primary health care unit is appointed to serve as the administrative, technical and supportive supervision link to their health posts [7]. Each health post is staffed by two Health Extension Workers (HEWs) and, to extend their reach in mobilizing communities and households, each kebele (community) includes a network of women volunteers who form the WDA, also known as Health Development Army. Since 2008, The Last Ten Kilometers Project (L10 K) has been working to improve coverage of effective reproductive, maternal, newborn and child health services, and to strengthen the skills of HEWs in 115 of 408 woredas (districts) across four regions of Ethiopia, covering about 19% of the country’s population: Amhara, Oromia, Tigray, and Southern Nations, Nationalities, and Peoples’ Region. The L10 K Platform in the 115 woredas included CBDDM, a surveillance system of households to ensure continuum of care for reproductive, maternal, newborn and child health services; Family Conversations, a forum conducted at the house of a pregnant woman with her family members during the antenatal period, to reinforce birth preparedness; and Birth Notification to ensure early postnatal care. (For details on the Platform please see Additional files 1–3 for the first paper in this supplement and two of the other papers in this supplement on the CBDDM and Family Conversations strategies [28, 32, 33]). In October 2012, the L10 K Project introduced a program in 91 woredas to improve basic emergency obstetric and newborn care (BEmONC) through training, mentoring, provision of equipment and supplies, and addressing barriers for improved infection prevention practices. The L10 K Project complemented Ethiopian government initiatives to improve maternal and newborn health outcomes, which included infrastructure expansion of primary health care units, strengthening the referral system, procurement of ambulances to provide free transport to laboring women, maternal death surveillance and response, training of health care providers in basic maternal and neonatal care, and fee waivers and exemptions for maternal and child health care services [34]. To improve demand and quality of these services, the PCQI strategy was implemented from October 2012 in all 93 intervention kebeles communities) across 14 woredas where BEmONC was also initiated. The primary focus of PCQI in the first 18 months was on maternal care in two PCQI cycles (cycles described below); this was subsequently extended to include newborn care in the third cycle, for 6 months. The PCQI strategy aimed to achieve its goal by facilitating community involvement in defining, implementing and monitoring the quality improvement process. Figure 1 shows a conceptual framework for the strategy. Increasing access and quality of ANC, delivery and perinatal outcomes began with an understanding of the barriers to quality care from provider and community perspectives. By strengthening communication processes on issues related to the quality of maternal and neonatal care, it was expected that enhanced interactions between the communities, HEWs, health care providers and woreda health offices would lead to recognition of a shared responsibility in improving maternal and newborn care behaviors and practices. Conceptual framework of the PCQI strategy PCQI involves a seven step cyclical process to achieve outcomes, as shown in Fig. 2.; 1) Selecting the primary care unit and holding a launching workshop with key stakeholders to build consensus. 2) Identifying and meeting community representatives (pregnant women, husbands, mother in laws; religious leaders and WDA members). 3) Explore quality meeting: conducting meetings with community representatives and biannual meetings at the facility level to identify major barriers to accessing services and gaps in service provision. 4) Bridging the gap workshops bringing community representatives and health workers from the health center together, to present their own perspective on barriers and service gaps (e.g. low care seeking behaviour; a health post not providing 24 h services because the HEWs live outside the kebele; communities unable to reach the health center due to poor roads and lack of transport; disruption of drugs and basic supplies; little confidence in giving birth at a health facility; and low levels of trust in the community). 5) Development of strategies and a joint action plan to address these barriers and gaps. 6) Implementation of the identified strategies (e.g. building HEW residences to enable HEWs to provide 24 h services; outreach sites; maintaining roads to facilitate transportation of pregnant mothers; preparing local stretchers for transporting women in labor and organizing youth groups to carry women in labor; supporting HEWs to inform the community about their schedule regularly; arranging a labor ward tour for women in their third trimester; training WDA members on the proper use of the family health card; promoting timely provision of drugs and supplies by woreda health offices). 7) Monthly review meetings of the performance of each strategy. Participatory Community Quality Improvement cycle This study was nested in a broader program evaluation for the L10 K Project and drew from the before-and-after household surveys conducted in January 2010–February 2011 and January 2014–February 2015, comparing areas with both PCQI and BEmONC strengthening in addition to the L10 K Platform, to the areas with BEmONC strengthening with the L10 K Platform alone. As indicated earlier, the L10 K Platform included the CBDDM and Family Conversation strategies and Birth Notification. The endline household survey was conducted after 26 months of PCQI intervention activities. The sample size for the PCQI intervention area was based on precision and not based on detecting effect estimates of the PCQI strategy. The parameters of the sample size estimation were: 50% expected prevalence, 95% confidence interval (CI) with ±6 percentage-points precision, 1.5 cluster survey design effect, with number of respondents per cluster set at 12. Thus, 34 primary sampling units were needed to obtain the sample size for the intervention area. The study participants were women of reproductive age who had a live birth in the 12 months before the survey. About 324 kebeles from the L10 K intervention areas were visited during both the survey periods of the broader evaluation (Table A2 in Additional file 1 for the first paper in this supplement [28]). These included 34 required kebeles from the PCQI areas and 82 kebeles that met the comparison group criteria. Thus, the sample sizes for propensity score matching (PSM) were 408 women from the intervention area and 990 women from the comparison area. In few cases the interviewers mistakenly interviewed more than 12 women from a kebele which resulted in six more women than the expected 984 (=12 X 82) respondents in the comparison areas. The broader L10 K evaluation was a two-stage cluster survey, stratified by administrative regions and the L10 K Project strategy (including PCQI). Kebeles were selected as primary sampling units (clusters) with the probability proportionate to its population size. At the second stage, the sampling strategy described by Lemeshow and Robinson was used to select the household with the target respondents [35]. Accordingly, the first household was selected randomly from the middle of the kebele and then every fifth household was visited, moving away from the middle, and if the household had women with children aged 0 to 11 months old they were interviewed, after seeking their consent. Twelve women were interviewed from each kebele to obtain information on their socio-demographic background and the maternal and newborn health care behavior and practices associated with their most recent pregnancy and childbirth. The health post of the sampled kebeles was visited, the HEWs interviewed, and the health post records reviewed to obtain information on HEW to population ratio (Additional files 1–3 for the first paper in this supplement [28]). The outcome indicators of interest were household and provider maternal and newborn health care behaviors and practices associated with the most recent childbirths among women with children aged 0 to 11 months. These were measured by the household survey. The definitions of the indicators are shown in Table 1. Definition of maternal and newborn health care indicators The independent variables of interest were the indicator variables for each study arm and survey period and the respondent’s age, education, marital status, parity, religion, household wealth, distance of the respondent’s household to the nearest health facility, administrative region and HEW density of the sampled kebele. The wealth index score was constructed for each household using the principal component analysis of household possessions (electricity, watch, radio, television, mobile phone, telephone, refrigerator, table, chair, bed, electric stove and kerosene lamp), and household characteristics (type of latrine and water source). The index was created among all respondents in the larger dataset from which the data for this study were extracted. The households of the larger survey were ranked according to the wealth score and then divided into five quintiles [36]. The WDA density in a kebele was the ratio between the total number of households and the number of active WDA team leaders in that kebele. Active WDA team leaders were those who had met with a HEW and discussed Health Extension Program issues during the 3 months preceding the survey. First, we compared individual, household and kebele-level sample characteristics measured in the follow-up survey across study arms using Pearson’s chi-squared statistics adjusted for cluster survey design effects. Similar statistical tests were done to 1) compare the outcome variables between the study arms during the baseline and the follow-up surveys; and 2) to assess statistically significant changes in the outcome variables during the observation period within each of the study arms. Stata 14.2 was used for the statistical analysis conducted for this study [37]. Propensity score matching was used to identify comparison individuals that were similar to those in the intervention area at follow-up, in terms of the background characteristics of the respondents, including kebele level estimates of the outcome of interest at baseline. Propensity scores are the probabilities of participation in the intervention and were estimated using logit models for each of the seven outcomes of interest, using the baseline kebele-level estimate of the given outcome as a covariate along with the background characteristics of the respondents in each case. We assessed average treatment effects (ATEs) of PCQI on the outcomes of interest using Stata’s ‘teffects psmatch’ procedure [38, 39]. The method imputes missing potential outcomes for each participant by using an average of the outcomes of similar participants that receive the other treatment level. The ATE was then calculated by taking the average of the difference between the observed and potential outcomes for each participant. To assess the adequacy of the matching, we assessed the balance of covariates across study arms after matching. Balance was considered adequate if the standardized differences of the covariates between the study arms were less than 10% after matching [40]. A minimum of one-to-one match per participant was considered adequate if the balancing property was satisfied. If the one-to-one match did not satisfy the balancing property, then the minimum number of matches per participant was incrementally increased until the balancing property was satisfied [40]. The method selected an extra match per participant if the propensity score was tied. ATEs are presented from propensity scored matched models that satisfied the balancing property.