Effectiveness of participatory community solutions strategy on improving household and provider health care behaviors and practices: A mixed-method evaluation

Introduction We implemented a participatory quality improvement strategy in eight primary health care units of Ethiopia to improve use and quality of maternal and newborn health services. Methods We evaluated the effects of this strategy using mixed-methods research. We used before- and-after (March 2016 and November 2017) cross-sectional surveys of women who had children 0-11 months to compare changes in maternal and newborn health care indicators in the 39 communities that received the intervention and the 148 communities that did not. We used propensity scores to match the intervention with the comparison communities at baseline and difference-in-difference analyses to estimate intervention effects. The qualitative method included 51 in-depth interviews of community volunteers, health extension workers, health center directors and staff, and project specialists. Results The difference-in-difference analyses indicated that 7.9 percentage points (95% confidence interval [CI]: 1.8-13.9%) increase in receiving skilled delivery care between baseline and follow-up surveys in the intervention area that is attributable to the strategy. The intervention effect on postnatal care in 48 hours of the mother was 15.3% (95% CI: 7.4-23.2). However, there was no evidence that the strategy affected the seven other maternal and newborn health care indicators considered. Interview participants said that the participatory design and implementation strategy helped them to realize gaps, identify real problems, and design appropriate solutions, and created a sense of ownership and shared responsibility for implementing interventions. Conclusions Community participation in planning and monitoring maternal and newborn health service delivery improves use of some high-impact maternal and newborn health services. The study supports the notion that participatory community strategies should be considered to foster community-responsive health systems. Ethiopia has a three-tiered health system designed to deliver services. The first level is primary health care, which serves an administrative district (woreda) with an average population of about 100K. The second level is a general hospital that serves the catchment population of approximately 10 woredas (about 1 million people), and the third level is a specialized hospital that serves the catchment population of five general hospitals (about 5 million people). In rural woredas, primary care comprises a primary hospital and a primary health care unit (PHCU) of one health center and five satellite health posts for every 25K people in the woreda [40]. Within the PHCU is Ethiopia’s flagship Health Extension Program (HEP), which establishes one health post and deploys two health extension workers (HEWs) in each kebele (a community of about 5,000 people) in the country to provide basic promotive, preventive, and curative health services [41]. To extend the reach of the HEP and mobilize the community and households, the Federal Ministry of Health (FMOH) established a network of women’s development army (WDA) members. Each WDA member is assigned to five households and encourages families to adopt and practice healthy behaviors [42]. The FMOH is committed to achieving the health-related Sustainable Development Goals (SDGs). The targets for improving reproductive, maternal, newborn, and child health (RMNCH) outcomes of its Health Sector Transformation Plan (HSTP) are aligned with the SDGs of reducing maternal and newborn mortality rates [43, 44]. To achieve these objectives, the HSTP aims to strengthen health systems to provide universal access to high-quality promotive, preventive, curative, and rehabilitative services. One of the key aspects of ensuring universal access to health services is engaging communities in the governance and accountability of their health system by monitoring program performance and ensuring service quality [45]. JSI Research & Training Institute, Inc. (JSI), with funding from the Bill & Melinda Gates Foundation, implements the Last Ten Kilometers (L10K) 2020 project strategies to involve communities in improving high-impact RMNCH care behavior and practices in 115 woredas in four of the most populous regions of the country (Amhara, Oromia, Southern Nations, Nationalities and Peoples [SNNP], and Tigray), covering about 19 million people, to help meet HSTP MNH targets. Its strategies include community-based data for decision-making (CBDDM), family conversation, and birth notification [46]. CBDDM is used to identify pregnant women and ensure they receive ANC, intrapartum care, and PNC for themselves and their infants [46]. L10K 2020 also introduced a birth notification strategy to promote postnatal care. Family conversation is a forum conducted at the home of a pregnant woman with her family members and relatives who are encouraged to support her during pregnancy, labor, delivery, and the postpartum period to promote birth preparedness and essential newborn care [47]. The project implemented the PC-Solutions strategy in eight of the 115 L10K 2020 Platform woredas (two in each region) between March 2016 and October 2017. This integrated intervention joined communities (including health posts) and facilities (health centers) and built on two previous JSI interventions; Participatory Community Quality Improvement and Early Care-Seeking and Referral Solutions [37, 48, 49]. PC-Solutions emphasized quality of care throughout the primary health care level by including communities, along with providers and managers at the primary health care level and the woreda health office, as a critical constituency. The PC-Solutions strategy is a four-step QI process (plan-do-study-act), for MNH services provided at PHCUs (Fig 1). In the first step, a joint situational analysis was conducted in the PHCUs using workflow mapping, client exit interviews, document review, and focus group discussions with mothers and WDA members. Following the assessment, a meeting with community members, HEWs, health center staff, woreda health office staff, and referral hospital staff was held to discuss assessment findings, consolidate points, and identify priority problems and their solutions. Challenges identified during the assessment included delayed ANC booking, suboptimal use of PNC, and poor quality ANC, obstetric care, and PNC. The main problems were: 1) delayed identification of pregnant women and linking them to health center and health post/HEW; 2) inability of health workers to use partograph; and 3) delayed notification of recently delivered women to HEWs for PNC. Community members also mentioned cultural taboos on disclosing pregnancy before a certain amount of time (i.e., people believe that the pregnancy might end in miscarriage if mothers disclosed earlier); lack of awareness (even among WDA members) of the importance of early ANC booking; suboptimal use of PNC; and late notification of births to the HEWs hindering ability to initiate PNC immediately following birth. Encouraging early ANC booking, increasing use of PNC, and improving quality of ANC, obstetric care, and PNC were PC-Solutions’ priorities. Its interventions included early identification and notification system of pregnancy and postpartum mothers; introduction of ANC defaulter tracing mechanism through the WDA members; using mentors and peer learning for onsite partograph training for health care providers; introduction of automated monitoring tools at PHCUs to use data for decision making; and establishing health facility QI teams that included community members. The strategy also included monthly follow up and coaching visits from L10K 2020; monthly QI meetings at health centers and in communities for health post staff, HEWs, and WDA members to review data and progress; quarterly learning sessions; and QI refresher training for facilitators (Table 1). In the first plan of the PC-Solution strategy, early ANC and PNC, continuity of ANC visits, and partograph use were prioritized and implemented over two years. Intrapartum and newborn care quality were introduced after a reassessment at the end of fiscal year 2017. Quality improvement teams were formed at the health center and community levels and included health center service providers, HEWs, WDA members, and local administrators. The QI team collated and triangulated administrative data from health centers and posts to inform QI cycle plan and study fora. The overall QI approach of PC-Solutions included joint “planning and acting” and independent kebele- and the health center level “do and study” fora. The kebele-level do and study cycles were facilitated by health center staff with participation of the kebele-level QI team. Community members identified bottlenecks and solutions and helped implement and monitor the process to improve facility quality and performance. We evaluated the effects of PC-Solutions strategy using mixed-methods research. Four rounds of cross-sectional surveys of all eight health centers were conducted in March 2016, October 2016, April 2017, and October 2017. A pre-/post-test nonequivalent group study design was nested within the household surveys of women with children 0–11 months conducted in March 2016 and November 2017 to monitor MNH care behaviors and practices in the 115 L10K 2020 intervention areas [50]. To evaluate the effectiveness of the PC-Solution strategy, changes in household MNH care behavior and practices between the two surveys were compared between L10K 2020 Platform areas with and without the PC-Solutions strategy. Researchers choose a programmatic qualitative research design to answer the qualitative objectives. The interview technique was face-to-face in-depth interviews (IDIs) of WDA members, HEWs, health center directors, health center staff, and L10K 2020 QI specialists from four PHCUs. The qualitative study was conducted in September 2018. All eight health centers were visited for the facility survey. Data were collected through interviews with providers and a review of patient records and service statistics. For the household surveys, the sample size was powered to detect 10 percentage-points difference between two survey periods for an indicator with alpha error set at 0.05; beta error set at 0.20; and cluster survey design-effect set at 2.0 for the comparison area and 1.0 for the intervention area. The point estimates for an indicator at baseline and follow-up were assumed to be 45% and 55%, respectively, to yield the largest sample size to detect the desired change. Accordingly, the sample size for the intervention area was determined to be 400 women with children ages 0–11 months; for the comparison area, the sample size was 800 women with children ages 0–11 months. The household surveys employed a two-stage cluster sampling method stratified by program domain and region. Within each stratum, kebeles were selected as primary sampling units with the probability of selection being proportionate to population size (first stage); at the second stage, the sampling strategy described by Lemeshow and Robinson (1985) [51] was used to select households with target respondents and interview them. To do so, a kebele was sub-divided into three equal segments (sub-kebeles) and four respondents from each segment were interviewed. To identify the first respondent, the interviewers went to the population center of the segment (the point in the segment where the population is about equally distributed on all sides), spun a pen on the ground, and chose the first household in the direction that the pen pointed after it stopped spinning. Consecutive households were visited until the desired sample size was achieved, moving away from the middle of the segment. If the household had women with children 0–11 months and they consented, they were interviewed. Kebeles visited for data collection during the baseline survey were revisited during the follow-up survey. A total of 39 kebeles in the intervention area were selected for the study. The strategy estimated that about 80 kebeles from the comparison area would be available to obtain the required sample size. However, as the L10K 2020 innovations were not scaled-up to other woredas as initially planned, the domain of the comparison area included more woredas. Therefore, the number of kebeles representing the comparison area was 148, and the power of the sample was greater than 80% to detect a minimum of 10 percentage points intervention effect. The final sample size of respondents at baseline and follow-up were 2,268 (473 interventions and 1,795 comparison) and 2,244 (468 intervention and 1,776 comparison), respectively. Data were collected using a structured interview questionnaire (S2 Appendix) designed into an Android mobile application SurveyCTO collect [52]. An interview guide with open-ended questions was used to capture qualitative information from informants (S3 Appendix). We recruited four research assistants, (one per region), who spoke the local language; who have a health background; and who have experiences in qualitative research, to do the interview. We oriented research assistants to maintain neutral so as to not influence the participants’ responses. Moreover, we thoroughly discussed with the research assistants on interview techniques to engage participants throughout the interview and to get a truthful and honest answer. Stratified purposive sampling schemes were used to obtain in-depth information. First, we selected one intervention PHCU from each region to gather detail and contextually relevant data. Then, in each PHCU, staff from the health center and all health posts and selected active WDA members in catchment kebeles were recruited for the study. The research team interviewed the PHCU director, PHCU staff who were actively participating in the implementation of the PC-Solutions strategy, and those who facilitated the community-level QI cycle. We also interviewed the L10K 2020 technical specialists for the PC-Solutions strategy in each region. Through the facilitation of the PHCU director and regional staff, the research team approached HEWs and WDA members and invited them to participate in IDIs at the health post and their home, respectively. Theoretical sampling technique was used to collect rich information from community health workers until saturation of categories with data is achieved. In-depth interviews of HEWs and WDAs were conducted until saturation of information is reached. All health workers who were actively participating in the implementation of the PC-Solutions strategy and L10K 2020 QI specialists were included. Accordingly, 51 IDIs were conducted with WDAs, HEWs, health center directors, health center staff and L10K 2020 QI specialists (Table 2). All recruited participants participated, no one refused to participate in the study. The dependent variables of interest were the household and provider MNH care behaviors and practices that were expected to be affected by the intervention measured by the household survey. MNH care indicators and facility readiness and performance definitions are in Table 3. The independent variables that were considered as potential confounders were the individual-, household-, and kebele-level sample characteristics and administrative regions. The individual-level characteristics considered were age, education, marital status, parity, and religion; the household-level characteristics were wealth and distance of the respondents’ household from the nearest health facility; and the kebele-level characteristic considered was region. The wealth index score was constructed for each household with the principal component analysis of the household possessions (electricity, watch, radio, television, mobile phone, telephone, refrigerator, table, chair, bed, electric stove, and kerosene lamp), and household characteristics (type of latrine and water source). The households were ranked according to the wealth score and then divided into five quintiles [53]. Stata 15.1 was used for the statistical analysis conducted for this study [54]. The health facility indicators were presented by survey periods. The background characteristics of household respondents were compared between study arms at baseline and at follow-up survey periods using Pearson’s chi-squared statistics. Similarly, the unadjusted MNH indicators were compared. To estimate the adjusted intervention effects, the propensity scores were first estimated for each kebele using a logit model that predicted the kebeles in the intervention area at baseline. The covariates of the logit model were kebele averages of individual and household characteristics at baseline, kebele averages of MNH care behavior and practice indicators at baseline, kebele characteristics at baseline, and administrative regions. Covariates that had less than 0.2 p-value in the logit model were dropped using stepwise-backward selection [55, 56]. The final logit model from the stepwise procedure included the following covariates: education, religion, administrative region, first ANC, ANC in the first trimester, complete ANC, ANC experience score, ANC in the first and last trimester, PNC within 48 hours at home and at facility for the mother, and home birth notification. Intervention and comparison kebeles with similar propensity scores at baseline were coded so that they could be identified as similar. To assess the adequacy of the matching, t-tests were performed to ensure that the covariates of the final logit model were not statistically significantly (p>0.1) between the intervention and the control kebeles, after accounting for the matched kebeles. Finally, intervention effects (difference-in-difference) was estimated from kebele-level random effect models predicting the outcome of interest with indicator variables for study arm, survey period, the interaction term between study arm and survey period, and for the kebeles that matched between the intervention and comparison areas (dummy variables) as the predictors. Stata’s ‘margins’ command was used to obtain adjusted estimates of the outcomes of interest according to study arm and survey period and the difference-in-difference (DiD) (i.e. intervention effects). For the qualitative component of the study, audio records from IDIs were transcribed verbatim. The data were analyzed thematically. The transcript texts were manually coded. Then, themes were derived from the data coded. The codes, categories and the concepts emerged from an interview group were verified by linking the emerging categories with the data received from another group of informants to improve the trustworthiness of the qualitative data analysis. These categories were also linked to quotes from the research informants to ensure the reliability of the study. During reporting, participant quotations are presented to illustrate the themes/findings. Ethical clearances for the surveys were obtained from the ethical review boards of Amhara, Oromia, SNNP, and Tigray Regional Health Bureaus, and JSI. All participants were informed about the purpose of the study; benefits and hazards of the study were explained to all study participants, and each participant was notified of his/her right to opt out when responding to questions. Verbal consent was sought and documented before conducting any interviews. If the respondent was younger than 18 years old, consent was sought from her husband or guardian. Because the majority of the respondents were not expected to be able to read or write; written consent was not sought. If the respondent agreed to be interviewed after listening to the consent statement, the interviewer marked the questionnaire as consent given below the consent statement and signed below that. The interviewer continued with the interview only after receiving and documenting consent. The survey protocol submitted to the ethical review committee included the study questionnaire with the statement that described the consent-obtaining procedure. Moreover, the information obtained from the research participants was kept private (codes were used during reporting of the IDI quotes).