Implementation of a community-based intervention in the most rural and remote districts of Zambia: A process evaluation of safe motherhood action groups

Background: A community-based intervention known as Safe Motherhood Action Groups (SMAGs) was implemented to increase coverage of maternal and neonatal health (MNH) services among the poorest and most remote populations in Zambia. While the outcome evaluation demonstrated statistically significant improvement in the MNH indicators, targets for key indicators were not achieved, and reasons for this shortfall were not known. This study was aimed at understanding why the targeted key indicators for MNH services were not achieved. Methods: A process evaluation, in accordance with the Medical Research Council (MRC) framework, was conducted in two selected rural districts of Zambia using qualitative approaches. Focus group discussions were conducted with SMAGs, volunteer community health workers, and mothers and in-depth interviews with healthcare providers. Content analysis was done. Results: We found that SMAGs implemented much of the intervention as was intended, particularly in the area of women’s education and referral to health facilities for skilled MNH services. The SMAGs went beyond their prescribed roles to assist women with household chores and personal problems and used their own resources to enhance the success of the intervention. Deficiencies in the intervention were reported and included poor ongoing support, inadequate supplies and lack of effective transportation such as bicycles needed for the SMAGs to facilitate their work. Factors external to the intervention, such as inadequacy of health services and skilled healthcare providers in facilities where SMAGs referred mothers and poor geographical access, may have led SMAGs to engage in the unintended role of conducting deliveries, thus compromising the outcome of the intervention. Conclusion: We found evidence suggesting that although SMAGs continue to play pivotal roles in contribution towards accelerated coverage of MNH services among hard-to-reach populations, they are unable to meet some of the critical sets of MNH service-targeted indicators. The complexities of the implementation mechanisms coupled with the presence of setting specific socio-cultural and geographical contextual factors could partially explain this failure. This suggests a need for innovating existing implementation strategies so as to help SMAGs and any other community health system champions to effectively respond to MNH needs of most-at-risk women and promote universal health coverage targeting hard-to-reach groups. The study was conducted in two remote districts, located in Luapula and the northern provinces of Zambia. The districts are among the four districts for the HPP where the SMAGs programme was implemented [15]. To select the study districts, we first stratified the districts into two provinces. Within each of the two provinces, we randomly selected one of the two intervention districts by flipping a coin. From each of the selected districts, two intervention health facilities were randomly selected using a lottery method, where all the facilities were assigned numbers, after which two numbers were selected at random. A process evaluation was conducted retrospectively, in accordance with the Medical Research Council (MRC) framework [19, 20], between November 2016 and January 2017, using focus group discussions (FDGs) and in-depth interviews (IDIs). Process evaluations have been reported as an essential part of community-based interventions [19, 21], needed to provide insight on how well programme activities are implemented, and performing within the context in which implementation occurs [22]. According to Moore and colleagues [19], effect sizes alone may not inform policy and implementers on how such community-based interventions may be replicated or reproduced in specific contexts. Moore et al. further argued that process evaluations are needed to assess fidelity and quality of implementation, as well as to identify causal mechanisms and contextual factors associated with the variations in the outcomes of interventions. The UK MRC framework [21] was adopted to guide the identification of relevant key constructs and to generate evaluation questions in this study. According to Moore et al. [19], despite a need to understand casual assumptions that underpin an intervention in complex interventions such as the HPP project, there is also a need to understand how the intervention worked by scrutinising its plausibility and the relations between implementation, mechanisms of impact and context. The SMAG intervention was regarded as complex because it comprised multiple interacting components and a number of targets to be met. According to the MRC framework, an intervention may have limited effects or positive outcomes due to its implementation processes such as fidelity, whether the intervention was implemented as intended or the degree to which an intervention is delivered as intended; the dose; the quantity of the intervention implemented; and its reach, whether the intended audience comes into contact with the intervention or not [23]. While the implementation context includes anything external to the intervention that may act as a barrier or facilitator to its implementation [19]. Further, the mechanism of impact guides an understanding of how an intervention was delivered and how the effects of the intervention occurred. An illustration of these key constructs and the assumptions on their interaction with the intervention is provided in Fig. 1. Key constructs of the process evaluation and the relations among the constructs [19] Existing evidence shows that the outcome of a complex community-based intervention can be influenced by the interactions between the stated three key constructs, namely implementation, context and mechanisms [19, 21]. The logic ‘inputs-processes-outputs-outcomes-impact model’ was used as a theory of change to guide the implementation of the intervention. Figure 2 describes the inputs, outputs (activities, participation) and their links to outcomes. Based on the model, the inputs included implementation plans, human resources, funding and working with district health teams. The processes included training of SMAGs and procurement of supplies, including bicycles and medicines, and the creation of data collection tools/systems that would facilitate the development of the community Health Management Information Systems (HMIS). These processes were expected to lead to short-term results that were expressed as output indicators, such as numbers of CHWs trained and referrals conducted. It was also assumed that the processes of the intervention would ultimately lead to medium-term outcomes of the intervention based on baseline coverage data, such as the proportion of mothers receiving at least four ANC visits during pregnancy. Finally, the impact was the long-term goal of the project that would include a reduction in neonatal, infant and maternal morbidity and mortality. However, it was also noted that there would be external factors likely to interact with this theory of change. Logical model for the Health for the Poorest Populations project Participants engaged in the intervention were purposively sampled for in-depth interviews and focus group discussions. Healthcare providers were purposively selected for in-depth interviews based on their active involvement in maternal and neonatal health as well as in the intervention. Focus group discussions were conducted with women and SMAGs who were purposively selected. The inclusion criteria for focus groups with women were women of reproductive age, with children less than 1 year old and living within the study community during their most recent pregnancy. The SMAGs were included in the study with the help of healthcare workers at the facility level if they were above the age of 18 years, both male and females, working within the communities under study on the implementation of the intervention and living either within or beyond 5 km radius from the health facility. A total of 78 participants were interviewed, 34 SMAGs, 36 mothers and 8 healthcare providers from Samfya and Luwingu districts. Eight in-depth interviews were conducted with healthcare providers, using qualitative research techniques to explore issues related to the implementation of the intervention, such as referral practices and supervision at the community level. In addition, eight FGDs were conducted, two from each facility. At each of the four facilities, one FGD was with SMAGs and another with mothers. Trained research assistants with experience in qualitative studies collected the data, 1 year after the intervention. Focus group discussions were conducted by a pair of research assistants, who were of the same gender and fluent in the local language (Bemba). One research assistant facilitated the sessions while the other one managed audio recordings and took field notes. The research assistants underwent a 2-day training prior to the data collection and were supervised by one of the co-authors (CJ). The data collection tools were piloted in a similar facility not included in the study (Additional file 1). The average duration of FGDs and KIIs was 45 min. The interviews were delivered on a face-to-face basis, at the health facilities. Informed consent was obtained from all the participants, and digital voice recorders were used to document the interviews and discussions. Recorded data were transcribed verbatim, and translated from Bemba to English, supplemented with field notes. All transcripts were assigned a unique identifier and imported into NVivo 13 for data management and analysis. Data was coded by two individuals, a trained research assistant and one of the co-authors (CJ). An iterative inductive thematic approach [24] was used through repeated rounds of reading and re-reading to clarify coding differences and to ensure consistency for subsequent analyses. Coders first independently listened to some recordings, reviewed a sample of the transcripts and began to formulate draft codes and themes. The researchers then met after coding the first six interviews to discuss the coding. Discrepancies were discussed until consensus was reached. Coding meetings with the research team and an experienced research assistant were held every week to create a mutual understanding of codes and refine the coding framework. The two coders examined and assigned sections of text to codes, representing themes or subthemes. Extracts of data were coded and memos were written to record emerging impressions of the data. Coded data extracts were further discussed among all the authors and merged into categories before refining them into themes. To further verify our results, we returned to the raw data. To enhance study validity, triangulation of different data sources (FGDs and IDIs) between different respondent groups was done by cross-examining the data [25, 26]. Triangulation is a recognised method to increase the credibility of data analysis [25]. This was achieved through data triangulation whereby the perspectives of the different respondent groups were explored. We also maintained a detailed audit trail of all decisions through a codebook, coding discussions and meetings.