A process evaluation plan for assessing a complex community-based maternal health intervention in Ogun State, Nigeria

Background: Despite increased investment in community-level maternal health interventions, process evaluations of such interventions are uncommon, and can be instrumental in understanding mediating factors leading to outcomes. In Nigeria, where an unacceptably number of maternal deaths occur (maternal mortality ratio of 814/100,000 livebirths), the Community Level Interventions for Pre-eclampsia (CLIP) study (NCT01911494) aimed to reduce maternal and neonatal mortality and morbidity with a complex intervention of five interrelated components. Building from previous frameworks, we illustrate a methodology to evaluate implementation processes of the complex CLIP intervention, assess mechanisms of impact and identify emerging unintended causal pathways. Methods: The study was conducted from 2013-2016 in five Local Government Areas in Ogun State, Nigeria. A six-step approach was developed to evaluate key constructs of context (external factors related to intervention), implementation (fidelity, dose, reach, and adaption) and mechanisms of impact (unintended outcomes and mediating pathways). The steps are: 1) describing the intervention by a logic model, 2) defining acceptable delivery, 3) formulating questions, 4) determining methodology, 5) planning resources in context, lastly, step 6) finalising the plan in consideration with relevant stakeholders. Results: Quantitative data were collected from 32,785 antenatal and postnatal visits at the primary health care level, from 66 community engagement sessions, training assessments of community health workers, and standard health facility questionnaires. Forty-three focus group discussions, 38 in-depth interviews, and 23 structured observations were conducted to capture qualitative data. A total of 103 community engagement reports and 182 suspected pre-eclampsia case reports were purposively collected. Timing of data collection was staggered to understand feedback mechanisms that may have resulted from the delivery of the intervention. Data will be analysed using R and NVivo. Diffusions of innovations and realist evaluation theories will underpin analysis of the interaction between context, mechanisms and outcomes. Conclusion: This comprehensive approach can serve as a guide for researchers and policy makers to plan the evaluation of similar complex health interventions in resource-constrained settings, and to aid in measuring ‘effectiveness’ of interventions and not just ‘efficacy’. Trial registration: This research is a part of the Community Level Interventions for Pre-eclampsia Study, NCT01911494. The trial is registered in Clinicaltrials.gov, the URL is https://clinicaltrials.gov/ct2/show/NCT01911494 The trial was registered on June 28, 2013 and the first participant was enrolled for intervention on March 1, 2014. Five Local Government Areas (LGAs) in Ogun State, Nigeria were chosen to receive the CLIP intervention by stratified random sampling. The CLIP (Community Level Interventions for Pre-eclampsia) intervention was delivered as part of the CLIP cluster randomized pilot trial from March 2014 to May 2015 in two Local Government Areas, Yewa South and Remo North, and later expanded to an additional three Local Government Areas(I,e, Ijebu North East, Odeda, and Ogun Waterside) from May 2015 to January 2016 as part of the definitive CLIP cluster randomized trial. In Nigeria, the primary implementers of the intervention were the community health care providers- community health extension workers (CHEWs), health assistants (HAs) and staff nurses. The process evaluation protocol covers data gathered during the Feasibility Study (2013–2014) [11], and during delivery of the CLIP intervention (2014–2016). The Research ethics boards at UBC Children’s and Women’s Health Centre of British Columbia and Olabisi Obabanjo University Teaching Hospital in Sagamu Nigeria provided ethical approval for the CLIP Cluster Randomized Controlled Trial (Number: H12-03497). The methods used to develop this process evaluation were adapted from Saunders et al [12] and tailored to the CLIP intervention in accordance with the Medical Research Council guidance [5, 8, 13]. Six steps were undertaken to: (i) to describe the intervention using a logic model to represent intervention activities, intended outcomes, theoretical constructs, and mediating factors [12]; (ii) to define complete and acceptable delivery of the intervention, in order to understand how the intervention may interact with the external MRC framework of process evaluation [5–9, 7, 13–17]; (iii) to develop process evaluation questions (iv) consider the relevant program resources (v) develop data management strategies (i.e., data sources, timing, and planning of data collection tools) using mixed-methods to answer the questions outlined in Step iii [10]; and (vi) to finalise the evaluation plan within an interdisciplinary team in collaboration with relevant stakeholders [9]. Quantitative data (such as that obtained using the PIERS on the Move mHealth platform), trial logs to monitor delivery of the intervention (community engagement logs, staff training logs, pre-post test questionnaires, drugs and devices tracking logs), observations checklists, budgets, and facility assessment data) will be analysed using simple descriptive analyses using Microsoft Access or R. Qualitative data (focus group discussions, key informant interviews, non-participant observations, pre-eclampsia case reports, community engagement field reports) will be analysed using thematic analysis in NVivo qualitative software. The use of established social theories is widely encouraged for process evaluations to allow for comparisons [8, 9, 18]. Therefore, building upon the Nigerian CLIP Feasibility Study [11], an adaptation of the diffusion of innovation theory [18, 19] was used to assess the CLIP intervention interacts with the system antecedents (context) to diffuse with the system (health system and community) for adoption by users (health workers and participants who receive the intervention). Realist theory was used to expand on the interaction between context and mechanisms to analyse identified mechanisms of action. The interaction of how ‘mechanisms’ and ‘context’ interact to produce ‘outcomes’ is represented in Fig. 2. Adaptability will be assessed to evaluate pragmatic contextual factors with delivery of implementation. Constructs of process evaluation for the CLIP intervention in Ogun State: The key functions assessed will be implementation (the infrastructure through which intervention is delivered, how it is delivered and the ‘what’ ‘quantity and quality’ of intervention), mechanisms of impact (how interaction between intervention activities and participants effect outcomes), and context (evaluating external factors which shape or may be shaped by intervention). As evident, these functions are non-linear and mutually-informative