Understanding the influence of the MomConnect programme on antenatal and postnatal care service utilisation in two South African provinces: A realist evaluation protocol

Introduction Timely antenatal care (ANC) and postnatal care (PNC) attendance decrease maternal and child mortality by improving maternal and child health (MCH) outcomes. Mobile health or mHealth has been identified as an effective way of improving the uptake of MCH services. The MomConnect programme is an mHealth initiative launched by the National Department of Health of South Africa in August 2014 to support MCH. Although widely used, there is a limited understanding of how, why, for whom and under which health system conditions, the implementation of MomConnect improves the health-seeking behaviour of pregnant women and mothers of infants in ANC and PNC facilities. This paper describes the protocol for a realist evaluation of the MomConnect programme, to provide a theory-based understanding of how, why and under what healthcare conditions the MomConnect programme works or not. Method and analysis We will use the realist evaluation approach through its research cycle conducted in three phases. In phase I, a multimethod elicitation study design will be used, including a document review, key informant interviews and a scoping review to formulate an initial programme theory of the MomConnect intervention. Content and thematic analytic approaches will be used to analyse the data that will be fitted into a realist framework to formulate the initial programme theory. In phase II, a multi-case study design will be applied using a multimethod approach in two South African provinces. In each case, a theory-testing approach underpinned by the hypothetico-deduction analytic model will be used to test the initial programme theory. Surveys, interviews and focus group discussions will be conducted with various programme actors and analysed using appropriate methods. Phase III will entail refining the tested/modified programme theory through cross-case analysis. Expected outcomes An improved understanding of how and why the MomConnect intervention improves the health-seeking behaviour of pregnant women and mothers of infants, and the health system conditions that influence its implementation. Ethics and dissemination Ethics approval was granted by the Stellenbosch University Ethics Committee (S18/09/189). The protocol has been designed and the study will be conducted in line with the principles of the Declaration of Helsinki (1964). The study will be guided by the realist evaluation approach, which was introduced through the seminal work of Pawson and Tilley36 to address the question: ‘What works, for whom, why, in what situation, and how?’ with regards to intervention, programmes and policies. The philosophical ontology of realist evaluation is realism, and its epistemological foundation lies in scientific realism.36 Realist evaluation belongs to the family of theory-driven approaches of evaluation. In this light, the approach aims to develop and refine hypotheses of generative causality explicating why and how intervention inputs bring about changes in key outcomes.37 The proposed study will be developed following three different phases, as described in figure 2, and reported following the guidelines for reporting realist evaluation studies.37 Study design showing phases I to III. Adapted from Mukumbang et al and Van Belle et al 43 50 (Source: study author). ICAMO, Intervention-Context-Actor-Mechanism-Outcome. Phase I will address the first objective of the study re-formulating the initial programme theory (IPT) of how the MomConnect programme was expected to work for different actors (designers, health workers and MCH clients). A multimethod elicitation study will be conducted using data collected from the following sources: document review, exploratory study with key informant interviews and a scoping review.38 Since phase I of the study requires many substudies, it is estimated to take between 12 and 18 months depending on funding availability. The document review will explore documents such as the action plans, staff meeting reports and other minutes; any literature on MomConnect and the national monitoring and evaluation report. Permission will be requested from the MomConnect monitoring and evaluation team to access these documents and explore how MomConnect was developed, who participated and with what intended objectives. Document analysis,39 as a methodological process for review and evaluation, will be used to examine and interpret MomConnect documents, including capturing the meaning, gaining understanding and developing empirical knowledge on how the intervention was developed and implemented.39 The Intervention-Context-Actors-Mechanism-Outcomes (ICAMO) heuristic tool40 will be used to guide a content analysis approach. An exploratory qualitative analysis of programme managers and designers’ assumptions and perspectives will be undertaken simultaneously.41 In-depth interviews (IDIs) with 10–15 key informants, including programme designers (those who assisted in programme conceptualisation) and managers (those who assisted in programme implementation and coordination) will be conducted to explore their expectations on how MomConnect was supposed to work. Key informants will be selected using purposive sampling, and face-to-face interviews will be conducted using an interview guide with each key informant. The IDIs with key informants will be audiorecorded and transcribed verbatim. Summary sheets and field notes will be written up for each interview at the end of the day. Thematic analysis based on the generic inductive approach41 will be conducted using Atlas.ti software V.8.0. The scoping review of research conducted on mHealth and MCH will be performed by searching the following databases: Academic research complete, Medline, Pubmed, Scopus, Health System Evidence and Google scholar using medical subject headings (MeSH) terms. The following MeSH terms combinations (Boolean phrases) will be used to search the identified databases: [‘mHealth’ AND ‘maternal health’], [‘mobile phone’ AND ‘maternal health’ AND ‘child health’], [‘mHealth AND ‘maternal health services’], [mHealth PRE/15 maternal] and [mHealth PRE/15 maternal AND child AND health]. A thematic analysis42 will be used to explore the various modalities of mHealth, relevant context conditions, possible generative mechanisms and important outcomes reported in other studies conducted on mHealth interventions and MCH. Using abductive reasoning, we will conduct configurational mapping informed by the ICAMO heuristic tool to formulate tentative models of how and why mHealth programmes work in general. A configurational mapping approach40 guided by the ICAMO heuristic tool will be used to synthesise the information gleaned from the document review, qualitative exploration study with key informants and scoping review, to formulate the IPT that will be tested in phase II.40 The theory formulated will be informed by various abductive reasoning through deductive, inductive and retroductive reasoning forms of inferences making. The IPT formulated in phase I will be tested in the selected cases using a multimethod study design. In each case, a theory refining research approach will be used, underpinned by the hypothetico-deduction analytic model informed by data from both quantitative and qualitative methods.43 44 Quantitative assessments will be performed using a cross-sectional design, whereby a survey will be used to explore how socioeconomic characteristics influence the uptake of MCH to determine for whom the intervention works. The cross-sectional study could identify possible demi-regularities and contrastives related to the MCH uptake. The qualitative approaches will help us explore the different mechanisms and various context conditions by which the outcomes are generated. A representative sampling frame from the DHIS was used to obtain the study sample based on ANC first visits before 20 weeks for June 2016. This period was the latest month in the master frame data of DHIS which could be used to calculate the sample. All the districts in each province were drawn to identify districts with the highest and lowest ANC first visits before 20 weeks. In GT, West Rand and Johannesburg districts have the highest (69.0%) and lowest (57.3%) rates, respectively. The Randfontein sub-district in West Rand District was identified as the best sub-district with the highest rate of ANC first visits before 20 weeks (80.3%), while Johannesburg A was the sub-district with the lowest rate in the Johannesburg district (50.6%). Similarly, in the FS Province the districts with the highest (73.4%) and lowest (69.1%) rates are Xhariep and Fezile Dabi districts, respectively. Naledi Sub-district (Xhariep District) was identified as the sub-district with the highest rate (80.4%) while Moqhaka Sub-district (Fezile Dabi District) was the sub-district with the lowest rate (55.0%). OpenEpi software and MS Excel were used to randomly select four facilities in each sub-district (see table 2). The sample size in each facility was calculated,45 assuming 50% prevalence of MCH services agreement per facility, a precision (d) of 0.10 and a 95% CI. The monthly estimated number of MCH visits for each facility and the proportion of ANC first visits before 20 weeks were used to calculate the sample size using an online calculator.45 In cases where the ratio of the sample size (n) to the population size (N) is >5%, finite population correction was used.46 The number of users found in all four facilities will be multiplied by a design effect of 1.5 to get the total number of participants (table 2). Selected sample by districts, sub-districts, facilities and participants *Estimated number of participants (based on proportional sampling). ANC, antenatal care. The study participants will include all pregnant women and mothers of infants registered under the MomConnect programme, who are ≥18 years of age, irrespective of parity (including stillbirths/miscarriages) and socioeconomic status. They will be identified through the MomConnect registration registers at facility level. HCPs will include clinical staff in charge of ANC and PNC at facility level and will be selected based on their prior experience with the MomConnect programme. A structured questionnaire survey tool has been developed (online additional file 1) to collect quantitative data from pregnant women and mothers of infants registered with the MomConnect programme. This tool will assess their understanding of how psychological determinants, sociocultural context and structural context influence their uptake of MCH services. An appointment will be made with each participant, using the contact details captured in the MomConnect database, to invite them to the facility to participate in the study. In addition, the survey instruments will be administered telephonically to those participants who are unable to visit the facility during the study period. bmjopen-2019-029745supp001.pdf Furthermore, a facility assessment questionnaire (online additional file 2) will be administered to HCPs at facility level to explore the structural and contextual attributes that may influence the uptake of ANC and PNC services. bmjopen-2019-029745supp002.pdf Qualitative assessments will include IDIs and focus group discussion (FGDs). An estimated 10–20 IDI will be conducted with HCPs at facility level to explore their perceptions on the resources, implementation processes and programme uptake of the MomConnect programme. See online additional files 3, 4, and 5 for interview and FGD guidelines with patients and HCPs. Four FGDs (one per facility) consisting of between 10 and 15 participants will be conducted by the field workers and the principal researcher to explore participants’ perceptions regarding the uptake of MCH services and expectations of the MomConnect programme. Daily activities in selected ANC and PNC facilities will be observed as well. IDIs and FGDs will be audiorecorded and transcribed verbatim to be analysed using Atlas.ti 8.0. The interview guide and survey questionnaire are designed in English but will be translated into the local language used in the different study settings (Afrikaans, Setswana, Sesotho, Zulu, Xhosa) and back-translated to English. Data gathered from the above sources will be transcribed, translated and back-translated in preparation for analysis. In each case, a theory refining research approach underpinned by the hypothetico-deduction analytic model will be used for both quantitative and qualitative studies.43 44 The hypothetico-deduction approach is the most appropriate when testing an existing theory or a theory formulated a priori. This approach allows for various aspects and the entire hypothesis or initial theory to be examined in light of the new evidence that is emerging in the various cases. Inferential and descriptive analyses of the quantitative data including proportions and frequencies will be performed using Stata V.15. Socioeconomic and demographic data will be used as independent variables while the uptake of MCH will be used as the dependent variable, which will allow us to determine for whom the intervention works. For bivariate analysis, categorical data will be analysed using Χ2 test. Logistic regression models will be constructed to assess the effects of independent variables as predictors of dependent outcomes of interest. The power of each dependent variable will be tested before the variable is included in the model, and variables with positive and negative outcomes will be maintained. A p value <0.05 will be used to indicate statistical significance along with a 95% corresponding CI. For qualitative data, IDI and FGDs will be analysed separately using a thematic content analysis approach to identify and make explicit the mechanism by which observed outcomes are generated using the ICAMO framework. FGDs and IDI transcripts will be uploaded onto Atlas.ti 8.0 for analysis. Deductive and inductive thematic analyses will be applied to analyse the data collected through observation, IDIs and FGDs following these seven steps: (1) Familiarising with the data set; (2) development of a coding framework, (3) coding a portion of the dataset for each case study, (4) testing code reliability, (5) identifying initial themes emerging from the data, (6) using the code manual to apply codes to the entire script and (7) connecting codes into themes through abstraction. In-case analysis36 will be conducted using retroductive reasoning47 to modify the ICAMO elements through configuration mapping based on the data obtained from each case to modify the IPT. In other words, ICAMO configurations will be modified into case-based programme theories (for each of the four facilities). Each case-based modified theory will be tested to check their explanatory power through the process of counterfactual thinking towards a functional theory.48 ICAMO matrices will be used to present data for each higher-level outcome of concern. A cross-case analysis of the four case studies will be conducted using retrodictive reasoning to construct ICAMO matrices (ICAMO configuration obtained from each of the four case studies) to obtain a refined programme theory or model. Retrodiction is used to examine the similarities and differences between the theories or models obtained from the different cases. The retrodictive cross-case analysis will allow us to obtain a more refined programme theory to the IPT and the case-specific theories. A refined theory is a clear or functional explanation theory that can be used to give details of the programme elements and their roles in orchestrating the observed outcomes in certain context conditions for the different actors involved. This refined theory, although obtained through analytic generalisation, remains close enough to the observed data, yet provides explanations that are sufficiently general to explain outcomes across settings and social activities.36 The RAMESES II guideline for conducting and reporting realist evaluation37 will be used to ensure quality control in the study. First, to elicit the IPT, all the above steps will be followed to ensure the trustworthiness of data collected from various sources, thus capturing a wide range of intended and unintended outcomes, context-mechanism interactions and relevant actors. In phase II, to assess the reliability of data collection, a pilot study will be conducted in two healthcare facilities selected for convenience in the sub-district with the highest and lowest rates of ANC first visits before 20 weeks in GT and FS, respectively. The pilot will include HCPs, pregnant women and mothers. At all levels of the study, quality control and credibility will be assured through data familiarisation by all the investigators and discursive interactions. Ethics approval has been granted by the Health Research Ethics Committee of the Stellenbosch University (Ref No: S18/09/189). An approval to conduct the study in the selected facilities is currently being processed by the relevant provincial Department of Health. Consent forms will be used to obtain permission from study participants before data collection. Personal information will be protected by not disclosing names during data analysis or reporting. Different papers will be published from the study, and the results will be presented in academic open day, national and international conferences.