Menu
Menu
Menu
Menu
Two reasons for the lack of success of programs or interventions are poor alignment of interventions with the causes of the problem targeted by the intervention, leading to poor efficacy (theory failure), and failure to implement interventions as designed (program failure). These failures are important for both public health programs and randomized trials. In the Sanitation Hygiene and Infant Nutrition Efficacy (SHINE) Trial, we utilize the program impact pathway (PIP) approach to track intervention implementation and behavior uptake. In this article, we present the SHINE PIP including definitions and measurements of key mediating domains, and discuss the implications of this approach for randomized trials. Operationally, the PIP can be used for monitoring and strengthening intervention delivery, facilitating course-correction at various stages of implementation. Analytically, the PIP can facilitate a richer understanding of the mediating and modifying determinants of intervention impact than would be possible from an intention-to-treat analysis alone.
We collect process evaluation data through record reviews, structured observations, interviews with VHWs and interviews with study participants at outcome measurement time points [31]. All enumerators, research nurses, and supervisors were trained and standardized on the various methods. VHWs maintain a module-delivery schedule for each mother they recruit into SHINE. These schedules specify which module should be delivered when, as well as allowable and acceptable windows around the target date. VHWs record when the module was delivered. VHWs also maintain registers in which they record their activities, such as prospective pregnancy surveillance. Supervisors routinely inspect this documentation and collect these data from each VHW during their scheduled monthly supervisory contacts. These data will be used to characterize supportive supervision (frequency of VHW-supervisor contacts) and VHW performance. VHW supervisors conduct structured observations of all VHWs to assess and document VHW interactions with study participants and adherence to behavior change intervention protocols. For each VHW, these observations are conducted during the first delivery of each new behavior change intervention module and quarterly thereafter. The assessment tools consist of Likert-type, multiple-choice, dichotomous, and subjective qualitative items that are used to assess specific behaviors of VHWs. Measures of VHW performance, such as lesson delivery scores, will be derived from these data. Research staff (part-time enumerators) administer a questionnaire to each VHW (following their informed consent as a research subject) 3 times, at baseline, midline, and endline. Data on sociodemographic, supervisory and motivational characteristics [14], curriculum knowledge, [18] and goal-setting capacity are collected. Research nurses administer questionnaires to participating women during 2 antenatal and 5 postnatal visits between recruitment at approximately 14 weeks of gestation and 18 months postpartum. Data collected include sociodemographic information, exposure to behavior change interventions, curriculum knowledge, maternal capabilities for caregiving [38], and WASH and infant feeding behaviors. A questionnaire module ascertains different indicators of household water access: source, type, walking time [44], distance of water for drinking and water for uses other than drinking, and 24-hour recall of household water collection. A composite measure of knowledge-sharing efficacy [18] will be derived from combining data on the curriculum knowledge of participating women with curriculum knowledge of VHWs, to assess VHW performance in knowledge sharing. Also, we will explore the computation of separate WASH and infant feeding behavior scores incorporating the behaviors promoted by the SHINE interventions. Relative socioeconomic (wealth) status will be derived using a principal components analysis that includes data on household assets, income, expenditures, and access to agricultural land at the time of the baseline household visit. A summary of the data collected, data sources, the indicators derived, and timing of data collection is presented in Supplementary Appendix Table 1. The full PIP, from randomized treatment allocation to reduced childhood stunting and anemia, elucidates several intermediate steps, a number of potential modifiers at each step, and different potential measures to characterize each step (including of FOI at delivery/receipt steps such as between the VHW and caregiver or between the caregiver and infant). Above and elsewhere [31], we describe our efforts to collect data that characterize this complex system. However, without making a large number of assumptions, it is infeasible to model this full PIP in a single statistical analysis. Instead, we will carry out a series of separate “partial” analyses that, when taken as a whole, test the theorized links in the PIP [7]. The statistical approaches we use complement the analysis plan for the primary outcomes of the trial [31], applied to the intermediate outcomes in the PIP. More specifically, we will conduct analyses of intermediate outcomes at each step along the PIP: (1) VHW performance capacity; (2) VHW performance; (3) maternal behavioral determinants/capacity; and (4) maternal behavior/performance. We will employ 2 analytical approaches in these analyses: (1) ITT based on the original randomized design and examining each intermediate outcome separately as an endpoint; and (2) per-protocol analyses linking together intermediate steps and conditional on specific prior outcomes or achievements in an earlier step, such as high FOI. For both approaches we will, via interactions, explore the role of pre-specified modifiers. Examples and potential hypotheses to be explored are presented in Table Table11. Potential Program Impact Pathway Hypotheses and Their Estimation Strategies Abbreviations: FOI, fidelity of implementation; IYCF, infant and young child feeding; PIP, Program Impact Pathway; VHW, village health worker; WASH, water, sanitation, and hygiene. The ITT analyses will examine the impact of the randomized interventions on an intermediate outcome, one at a time, treating that outcome as an endpoint [15, 20, 22], as well as assessing the role of modifying effects on it. For example, the second intermediate domain of the PIP is VHW performance. We hypothesize that a VHW’s performance of SHINE tasks (completion of module delivery visits, knowledge transfer) will differ according to their treatment assignment, and that the performance of VHWs assigned to implement both the WASH and IYCF interventions will be lower than that of VHWs assigned to implement only the WASH or IYCF interventions. Further downstream, we hypothesize that for the WASH intervention, mothers in households with greater access to water will practice hand washing with soap to a greater extent than households with less access to water. These analyses exploit the randomized design, and ITT estimates will be estimated as described [31] and will provide estimates of the average effect of the interventions (ie, the ITT effects) on the intermediate outcomes according to our program theory (PIP). Collectively, these analyses will address (1) the extent to which each of the 4 intermediate sequential processes were achieved; and (2) what the modifiers of those processes were, including whether the effects were modified by predetermined characteristics. The per-protocol analyses go beyond these intermediate outcome ITT estimates to examine movement along the PIP—that is, the linkages from earlier to later steps in the chain including, in particular, the final outcomes. For example, linking VHW performance capacity to actual VHW performance. Per-protocol analyses will also explore the linkages from earlier steps in the chain to the final outcomes, such as linking FOI of VHW delivery and stunting and anemia (to ascertain the effects among those who received the treatment as intended), and linking FOI of maternal/caregiver delivery to stunting and anemia (effects among those who tried and maintained the treatment behaviors). Conditional on having delivered/received the intervention relevant to the participant’s treatment arm (as defined by indicators for FOI), we will examine the association between the intervention and the outcome in a later step of the PIP, as well as with the final outcomes of the trial. As with the first set of ITT analyses, potential modifiers at each stage can be assessed using interactions. Depending on the starting point, the per-protocol analyses will be based on our categorization of FOI into 2 types—VHW and caregiver. In the first of these, FOI of VHW delivery, we classify participants who had at least 10 of the 15 VHW SHINE scheduled visits, starting at 24 weeks of gestation as having high/adequate fidelity. We standardized the number of VHW visits (15 module delivery contacts) across treatments to ensure that the content, rather than the number of contacts, is what differentiates the treatment groups. A visit is therefore defined by having contact at a scheduled behavior change intervention delivery visit. For FOI of caregiver delivery, we will develop separate and combined compliance indices for the WASH and infant feeding behaviors and apply a similar condition of at least two-thirds of the behaviors implemented. In contrast to the ITT, for these analyses the estimation sample is limited to those following protocol, and for whom effects are hypothesized to be larger. A limitation to this approach is that it no longer fully exploits the randomized design and therefore weakens causal inference. A benefit to this approach, however, is that it allows us to explore more directly the links between improved WASH and infant feeding practices themselves and the final outcomes. In particular, evidence on the linkages along the intermediate stages of the PIP, as well as any dose-response associations in the relationships between VHW delivery of interventions and the final outcomes, can provide additional plausibility to any observed ITT effects. Furthermore, identifying the drivers of effect heterogeneity can elucidate the circumstances, persons, and contexts in which any such effects are likely to be greatest.
N/A
