Implementation science protocol for a participatory, theory-informed implementation research programme in the context of health system strengthening in sub-Saharan Africa (ASSET-ImplementER)

Objectives ASSET (Health System Strengthening in sub-Saharan Africa) is a health system strengthening (HSS) programme involving eight work-packages (ie, a research study that addresses a specific need for HSS) that aims to develop solutions that support high-quality care. Here we present the protocol for the implementation science (IS) theme within ASSET (ASSET-ImplmentER) that aims to understand what HSS interventions work, for whom and how, and how IS methodologies can be adapted to improve the HSS interventions within resource-poor contexts. Settings Publicly funded health facilities in rural and urban areas in in Ethiopia, South Africa, Sierra Leone, and Zimbabwe. Participants Research staff including principal investigators, coinvestigators, field staff, PhD students, and research assistants. Interventions Work-packages use a mixed-methods effectiveness-effectiveness hybrid designs. At the end of the pre-implementation phase, a workshop is held whereby the IS theme, jointly with ASSET work-packages apply IS determinant frameworks to research findings to identify factors that influence the effectiveness of delivering evidence-informed care. Determinants are used to select a set of HSS interventions for further evaluation, where work-packages also theorise selective mechanisms. In the piloting and rolling implementation phase, work-packages pilot the HSS interventions. An iterative process then begins involving evaluation, reflection and adaptation. Throughout this phase, IS determinant frameworks are applied to monitor and identify barriers/enablers to implementation. Selective mechanisms of action are also investigated. Implementation outcomes are evaluated using qualitative and quantitative methods. The psychometric properties of outcome measures including acceptability, appropriateness and feasibility are also evaluated. In a final workshop, work-packages come together, to reflect and explore the utility of the selected IS methods and provide suggestions for future use. Structured templates are used to organise and analyse common and heterogeneous patterns across work-packages. Qualitative data are analysed using thematic analysis and quantitative data are analysed using means and proportions. Conclusions We use a novel combination of IS methods at a programmatic level to facilitate comparisons of determinants and mechanisms that influence the effectiveness of HSS interventions in achieving implementation outcomes across different contexts. The study also contributes conceptual development and clarification at the underdeveloped interface of IS, HSS and global health. The ASSET-ImplementER theme is considered minimal risk as we only interview researchers involved in the different work-packages. To this effect we have received approval from King’s College London Ethics Committee for research that is considered minimal risk (Reference number: MRA-20/21-21772). ASSET is working on three care platforms: (1) integrated primary care; (2) maternal and newborn care; and (3) surgical care, across four sub-Saharan African countries: Ethiopia, Sierra Leone, South Africa and Zimbabwe. Within the three care platforms are eight work-packages (table 1). Description of the ASSET work-packages for the different healthcare platforms; implementation research cuts across all of them (ASSET-ImplementER theme) TB, Tuberculosis; WP, work-package. ASSET-ImplementER will be embedded within the timelines for ASSET (2017–2022). The work-packages use mixed-methods throughout ASSET to select HSS interventions in the pre-implementation phase and evaluate the intervention through ‘effectiveness–implementation hybrid’ designs in the piloting and rolling implementation phase. Hybrid designs are essential with implementation research as they blend the components of study designs used to evaluate clinical effectiveness, with those of implementation study designs that focus on the evaluation of the influence of context on the effectiveness of HSS interventions.35 Throughout ASSET, the ASSET-ImplementER stream uses mixed-methods including workshops, semistructured interviews and documentary analyses, to standardise, record and synthesise findings from the implementation component of the different work-packages. Findings include information from the different frameworks such as context, intervention, selected HSS interventions and implementation outcomes. Figure 1 describes the flow of methods for the ASSET-ImplementER theme. Flow of methods for the ASSET-ImplementER theme. 1Note: AIM, IAM and FIM have demonstrated promising psychometric properties in high-income settings.41 AIM, Acceptability of Intervention Measure; FIM, Feasibility of Intervention Measure; HSS, health system strengthening; IAM, Intervention Appropriateness Measure; LMICs, low-income and middle-income countries; ToC, Theory-of-Change; WP, work-package. To standardise methods and facilitate cross-site comparisons across ASSET, work-packages select contextual and behavioural determinants and implementation outcomes from a defined set IS frameworks. These frameworks are relevant to the programme as a whole, yet at the same time account for the specific characteristics of the different work-packages. Table 2 describes the frameworks that the various work-packages use and how this is relevant to the overall ASSET programme. Selected implementation science frameworks and theories used within ASSET-ImplementER BCW, Behaviour Change Wheel; CFIR, Consolidated framework for Implementation Research; CICI, Context and Implementation of Complex Intervention; HSS, health system strengthening; TDF, Theoretical Domains Framework. The Consolidated Framework of Implementation Research (CFIR) is a determinant framework that is used as it provides an overview of a broad range of determinants that influence implementation effectiveness, such as the inner setting (ie, characteristics of the health facility), characteristics of the intervention (eg, complexity and adaptability) and implementation processes (eg, regular feedback about progress and quality of implementation).36 The Context and Implementation of Complex Intervention (CICI) framework is another framework that we use as it offers a detailed approach to identifying determinants from the external context (eg, sociocultural, socioeconomic, political, epidemiological, ethical and legal) that are known to influence implementation effectiveness that are particularly relevant to LMICs.37 It is expected that both of these frameworks will provide a detailed spectrum of determinants that are relevant to the ASSET programme. However, the frameworks are not exhaustive: any determinant identified that is not a part of either frameworks will be documented as such. Further to contextual determinants, it is also important to understand in some detail the characteristics of the users who deliver the healthcare. Changing behaviour ingrained in both individuals working within health systems and users of the health system will help to ensure the adoption and longer-term sustainability of the HSS interventions. To address this, we will use the Theoretical Domains Framework (TDF), a determinant framework that brings together evidence-based determinants of behaviour.38 We also use the Behaviour Change Wheel (BCW) that explicitly maps behavioural-change interventions onto determinants of behaviour identified with the TDF.39 Each work-package selects the contextual and behavioural determinants, implementation outcomes and HSS interventions from the different frameworks that are relevant to their aims and objectives. Table 3 describes the objectives of the different IS frameworks across the different phases of ASSET. However, identifying determinants that influence the effectiveness of HSS interventions in delivering evidence-informed practices is not enough to address complexity associated with HSS. We will therefore also explore how context influences the mechanisms introduced by the HSS interventions on implementation outcomes. Application of implementation science frameworks across asset HSS interventions and research phases CFIR, Consolidated framework for Implementation Research; CICI, Context and Implementation of Complex Intervention; HSS, health system strengthening; TDF, Theoretical Domains Framework. To evaluate the effectiveness of the implementation efforts, ASSET uses a combination of implementation outcomes as defined by the framework by Proctor et al.40 The selected implementation outcomes are aligned with the aims and objectives for the different work-packages. Many measures used to evaluate implementation outcomes have not been validated, making it difficult to compare the effectiveness of alternative interventions both within and between studies.41 However, three implementation outcomes measures including Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM) and Feasibility of Intervention Measure (FIM) have demonstrated promising psychometric properties in high-income settings.41 Work-packages test the applicability of these measures in low-resourced settings by adapting them to their specific context. Each measure includes four questions measured on a Likert scale (completely disagree, disagree, neither agree nor disagree, agree, completely agree). Questions for each measure will be translated and back translated to ensure linguistic and cross-cultural equivalence. The questions will then be reviewed and cognitive testing with local participants performed prior to the tools being used. Initially we use the EPOC taxonomy for HSS interventions to label the different interventions selected by work-packages.33 If an appropriate intervention is not included in this taxonomy, work-packages use the ERIC taxonomy for implementation strategies to label the intervention.30 To help alleviate the ambiguity in reporting implementation strategies and HSS interventions, we also report any important differences and similarities between the two taxonomies. To provide consistency in labelling of the behavioural-change interventions, we use the taxonomy of behavioural-change interventions that is also aligned with the TDF and the BCW.42 However, there are important limitations to applying the TDF and BCW to help select interventions that improve the quality of people-centred care that also address issues such as illness-related stigma. As an example, these behavioural-change methods were not specifically designed to address the issues surrounding implementation research in low-resource setting such as stigma, that ASSET is focusing on. As such the behavioural-change interventions in this taxonomy may not be appropriate. To address this issue and similar issues with the other taxonomies, where the HSS interventions, quality improvement strategy or behavioural-change intervention is not described in any taxonomy, we clearly describe the intervention and highlight the issue with the selected taxonomy not describing our intervention. To ensure consistent and accurate reporting of implementation studies, work-packages will apply the Standards for Reporting Implementation Studies.43 At the end of the pre-implementation phase of ASSET, a 2-day workshop is held involving relevant participants from all work-packages (ie, principal investigators, coinvestigators, field staff and research assistants). The objectives of the workshop are to: (1) standardise the implementation methods being applied across the different work-packages; and (2) collect data from each work-package on contextual and behavioural determinants associated with the delivery of high-quality evidence-informed care and the associated HSS interventions and behavioural-change interventions that were selected to help deliver this care. Findings from this exercise will help to address our first and second objectives of contrasting and comparing contextual and behavioural determinants between the different work-packages and the selected HSS interventions. Using data collection tools that identify and describe determinants of the implementation process, participants within the different work-packages apply findings from the formative research to identify and record barriers and/or enablers to the delivery of high-quality care. Participants then use the selected determinants to theorise how potential HSS interventions, described in the EPOC taxonomy, can assist in delivering high-quality evidence-informed care relevant to their work-package. Specifically, work-packages theorise and describe the mechanisms behind how the different determinants interact to produce a consequence (ie, poor quality of care, delayed access to care, loss to follow-up, lack of people-centred care). Similar data collection methods and forms are applied to label and record determinants of behaviours that influence the delivery of evidence-informed care. The identified determinants are then used to select and record behavioural-change interventions. Templates for the workshops can be found in online supplemental appendices 1 and 2. bmjopen-2021-048742supp001.pdf bmjopen-2021-048742supp002.pdf To understand our third objective of the pre-implementation phase of ASSET (ie, how contextual determinants influence investigators in selecting HSS interventions, implementation outcomes), the workshop is followed-up with semistructured virtual interviews involving investigators from the different work-packages. The purpose of the workshop is to help understand the ‘how and why’, which led them to select the HSS interventions and associated strategies to facilitate their implementation (or virtual focus groups if more feasible for country teams). The same interviews/focus groups are also used to review and finalise the findings from the workshop and to understand if there are any gaps in evidence that will require the review any additional documentation. We anticipate including a minimum of two investigators from each ASSET work-package (16 investigators minimum sample) to reach saturation of the thematic areas that emerge from their responses. A guide for the interviews can be found in online supplemental appendix 3. bmjopen-2021-048742supp003.pdf Throughout the piloting and rolling implementation phase, workshops are held with each work-package team, involving relevant participants (principal and coinvestigators, field staff, research assistants and PhD students). The purpose of the workshops is to guide investigators on how to design the data collection tools including interview guides, using the IS determinant frameworks, HSS taxonomies and implementation outcomes. In doing so, we hope to ensure findings are standardised and therefore comparable across work-packages. At the end of the piloting and rolling implementation phase, a separate workshop will be held with each of different work-package teams to feedback the following information collected using the data collection tools: (1) the effectiveness of HSS interventions on standardised implementation outcomes, (2) the influence of context on the effectiveness of HSS intervention in delivering evidence-informed care and (3) the direct influence of context on the mechanisms introduced by the intervention to produce change. Participants are strongly encouraged to also theorise how the identified determinants interact with one another to produce mechanisms and the identified outcomes. It is through these workshops that we will be able to standardise the labelling of contextual and behavioural determinants to allow cross site comparisons on their influence of the standardised implementation outcomes (AIM, IAM, FIM) between the different work-packages (objective 4). At the end of the piloting and rolling implementation phase, another workshop lasting 1 day and involving investigators from all work-packages is used to reflect on the different IS methodologies and suggest improvements for further use. Specifically, we seek investigators’ opinions about the extent to which different frameworks capture the complexity of how the health system impacts on the overarching problem each work-package is trying to address. Again, we anticipate including a minimum of two investigators from each ASSET work-package (16 investigators minimum sample) to reach saturation of the thematic areas that emerge from their responses. All meetings will be audio-recorded with descriptive notes of discussion. Findings from these workshops are used to suggest how frameworks can be adapted to capture this complexity (objective 5). In work-packages that had available research capacity (WP1, WP2, WP5, WP6, WP8), staff delivering the interventions (ie, nurses, community health workers) are asked to complete the linguistically adapted tools for the three outcome measures, including AIM, IAM and FIM.41 Approximately 60 staff members from each of the participating work-packages contribute to these surveys. A subset of these participants are then interviewed to discuss the usefulness of the data collection tools. In particular, we review methods to capture the influence of context on both the effectiveness of the HSS interventions in influencing implementation outcomes as well as the influence of context on influencing the mechanisms introduced by the HSS interventions on implementation outcomes. Participants are also questioned on recommendations to improve our ability to capture the influence of context on implementation outcomes (Objective 5). Theory-of-Change (ToC) methodology is a participatory approach involving key stakeholders that allows the articulation of the ‘theory’ of how a complex interventional programme will work in reality, describing the necessary interventions to bring about the change, as well as the assumptions inherent to the programme and importantly the context of implementation.44 ASSET work-packages are developing ToCs to support their implementation and evaluation planning. ToCs are effectively programme theories, contextualised within each one of the ASSET work-packages, offering an overview of how the selected HSS interventions are theorised to achieve specific implementation outcomes. ToCs also include information on key assumptions and work-package context. Each work package, including all relevant stakeholders, develops an initial ToC in the pre-implementation phase. This programme theory is then adapted throughout the phases. We examine whether and how the ToCs align with findings from both the pre-implementation and piloting and rolling implementation phases of the different work-packages including determinants identified using the IS frameworks outlined in table 3. We will also use ToC to better articulate how context influences mechanisms introduced by the HSS interventions described by each work-package (Objective 4). The outcome measures AIM, IAM and FIM that have demonstrated promising psychometric properties in high-income countries are tested for similar properties including substantive and discriminant content validity (the extent to which a measure is judged to be reflective of a construct of interest), and interitem consistency (extent to which scale items are scored in a similar manner) by the different work-packages45 to determine their relevance in low-resource settings. Measures are adapted to the local context, translated and back translated. To test for substantive validity (extent to which a measure is reflective of the construct of interest) and discriminant content validity (extent to which a measure that is not supposed to be related is actually unrelated), different cadres of workers who are responsible for delivering the intervention in the different work-packages assign 31 items reflecting the three constructs, to each of the three constructs and rate their confidence in the assignments in order test. The Wilcoxon one-sample signed-rank test or t-test (as appropriate) is used to determine whether items measured the intended construct, or whether items measured a combination of constructs. Hochberg’s correction is used to correct for multiple tests.46 Intraclass correlation coefficients using a two-way mixed-effects model to assess the level of agreement in item assignments among all participants, and also within key stakeholder groups, across 31 items and for each construct.47 The same data are also used to assess the factorial validity of the three measures, initially through exploratory factor analyses.41 We assess interitem consistency by computing Cronbach’s alpha for each of the four-item scales. For each measure, we also calculate means and SDs. Higher scores represent more favourable responses. If the measures demonstrate adequate psychometric properties, they are used to facilitate cross-site comparisons across the different work-packages. For each work-package, a thematic analysis is used to analyse the qualitative data collected in semi-structured interviews and workshops that identify key issues pertinent to using IS frameworks and implementation outcome measures for HSS in low-resourced contexts. At the end of each phase of research, data are collected from each of the work-packages (ie, contextual and behavioural determinants, mechanisms, selected HSS interventions and implementation outcomes) and entered into a template created in Excel software (vs 16.49). As an example, each work-package records information on identified barriers/enablers, associated data source, relevant IS framework, EPOC HSS interventions and implementation outcomes. Table 4 demonstrates an example of the Excel template. Sample template used to synthesise findings from the workshops and interviews CICI, Context and Implementation of Complex Intervention; EPOC, Effective Practice and Organisation of Care; ERIC, Expert Recommendations for Implementing Change; TDF, Theoretical Domains Framework. We adapt the matrixed multiple case study approach to analyse and synthesise the data we record in the templates.48 This method facilitates comparisons between relevant work-packages, organising, analysing and presenting common and heterogeneous findings across implementation sites. Such an approach also aims to create generalisable knowledge regarding what and how local factors influence implementation. The matrixed multiple case study approach uses a combination of quantitative and qualitative methods that will allow us to identify associations between specific implementation processes and contextual factors on the one hand, and implementation outcomes on the other. Initially, data are analysed separately for each work package. Quantitative data are analysed using means and proportions. Given the heterogeneous nature of the care platforms and associated work-packages, context and selected HSS interventions, it will not be useful to compare quantitatively the influence of context on the effectiveness of HSS interventions on implementation outcomes between the work-packages. Instead, we use a qualitative approach that aims to understand why implementation outcomes were similar or different by describing the associated HSS interventions and contextual and behavioural determinants. Results from the matrixed multiple case study approach as well as findings from the interviews and the final workshop will be triangulated to identify points of convergence and divergence between different work-packages. Using these methods, we analyse how each work-package defined and operationalised different IS constructs (eg, context, intervention mechanisms, HSS interventions, behaviour change techniques), and how implementation theories and frameworks were used to support specific HSS interventions throughout implementation. The final part of our analysis integrates these elements to offer an overarching understanding of how IS frameworks have been operationalised across all ASSET work-packages and whether the use of these frameworks offered added utility (from a design, implementation or evaluation perspectives). Patients and the public were not involved in the designing/writing protocol for this protocol. However, extensive participatory methods that involve both the patients and public will be used by the individual work-packages to design, select and evaluate the HSS interventions for ASSET.