Menu
Menu
Menu
Menu
Introduction: The Integrated District Evidence-to-Action program is an audit and feedback intervention introduced in 2017 in Manica and Sofala provinces, Mozambique, to reduce mortality in children younger than 5 years. We describe barriers and facilitators to early-stage effectiveness of that intervention. Method: We embedded the Consolidated Framework for Implementation Research (CFIR) into an extended case study design to inform sampling, data collection, analysis, and interpretation. We collected data in 4 districts in Manica and Sofala Provinces in November 2018. Data collection included document review, 22 in-depth individual interviews, and 2 focus group discussions (FGDs) with 19 provincial, district, and facility managers and nurses. Most participants (70.2%) were nurses and facility managers and the majority were women (87.8%). We audio-recorded all but 2 interviews and FGDs and conducted a consensus-based iterative analysis. Results: Facilitators of effective intervention implementation included: implementation of the core intervention components of audit and feedback meetings, supportive supervision and mentorship, and small grants as originally planned; positive pressure from district managers and study nurses on health facility staff to strive for excellence; and easy access to knowledge and information about the intervention. Implementation barriers were the intervention’s lack of compatibility in not addressing the scarcity of human and financial resources and inadequate infrastructures for maternal and child health services at district and facility levels and; the intervention’s lack of adaptability in having little flexibility in the design and decision making about the use of intervention funds and data collection tools. Discussion: Our comprehensive and systematic use of the CFIR within an extended case study design generated granular evidence on CFIR’s contribution to implementation science efforts to describe determinants of early-stage intervention implementation. It also provided baseline findings to assess subsequent implementation phases, considering similarities and differences in barriers and facilitators across study districts and facilities. Sharing preliminary findings with stakeholders promoted timely decision making about intervention implementation.
We used an extended case study design that combined case-oriented and variable-oriented approaches, in which the CFIR was embedded. The case-oriented approach focused on describing the specificities of each case (district) using the CFIR constructs and subconstructs. The variable-oriented approach focused on exploring the similarities and differences across the cases with reference to each construct or subconstruct (“variable”). The CFIR informed the development of data collection tools and the approach to analysis, while the extended case study approach16–18 guided the interpretation of findings. We collected data in 4 districts of Mozambique (Báruè and Gondola in Manica Province, and Beira and Búzi in Sofala Province), in November 2018, immediately after the first year of implementing the IDEAs program. Mozambique’s national health system is organized into 4 levels of health care attention (primary, secondary, tertiary, and quaternary) with corresponding geographic levels of management (rural or urban, city or district, province, and central). The district, which in some cases is also a municipality, is the most basic level of administrative and financial management of Mozambique’s health system, instead of the health facility. For this reason, the district served as the IDEAs program’s basic level of intervention. The sampling plan consisted of multiple stages. First, we randomly selected 4 of the 12 intervention districts, 2 in each of the 2 provinces where the intervention was being implemented. Second, we purposively selected participants for in-depth individual interviews (IDIs) and focus group discussions (FGDs). We selected 2 provincial health managers (MCH supervisor and head of planning and cooperation) in each of the 2 provinces, 2 district health managers (health director and MCH supervisor) in each of the 4 districts, and 12 MCH nurses—1 from each of the 3 facilities receiving direct intervention support in each of the 4 districts in the semester leading up to data collection. We selected 1 MCH manager from each of the 12 above-mentioned health facilities to participate in FGDs, regardless of having received direct intervention support. All participants were eligible for the IDI or FGD if they had in-depth knowledge of the intervention. Provincial and district managers’ in-depth knowledge was defined as having participated in intervention activities for at least 12 months at the time of IDI or FGD. Health facility managers’ and nurses’ in-depth knowledge was defined as having attended at least 2 audit and feedback meetings before study participation. At all levels, higher-ranking managers were prioritized for participation but were replaced by lower-ranking ones if they were unavailable at the time of study participation. Each health facility sent only 1 manager and 1 nurse to the district meeting, so we interviewed the 1 present at that meeting if they were eligible. IDIs with provincial managers provided an overview of implementation in the province, while FGDs provided district perspectives. IDIs with district managers and FGDs with MCH managers from facilities that received direct intervention support aimed to provide in-depth understanding of intervention implementation in each district. All IDIs and FGDs were conducted in person and included 41 participants (22 from IDIs and 19 from 2 FGDs). Most participants were nurses (n=15, 36.6%), MCH facility managers (n=14, 34.1%), and women (87.8%) (Table 1). We reviewed documents that described intervention plans, implementation reports, and presentations made during audit and feedback meetings, and we audio-recorded 88% of the included IDIs and FGDs (n=22/24). We conducted 2 FGDs with MCH facility managers. Participants in Study on Integrated District Evidence-to-Action Program to Reduce Under-5 Mortality in 4 Districts, Mozambique Abbreviations: FGD, focus group discussion; IDI, in-depth individual interview; MCH, maternal and child health. We did not interview MCH nurses from 2 health facilities, because 1 was unavailable for interviewing and another was not eligible because she was new to the intervention. Participants in 2 IDIs and 1 FGD did not consent to audio-recording. We removed 2 FGDs from analysis because of protocol violations: 1 was conducted with MCH managers from facilities not included in the intervention, and the other was conducted with participants who had already completed IDIs. Interviews were conducted after study participants gave their written informed consent. They gave consent separately for documenting the interviews using notes and audio-recording. Audio-recordings and notes were assigned individual alphanumeric codes that protected the identification of each key informant. Before preparing this article, study investigators obtained key informants’ feedback on preliminary study findings developed from IDI and FGD notes. To protect the identity of study participants, we replaced district names with codes (A, B, C, D) and used general participant categories, such as “provincial or district manager” (without mentioning the province or district), and “nurse,” or “nurse manager” (without specifying the health facility). Two teams of 3 interviewers with academic training in the social sciences, humanities, or public health research and 3 or more years of research experience in MCH, supervised by 2 study investigators, conducted the assessment over 3 weeks in November 2018. Before conducting data collection, teams received 5 days of training on procedures in human subjects research and data collection and management, and data collection and supervision instruments were pretested. Team composition, supervision, and debriefing meetings were used to improve data quality and analysis. IDIs and FGDs were conducted in Portuguese. For each IDI, 1 team member conducted the interview while a second member documented it through field notes and audio-recording, if participants consented to the procedure. FGDs were run by all 4 team members: 1 facilitator, 2 notetakers, and an observer (study investigator). Study investigators conducted supportive supervision by observing at least 1 interview led by each interviewer, where they observed the quality of rapport and interview techniques, time spent during the interview, and how participant anonymity and data confidentiality were ensured. At the end of each supervised interview, the study investigator provided feedback to the interviewer and, if needed, discussed with them how to improve forthcoming interviews. At the end of each day, the study investigator led a team debriefing meeting to share supervision feedback, review findings, and conduct preliminary data analysis. The team also identified potential protocol violations or adverse events and other challenges and planned for the following day of data collection. As a first step to assess the validity of our findings,19 at the end of the data collection period, the teams presented preliminary district reports to district managers and the HAI implementation team in Mozambique. Each report contained key findings organized around the core intervention characteristics and the barriers and facilitators to successfully implementing the intervention, which participants had identified. In-depth data analysis was consensus-based and iterative, following a mixed deductive and inductive approach, using a predefined codebook containing CFIR constructs and subconstructs organized around the 5 original conceptual domains (https://cfirguide.org/tools/tools-and-templates/). Data analysis was conducted using ATLAS.ti 8.4, where data was stored and entries into the codebook, including definitions of conceptual entities (domains, constructs, and subconstructs), were made. After audio-recordings were transcribed, for 2 months between May and August 2019, 3 study investigators with experience in qualitative research methods and CFIR application analyzed data from IDI and FGD transcripts and notes and from documents containing intervention details. To ensure consistency in the analysis, the 3 investigators standardized analysis procedures in a daylong workshop. Thereafter, 2 investigators independently analyzed the same data (transcripts, notes, and documents) for each study facility and district and met daily to resolve discrepancies in coding and to jointly produce district-specific memos. Data analysis started with 12 constructs that study investigators had prioritized based on their research experience using the CFIR and working in the intervention geographic area (deductive approach). These constructs included “linkages among intervention components,” a construct that was not originally in the CFIR but that was based on the investigators’ experience using the CFIR. Study investigators were open to adding other CFIR constructs and subconstructs that they had not anticipated but that emerged during data analysis (inductive approach). This iterative approach allowed for adding 4 unanticipated CFIR constructs (Table 2). When consensus was not reached, a third investigator (tiebreaker) with experience using the CFIR resolved differences. Then, 1 of the 2 original investigators entered the agreed-upon codes and ratings in ATLAS.ti, after which the 2 original investigators met to write district memos. CFIR Constructs and Subconstructs Used to Analyze Study on Integrated District Evidence-to-Action Program to Reduce Under-5 Mortality in 4 Districts, Mozambique Abbreviation: CFIR, Consolidated Framework for Implementation Research. Ratings followed an approach developed by Damschroder et al., which defines the valence and strength of each CFIR construct or subconstruct.11,12 Valence denotes the positive or negative influence of the construct or subconstruct on implementation,11,12 which we defined as a facilitator or a barrier. Strength indicates (1) the level of emphasis, which is determined by the descriptive language participants used; (2) whether concrete examples were provided; and (3) the level of participant agreement on language and/or examples.11,12 Positive valence is indicated by +, and its strength can be weak (+1) or strong (+2), whereas negative valence is indicated by -, and its strength can be weak (-1) or strong (-2).11 Valence of constructs and subconstructs can also be neutral (0) if they have unclear directional influence, and their influence can be mixed (X) if the positive and negative influences cancel each other out.11,12 A construct or subconstruct was deemed significant in a district if at least 2 participants mentioned it and was considered important to the evaluation if it was mentioned in at least 2 districts. We also adapted and expanded the CFIR analysis process by using an extended case study approach, including a case-oriented and a variable-oriented approach.16–18,20 Using the case-oriented approach, the investigators coded and wrote memos for each IDI or FGD; using the variable-oriented approach, investigators selected constructs and subconstructs and wrote memos by site (district). Site-specific memos included construct and subconstruct ratings, with a narrative justifying the ratings and providing details (e.g., participant quotes) as appropriate. We then presented district-specific reports, in lay language in Portuguese, to each district to (1) assess validity of our findings through participant feedback, (2) promote evidence-based decisions about adjustments to the intervention, and (3) fulfill an ethical obligation of returning research findings to study participants. To prepare this article, we synthesized the key findings from district reports, highlighting differences and commonalities in barriers and facilitators, wherever relevant, and noting where data was not enough to warrant a conclusion on whether certain themes worked as facilitators or barriers. We adapted and expanded the CFIR analysis process by using an extended case study approach, including a case-oriented and a variable-oriented approach. The study was approved by the institutional review board of the University of Washington (IRB#STUDY00003926), Mozambique’s National Bioethics Committee for Health (Comité Nacional de Bioética para a Saúde-CNBS-IRB00002657), and the Ministry of Health, after endorsement from Manica and Sofala Provincial Health Directorates.
N/A
