Exemplars in vaccine delivery protocol: A case-study-based identification and evaluation of critical factors in achieving high and sustained childhood immunisation coverage in selected low-income and lower-middle-income countries

Introduction Increases in global childhood vaccine delivery have led to decreases in morbidity from vaccine-preventable diseases. However, these improvements in vaccination have been heterogeneous, with some countries demonstrating greater levels of change and sustainability. Understanding what these high-performing countries have done differently and how their decision-making processes will support targeted improvements in childhood vaccine delivery. Methods and analysis We studied three countries – Nepal, Senegal, Zambia – with exemplary improvements in coverage between 2000 and 2018 as part of the Exemplars in Global Health Programme. We apply established implementation science frameworks to understand the € how’ and € why’ underlying improvements in vaccine delivery and coverage. Through mixed-methods research, we will identify drivers of catalytic change in vaccine coverage and the decision-making process supporting these interventions and activities. Methods include quantitative analysis of available datasets and in-depth interviews and focus groups with key stakeholders in the global, national and subnational government and non-governmental organisation space, as well as community members and local health delivery system personnel. Ethics and dissemination Working as a multinational and multidisciplinary team, and under oversight from all partner and national-level (where applicable) institutional review boards, we collect data from participants who provided informed consent. Findings are disseminated through a variety of forms, including peer-reviewed manuscripts related to country-specific case studies and vaccine system domain-specific analyses, presentations to key stakeholders in the global vaccine delivery space and narrative dissemination on the Exemplars.Health website. The purpose of this study is to assess ‘how’ and ‘why’ some countries have succeeded in achieving significantly improved coverage rates between 2000 and 2018, and to provide actionable recommendations for improving national and subnational vaccine coverage. This study focuses on critical policy and programmatic innovations that drove changes to vaccine coverage and equity across the three countries of interest, and specifically investigates ‘how’ and ‘why’ these innovations were implemented. Our research consortium includes Emory University, Georgia Institute of Technology, the University of Delaware, the Center for Molecular Dynamics in Nepal, the Center for Family Health Research in Zambia, the Institut de Recherche en Santé de Surveillance Epidemiologique et de Formation (Institute for Health Research, Epidemiological Surveillance and Training) in Senegal. Three exemplar countries—Nepal, Zambia and Senegal—were selected based on available data and expert review. Countries were eligible for inclusion if, in the year 2000, (1) their population exceeded 5 million and (2) the World Bank classified them as low income. Forty-seven countries met these criteria. Two analyses were performed to identify exemplars from the eligible countries based on measured coverage of DTP1 and DTP3: direct estimates of the compound annual growth rate (CAGR) of vaccine coverage and a segmentation analysis based on coverage, dropout rates and country conflict status (figure 1). Taken together, DTP1 and DTP3 serve as common proxies for the function of the vaccine delivery system in each country, as DTP1 can indicate how many children are reached by the immunisation system, and DTP3 can indicate how many children the programme has continued to reach.13 14 Country filtering process, of which 47 countries met the growth criteria. CAGR, compound annual growth rate; DTP, diphtheria, tetanus, pertussis; IHME, Institute of Health Metrics and Evaluation; LIC, low-income countries. The CAGR analysis used both WUNEIC and Institute of Health Metrics and Evaluation (IHME) data.5 15 For the above-mentioned 47 countries, we calculated CAGRs for each country, with both WUENIC and IHME data, from 2000 to 2016. CAGR calculations used 3-year rolling averages. We found the highest-performing countries by applying predetermined cutoffs by data source; the cut-off percentage depended on the overall performance of the group. The WUENIC data had a CAGR cut-off of 0.9%, indicating a 9% increase over 10 years, and the IHME data had a CAGR cut-off of 0.5%, indicating a 5% increase over 10 years. Seventeen countries met both the WUENIC and IHME CAGR cut-off percentage. The segmentation analysis used the rolling 3 year averages obtained from WUENIC data.10 Five segments were created by analysing and ranking DTP1 coverage, DTP3 coverage, dropout rates and conflict. The segments were classified as follows: Segment 1 countries had ‘proven themselves’ with national DTP3 coverage greater than 90%; Segment 2 included countries that were ‘on the right track’ with national coverages of DTP3 less than or equal to 90%, but DTP1 greater than 80% and a dropout rate greater than 10%; Segment 3 included countries that were ‘getting children back into the system,’ with national coverages of DTP3 90%, DTP1 80% and a dropout rate 10%; Segment 4 included countries that were still ‘building essentials’, with national coverages of DTP3 90%, DTP1 80% and no conflict at time of selection; and Segment 5 included countries with ongoing conflict at time of selection. Exemplar countries were identified as those meeting all three of the following criteria: (1) The country was in segments 3, 4 or 5 at any time during the period 2005–2010; (2) The country progressed to either segment 1 or 2; and (3) The country stayed in segment 1 or 2 for at least 3 years (figure 2). Segment analysis logic. DTP, diphtheria, tetanus, pertussis. The shortlist of possible exemplar countries, based on both analyses, had 13 countries (table 1). The final three countries were selected to represent geographical diversity (South Asia, East Africa, West Africa), as these regions have the majority of unvaccinated children globally. The democracy index, as defined by the 2018 Democracy Index, was used for framing the country selection and for exclusion criteria.16 Final exemplar countries were selected in conjunction with our technical advisory group (TAG). Additional country selection criteria considered during study planning and rationale for final selection, as of 2018 *Terms from the Economist Democracy Index 2018, and briefly defined as follows: Flawed Democracies have free and fair elections, and basic civil liberties are respected even through problems and weaknesses in the system; hybrid regimens have elections with irregularities, contain weaknesses in the system and typically contain a weak civil society; Authoritarian Regimens do not have free and fair elections, if they occur at all, and infringe on civil liberties, along with repressing criticism and censoring dissenters.16 †As of the 2020 Democracy Index Report, Senegal is now considered a ‘Hybrid Regimen’.20 CAGR, compound annual growth rate; DTP, diphtheria, tetanus, pertussis. We conducted research at different levels of the healthcare system for each country: the national level, three subnational regions/provinces and three districts per region/province for a total of nine districts. Our predetermined subnational region selection criteria differed by country, but one region in each country contained the capital city of the country, with the other two regions stratified on factors determined with input from the local study team (eg, high/low subnational immunisation coverage, rural/urban, road access/lack of road access, ethnic/religious minority/majority). Changes in subnational immunisation coverage over time were assessed using district-level data (figure 3A–C). Districts were selected based on country specific CAGR and DTP3 percentile cutoffs. Historical patterns of subnational DTP3 vaccine coverage in the three identified exemplar countries: Nepal (A), Senegal (B) and Zambia (C). DTP, diphtheria, tetanus, pertussis. Alongside our network of in-country and regional collaborators and networks, we identify a comprehensive list of key stakeholders to include in data collection. We aim to identify both individuals who were in the related positions at the time of data collection, and those who previously held such positions to assess how programmatic changes were implemented and adapted over time. The generalised list of positions is documented in table 2; due to local context and health system structure, specific positions may differ by country. Specific categories and titles, and the number of related data collection activities, will be presented alongside country-specific analyses. General summary of key informant and focus group participants by roles within the vaccine system *Includes volunteer community health workers, female community health volunteers, vaccinators, bajenu gox and neighbourhood health committee members. FGD, focus group discussion; KIIs, key informant interview. We formed a TAG consisting of experts in global health, vaccination delivery, vaccine confidence, and LIC and LMIC health systems to facilitate interpretation and dissemination of findings. The engaged stakeholder groups include WHO, UNICEF, CDC and Gavi. Engagement of the TAG is an ongoing process, with meetings convened for discussion at key decision points—including, but not limited to, input on final country selection, review of preliminary findings, review of context around key findings, and the current development of plans for dissemination This project uses several frameworks, which guided the development of tools and areas of inquiry. These overarching frameworks were taken from literature on vaccine delivery and implementation science. Implementation science is a growing field with the focus on applying evidence-based research findings into routine practice. Additional cross-cutting analyses use discipline-specific frameworks based on and extrapolated from the existing literature. The primary outputs of this study are country-level case studies, with additional cross-topic synthesis as possible. Our conceptual model organises the complex interplay of barriers and factors impacting global childhood vaccine coverage, based on the work of Phillips et al9 and LaFond et al,10 and a broader review of the vaccine confidence and coverage literature (figure 4). Specific input was provided by our multidisciplinary team of public health, behavioural science, implementation science, political science, public policy and systems science and engineering researchers. This novel framework serves as a guiding summary of the key issues for consideration in each country. The research is driven by the findings from each country (see Research Activities below), with no preconceptions regarding specific practices or interventions. An initial scoping visit for each exemplar country was used to gather preliminary feedback about the immunisation programme, historical challenges and interventions, and key stakeholders’ initial impressions about reasons for success. These findings were then compared with the overall framework in figure 4 to identify specific areas in which additional focus was needed during the main research activities. Conceptual framework of drivers of vaccine delivery, derived from scoping visits, Phillips et al,9 and LaFond et al.10 The goal of this project is to provide evidence-based, actionable recommendations to country and global stakeholders, with a focus on new insights to exemplary performance of vaccine delivery. Our initial scoping visits identified key historical barriers and interventions in each country; the focus of this research is understanding the ‘how’ and ‘why’ related to the adoption of each of these interventions or activities. Interventions may have been developed by stakeholders within each country (ie, endogenous innovation) or may be adaptations of higher-level guidance, such as local implementation of WHO guidance (ie, exogenous adaptation). For each intervention or programme—defined here as a solution developed and delivered by the country stakeholders (‘what’)—there is an iterative process between identifying the problem to be addressed (‘why’) and developing mechanisms for change, in other words ‘how’ the change could come about (figure 5). Understanding the interplay between ‘how,’ ‘why’ and ‘what’ can help identify actionable recommendations that may be useful for countries to consider when evaluating improvement in their vaccination systems. Mapping the ‘how’ and ‘why’ behind an intervention. A combination of two implementation science frameworks was applied to develop tools for data collection. Application of these frameworks directed our inquiry towards key domains of the historical decision-making and implementation process. Consolidated Framework for Implementation Research (CFIR) is a framework of five interrelated domains (intervention, outer setting, inner setting, individual characteristics and process of implementation) which influence the effectiveness of intervention implementation, and promote hypotheses of ‘what works where and why across multiple contexts.’17 We identified constructs within CFIR for focus within our tool development—including motivation, decision-making processes, mechanism for change, and the process and environment of development and delivery—in addition to inquires of events and policies most relevant to the success of Exemplar countries. The CFIR framework guides our examination of ‘what they did,’ ‘why they did it’ and ‘how they did it,’ at national, regional, district and local levels in order to understand diverse contexts and perspectives within each of the exemplar countries. This allows us to systematically organise our findings, and better interpret the similarities and differences both across and between exemplar countries. The Context and Implementation of Complex Interventions (CICI) framework was applied in addition to CFIR to address contextual factors and the interdimensionality missing from the CFIR framework; both framed our initial thinking about the vaccine delivery system.18 Both CFIR and CICI frameworks guided the development of an iterative data collection tool that could be applied consistently across diverse contexts and settings. Qualitative data collection was guided by semistructured key informant interview (KII) guides for use with health officials, external stakeholders and community leaders, and focus group discussion (FGD) guides for use with fathers, mothers, grandmothers and community health workers. These instruments explore the following CFIR and CICI domains: intervention characteristics, outer setting, inner setting, characteristics of individuals, process and context.17 Qualitative data collection was intended to limit the time burden for KII or FGD participants to no longer than 1 hour, although some data collection took longer—up to 2 hours or more—based on the richness of the discussion. An initial KII guide was developed for scoping visits and was revised post visit to ensure data was captured within the domains of interest raised in those KIIs. Our overarching goal was to gather information from participants about ‘how’ and ‘why’ interventions were developed, adapted and implemented, and how they led to an increase in vaccination coverage. The guides were developed by the research team and refined through iterative review after completion of data collection in each country. Prior to beginning both in-depth data collection and review of relevant literature, we conducted a 2-week scoping visit in each country to (1) meet with and select in-country partners; (2) discuss key factors of change for further exploration (eg, identify the ‘what’ items for exploration of ‘how’ and ‘why’) and (3) prepare for in-depth country research activities (eg, establish local partnerships, start ethical reviews, research activity logistics). We conducted 10-day training workshops with our local research partners prior to the start of data collection in each country. In addition to training on study materials and methodology, we reviewed the materials alongside our in-country research partners to aid in any translation and adjust content for country context. We conducted both KIIs and FDGs, as appropriate, with data collection occurring at the national level, subnational levels and community stakeholders at subnational levels (table 2). KIIs and FGDs took place in offices, clinics and community centres. All activities took place in a location deemed private, safe and comfortable by the participants. Qualitative data collection activities were conducted in person with trained facilitators and note-takers, when possible. Conditions for in-person research relative to the COVID-19 pandemic necessitated adjustments to maximise the quality of data collection and participant and researcher safety. FGDs consisted of 6–8 participants. FGDs were held in the communities, organised by type of participant (eg, fathers will be in one group), and consisted of groups of fathers, mothers, grandmothers and community health workers. Partner organisations or community health workers identified the FGD participants. With permission from KII and FGD participants, interviews were recorded to ensure capture of all information. Recordings were transcribed verbatim from the local language by local research assistants and translated to English manually, or translated using Google Translate (for French), with verification by a fluent bilingual speaker. All documents with transcriptions were only accessible to researchers named on the IRB. All transcribed documents required a code to access. All research files, recordings and transcriptions in-country were saved on password-protected computers. Recordings were removed from recorders at the end of every day, deleted once uploaded onto password-protected computers and saved to HIPAA-compliant storage in folders only accessible to the study team. All recordings have been removed from computers and servers following transcription and verification of accuracy. Interviewees’ names and contacts were deidentified, and all information will be used without mentioning their names. Documents that may link participants to their identifier code will be stored in separate locations. Data were coded using MAXQDA V.20 (Berlin, Germany) and analysed thematically by specific aim, research question and framework-specific construct(s). The initial analysis for each country consisted of a case study, specific to that country, identifying the key drivers of improvements in vaccine coverage. This broad case study served as a starting point for more detailed topic-specific analyses and manuscripts. For key factors identified in multiple countries, a cross-country synthesis will be conducted to identify similarities and differences in implementation across study countries. Quantitative data were gathered through freely obtained information on Ministry websites or data given from Ministry or other partners, such as the WHO, UNICEF and CDC. This quantitative analysis investigates vaccine coverage through a review of the health spending and economic growth trends from LICs and LMICs. Selected exemplar countries are compared with this grouping to determine what factors made exemplar countries stand apart from their peers. Analysis will use cross-country and multiyear mixed-effects regression models to statistically test financial, economic, development, demographic and other country-level indicators. A key component of this research will be to identify factors that may have been associated with improvements in vaccine coverage that are not commonly used as indicators of immunisation. This can include general health systems strengthening and improvements in funding for public health, as well as improvements in maternal and child health that may have driven support for immunisation services.19 We consulted with a TAG, but did not directly solicit patient or public involvement in the development of this research project.