Systems analysis and improvement to optimize pMTCT (SAIA): A cluster randomized trial

Background: Despite significant increases in global health investment and the availability of low-cost, efficacious interventions to prevent mother-to-child HIV transmission (pMTCT) in low- and middle-income countries with high HIV burden, the translation of scientific advances into effective delivery strategies has been slow, uneven and incomplete. As a result, pediatric HIV infection remains largely uncontrolled. A five-step, facility-level systems analysis and improvement intervention (SAIA) was designed to maximize effectiveness of pMTCT service provision by improving understanding of inefficiencies (step one: cascade analysis), guiding identification and prioritization of low-cost workflow modifications (step two: value stream mapping), and iteratively testing and redesigning these modifications (steps three through five). This protocol describes the SAIA intervention and methods to evaluate the intervention’s impact on reducing drop-offs along the pMTCT cascade.Methods: This study employs a two-arm, longitudinal cluster randomized trial design. The unit of randomization is the health facility. A total of 90 facilities were identified in Côte d’Ivoire, Kenya and Mozambique (30 per country). A subset was randomly selected and assigned to intervention and comparison arms, stratified by country and service volume, resulting in 18 intervention and 18 comparison facilities across all three countries, with six intervention and six comparison facilities per country. The SAIA intervention will be implemented for six months in the 18 intervention facilities. Primary trial outcomes are designed to assess improvements in the pMTCT service cascade, and include the percentage of pregnant women being tested for HIV at the first antenatal care visit, the percentage of HIV-infected pregnant women receiving adequate prophylaxis or combination antiretroviral therapy in pregnancy, and the percentage of newborns exposed to HIV in pregnancy receiving an HIV diagnosis eight weeks postpartum. The Consolidated Framework for Implementation Research (CFIR) will guide collection and analysis of qualitative data on implementation process.Discussion: This study is a pragmatic trial that has the potential benefit of improving maternal and infant outcomes by reducing drop-offs along the pMTCT cascade. The SAIA intervention is designed to provide simple tools to guide decision-making for pMTCT program staff at the facility level, and to identify low cost, contextually appropriate pMTCT improvement strategies.Trial registration: ClinicalTrials.gov NCT02023658. © 2014 Sherr et al.; licensee BioMed Central Ltd.

The SAIA intervention entails mentored, iterative application of a systems analysis tool and related improvement approach to provide facility-level pMTCT staff and managers with a holistic view of their system’s performance, identify which steps in the pMTCT cascade are the highest priority for improvement and which bottlenecks are modifiable, and test contextually appropriate solutions. By mentoring facility staff to identify and test solutions, it is expected that this overall analysis and optimization process will lead to rapid and sustainable improvements in pMTCT service delivery, quantified using routinely available indicators. The findings coming from three diverse countries in sub-Saharan Africa, coupled with additional implementation process measures, will provide practical results that are directly applicable to pMTCT programs in diverse settings. This intervention incorporates systems engineering techniques that provide a systems view combined with iterative improvement cycles, which have been designed to holistically capture complex real-world systems and improve their functioning. These methodologies are also intended to be user friendly and simple to apply in order to engage frontline health workers and facility managers in the process of identifying bottlenecks and solutions, and testing sustainable solutions that are within their scope of control. The pragmatic study design and analytical approach represent an appropriate and robust attempt to evaluate the study intervention. A cluster randomized design is feasible and efficient, will adequately enable the detection of improvements in pMTCT performance associated with a facility-based intervention, and will reduce the risk of potential confounding related to assignment of health facilities to receive the intervention. Furthermore, the inclusion of three diverse countries in sub-Saharan Africa will improve the relevance and applicability of trial results in high need countries. This study employs a two-arm, 1:1 cluster randomized trial design to assess the effectiveness of the SAIA intervention. The unit of intervention is the health facility. A total of 36 health facilities are part of the trial, including 18 intervention and 18 comparison facilities (six intervention and six comparison facilities per country). The SAIA intervention is applied at the facility-level. Through improved pMTCT services, it aims to improve outcomes for pregnant and postpartum mothers and their infants. Study facilities are split evenly by intervention and comparison group across three sub-Saharan African countries—Côte d’Ivoire, Kenya and Mozambique (Figure 1). These three countries were chosen to provide heterogeneous implementation settings in order to generate evidence on the SAIA intervention across multiple countries from diverse geographic regions, with different health sector designs, varying levels of resource investments, and different patterns of HIV burden. Map of the Systems Analysis and Improvement Approach (SAIA) study countries. Intervention and comparison facilities split equally across the three study countries (six intervention and six comparison facilities per country). Study facilities are located in three northern regions of Côte d’Ivoire (Gbêké, Hambôl, Poro-Tchologo-Bagoue) which had an estimated adult HIV prevalence of 4.4%, 4.4%, and 2.5% respectively in 2011 – 2012 [45]. PMTCT services in Côte d’Ivoire are provided primarily through public sector health facilities, as well as some private facilities. In 2011, 734 health facilities actively provided PMTCT services in Côte d’Ivoire, of which 113 are located in the study regions. The World Health Organization’s PMTCT Option B regimen was adopted as national policy in November 2012. Study facilities were selected from Nairobi city and Coast province, which had an estimated adult HIV prevalence of 4.9% and 4.3%, respectively, in 2012 [46]. PMTCT services in Kenya are primarily provided through public sector and private not-for-private health facilities, and use a mixed approach that includes the World Health Organization’s Option B + strategy in high volume facilities, with Option B and A in lower volume facilities. Study facilities were selected from Dondo and Nhamatanda districts, and Beira City, located along the heavily populated Beira corridor in Sofala province, Mozambique. Sofala has an estimated average adult HIV prevalence of 15.5%, which is higher in densely populated, urban areas [47]. The Mozambique National Health Service has a broad network of health facilities, and is the principal provider of formal health services in Sofala, providing over 98% of outpatient services in the province [48]. Utilization of primary healthcare services, including maternal and child health services, is high, and an estimated 95% of women have at least one ANC visit, and 71% deliver in institutional settings [49]. Since the launch of multiple HIV prevention, care and treatment strategies between 2002 – 2004, HIV services (including pMTCT) have been rapidly scaled-up to achieve geographic coverage in an integrated fashion [50]. The Ministry of Health uses a mixed approach for pMTCT, including the World Health Organization’s Option B + in facilities with current cART capacity, and Option A in the remaining facilities. All public and non-profit health facilities with pMTCT services in the study region in each country were considered for inclusion. Facilities were excluded if they were more than 20 kilometers from a main transport corridor to ensure frequent contact with the study team. Facilities were also excluded if an ongoing prospective study or similar systems analysis and enhancement technique were being implemented, and if facility managers and/or staff were unwilling to participate in the study. Consent for inclusion in the trial was sought from facility leadership after the randomization process. A total of 90 health facilities providing pMTCT (30 per country) provided the initial sample frame for this study (Figure 2). Study leadership from each country (including the project Principal Investigator) met in November 2013, in Beira, Mozambique, to define intervention and comparison groups. After removing facilities that did not meet the eligibility criteria, the remaining 55 facilities were ranked by country according to volume of first ANC visits. The top 12 facilities were selected in each country, and split into two groups of six facilities (upper and lower 50%, according to ANC volume). Three facilities from each group were assigned to receive the intervention and three to not receive the intervention in each country (using the random number generator in Excel®), totaling six intervention and six comparison facilities per country (18 intervention and 18 comparison facilities in total). Facility eligibility and randomization. ANC: Antenatal care; KM: Kilometer; PMTCT: prevention of mother-to-child transmission (of HIV). The SAIA intervention is a facility-based, five-step, iterative process designed to guide pMTCT staff and facility-level managers in understanding and improving their pMTCT services. The intervention incorporates tools to enable a systems view of pMTCT performance from ANC through postpartum care for mother-infant pairs and to discern modifiable barriers on which to focus improvement efforts, followed by continuous quality improvement cycles to guide facility personnel in the identification, implementation, and rapid evaluation of appropriate facility-level solutions (Figure 3). Five steps of the Systems Analysis and Improvement Approach (SAIA intervention. An essential step for systems improvement is to engage facility-level pMTCT staff and managers in understanding their system’s performance and using performance data to identify and prioritize areas for improvement. A pMTCT cascade analysis tool (PCAT) was developed to provide this systems view and highlight priority areas to address within the pMTCT cascade, based on a similar approach representing the linkage between HIV testing and combination antiretroviral services in Mozambique (Figure 4) [51]. This Microsoft Excel©-based tool uses routinely collected facility data to calculate the number and proportion of women and children flowing through each step of the pMTCT cascade, broken down into the flow from ANC through birth, and the subsequent postpartum period. The PCAT is also automated to calculate the number lost at each step (difference between those eligible for and those successfully passing through each step), and to estimate the additional number of women and exposed infants who would complete all pMTCT steps if each step were individually improved; this calculation assumes that drop-offs for all other steps remain constant and rates of eligibility to pass to next steps are similar among those who do and do not pass through each step. As the first step of the SAIA intervention, the cascade analysis tool is used by study personnel to guide a discussion with facility-level staff to determine which steps in the pMTCT cascade are likely to have the largest effect in order to optimize overall pMTCT services. PMTCT Cascade Analysis Tool (PCAT). Demonstrates number lost and potential gains per step (if that step improved to 100%, holding the other steps constant) for the ANC➔maternity (yellow), and postpartum (blue) cascades. ANC: Antental care; cART: Combination anti-retroviral therapy; dx: Diagnosed; PPO: Prophylaxis. To support facility-level staff in identifying specific bottlenecks in their pMTCT system, study teams will work with staff from the ANC, maternity, postpartum, and at-risk child care settings to map the existing flow of mother-infant pairs across these services (see Figure 5 for an example of two ANC flow maps from a large, urban and a smaller, rural facility in central Mozambique). By working with facility staff to explicitly describe the sequential, linked processes of care delivery at their facility, systems inefficiencies and potential solutions are highlighted, and health workers and managers are directly engaged in process improvement efforts. Each intervention facility’s existing pMTCT flow will be described on paper with input from multiple pMTCT staff, then be transferred to a computerized version using Microsoft Visio, and subsequently be discussed with pMTCT staff to confirm that it accurately represents the care processes in place and to identify inefficiencies that can be modified within each step. Example of pMTCT process maps from two facilities in Sofala Mozambique. Legend: Maps are from a medium-sized rural health center (Tica) and large urban health center (Munhava) in 2009, and demonstrate the flow of women from entry into antental care through receipt of antiretroviral prophylaxis or combination antiretroviral therapy. The final three steps of the SAIA intervention build off of continuous quality improvement methodologies. In step three, the PCAT analysis and process maps will be a focal point for brainstorming solutions with facility-level staff, mentored by study assistants trained in the SAIA intervention (training includes both didactic introduction to the intervention, and practical application of the tools during a one-week pilot phase in each country). It is expected that these solutions will lead to more efficient progression across the pMTCT cascade through process flow adaptation and simplification, task restructuring, service integration, and/or job aid introduction. Workflow adaptations will be selected based on their potential to lead to rapid and sustainable improvements in the targeted step in the pMTCT cascade, and feasibility of implementation, including being within the sphere of influence of facility management and pMTCT staff (Table 1). An implementation plan for the innovation will be described in writing by facility and study personnel, including a future state process map that reflects processes after the modification, to ensure consensus among facility staff on the contents of the solution, as well as clarity of operational design and roles and responsibilities among facility personnel. After identifying and defining the adaptation to be implemented, facility staff will implement the proposed changes. Illustrative facility-level workflow modifications ANC: antenatal care; cART: Combination anti-retroviral therapy; CD4: Cluster of differentiation 4; HIV: Human immunodeficiency virus; PCR: Polymerase chain reaction. Using routinely reported pMTCT cascade data from the pMTCT step that was selected for improvement, facility staff will monitor improvements in relevant indicators for two to six weeks following the introduction of the facility-specific workflow adaptation. Monitoring data will be analyzed visually and using descriptive statistics after a short time period, relying on improvements in the absolute increase in the proportion of women or infants successfully progressing through the step of interest to capture large, rapid improvements associated with the introduction of the adaptation. Systems engineering process improvements are by definition iterative, entailing ongoing testing of innovations that respond to contextually specific, modifiable barriers. Facility staff will repeat steps one through five at the end of each change cycle, focusing on identifying new approaches to modify previously identified barriers, or if the first cycle was successful, focusing on improving priority bottlenecks identified in a repeated systems analysis. To ensure intervention adequacy and the implementation process, the SAIA intervention was piloted in five facilities for six months in Sofala, Mozambique. Through the piloting phase, the PCAT was adapted and translated from Portuguese into French and English, a toolkit was developed to support introduction of the intervention, and standard operating procedures were written to guide the implementation process and data collection procedures for the trial. The SAIA intervention is designed to be introduced to each study facility over a four-day period (Table 2). On day one, trained study nurses hold a meeting with representatives from sectors of the health facility relevant to pMTCT, which may include ANC, maternity, postpartum care, at-risk child care, laboratory, and pharmacy staff. The specific composition depends on the individual health facility structure. At this initial meeting the study objectives and core components of the intervention are introduced, a calendar of activities for the week and across the six-month intervention period are developed, and PCAT results populated with the facility’s data from the most recent six months shared. On days two and three, study nurses work with facility staff to develop process maps for relevant services (ANC, maternity, postpartum care, and at-risk child care) with up to two health workers from each sector (including the sector in-charge and a second staff member who is knowledgeable about the health facility and are willing to participate in the exercise). A feedback meeting is held at the end of day three to have staff from each sector share their process maps, prioritize which sector to target with the initial improvement cycle, and brainstorm potential solutions. On day four, study nurses work with facility staff to prioritize solutions, and develop an implementation plan that explicitly records: the change(s) to be tested; the individual(s) responsible for each change; metrics to assess service improvement; a future process map to reflect the innovation; and a follow-up visit schedule. Subsequent follow-up visits are planned weekly for the first four weeks, bi-weekly for the next eight weeks, and subsequently according to the schedule agreed upon by facility staff and based on support needs. SAIA introduction schedule ANC: Antenatal care; HIV: Human immunodeficiency virus; PCAT: PMTCT cascade analysis tool; SAIA: Systems Analysis and Improvement Approach. It is expected that each change cycle will require four to six weeks of testing to allow for sufficient time to detect desired effects on the pMTCT cascade, but to be short enough to allow for multiple iterations during the six-month intervention period. After each cycle, study staff will work with facility staff to review the original implementation plan, describe what actually occurred (including review of planned metrics to assess improvements), discuss what worked well and why, and identify what could be improved and how. The five-step SAIA intervention cycle will again be repeated to review PCAT data, update process maps, and develop the next change cycle. It is expected that between four and six change cycles will be implemented in each facility over the six-month intervention period. The total data collection and implementation period is designed to cover 21 months, which includes 12 months of pre-intervention data collection, six months of intervention implementation, and three months of post-intervention data collection. The 12-month timeframe for pre-intervention data collection was chosen to generate stable baseline study outcomes despite monthly temporal fluctuations in these measures. A total of six months was selected for implementing the study intervention to provide sufficient time for the intervention to be adopted and lead to improvements, to allow for a sufficient number of iterative system analysis and improvement cycles to lead to sustainable performance improvements (assuming that each cycle will last on average four to six weeks, this timeframe will allow for between four and six cycles per facility), and to be feasible to complete within the specified funding timeline. The three study outcomes include the proportion of women screened for HIV during their first ANC visit; the proportion of HIV infected women receiving antiretroviral medications during pregnancy (either bi-prophylaxis using AZT or combination antiretroviral therapy); and the proportion of infants born to HIV-infected women screened for HIV at six weeks postpartum (or two months in the case of Côte d’Ivoire, following national norms) (Table 3). These outcome measures were selected because they reflect successful progression through steps in the pMTCT cascade, they are likely sensitive to system-level improvements, and they represent steps that, if changed, would meaningfully alter patterns of HIV transmission or access to maternal and infant HIV care. Furthermore, these measures reflect national pMTCT protocols, and are therefore well understood and available in all three study countries. Data will be collected over 12 months before the intervention, during the six-month intervention, and three months after the intervention in both intervention and comparison facilities (21 time points and 36 measurements per time point). pMTCT outcome measures ANC: Antenatal care; ARV: Antiretroviral; AZT: Azidothymidine; cART: combination antiretroviral therapy; HIV: Human immunodeficiency virus; PCR: Polymerase chain reaction. All primary outcome measures are calculated as a monthly average (continuous percentage between 0% and 100%). Notably, outcome measures for study measures on use of appropriate ARVs in pregnancy for prophylaxis or initiation of cART, and infant HIV determination, include numerators and denominators from different time periods due to time progression between service contacts that can be represented by available data. For the use of appropriate ARVs in pregnancy, the delay between the service contact in the numerator (the number of pregnant women receiving AZT or cART) and denominator (number of women testing HIV positive in ANC) is estimated to be two months. This two-month delay reflects a mean gestational age of first ANC visit of 25 weeks, or approximately three months before initiation of AZT or cART. The recommended timeframe for PCR testing of infants is on average five months after the first ANC visit, where the majority of pregnant women undergo HIV testing. Facility-level covariates include those that were found to be associated with pMTCT performance in analyses that are part of the wider research project that includes the SAIA intervention trial, including patient volume, staffing levels, availability of laboratory tests, geographic location (urban or rural), availability of community-based support groups, year of pMTCT initiation, and pMTCT approach (Option B+/B/A). Further qualitative data will be collected during the implementation process, and at the end of the trial, to facilitate the interpretation of the primary study results and increase the applicability of the SAIA intervention to other settings. The qualitative implementation process measures were designed based on the domains of the Consolidated Framework for Implementation Research (CFIR) [52], and intended to provide contextual information on factors that are important for the intervention’s acceptability, penetration and adoption by pMTCT staff and facility leadership, as well as factors impacting the quality, feasibility, and sustainability of the intervention’s application. Descriptive information will be collected on how the SAIA intervention was introduced, facility-level innovations that were proposed for testing, what was actually implemented, what went well and why, and what could be improved. Likert scale measures (scaled between 1 to 10) as well as open-ended questions will focus on manager and staff impressions on the acceptability of the intervention, utility of the intervention, leadership support, involvement of other agencies, intervention timing, staff capability to implement the intervention, clarity of roles to implement the intervention, ability of the intervention to meet staff and patient needs, and what the intervention fails to address. Health facility registries from ANC, maternities, and postpartum child care services are the source for monthly performance measures pre-, during and post-intervention at the 36 study facilities. Data extraction from registries is performed by two trained study team members in each facility, and compared for consistency. In cases of inconsistency, data collection procedures are redone, until monthly totals are in agreement. All project data are double-entered into a Microsoft Access© database by each in-country study team and sent to the Seattle-based support staff for warehousing. Additional quality control procedures are carried out on a monthly basis on the centralized database to identify cases of missing data or outliers. Additional facility-level descriptive factors will be collected via interviews with facility managers and direct observation by trained study personnel using a data collection instrument designed for the purposes of this study. Covariates cover information on the facility (size, geographic location, public/non-profit managed), staffing levels and structure, service utilization, available auxiliary services (laboratory, pharmacy, community outreach), and other similar factors that may affect both the adoption and effects of the intervention. In addition to the routinely reported administrative data, in-country study teams will collect descriptive implementation process data on an ongoing basis using data collection instruments designed for the purposes of this study. A 14-item questionnaire using Likert scales (ranging from 1 to 10) and open-ended questions was developed to gather data relevant to the CFIR domains, and will be applied weekly for the first four weeks of the SAIA intervention, followed by monthly for the remaining five months, with facility managers and a rotating group of facility nurses. This questionnaire will be complemented by field diaries maintained daily by study teams, and post-intervention key informant interviews with study assistants and in-country study managers. The primary analysis approach to assess the impact of the SAIA intervention will rely on paired t-tests comparing the average outcome measures over the 12 months preceding the intervention, compared with the average over the three months directly following the intervention period, for the three study outcomes. As a secondary analysis approach, segmented regression analysis that controls for baseline outcome levels and trend to compare the monthly averages before versus after the intervention will be carried out for all three study outcomes. Change points will include a node at the beginning of the intervention period to reflect the introduction of the intervention; two additional terms will be added to the model at the end of the six-month intervention period to indicate the level change and trend change post intervention. To account for a three-month lag in the impact of the intervention, data from the first three months post initiation of the intervention will be excluded from analysis. Sub-analyses will be carried stratified by country and facility size to assess subgroup effects. Because facilities will be randomly selected and allocated to intervention and control facilities, we do not expect to perform adjusted analyses. However, descriptive characteristics will be compared between intervention and comparison facilities using student’s t-test and chi-square tests to assess potential imbalance between study arms, and subsequent analyses will consider covariates significant in bivariate analysis for inclusion in the final model in a stepwise fashion. In addition, because outcomes are reported as a monthly summary measure at the facility level, no additional procedures will be implemented to control for potential intra-class correlation at the cluster (health facility) level. Autocorrelation will be assessed using the Durbin-Watson statistic, and adjusted for if appropriate. Descriptive analyses will include visual inspection of data for time trends across the study facilities, as well as within each facility, to identify trends, assess whether specific workflow adaptations led to improvements in the pMTCT cascade, and describe the sustainability of effects associated with individual adaptations. Those adaptations that are found to have dramatic or sustained effects will be documented as part of identifying best practices for pMTCT improvement. Exploratory bivariate analyses using student’s t-tests and chi-square tests will be carried out among facilities that showed large improvements in study outcomes during the six-month implementation period to identify descriptive factors that may be associated with adoption or may mediate the impact of the SAIA intervention. Analysis of the implementation process data will include aggregate reporting of individual items from the Likert scales, as well as disaggregated analysis by study country, and by facilities found to have relatively better success with the intervention. The constant comparison method will be used to describe responses to qualitative data from open-ended questions, field diaries, and key informant interviews. Sample size estimates are based on the health facility as the unit of analysis, and were calculated to detect the change in the proportion of HIV-infected pregnant women receiving antiretroviral medicines in pregnancy (arguably the most important endpoint of the study). Using program data from 2009 to 2010, we estimated a mean pre-intervention baseline of 50%, and predicted a mean post-intervention level of 70% for intervention facilities (mean change score = 20%) and 50% for control facilities (mean change score = 0%), and a standard deviation of yearly change scores of 11%. Assuming α = 0.05 and β = 0.80, the sample size is sufficient to detect an 11% and a 20% difference in mean change scores across the three countries and at the country level, respectively (Table 4). Sample size and detectable change in study outcomes This study was approved by the institutional review boards of the Ministry of Health of Côte d’Ivoire, Kenyatta National Hospital, and the Ministry of Health of Mozambique, and was determined to qualify for exempt status by the human subjects division at the University of Washington. The study was registered with ClinicalTrials.gov ({“type”:”clinical-trial”,”attrs”:{“text”:”NCT02023658″,”term_id”:”NCT02023658″}}NCT02023658). Implementation of the SAIA intervention began in January 2014 in all study countries, and is being sequentially introduced in the 18 intervention facilities (six in each study country). At the time of submission, the intervention has been introduced in nine facilities (four in Côte d’Ivoire, two in Kenya, and three in Mozambique).

N/A

• Dima Health