Effectiveness of a multicomponent safe surgery intervention on improving surgical quality in Tanzania’s Lake Zone: Protocol for a quasi-experimental study

Introduction: Effective, scalable strategies for improving surgical quality are urgently needed in low-income and middle-income countries; however, there is a dearth of evidence about what strategies are most effective. This study aims to evaluate the effectiveness of Safe Surgery 2020, a multicomponent intervention focused on strengthening five areas: leadership and teamwork, safe surgical and anaesthesia practices, sterilisation, data quality and infrastructure to improve surgical quality in Tanzania. We hypothesise that Safe Surgery 2020 will (1) increase adherence to surgical quality processes around safety, teamwork and communication and data quality in the short term and (2) reduce complications from surgical site infections, postoperative sepsis and maternal sepsis in the medium term. Methods and analysis: Our design is a prospective, longitudinal, quasi-experimental study with 10 intervention and 10 control facilities in Tanzania’s Lake Zone. Participants will be surgical providers, surgical patients and postnatal inpatients at study facilities. Trained Tanzanian medical data collectors will collect data over a 3-month preintervention and postintervention period. Adherence to safety as well as teamwork and communication processes will be measured through direct observation in the operating room. Surgical site infections, postoperative sepsis and maternal sepsis will be identified prospectively through daily surveillance and completeness of their patient files, retrospectively, through the chart review. We will use difference-in-differences to analyse the impact of the Safe Surgery 2020 intervention on surgical quality processes and complications. We will use interviews with leadership and surgical team members in intervention facilities to illuminate the factors that facilitate higher performance. Ethics and dissemination: The study has received ethical approval from Harvard Medical School and Tanzania’s National Institute for Medical Research. We will report results in peer-reviewed publications and conference presentations. If effective, the Safe Surgery 2020 intervention could be a promising approach to improve surgical quality in Tanzania’s Lake Zone region and other similar contexts. The study design includes two main elements: (1) a longitudinal, controlled quasi-experimental study examining changes in surgical quality processes as well as postoperative and postnatal complications; and (2) an in-depth, longitudinal qualitative study exploring factors that distinguish high-performing facilities with the most improvement in surgical quality processes. The study will have 3-month preassessment, 9-month intervention and 3-month postassessment periods (figure 2). Project timelines. The quantitative results will illuminate the impact of the intervention, while studying ‘positive deviants’ with the most improvement in adherence to surgical quality processes measured by safety as well as teamwork and communication scores using qualitative methods will identify the factors that allow facilities to achieve top performance. Combined, the results will contribute to knowledge about the effectiveness of strategies to improve surgical quality in Tanzania’s Lake Zone region and other similar contexts. Safe Surgery 2020 intervention will be introduced at 10 intervention facilities, while 10 control facilities will not receive the intervention. The intervention will be delivered at the facility level, with the primary outcome measured at the individual level (ie, surgical provider or patient). The setting includes five regions surrounding Lake Victoria, Tanzania: Geita, Simiyu, Shinyanga, Kagera and Mara. Collectively, they have an estimated population of 9 060 348 (18.5% of the nation’s population); 28.2% live below the poverty line and 67.0% live in rural areas.31 Ten facilities within Mara and Kagera regions have been selected as intervention sites based on a feasibility assessment conducted by Safe Surgery 2020 partners in October 2017 using a priori selection criteria: regional, district or health centre facilities with a surgical volume of at least 50 major surgeries annually; perceived quality improvement (QI) culture; availability of basic infrastructure to support the implementation of QI processes; willingness of facility leadership and surgical teams to participate; and site accessibility for the research team. Ten sites in Geita, Shinyanga and Simiyu have been selected as control sites based on their similarity to the intervention sites on community socioeconomic characteristics, facility level, type of surgical cases and total surgical volume. Facility characteristics are outlined in table 1. Characteristics of intervention and control facilities, 2018 Results of the feasibility assessment found that there are seven surgeons and no anaesthesiologists across the study facilities, surgery is largely provided by generalist physicians and assistant medical officers with 5 years of clinical training and anaesthesia is provided by nurse anaesthetists. Infrastructure, such as blood, clean water, oxygen and electricity, is not consistently available. Only two of the three Bellwether procedures, caesarean sections and laparotomies, are performed, with no open fracture fixations performed. Most facilities lack postoperative recovery areas and intensive care units. The Surgical Safety Checklist is not in use at any facility. All surgical inpatients who had a major surgery and postnatal women (based on WHO consensus definition) who underwent C-sections or delivered vaginally32 at study facilities will be enrolled and followed during their inpatient stay, up to 30 days. All patients who have a surgical procedure in the major operating theatre are operationally defined as having a major surgery whether performed under local or general anaesthesia. Patients from the study population will not be followed after discharge. Paediatric patients under the age of 5 years, minor surgeries, visiting surgeons’ patients, antenatal patients, women with spontaneous abortions and surgical outpatients will be excluded. Excluded minor surgeries are provided in online supplementary appendix 1. bmjopen-2019-031800supp001.pdf Patients were not involved in the conception and design of this study. Surgery 2020 is a multicomponent intervention that acts across the surgical system to strengthen five surgical quality areas: leadership and teamwork, safe surgery and anaesthesia practices, sterilisation, data quality and infrastructure, with staggered implementation over 9 months. The intervention will be delivered through a series of training sessions attended by a multidisciplinary surgical team from each intervention facility. During the leadership and teamwork training, surgical teams will identify surgical QI priorities by considering impact, influence and ability; identify root causes using QI techniques such as the fishbone diagrams or ‘Five Whys’ to solve problems and develop a QI plan to implement at their facility over 9 months.33 34 Accurate recording and reporting of outcome measures is key for feedback on performance, thus encouraging surgical QI and patient safety.35 The data quality strengthening intervention will improve the data accuracy and enable mentors to form feedback loops with surgical teams to identify gaps in quality. Intervention facilities will also develop a Facility Accelerator Fund grant proposal to access a US$10 000 grant to address infrastructural barriers to implementing their QI plan. A mentoring team including a surgeon, anaesthesiologist, obstetrician and a theatre nurse from the zonal hospital will visit each intervention facility bimonthly to reinforce the training and to support the surgical teams in achieving their QI plan. Mentoring will also occur virtually through Project ECHO (Project Extension for Community Healthcare Outcomes),36 where specialists from the zonal hospital and international faculty will provide didactic clinical updates, demonstrate skills and techniques, and provide weekly advice on difficult cases using video conferencing. Intervention facilities will learn from each other’s experiences through a WhatsApp group and an annual knowledge sharing session. To promote sustainability, facilitating leadership support, buy-in at all levels, and a QI culture and structure will be emphasised.30 Safe Surgery partners will offer structured feedback through facility-level baseline results and data on performance and coaching through mentorship visits. Safe Surgery 2020’s collaboration with the ministry to develop the National Surgical Obstetric and Anaesthesia Plan further explores sustainable financing mechanisms for QI at scale.37 The intervention is shown in figure 3; additional information is provided in online supplementary appendix 2. Safe surgery interventions. bmjopen-2019-031800supp002.pdf Short-term outcomes include surgical quality processes related to safety, teamwork and communication, and data quality. Medium-term outcomes are rates of SSIs, postoperative sepsis and maternal sepsis, up to a 30-day postoperative or postnatal hospitalisation period. The outcomes are described in box 1. Twenty-five Tanzanian medical doctors will be recruited as data collectors and trained by the research team for 4 days in the identification and classification of surgical process measures and complications, study data collection tools and Research Electronic Data Capture (REDCap).38 Data collectors will be placed at each of the 20 study sites during the 3-month preintervention and 3-month postintervention period. Data collectors will work full time and will be compensated. They will collect data prospectively on each operation and birth using three standardised data collection tools. Data will be collected first using paper-based tools and transferred electronically into REDCap daily. Data collection tools are provided in online supplementary appendix 3. bmjopen-2019-031800supp003.pdf Data collectors will observe each operation and record the surgical teams’ adherence to essential surgical safety as well as teamwork and communication quality processes using the Surgical Safety Checklist Observation Tool adapted from Huang et al.39 The tool contains 38 yes/no or not applicable items in the following categories: sign-in (13 items), time out (14 items), sign out (6 items) and additional items (5 items). Case information, including demographics, clinical and procedural information, will be entered for each patient from the medical record. Data collectors will follow patients prospectively during their in-patient stay, up to 30 days, for SSI, postoperative sepsis and maternal sepsis. Patient demographics will be collected and outcomes will be identified based on clinical symptoms through daily surveillance during patient rounds, communication with clinical staff and patient chart review. If symptoms are identified, additional screening tools will be administered to collect information on patient demographics, medical history, symptoms and severity of infection and antibiotic treatment. The SSI Tool was adapted from the Protocol for the Surveillance of Surgical Site Infection, Public Health England (2013) and based on the CDC criteria for diagnosing and classifying SSIs.26 40 The Postoperative Sepsis Screening Tool and Maternal Sepsis Tool were adapted for low-resource settings from the Surviving Sepsis Campaign guidelines, which are based on the Second (2001) International Consensus on Sepsis.41–44 The definitions for diagnosing and classifying SSIs, postoperative and maternal sepsis are provided in online supplementary appendix 4. bmjopen-2019-031800supp004.pdf Paper-based data quality will be assessed retrospectively in the intervention regions only; medical records of all patients diagnosed with SSI, sepsis and maternal sepsis will be reviewed before and after the intervention. The questionnaire includes 18 yes/no questions related to documentation of vital signs, SSIs/sepsis symptoms, patient history, daily progress notes, doctors’ orders, partogram utilisation, discharge details and postoperative notes. The tool was informed by WHO Western Pacific Region’s Medical Records Manual: A Guide for Developing Countries45 and measures change in the completeness of medical records for patients diagnosed with SSIs, postoperative sepsis and maternal sepsis. A data quality assurance system will focus on four data quality components: accuracy (correct data values), completeness (no missing data elements), reliability (data collected consistently across study sites) and timeliness (data recorded and reported the same day, on a near real-time basis). Our processes for achieving high-quality data include training data collectors, a standardised operating procedure manual, an electronic data capture with built-in quality controls and weekly data quality checks at each study site by the field research team. Based on the feasibility assessment results, we anticipate an average monthly surgical volume per site of 75 cases in intervention regions and 90 cases in control regions. Over the 3-month baseline period, we assume that this will result in at least 2250 enrolled patients in intervention and control regions, respectively. The sample size was calculated to provide 80% power based on the Cochran-Armitage test for detecting a 5% reduction in the rate of SSIs, a 3% reduction in the sepsis rate and a 1.5% reduction in the maternal sepsis rate postintervention. These sample sizes will provide 80% statistical power to detect a difference in average Surgical Safety Checklist adherence of 5% between the control and intervention arms using Student’s t-test, assuming an SD in adherence of 20% (standardised effect size=0.25). All statistical tests assume a two-tailed alpha level of 0.05. Power analysis was performed using nQuery Advisor software (V.8.0, Statistical Solutions, Cork, Ireland). To evaluate the impact of the intervention and compare changes in outcome rates over time, we will conduct difference-in-differences analysis. The slope fitted in the mixed-effects generalised linear model will be used to evaluate the difference-in-differences of SSI, sepsis, maternal sepsis and Surgical Safety Checklist adherence rates. For the analysis of SSI, postoperative sepsis and maternal sepsis, we will assume a binomial distribution while considering region, facility and patient as random effects. Both C-sections and vaginal deliveries will be included in the maternal sepsis rate calculation although stratified analysis by mode of delivery will be performed. Given that the power calculation is performed based on surgical cases alone, we are unlikely to be powered to detect a change in maternal sepsis rates only among women who undergo C-sections. For the analysis of Surgical Safety Checklist adherence, we will consider a normal distribution with region, facility and patient as random effects. The association between Surgical Safety Checklist adherence and SSI, postoperative sepsis and maternal sepsis will be evaluated by treating Surgical Safety Checklist adherence as a categorical variable. Collapsed categories for Surgical Safety Checklist adherence will be defined based on tertile (low, moderate and high adherence) categories. Covariates in all models will include intervention region, age, sex, procedure type, wound class, ASA score and length of surgical procedure. Adjusted ORs, 95% CIs and p values based on the Wald test will be presented to estimate the independent association of each factor with SSI, sepsis, maternal sepsis and adherence to the Surgical Safety Checklist. In secondary analyses, outcome measures will be stratified by health centre, district health facility and regional health centre to estimate differences in change rates by facility level, controlling for facility-level influences on outcomes. All statistical analyses will be performed using Stata software (V.15.0, StataCorp). A two-tailed significance level of p<0.05 will be used as the criteria for statistical significance. We will use a positive deviance approach with information-rich cases to understand the factors that distinguish facilities with the most surgical quality improvement.46 47 To maximise our learning, we will use a purposeful sample of six facilities as polar examples. We will stratify our sample of 10 facilities into higher and lower performing facilities based on quantitative findings. Higher performers will be defined as the three facilities with the most improvement in surgical quality processes measured by safety as well as teamwork and communication scores (change in percentage points). Lower performers will be defined as the three facilities with the least improvement in surgical quality processes. We will analyse differences between higher and lower performing groups using interview data collected at baseline, midline and conclusion. The interviews will explore contextual factors, implementation processes, facilitators and barriers to implementation as well as their acceptability. Interviewees will include a facility leader and three members of the surgical team at each site. Semistructured interviews will be conducted using a standard interview guide, digitally recorded based on participant consent, transcribed and entered into NVivo software (V.12; QSR International) for coding. We will use the constant comparison method to analyse the qualitative data between higher and lower performing facilities.46 48 First, study team members will review and discuss responses to assign preliminary codes to represent recurring themes. We will establish intercoder reliability by having two independent researchers code and compare results for an initial set of transcripts, with any disagreements resolved through discussion. Codes will be refined to represent the themes reflected in the data. We will reanalyse the transcripts using the final coding scheme and then rate the strength of each facility on each theme on a 4-point scale. We will then compare each theme across the higher and lower performing groups to identify the factors that differentiate higher versus lower performing facilities.