Shedding light on quality of care: A study protocol for a randomized trial evaluating the impact of the Solar Suitcase in rural health facilities on maternal and newborn care quality in Uganda

Background: Continued progress in reducing maternal and newborn morbidity and mortality in low-income countries requires a renewed focus on quality of delivery care. Reliable electricity and lighting is a cornerstone of a well-equipped health system, but most primary maternity care facilities in sub-Saharan Africa are either not connected to the electrical grid or suffer frequent blackouts. Lack of reliable electricity and light in maternity facilities may contribute to poor quality of both routine and emergency obstetric and newborn care, by hindering infection control, increasing delays in providing care, and reducing health worker morale. The “Solar Suitcase” is a solar electric system designed specifically for maternity care facilities in low-resource environments. The purpose of this trial is to evaluate the impact of the Solar Suitcase on reliability of light, quality of obstetric and newborn care, and health worker satisfaction. Methods: We are conducting a study with 30 maternity care facilities in rural Uganda that lack access to a reliable, bright light source. The study is a stepped wedge cluster randomized controlled trial. Study facilities are identified according to predefined eligibility criteria, and randomized by blocking on baseline covariates. The intervention is a “Solar Suitcase”, a complete solar electric system that provides essential lighting and power for charging phones and small medical devices. The primary outcomes are the reliability and quality of light during intrapartum care, the process quality of obstetric and newborn care, and health worker satisfaction. Outcomes will be assessed via direct clinical observation by trained enumerators (estimated n = 1980 birth observations), as well as interviews with health workers and facility managers. Lighting and blackouts will be captured through direct observation and via light sensors installed in facilities. Discussion: A key feature of a high quality health system is appropriate infrastructure, including reliable, bright lighting and electricity. Rigorous evidence on the role of a reliable light source in maternal and newborn care is needed to accelerate the “electrification” of maternity facilities across sub-Saharan Africa. This study will be the first to rigorously assess the extent to which reliable light is an important driver of the quality of care experienced by women and newborns. Trial registration: ClinicalTrials.gov: NCT03589625 (July 18, 2018); socialscienceregistry.org: AEARCTR-0003078 (dated June 16, 2018). Uganda has a population of about 34.6 million people, with 48% below the age of 15, and an estimated 79% living in rural areas [23]. About 29% of households in Uganda have access to electricity, though only 18% of rural households do [24]. The total fertility rate in 2017, at 5.5 children per woman, was higher than the average in low income countries (4.6 children) [25]. Nearly all women (97%) receive antenatal care from a skilled provider and 73% of births are delivered in a health facility [24]. The maternal mortality ratio for Uganda is 336 deaths per 100,000 live births for the period of 2009–2016. The perinatal mortality rate—defined as stillbirths after seven completed months gestation and deaths within the seven days after birth—is an indicator directly linked to the quality of antenatal and intrapartum care. In Uganda, the perinatal mortality rate is 38 deaths per 1000 pregnancies for the period of 2011–2016 [24], and this rate has remained stagnant since 2006 [26]. A study on delays in newborn care in Uganda found that delays in receiving quality care at the health facility was the second major contributor to newborn deaths, and that facilities were ill-equipped to care for newborns [27]. Uganda’s health system is divided into public and private sectors [28]. The public sector includes national and regional hospitals and a District Health system composed of a tiered system of health centers (HCs). The health center tiers include: (1) HC I’s, community-run health volunteers, (2) HC IIs, first level of interface between the formal health sector and the community providing outpatient services and some maternity services where necessary, (3) HC III’s, offering outpatient and inpatient services, including uncomplicated maternity care services, and (4) HC IV’s, offering provision of emergency medical, surgical, and obstetrical care (such as caesarean section) [28, 29]. Recent, high quality data on health facility access to reliable light and electricity in Uganda is sparse. Research has generally found that while most HC IV’s have access to the central supply electricity grid, most HC IIs and IIIs do not, and often rely on kerosene lamps for much of their lighting needs [30–32]. While kerosene lamps can provide ambient lighting, it is not sufficient for providing the focused, bright light that many elements of maternity care requires [10]. In addition, even among facilities connected to the grid, 74% had frequent interruptions lasting more than two hours a day [33]. The intervention is a “Solar Suitcase” manufactured by the non-governmental organization We Care Solar. The We Care Solar Suitcase® is a complete solar electric system that provides essential medical lighting and power for charging phones and small medical devices (Fig. 1). The system contains a photovoltaic solar panel installed on the roof of a health facility; a 12 V lithium ferrous phosphate battery; high-efficiency, moveable light-emitting diode (LED) lights for maternity rooms; and two rechargeable LED headlamps. In addition, it contains a fetal Doppler with rechargeable batteries, two 12 V DC accessory sockets, two USB ports for charging cell phones, and a AA/AAA battery charger. Because the Solar Suitcase includes several components, the evaluation will test the overall impact of all components on outcomes and it will not be possible to determine the impact of any individual component. Image of The We Care Solar Suitcase® and component parts. Notes: Copyright by We Care Solar. Written permission was given by We Care Solar to reproduce this image Installations will be done by a local solar contracting firm based in Uganda. Depending on the facility, each installation takes 3–6 h. One Solar Suitcase will be installed in each facility, with 2–4 overhead LED lights for each delivery room, depending on its size. To ensure consistent and appropriate use of the installed Suitcase, installers will teach health workers how to use and maintain the Solar Suitcase and all accessories on the day of installation and in subsequent check-ins, which will be done in-person or over the phone as needed. The contractor will also follow up with maintenance requests. Health facilities will not incur any costs during the study for installation, operation, or maintenance of the Solar Suitcase. The study is designed as a stepped wedge cluster-randomized controlled trial. The intervention is delivered at the cluster (i.e. facility) level and the primary outcome is measured at the birth event level (i.e. mother and infant). Facilities are randomized into one of two groups of 15 facilities. The stepped wedge design ensures that both groups of facilities eventually receive the intervention, but the implementation is staggered in a random fashion allowing a rigorous evaluation to take place. All facilities are observed before and after receiving the intervention. Regardless of study findings, at the end of the study all facilities will keep the Solar Suitcase and will be provided with contact information in case of needed repairs. The study timeline (Fig. 2) includes a baseline (pre-intervention) observation period of about 4–6 weeks, followed by installation of the Solar Suitcase in the first group of facilities. After installation, there is an exposure period of about 6 weeks so that facility staff can become comfortable with the Solar Suitcase. Following this period, there is a midline observation period of 4–6 weeks, which serves as a post-intervention observation for group 1 and a second pre-intervention observation for group 2. Finally, installation will occur in the second group of facilities, followed by another 6 week exposure period and then an endline observation of 4–6 weeks, serving as a second post-intervention observation for both groups. Flowchart of enrolment, randomization, and assessments during study period Study facilities are identified according to the following predefined eligibility criteria: i) facility is level II, III, or IV; ii) facility-reported overhead light is unreliable (zero or one functioning sources of overhead light in the maternity ward, which is interrupted at least a few times a week); iii) facility does not automatically refer out when there is no light; iv) facility is open 24 h a day; and v) medical officer in-charge (MOIC) at the facility is willing to participate. Enumerators will collect data at each facility for a trial period to verify study eligibility (i.e. to verify light reliability and source). The facilities will be chosen from districts in Central, Eastern, and Western regions of Uganda. Approval and support for working in these districts was granted by the Ministry of Health (MOH) of Uganda as well as the District Health Officers and Resident District Commissioners. For patients delivering at participating facilities, inclusion criteria includes consenting pregnant women aged 16 and older who are admitted for labor and delivery. Exclusion criteria includes pregnant women presenting for conditions other than normal labor and delivery, women who are immediately transferred to another hospital, women who deliver outside of the maternity ward, or women for whom the health worker objects to the observation. Enumerators will have a minimum of a diploma or certificate in Comprehensive Nursing or Midwifery from an accredited Uganda National Council of Higher Education. Enumerators will be trained for approximately 2 weeks on research protocol, data collection methods, and human subjects research. Facilities will be randomized to receive the intervention at the first step or the second step. Data collection will occur in three staggered waves. Randomization will be blocked by wave and by a measure of average adequate light during baseline (above or below the median for all facilities), and balanced on a 20-item quality of care index measured at baseline and average facility volume over the past three months. The algorithm will re-randomize until the balance statistic (Wilks’ lambda on a manova) exceeds 0.2. The allocation sequence will be generated by the study investigators using Stata version 15. Due to the nature of the intervention, neither participants nor researchers can be blinded to allocation. The following six types of data will be collected in this study. Quality of care will be assessed via direct clinical observation of deliveries by enumerators. Enumerators will complete a checklist for each delivery capturing the elements of care and timeliness of care provided to each eligible, consented patient. The checklist tool is adapted from the Maternal and Child Health Integrated Program Quality of Care Surveys and Short Observational Index [34]. Additional checklists are also used for cases of multiple birth, resuscitation of newborn with asphyxia, and observation of post-partum hemorrhage. Interviews will be conducted with all consenting medical staff who may be involved in labor and delivery. Information will be collected on basic sociodemographic characteristics, background and training, perception of electricity reliability in the facility, hours and duties, satisfaction and personal drive, and knowledge of maternal and newborn care. The facility survey includes a general facility assessment and the recording of facility registry data and will be conducted with the MOIC and/or the Head Midwife (maternal ward lead). This assessment will contain information on the facility infrastructure, staffing, services, fees, drug/supplies availability, and reported performance of routine and emergency signal functions. Recording of facility registry data will consist of collecting data from the MOH Health Management Information System (HMIS), including number of vaginal deliveries, number of cesarean deliveries, number of nighttime deliveries, number of referrals, and number of ANC visits. Information on availability and quality of light will be recorded in two ways. First, enumerators will observe and record sources of light, brightness of light, and timing in changes to sources of light on a data collection form. Second, light sensors will be installed in delivery rooms of health facilities, which will collect light voltage data for the duration of the study. For facilities connected to the grid, monitors will also be installed to detect whether the grid power is on or off. Data collection on availability and quality of light will be conducted during both daytime and nighttime hours. For one year after completion of birth observations, HMIS facility data on number of mothers presenting for delivery, number of nighttime births, number of live births, number of newborn deaths, number of newborn resuscitations, and number of ANC visits will be collected at quarterly visits to facilities. Enumerators will record qualitative observations of deliveries, including commentary on how light and electricity may have influenced the care provided, as well as contextual information about facility supplies, staffing, operations, and use of the Solar Suitcase components that may provide insight into interpretation of quantitative data. The evaluation of this study will include primary and secondary outcomes, as well as process measures and qualitative assessments of impact. Our study has more pre-specified outcomes than is typical for a randomized controlled trial because the important aspects of light (the brightness, the light source, or both) and the precise ways in which reliable electricity may affect quality of care are difficult to predict ex-ante. We thus chose several measures of light and several indices that capture various elements of quality of care and delays in providing care throughout labor, delivery and the immediate post-partum period. The primary outcomes include measures of adequate light, quality of care, and health worker satisfaction (Table 1). Measures of brightness are assessed by enumerators with standardized training as to what level of light defines each term. These outcomes will be evaluated during both daytime and nighttime hours, as facilities may experience dim conditions during the day and the Solar Suitcase provides a bright moveable light that can be used for difficult, detail-oriented work such as suturing. Primary outcomes 1. Average brightness of room during labor and delivery as measured by observer recorded questionnaire (range: 1 “pitch black” – 4 “very bright”) 2. Satisfactory light sourcea used for entire delivery as measured by observer recorded questionnaire (%). 3. Adequate lightb for duration of delivery as measured by observer recorded questionnaire (%). 1. 20-item quality of maternal care index of essential actions to be performed by provider during labor and delivery as measured by observer recorded questionnaire (%) 2. 37-item quality of maternal care index of essential actions to be performed by provider during labor and delivery as measured by observer recorded questionnaire (%) 3. 6-item delays in care index as measured by observer recorded questionnaire 1. Satisfaction with light & electricityc as measured by health worker survey (range 1 “strongly disagree” – 5 “strongly agree”) 2. Overall job satisfaction indexd as measured by health worker survey (range 1 “strongly disagree” – 5 “strongly agree”) Notes. a“Satisfactory” defined as observation that occurs during the day or observation that occurs during night and uses grid electricity, solar power, or functional generator (rather than kerosene lamp, candle, or torch) b“Adequate” light defined as light from a satisfactory source and is recorded by enumerator as “very bright” or “somewhat bright” (rather than “dim” or “pitch black”) cSatisfaction defined as agree/strongly agree with both statements: “I am satisfied with the availability and brightness of light in this facility” and “I am satisfied with the availability of electricity in this facility” dOverall satisfaction defined as average score of following items: “i) These days, I feel motivated to work as hard as I can. ii) Overall, I am satisfied with my job. iii) Overall, the morale level at my department is good iv) I plan on staying at this position for the next year” The 20-item provision of quality of care index is a validated measure developed for and evaluated in low- and middle-income settings [35]. It is composed of 20 indicators representing key dimensions of the quality of the process of intrapartum and immediate postpartum care in facility deliveries (Table 2). The index captures elements of quality including initial patient assessment, labor monitoring and delivery, and the immediate post-partum period (one hour after delivery). Research applying the index in three countries in sub-Saharan Africa found that it effectively discriminates between poorly and well-performed deliveries [35]. The 37-item index includes all 20 items plus an additional 17 items adapted from the Maternal and Child Health Integrated Program tool [34] (Additional file 1: Table A1). Both indices are constructed as the proportion of items performed per delivery and thus range from 0 to 1. Items composing the 20-item quality of maternal care index, Tripathi (2015) Notes: AMTSL = active management of the third stage of labor. Each indicator is scored as binary for health worker performed (1) or did not perform (0). Index score for each delivery is proportion of items performed. Reference for index is [35] The 6-item delays in care index measures the average of 6 measures of delays in time to provision of care (measured in minutes), and includes time between: facility arrival and first contact with health care worker, facility arrival and first examination, delivery and provision of uterotonic, delivery and assessment of perineal and vaginal lacerations, delivery and drying baby with towel, and delivery and initiation of breastfeeding. Finally, we include two measures of health worker satisfaction: (1) satisfaction with the availability and brightness of light and electricity (2-item index), and (2) overall job satisfaction including motivation, job satisfaction, morale, and plans to leave (4-item index). Items composing these indices and measurements are shown in Additional file 1: Table A2. Indices were adapted and extended from the World Bank Impact Evaluation Toolkit for Results Based Financing in Health [36]. Secondary outcomes are shown in Table A3 of Additional file 1 and include the individual components of the quality indices as well as other outcomes that may be affected by the provision of a reliable light source such as referrals out of the facility, patient volumes, APGAR scores, and suturing. Our sample will include 30 facilities. We expect to observe about 1980 patients (22 per facility at each of the three data collection points), and about 90 health workers (an average of 3 per facility). We used data collected from pilot facilities (that are not included in the study sample) to construct sample size and power estimates prior to study launch. We then updated these calculations with data collected at baseline with study facilities since this is likely to be a more accurate prediction of statistical power for study facilities. (Calculated effect sizes do not differ meaningfully if pilot data or baseline data is used.) We use this data to calculate means of outcome variables and the intraclass correlation (ICC). These power calculations use the full stepped wedge design (using the steppedwedge function in Stata v15), for 22 birth events per facility, 3 health workers per facility, 2 steps, 80% power and 0.05 alpha. We discuss here the minimum detectable effect sizes for one outcome within each domain of light, quality, and health worker satisfaction. We estimate that 61% of deliveries at baseline will be conducted with an adequate light source, with an intra-cluster correlation (ICC) of 0.20. Our minimum detectable effect size for this outcome is an increase of 13 percentage points. We estimate that health workers will be performing 44% of the essential quality of care items in the 20-item quality of care index (i.e. a score of .44) with an ICC of 0.4. Our minimum detectable effect size for this outcome is an increase of 11 percentage points (i.e. an increase to .54), representing an increase in the performance of about 2 items. We estimate that health worker satisfaction with light and electricity is a 2 (out of a 1–5 range) with an ICC of 0.3. Our minimum detectable effect size for this outcome is an increase of 0.62. Oversight of research in Uganda involving human subjects requires two levels of approval: (1) approval is required at the organizational level by Mildmay Uganda Research Ethics Committee and (2) approval is required at the national level by the Uganda National Council for Science and Technology in collaboration with the Uganda National Health Research Organization. Both levels of approval were obtained, alongside full review by the Harvard School of Public Health Institutional Review Board (IRB) and exemption from review by University College Dublin. Consent to participate is voluntary and participants, including facilities, health workers, and patients, can withdraw at any time. Written informed consent for all activities pertaining to the study will be obtained from the MOIC and all health workers who may be involved in labor and delivery prior to conducting observations and interviews. Written informed consent from patients will be obtained prior to or just after admission. Consent forms for patients will be available in seven local languages, with enumerators posted to facilities in areas where they speak the local language. If either the health worker or patient does not consent or objects to the observation after providing consent, the delivery observation will not occur. In such a case where the patient opts out, the care provided to the patient (and the baby) during the admission and after discharge will not be any different from what she, and the baby, would have received as a study participant. Data collection will take place using both paper surveys and electronically on tablets. Double data entry will be conducted and discrepancies will be reconciled. Logic checks for electronic entry will also be put into place to ensure linearity of timestamps and prevent contradictory data. Every birth event will be allotted a unique code. Observation data collection forms will be anonymous and no identifiers will be listed on these forms. No personal identifying information of women delivering will be collected on the paper or electronic questionnaires. The names of health care workers will be collected in order to link them to an identification number. The names and corresponding identification numbers will be kept securely in a locked cabinet. Results presented in reports and publications will always be aggregated in such a way that identification of facilities, health workers and patients is not possible. Data from tablets will be uploaded to an encrypted data storage cloud system managed by the research management team in Uganda and any data stored locally will be stored in encrypted folders. The PIs and research management team will have bi-weekly calls to discuss study progress. These meetings will be held to monitor data collection and discuss any adverse events, protocol violations, or protocol modifications. Any issues will be reported to the PIs within 24 h. In-person oversight of data collection from the management team will be ongoing throughout the study and data quality checks will take place daily. Additionally, there will be biweekly in-person group feedback sessions for enumeration staff to share challenges of conducting observations and discuss ways to overcome those challenges. There is no Data and Safety Monitoring Board, interim analyses, or stopping guidelines because there is no greater than minimal risk for participants, as deemed by the IRB. Analysis will be by intention-to-treat, with adjustments made for multiple hypothesis testing. Let Yijt be the outcome of interest for birth event i in facility j at time t, with t measured as the time-step. Djt is a binary variable for whether the Solar Suitcase had been installed at facility j at time t. The specification for the linear predictor representation is shown below: where Aj are facility fixed effects, Bt are calendar month fixed effects, Xjt is a matrix of facility-level controls, and ϵijt is the error term. The estimated impact of installation of the Solar Suitcase is β^. We will also analyze the impact of the Solar Suitcase at each step separately, i.e. Djt will be defined as a 3-level indicator (baseline, 1st follow-up post installation, 2nd follow-up post installation). In primary analyses, we will adjust for length of time spent observing at a facility in Xjt. In secondary analyses, we will adjust for additional baseline covariates to account for any baseline imbalances. Standard errors will be adjusted for clustering at the facility level and will be compared to those obtained from a wild cluster bootstrap. In a sensitivity analysis, we will conduct a multilevel regression with facility random effects instead of fixed effects, adjusting for randomization blocking variables and a variable accounting for length of time spent observing at a facility. For linear outcomes, analysis will consist of linear regression models. For count outcomes, analysis will consist of Poisson or negative binomial regression models. For binary outcomes, analysis will consist of logistic regression models. For the outcome of health worker satisfaction, the specification will be at the health worker level, rather than the birth-event level. We will conduct analysis using all of the pooled data and using data of deliveries occurring in the night-time only. We will also conduct a heterogeneity analysis, exploring whether bigger changes in light availability are associated with bigger improvements in quality of care. The study investigators will have access to the final trial dataset and interim datasets before the intervention is implemented. The study results will be disseminated to internal and external collaborators, funders, scientific media, and policymakers at the MoH. After manuscript publication, fully de-identified data may be shared with researchers upon request.