EHBB: A randomised controlled trial of virtual reality or video for neonatal resuscitation refresher training in healthcare workers in resource-scarce settings

Objective To assess the impact of mobile virtual reality (VR) simulations using electronic Helping Babies Breathe (eHBB) or video for the maintenance of neonatal resuscitation skills in healthcare workers in resource-scarce settings. Design Randomised controlled trial with 6-month follow-up (2018-2020). Setting Secondary and tertiary healthcare facilities. Participants 274 nurses and midwives assigned to labour and delivery, operating room and newborn care units were recruited from 20 healthcare facilities in Nigeria and Kenya and randomised to one of three groups: VR (eHBB+digital guide), video (video+digital guide) or control (digital guide only) groups before an in-person HBB course. Intervention(s) eHBB VR simulation or neonatal resuscitation video. Main outcome(s) Healthcare worker neonatal resuscitation skills using standardised checklists in a simulated setting at 1 month, 3 months and 6 months. Results Neonatal resuscitation skills pass rates were similar among the groups at 6-month follow-up for bag-and-mask ventilation (BMV) skills check (VR 28%, video 25%, control 22%, p=0.71), objective structured clinical examination (OSCE) A (VR 76%, video 76%, control 72%, p=0.78) and OSCE B (VR 62%, video 60%, control 49%, p=0.18). Relative to the immediate postcourse assessments, there was greater retention of BMV skills at 6 months in the VR group (-15% VR, p=0.10; -21% video, p<0.01, -27% control, p=0.001). OSCE B pass rates in the VR group were numerically higher at 3 months (+4%, p=0.64) and 6 months (+3%, p=0.74) and lower in the video (-21% at 3 months, p<0.001; -14% at 6 months, p=0.066) and control groups (-7% at 3 months, p=0.43; -14% at 6 months, p=0.10). On follow-up survey, 95% (n=65) of respondents in the VR group and 98% (n=82) in the video group would use their assigned intervention again. Conclusion eHBB VR training was highly acceptable to healthcare workers in low-income to middle-income countries and may provide additional support for neonatal resuscitation skills retention compared with other digital interventions. The study was conducted in Lagos, Nigeria and Busia, Western Kenya. Twelve healthcare facilities (nine secondary and three tertiary) were located in Nigeria while eight facilities were located in Kenya. The healthcare facilities were located in urban and semi-urban areas and all have maternal and newborn services with newborn bed capacity ranging from 2 to 80 beds and delivery and neonatal unit staffing capacity from 7 to 124 nurses (see online supplemental file 1). bmjopen-2020-048506supp001.pdf Study participants consisted of nurses and nurse-midwives assigned to labour and delivery, operating room and newborn care units. Site coordinators or research assistants requested contact numbers, units and wards of potential participants from head nurses at identified facilities. Research assistants contacted individuals to determine eligibility and obtained consent (see online supplemental file 2). bmjopen-2020-048506supp002.pdf Nurses and midwives who participate in deliveries and who provide neonatal resuscitation to inborn or outborn infants and provide study consent. Those who had attended a neonatal resuscitation training course in the 1 year preceding the study; individuals who did not provide neonatal resuscitation as part of their duties or would be unavailable or unwilling to participate in follow-up study activities throughout the 6-month postinitial training period. Study IDs generated for each country site were randomly assigned via a computer-generated algorithm to the VR, video and control groups by a US-based study coordinator. Participants were enrolled and assigned a study ID before the HBB course by local study coordinators. Each participant received an Android study phone, preloaded with permission-based access linked to their study ID, via the mobile Helping Babies Survive powered by District Health Information System (DHIS2) app (mHBS/DHIS2), to the participant’s assigned digital intervention. The data analysis team was blinded to the study assignments. The HBB provider course (second edition)27 was taught by study HBB master trainers as 1 day, 8-hour long sessions from December 2018 to August 2019. A 30 min orientation was provided on use of the mHBS/DHIS2 app, including how to access the assigned digital intervention. All participants had access to a digitised HBB provider manual through the mHBS/DHIS2 app. The VR group in addition accessed the eHBB VR simulations which consisted of three interactive three-dimensional simulation scenarios representing care of a newborn requiring routine care, some resuscitation and prolonged resuscitation with positive pressure ventilation. The features of eHBB VR have been previously described and the application is available for free download.26 28 The neonatal resuscitation video used by the video group featured preparation for delivery and the resuscitation of a newborn requiring positive pressure ventilation.29 None of the interventions required internet for use. A total of 274 HCWs participated in the in-person HBB training. Standardised knowledge and skills assessments were conducted by trained research assistants. The HBB knowledge check (15 of 18 multiple-choice questions, ≥80% required to pass) and bag-and-mask ventilation skill check (BMV; 14 of 14 items required to pass) were conducted precourse and postcourse along with the objective structured clinical examination (OSCE) A checklist on preparation for delivery and initial steps of resuscitation (9 out of 12 items and 3 required items to pass). In addition, the postcourse assessment included the OSCE B checklist on prolonged newborn resuscitation (17 out of 23 items and 6 required items to pass). HBB checklists are available for free download from the American Academy of Pediatrics.30 A demographic survey was completed (figure 1). Study diagram. BMV, bag-and-mask ventilation; HBB, helping babies breathe; LDHF, low-dose high frequency; VR, virtual reality. Following the course, participants were encouraged to use their assigned digital intervention weekly and to engage in standard bag-and-mask skills practice with a manikin at the HBB practice corner set up at their facility. Post-course assessments were repeated at 1, 3, and 6 months after the class. A follow-up survey was completed. Data were collected in person by study staff who had completed a HBB second edition master trainer course by experienced HBB master trainers. Staff used the mHBS/DHIS2 tracker app for offline data collection.26 The mHBS tracker app contained digitised HBB knowledge check, BMV skill check and OSCE A and OSCE B checklist and was used by the participants to report their HBB corner practice. The mHBS trainer app separately tracked educational interventions access and use. To standardise data collection and feedback to study participants, an enhanced neonatal simulator, called NeoNatalie Live (Laerdal Medical) was used for BMV. Compared with the low-fidelity NeoNatalie simulators used for HBB training (including the HBB practice corners in this study), NeoNatalie Live manikin can be programmed to simulate key physiological parameters, such as variable rates of lung stiffness and heart rate and provides auditory and visual cues, in the form of ‘crying’ and increased heart rate when the end-user provides BMV.31 In addition, brief automated feedback for the end-user is provided using a Bluetooth-connected tablet device at the end of the assessment as ‘well done’ or ‘needs improvement’ based on bag and mask performance. The use of the NeoNatalie Live manikin software enabled the correlation of observer collected metrics with manikin collected data. The automated feedback provided by Neonatalie Live was the only feedback provided following each assessment.31 Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research. We hypothesised that there will be at least a 20% difference in the proportion of subjects who pass OSCE B at the 6-month evaluation between the VR group or video group and the control group. A sample size of 83 subjects per group would provide 80% power to detect a difference in pass rates between groups if the true pass rates were 85% and 65%, respectively, based on a two-sided α=0.05. The required total sample size for the three groups (VR, video and control) was 249. We recruited 274 participants total to allow for 10% dropout over the 6-month follow-up period. An intention-to-treat analysis was performed, where participants were grouped according to their randomly allocated experimental group (VR, video or control) regardless of their actual exposure. Fisher’s exact test was used to test for any differences in pass rates among the three groups for each of the study evaluations: BMV skills assessment, and standardised simulations of routine care and initial resuscitation (OSCE A) and prolonged resuscitation (OSCE B). Post hoc pairwise comparisons and comparisons between demographic groups were also performed using Fisher’s exact test. Within-group comparisons of evaluation results between timepoints were performed using the sign test. Participant exposure to the interventions (time in the mHBS trainer app) and self-reported clinical activity during the follow-up period were compared between experimental groups using the Kruskal-Wallis test and Wilcoxon rank-sum test. All statistical calculations were conducted with the statistical computing language R (V.4.0.0; R Foundation for Statistical Computing, Vienna, Austria). Throughout, two-sided tests were used, with statistical significance defined as p<0.05.