Development of a clinical prediction model for perinatal deaths in low resource settings

Background: Most pregnancy-related deaths in low and middle income countries occur around the time of birth and are avoidable with timely care. This study aimed to develop a prognostic model to identify women at risk of intrapartum-related perinatal deaths in low-resourced settings, by (1) external validation of an existing prediction model, and subsequently (2) development of a novel model. Methods: A prospective cohort study was conducted among pregnant women who presented consecutively for delivery at the maternity unit of Zanzibar’s tertiary hospital, Mnazi Mmoja Hospital, the Republic of Tanzania between October 2017 and May 2018. Candidate predictors of perinatal deaths included maternal and foetal characteristics obtained from routine history and physical examination at the time of admission to the labour ward. The outcomes were intrapartum stillbirths and neonatal death before hospital discharge. An existing stillbirth prediction model with six predictors from Nigeria was applied to the Zanzibar cohort to assess its discrimination and calibration performance. Subsequently, a new prediction model was developed using multivariable logistic regression. Model performance was evaluated through internal validation and corrected for overfitting using bootstrapping methods. Findings: 5747 mother-baby pairs were analysed. The existing model showed poor discrimination performance (c-statistic 0·57). The new model included 15 clinical predictors and showed promising discriminative and calibration performance after internal validation (optimism adjusted c-statistic of 0·78, optimism adjusted calibration slope =0·94). Interpretation: The new model consisted of predictors easily obtained through history-taking and physical examination at the time of admission to the labour ward. It had good performance in predicting risk of perinatal death in women admitted in labour wards. Therefore, it has the potential to assist skilled birth attendance to triage women for appropriate management during labour. Before routine implementation, external validation and usefulness should be determined in future studies. Funding: The study received funding from Laerdal Foundation, Otto Kranendonk Fund and UMC Global Health Fellowship. TD acknowledges financial support from the Netherlands Organisation for Health Research and Development (grant 91617050).

The reporting of the study adheres to the TRIPOD guidelines (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis).11 First, we evaluated and updated an existing prognosis model for stillbirths in LMIC.10,12 Briefly, this model was developed in a retrospective cohort of 6,573 pregnant women and their babies in the Federal Medical Centre Bida, a tertiary hospital in Niger state, Nigeria, from January 2010 to December 2013. There were 443/6,956 (6·4%) stillbirths, defined as birth of a baby who died intrauterine after 20 completed gestational weeks. The original prediction model was developed using multivariable logistic regression and comprised of six predictors.12 After internal validation, the model showed excellent performance in terms of discrimination (C-statistic: 0·80, 95 % CI 0·78-0·83) and calibration in predicting stillbirths.12 The dataset collected for this study comprised a prospective cohort of labouring women at gestational age of ≥ 28 weeks, consecutively recruited as they presented for delivery at the maternity unit of Zanzibar’s tertiary hospital, Mnazi Mmoja Hospital, the Republic of Tanzania between October 2017 and May 2018. The following women were excluded from the study: confirmed intrauterine foetal death before or at the time of admission to the maternity unit and women who did not undergo intrapartum care in the hospital, i.e. women admitted for elective or emergency caesarean section or post-delivery. At the time of admission, trained research nurses collected routinely-measured predictors from antenatal care (ANC) card, history from the patient, in-patient file, and results of physical examination as assessed by routine nurses and they assessed outcomes mainly using in-patient files in the maternity and neonatal care units. For the predictor of maternal perception of foetal movement, a specific questionnaire was newly-developed. Data were recorded on a pilot-tested paper form, and visually inspected for inconsistencies and missing information before entry into a password-protected preformed electronic database (KobotoolBox). For validation of the existing model, the outcome was stillbirth, and defined as intrapartum death ≥28 weeks gestational age, in line with the WHO definition as this is more applicable in LMICs.13 We focused on intrapartum stillbirths (i.e. stillbirths who had a positive foetal heart rate on admission) because we aimed to build a model to reduce intrapartum-related deaths.14 For the new model development, the outcome was perinatal deaths, i.e. stillbirths and neonatal deaths before hospital discharge.15 Only prenatal and pre-delivery maternal and foetal characteristics were considered for prediction of intrapartum-related deaths (Table 1). For evaluation of the existing model, this study included all six predictors considered in the Nigerian model (i.e. place of residence, maternal occupation, maternal parity, bleeding and fetal presentation and maternal comorbidity) and were similarly defined (Table 1). In the original model, maternal comorbidity was an additive score of the following medical conditions: hypertension, pre-eclampsia, diabetes, impaired glucose tolerance, sickle cell disease, renal disease, thyroid disease, syphilis and pelvic inflammatory disease (PID) but in the validation dataset, all maternal comorbid conditions were captured except PID which was not available. Candidate predictors of perinatal death for new model development. Abbreviations: ANC = Antenatal care, HIV = Human Immunodeficiency Virus, As recommended in methodological papers, we based variable selection on “background knowledge …from previous studies in the same field of research, from expert knowledge or from common sense.”22 As such, for new model development the following candidate predictors were considered: all recorded six predictors from the Nigerian model as mentioned above, and five additional predictors that were identified from the literature and an international expert-based Delphi consensus (foetal movement by maternal perception, gestational age, fundal height, foetal heart rate on admission, meconium stained liquor),7,20 and five from clinical reasoning (previous caesarean, multiple gestation, number of ANC visits, prolonged rupture of membranes and antepartum haemorrage) (Table 1). The latter category were factors identified in the clinical setting through direct observations of the quality of labour care4 and the development process of the PartoMa labour management guidelines (the PartoMa Project).23 Determination of gestational age is notably challenging in low resource settings as most pregnant women do not have (an early) antenatal ultrasound and may also not recall their last menstrual period accurately.24 Thus, estimation of gestational age reflected the clinical reality whereby the most accurate available method of determination was used in the following order: 1) early ultrasound (up to 12 weeks), 2) the last menstrual period, 3) second trimester (up to 22 weeks), 4) 3rd trimester ultrasound.25 When none of these methods were available, gestational age was considered unknown and multiple imputation was used (see section on missing data). Precise data e.g. for gestational age, fundal height and foetal heart rate is difficult to obtain in these settings and thus categorisation of continuous data was used as a crude scale for these measurements, (Table 1). It has been recommended thatexternal validation studies should include at least 100, but preferably 200 or more outcome events.26 We aimed to include at least 200 events, in order to allow a sufficient sample size to develop a new model using more predictors with at least 10 events per candidate predictor. At MMH the stillbirth incidence was around 3.8%.27 Thus, the required sample size was 5,263 participants. With 12,000 births annually, this roughly corresponds to a seven-month period of data collection. Multiple imputation was applied to account for missing data using the MICE package in R. The imputation accounted for all candidate predictors and outcomes in the dataset. This resulted in 20 multiply imputed datasets.28,29 All analyses were repeated across the 20 datasets with pooling of estimates and their uncertainty measures using Rubin’s rules.30 Categorical variables were described using frequencies and percentages. As in the Nigerian study, all continuous data were summarized using medians and interquartile ranges (IQR) which allowed comparison of baseline characteristics between the Nigerian and Zanzibar datasets. Descriptive statistics were generated for the original data (before imputation) and the proportion of missing values was calculated for all candidate predictors. For the continuous variable of number of antenatal care visits, non-linear predictor-outcome association was explored using restricted cubic splines.31 For all (existing and newly developed) models, we assessed calibration and discrimination performance. Calibration was visually assessed using a calibration plot, comparing the agreement between observed frequencies of stillbirth (original and updated models) and perinatal deaths (new model) in the new dataset and the predicted risks. The ability of the models to discriminate between women with and without stillbirth (original and updated models) and perinatal death (new model) was assessed using the concordance (c)-statistic, which is equivalent to the area under a receiver operating characteristic (ROC) curve for prognostic models with binary outcomes.9,32, 33, 34, 35 The original model was adjusted to the new cohort using recalibration methods (adjustment of the intercept and adjustment of both the intercept and slope) previously described.34,36 For the new model development, multivariable logistic regression was used with all candidate predictors.35 This strategy is may be preferred over stepwise selection methods, which often lead to model instability and overfitting.9 All predictors (including all comorbidities) were entered individually in the initial model. Subsequently, hypertensive disorder and sickle cell were presented as individual predictors in the final model because they were the maternal co-morbidity with highest estimated risk. The remaining maternal conditions were combined into a comorbidity score (i.e. adding up of comorbidities). No interactions were identified clinically and so an additive model was used. This also reduces the risk of overfitting. Model optimism was assessed via bootstrap resampling.31 Briefly, the aforementioned prediction model was refitted (i.e. re-estimation of the coefficients) in 200 bootstrap samples, and the performance of these models was then evaluated in the original sample. This yielded a shrinkage factor which was used to adjust both the regression coefficients and c-statistic of the original model for optimism.37 All analyses were performed in R version 3·5·3 (The R Foundation for Statistical Computing, 2019).38 Methods previously described were used to derive a point score system for the newly developed prognostic model.39 Risk estimates were organised into clinically meaningful categories. An example is given which also illustrates the correspondence between the risks estimated by the multivariable model directly and those approximated by the points system. The study was approved by the Zanzibar Medical Research Ethical Committee (ZAMREC/0004/AGUST/17). Upon arrival to the admission room, a research nurse assessed the eligibility criteria of women. Written information in Kiswahili about the study was read out to the women by a research nurse. Women were then asked for their voluntary consent to participate in the study. Hardcopies of the data were stored in a locked office and electronic data were password protected. The funders of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study, and all the authors accept final responsibility for the decision to submit for publication.

• housseine, natasha
• rijken, marcus j.
• weller, katinka
• nassor, nassra haroub
• gbenga, kayode a.
• dodd, caitlin
• debray, thomas p.a.
• meguid, tarek
• franx, arie
• grobbee, diederick e.
• browne, joyce l.