A risk scoring tool for predicting Kenyan women at high risk of contraceptive discontinuation

Objective: We developed and validated a pragmatic risk assessment tool for identifying contraceptive discontinuation among Kenyan women who do not desire pregnancy. Study design: Within a prospective cohort of contraceptive users, participants were randomly allocated to derivation (n = 558) and validation (n = 186) cohorts. Risk scores were developed by selecting the Cox proportional hazards model with the minimum Akaike information criterion. Predictive performance was evaluated using time-dependent receiver operating characteristic curves and area under the curve (AUC). Results: The overall contraceptive discontinuation rate was 36.9 per 100 woman-years (95% confidence interval [CI] 30.3–44.9). The predictors of discontinuation selected for the risk score included use of a short-term method or copper intrauterine device (vs. injectable or implant), method continuation or switch (vs. initiation), < 9 years of completed education, not having a child aged < 6 months, and having no spouse or a spouse supportive of family planning (vs. having a spouse who has unsupportive or uncertain attitudes towards family planning). AUC at 24 weeks was 0.76 (95% CI 0.64–0.87) with 70.0% sensitivity and 78.6% specificity at the optimal cut point in the derivation cohort. Discontinuation was 3.8-fold higher among high- vs. low-risk women (95% CI 2.33–6.30). AUC was 0.68 (95% CI 0.47–0.90) in the validation cohort. A simplified score comprising routinely collected variables demonstrated similar performance (derivation-AUC: 0.73 [95% CI 0.60–0.85]; validation-AUC: 0.73 [95% CI 0.51–0.94]). Positive predictive value in the derivation cohort was 31.4% for the full and 28.1% for the simplified score. Conclusions: The risk scores demonstrated moderate predictive ability but identified large proportions of women as high risk. Future research is needed to improve sensitivity and specificity of a clinical tool to identify women at high risk for experiencing method-related challenges. Implications: Contraceptive discontinuation is a major driver of unmet contraceptive need globally. Few tools exist for identifying women who may benefit most from additional support in order to meet their contraceptive needs and preferences. This study developed and assessed the validity of a provider-focused risk prediction tool for contraceptive discontinuation among Kenyan women using modern contraception. High rates of early discontinuation observed in this study emphasize the necessity of investing in efforts to develop new contraceptive technologies and stronger delivery systems to better align with women's needs and preferences for voluntary family planning. We used data from the Mobile Data Collection for Contraceptive Use, Behaviors and Experience (mCUBE) study, a prospective cohort study of women's contraceptive experiences. Study participants were enrolled February–May 2018 while attending FP or maternal and child health clinics within 10 public health facilities in 5 counties of Western Kenya (Bungoma, Homa Bay, Kakamega, Kisumu and Nyamira). Women were eligible if they were ≥ 18 years old (or an emancipated minor ≥ 14 years old with a previous pregnancy); had daily access to a mobile phone with a Safaricom SIM card; were able to read and respond to SMS in English, Swahili or one of two local languages (Luo or Kisii) either alone or with the help of a trusted person; and were currently initiating, continuing or switching to a modern, reversible contraceptive method. Modern methods included injectables, implants, intrauterine devices or systems (IUDs), oral contraceptive pills (OCPs), emergency contraceptive pills, condoms, diaphragms, lactational amenorrhea, Standard Days Method and TwoDay Method [25]. Data were collected through structured SMS surveys operated by the Kenya-based company mSurvey (Nairobi, Kenya). Study staff administered an enrollment SMS survey, capturing information on sociodemographic characteristics, reproductive and contraceptive history, contraceptive use, fertility goals, and perceived quality and satisfaction with FP services. Participants received weekly follow-up SMS surveys for 24 weeks that captured information on contraceptive use, method type, reasons for switch or discontinuation (if applicable), side effects and healthcare utilization. Details on contraceptive method and discontinuation ascertainment are provided in the Online Appendix. All study procedures were approved by the Maseno University Ethical Review Committee. Participants signed a written consent form prior to any study procedures. The University of Washington's (UW's) Human Subjects Division (HSD) determined that ethical approval from UW was not required as the UW research team was not considered engaged in human subjects research; however, this specific analysis was approved by UW HSD. Contraceptive discontinuation was defined as a period of ≥ 2 consecutive weeks during which women self-reported that they were not currently using any modern contraceptive method. We defined discontinuation based on a ≥ 2-week period in order to capture short-term discontinuation episodes that have not been widely explored in the published literature. Method switches were considered as continuation unless a ≥ 2-week period elapsed with no modern method use. Our analytic sample comprised participants with complete baseline data for all risk factors considered and at least one complete observation during follow-up. Women who desired a pregnancy in the next year were excluded, as they were expected to be more likely to discontinue to become pregnant rather than for method-related reasons [26]. Potential predictors considered for the risk score included sociodemographic and clinical characteristics routinely collected in Kenyan FP clinics. Additional potential predictors not routinely collected (education, whether her spouse supported her contraceptive use, feelings about a hypothetical near-term pregnancy, side effects history and perceived quality of FP care) were also evaluated. For score development and validation, 75% were randomly assigned to a derivation cohort to select the prediction model and the remaining 25% to a validation cohort. Due to the relatively high level of interval censoring (3194/15,266 or 21% of weekly observations), we imputed weekly self-reported method use by carrying forward the last observation and carrying backward the next observation; we did not impute after a participant's final complete weekly report (Online Appendix). In the derivation sample, stepwise selection was used to identify the Cox proportional hazards model with the minimum Akaike information criterion [27]. If potential predictors were collinear in the full sample, the variable with a greater scientific basis for inclusion based on the published literature was included prior to model selection. Covariates considered in model selection are in Table 1. A full risk score model using all variables selected in the stepwise model as well as a simplified model comprising variables routinely collected in FP clinics was created. Sociodemographic, reproductive and FP characteristics of the derivation and validation cohorts at study enrollment Notes: p values obtained using χ2 test for proportion or Wilcoxon rank-sum test of medians for continuous measures. To construct risk scores, points were assigned to each variable by taking the ratio of its coefficient to the minimum coefficient in the multivariable Cox model rounded to the nearest integer [22,23]. We assessed predictive value of the risk score using receiver operating characteristic (ROC) curves and area under the curve (AUC) estimates at 12 and 24 weeks using an inverse-probability-of-censoring-weighting approach for right-censored data (Online Appendix) [28]. The 12- and 24-week time points were selected to assess rapid discontinuation after uptake and at the maximum follow-up time, respectively. Time-dependent sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were estimated, with optimal cut points defined by Youden's J statistic. Risk score performance was evaluated in the validation and full cohorts using time-dependent AUC-ROC analysis. Several sensitivity analyses were conducted. First, in addition to excluding women desiring pregnancy in the next year, the primary risk score was reconstructed additionally excluding women who desired a future pregnancy but were unsure when. Second, the full risk score was fit using a subdistribution hazard model [29], with discontinuation for pregnancy desire as a competing risk. We modeled missing reason for discontinuation using multiple imputation with chained equations [30]. Third, we explored an alternative methodology for variable selection using the Cox extension of the least absolute shrinkage and selection operator (LASSO) with a grouped penalty for categorical covariates (Online Appendix). Comparative performance of the stepwise- and LASSO-Cox risk scores is of interest, as LASSO may reduce overfitting compared to standard stepwise approaches [31]. Finally, we rederived the risk score using an alternative definition of discontinuation that required 4, rather than 2, weeks of method nonuse to assess robustness of our findings to selection of the discontinuation interval. Additional sensitivity analyses to assess the potential impact of measurement error in self-reported method use are presented in the Online Appendix. All analyses were conducted in Stata v15.1 (StataCorp, College Station, TX, USA) and R v3.6.2 (The R Project for Statistical Computing).