Operational lessons learned in conducting an international study on pharmacovigilance in pregnancy in resource-constrained settings: The WHO Global Vaccine safety Multi-Country collaboration project

The WHO Global Vaccine Safety Multi-Country Collaboration study on safety in pregnancy aims to estimate the minimum detectable risk for selected perinatal and neonatal outcomes and assess the applicability of standardized case definitions for study outcomes and maternal immunization in low- and middle-income countries (LMICs). This paper documents the operational lessons learned from the study. A prospective observational study was conducted across 21 hospitals in seven countries. All births occurring at sites were screened to identify select perinatal and neonatal outcomes from May 2019 to August 2020. Up to 100 cases per outcome were recruited to assess the applicability of standardized case definitions. A multi-pronged study quality assurance plan was implemented. The impact of the COVID-19 pandemic on site functioning and project implementation was also assessed. Multi-layered ethics and administrative approvals, limited clinical documentation, difficulty in identifying outcomes requiring in-hospital follow-up, and poor quality internet connectivity emerged as important barriers to study implementation. Use of electronic platforms, application of a rigorous quality assurance plan with frequent interaction between the central and site teams helped improve data quality. The COVID-19 pandemic disrupted data collection for up to 6 weeks in some sites. Our study succeeded in establishing an international hospital-based surveillance network for evaluating perinatal and neonatal outcomes using common study protocol and procedures in geographically diverse sites with differing levels of infrastructure, clinical and health-utilization practices. The enhanced surveillance capacity of participating sites shall help support future pharmacovigilance efforts for pregnancy interventions. A facility based, prospective observational study was conducted in 21 sites across six LMICs and one high-income country over a 12-month period between May 2019 to August 2020. Table 1 describes the study sites and their characteristics. The selected study outcomes were low birthweight, preterm birth, small for gestational age (SGA), stillbirth, in-hospital neonatal death, neonatal infection and postnatally diagnosed congenital microcephaly. As specified by the GAIA case definitions, surveillance was undertaken for three types of neonatal infections: meningitis, invasive bloodstream and respiratory infection. In addition to these outcomes, the applicability of the GAIA case definition for maternal immunization was also assessed. The maternal immunization exposure status, including date and time of vaccination, as well as batch details were collected (when available as part of routine patient documentation) for all vaccines administered in pregnancy and within 30 days prior to the last menstrual period. The process of site selection and network establishment has been described in a previous publication [14]. Study site characteristics. Abbreviations- BP: BP Koirala Institute of Health Sciences; GH: General Hospital; GMC: Government Medical College; GUH: General University Hospital; H: Hospital; IMS SUM: Institute of Medical Science and Sum Hospital; MC: Medical College; PC: Polyclinic; PH: Provincial Hospital; RH: Referral/Regional Hospital; RRH: Regional Referral Hospital; SKIMS: Sher-i-Kashmir Institute of Medical Sciences; TH: Teaching Hospital; UH: University Hospital; ZRH: Zonal Referral Hospital. For assessing background rates and minimum detectable risk, all study outcomes identified were included in the study. Assuming 50% of all cases would meet the case confirmation criteria, up to 100 cases per outcome were recruited per site to enable estimation of the proportion of cases meeting standardized case definition with a 20% relative precision. To minimize chances for bias, sites were requested to recruit the first two cases identified for each outcome every week. Fig. 1 describes the multi-level mechanisms for study management and scientific oversight. A diverse team of epidemiologists, biostatisticians, information technology (IT) and pharmacovigilance experts coordinated the study at the central level, while faculty members and research staff implemented the study at the site-level. Based on findings from a previous proof-of-concept multi-country collaboration study [10], National Focal Points (NFPs) were identified to facilitate study implementation in some countries. A seven member advisory committee maintained scientific oversight of the study. Study management and scientific oversight. The study protocol was developed in consensus with participating sites over a two-day investigator meeting in 2017. In addition to the study protocol, the requirements for national and local administrative clearances and training needs necessary for study initiation were discussed in detail among the central team, site investigators, and NFPs attending the investigator meeting. The master protocol was submitted to the WHO Ethics Review Committee (ERC) and by each participating site to local, national, or independent Ethics Committee (EC) as applicable. Additional regulatory clearances such as data transfer agreements and administrative approvals were sought based on country or site-level regulations and norms. Two to three-day country level workshops were conducted in-person using standardized training materials and support documents, such as the study standard operating procedures (SOP) and software application manuals. Feedback from each training was used to improve materials for subsequent trainings; for instance, a handy aide-mémoire was developed based on observed challenges in understanding certain aspects of the study. Following the workshop, sites completed a simulation exercise providing hands-on experience on study SOPs and data entry procedures. The central team provided additional training based on site performance during the simulation exercise. An android-based application, SOMAARTH III [16], was developed in-house at INCLEN and was used to implement study procedures and data collection. The application and electronic case report forms (e-CRFs) were tested internally by the study team and piloted at one study site prior to finalization. The SOMAARTH III application has built-in alerts for missing and invalid data, range-checks, automated skip logic, role-based access control, and dynamic form activation to minimize errors. The software allotted an auto-generated unique identification number to each registered birth, enabling tracking of mother–child dyads in a linked manner during the study. Study data could be collected and stored in offline mode, and only required internet connectivity for data submission. To aid assessment of eligibility for recruitment, a report module was introduced post study initiation for tracking the number of childbirths, and study outcomes recorded on a weekly basis. The data collection process is summarized in Fig. 2. Study data collection process. A data management plan was developed specifying the procedures for data collection, management, safety, privacy, sharing and archiving. A multi-pronged study quality assurance plan was implemented to ensure collection of quality data during the study (Fig. 3). Multi-pronged quality assurance mechanism. Data were reviewed remotely on a periodic basis to evaluate adherence to study protocol, data quality, and medical congruency by the central study team. Standardized templates and algorithms were developed using Stata version 15.1 [17] to automate data reviews. Discrepancies identified during the review were discussed and clarified with sites over teleconferences (held fortnightly in the first month and monthly thereafter). A mid-course country level call was also conducted with participation from all site teams and NFPs to review study progress, share experiences and discuss challenges. A centralized platform (JIRA®) was adopted for query generation, resolution, tracking and documentation [18]. In addition to remote monitoring, on-site visits were planned, initially to at-least 4 sites, to check compliance with study protocol and SOPs. A standardized template was developed to facilitate systematic conduct of on-site monitoring. All visiting team members were apprised of the objectives, provided a site performance report and a randomly chosen selection of recruited cases for comparison with source documents and validation. After each visit, a detailed report was prepared and discussed with the sites. Periodic feedback was sought from the scientific advisory committee. A quarterly newsletter was circulated to the sites, NFPs, and scientific committee to keep them informed about the status of the study. Independent, double programming of analysis was undertaken based on a pre-specified statistical analysis plan using R version 3.6.0 [19] and Stata version 15.1 [17]. Results from the double programming were matched and discrepancies identified were discussed and resolved. To identify and potentially address pandemic related challenges additional teleconferences were conducted with sites. Upon completion of data collection, sites were asked to complete a questionnaire aiming to understand the impact of the pandemic on study procedures and health care at study sites.