WHO global vaccine safety multi-country collaboration project on safety in pregnancy: Assessing the level of diagnostic certainty using standardized case definitions for perinatal and neonatal outcomes and maternal immunization

Standardized case definitions strengthen post-marketing safety surveillance of new vaccines by improving generated data, interpretation and comparability across surveillance systems. The Global Alignment of Immunization Safety Assessment in Pregnancy (GAIA) project developed standardized case definitions for 21 key obstetric and neonatal terms following the Brighton Collaboration (BC) methodology. In this prospective cohort study, we assessed the applicability of GAIA definitions for maternal immunization exposure and for low birth weight (LBW), preterm birth, small for gestational age (SGA), stillbirth, neonatal death, neonatal infection, and congenital microcephaly. We identified the missing data elements that prevented identified cases and exposures from meeting the case definition (level 1–3 of BC diagnostic certainty). Over a one-year period (2019–2020), all births occurring in 21 sites (mostly secondary and tertiary hospitals) in 6 Low Middle Income Countries and 1 High Income Country were recorded and the 7 perinatal and neonatal outcome cases were identified from routine medical records. Up to 100 cases per outcome were recruited sequentially from each site. Most cases recruited for LBW, preterm birth and neonatal death met the GAIA case definitions. Birth weight, a key parameter for all three outcomes, was routinely recorded at all sites. The definitions for SGA, stillbirth, neonatal infection (particularly meningitis and respiratory infection) and congenital microcephaly were found to be less applicable. The main barrier to obtaining higher levels of diagnostic certainty was the lack of sonographic documentation of gestational age in first or second trimester. The definition for maternal immunization exposure was applicable, however, the highest level of diagnostic certainty was only reached at two sites. Improved documentation of maternal immunization will be important for vaccine safety studies. Following the field-testing of these 8 GAIA definitions, several improvements are suggested that may lead to their easier implementation, increased standardization and hence comparison across studies. This was a prospective, descriptive, cohort study using a common protocol (S1) and routinely collected data at 21 sites in six LMICs (Ghana, Tanzania, Zimbabwe, Iran, India, Nepal) and one high-income country (HIC) (Spain), consisting of one primary care center, five secondary hospitals and fifteen tertiary hospitals. Each site had a maternity ward. Sites were selected using a 2017 study that employed site selection criteria and acceptable performance in a simulation exercise that tested capacity to access sufficient data of acceptable quality at the site level [25]. Table 1 lists characteristics of the participating sites. The first site started data collection on May 6, 2019, and the last site completed data collection on August 18, 2020. Detailed methods are available in the report [26]. Characteristics of participating sites. *8 months instead of 1 year at Grant GMC. AFRO: WHO African region; BP: BP Koirala Institute of Health Sciences; EMRO: WHO Eastern Mediterranean region; EURO: WHO European region; GH: General Hospital; GMC: Government Medical College; GUH: General University Hospital; H: Hospital; IMS SUM: Institute of Medical Science and Sum Hospital; MC: Medical College; NICU: neonatal intensive care unit; PC: Policlinic; PH: Provincial Hospital; RH: Referral/Regional Hospital; RRH: Regional Referral Hospital; SEARO: WHO South-East Asia region; SKIMS: Sher-i-Kashmir Institute of Medical Sciences; TH: Teaching Hospital; UH: University Hospital; ZRH: Zonal Referral Hospital. All births at the sites were prospectively recorded during a one-year period, and the following study outcomes occurring in the 28 days following birth were identified as part of routine care by the sites: LBW, preterm birth, SGA, stillbirth (antepartum or intrapartum), in-hospital neonatal death, neonatal infection (invasive bloodstream infection (BSI), respiratory infection or meningitis) and postnatally diagnosed congenital microcephaly. The outcomes were selected based on relevance in vaccine safety research and perceived ability to collect data on the outcome of interest. Cases were first identified by screening relevant data sources from the maternity and neonatal wards at the sites (e.g. labor room register, admission register, patient records; see full list in S2). Only study outcomes at the site were considered; no follow-up outside of the site was performed. Estimated rates of occurrence will be reported in a separate paper and are also accessible in the study report [26]. At each site, up to 100 cases of each study outcome were systematically recruited into the study (the first two cases per week, or all consecutive cases); informed consent was obtained from the mother. One hundred cases per outcome per site enabled the calculation of 20% relative precision around estimates of the proportion of cases meeting the GAIA definition, under the assumption that 50% of all cases met at least the lowest level definition. Exhaustive case report forms, including details on any vaccines received during pregnancy, were completed for recruited cases, based on existing routine medical records. Data sources included the mother’s antenatal care records, the antenatal care card, and inpatient records (full list in S2). Study site staff were trained on the study procedures. All the data were captured through an app-based electronic data capture system, SOMAARTH III [27] using tablets. Data quality was monitored centrally, and on-site monitoring visits and regular tele-conferences with sites were conducted. We developed algorithms for the GAIA definitions for LBW [19], preterm birth [20], SGA [21], stillbirth (antepartum and intrapartum stillbirth) [22], neonatal death [18], neonatal infection (bloodstream infection (BSI), respiratory infection, meningitis) [23], postnatally diagnosed congenital microcephaly [24] and maternal immunization [18] to assess the level of diagnostic certainty of the GAIA definition met by recruited cases, if any. Cases classified as Level 1, 2 or 3 were said to meet the GAIA definition. Level 1 represented the highest levels of diagnostic certainty (most specific, least sensitive), and Level 3 the lowest (least specific, most sensitive). Levels 4 and 5, if present, were not considered as those events did not meet the case definition. The GAIA definitions have been summarized in S3a. First, it assessed whether Level 1 criteria were met. If yes, then the case was considered classified to Level 1. If no, it assessed whether Level 2 criteria were met, and so on. For each definition, the applicability was assessed by calculating the proportion of cases or maternal immunization exposures meeting the GAIA definition and the proportion classified to each level, by site. The most common reasons for not meeting GAIA definitions or, for non-classification of level 3 cases to levels 1–2 were summarized (or described) for each outcome. We modified the GAIA definition so that criteria accepted at higher levels of diagnostic certainty (‘higher levels of evidence’) were also de facto acceptable at lower levels of diagnostic certainty. For example, in the case of LBW, we considered electronic scales (sufficient for levels 1 and 2) appropriate for a level 3 classification as well (S3b for further details). Several aspects of the maternal immunization definition were open to interpretation. For level 1, we interpreted ‘date/time’ as ‘date AND time’ and ‘details of vaccine’ as ‘lot number AND EITHER name of disease OR name of vaccine’. For level 2, we interpreted ‘details of disease’ as ‘name of the disease OR name of vaccine OR lot number’. For levels 1–2, primary sources such as the antenatal care card, vaccine card or vaccine register were required, and for level 3, secondary sources were accepted such as the patient case sheet or birth register. For each outcome, the proportion of recruited cases that met the GAIA definition was stratified by country, health facility level (primary, secondary, tertiary/referral), and health facility ownership (public/private). A Chi-square test was used to assess whether there were any significant differences between the categories. Double independent programming of all analyses was performed using R version 3.6.0 [28] by the company, P95 Epidemiology and Pharmacovigilance and Stata version 15.1 [29] by INCLEN Trust International. Output was compared and the differences were resolved. The study was approved by the WHO Ethics Review Committee (protocol ID: ERC.0003114), and by local and national committees as appropriate [26].