Improving measures of access to legal abortion: A validation study triangulating multiple data sources to assess a global indicator

Background Global mechanisms have been established to monitor and facilitate state accountability regarding the legal status of abortion. However, there is little evidence describing whether these mechanisms capture accurate data. Moreover, it is uncertain whether the “legal status of abortion”is a valid proxy measure for access to safe abortion, pursuant to the global goals of reducing preventable maternal mortality and advancing reproductive rights. Therefore, this study sought to assess the accuracy of reported monitoring data, and to determine whether evidence supports the consistent application of domestic law by health care professionals such that legality of abortion functions as a valid indicator of access. Methods and findings We conducted a validation study using three countries as illustrative case examples: Argentina, Ghana, and India. We compared data reported by two global monitoring mechanisms (Countdown to 2030 and the Global Abortion Policies Database) against domestic source documents collected through in-depth policy review. We then surveyed health care professionals authorized to perform abortions about their knowledge of abortion law in their countries and their personal attitudes and practices regarding provision of legal abortion. We compared professionals’ responses to the domestic legal frameworks described in the source documents to establish whether professionals consistently applied the law as written. This analysis revealed weaknesses in the criterion validity and construct validity of the “legal status of abortion”indicator. We detected discrepancies between data reported by the global monitoring and accountability mechanisms and the domestic policy reviews, even though all referenced the same source documents. Further, provider surveys unearthed important context-specific barriers to legal abortion not captured by the indicator, including conscientious objection and imposition of restrictions at the provider’s discretion. Conclusions Taken together, these findings denote weaknesses in the indicator “legal status of abortion”as a proxy for access to safe abortion, as well as inaccuracies in data reported to global monitoring mechanisms. This information provides important groundwork for strengthening indicators for monitoring access to abortion and for renewed advocacy to assure abortion rights worldwide. This is a cross-sectional, observational study design using multiple sources of data. We collected secondary data through policy review and primary data through cross-sectional survey of healthcare providers to address two validation questions, respectively: Three LMIC research settings (Argentina, Ghana, and India) were purposively selected for the larger research project of which this study is part, based on geographic diversity across those world regions reflecting the highest burden of maternal mortality and demonstrated local research capacity. Primary data were collected in four districts/provinces of each country that were selected systematically using a multi-stage standardized sampling plan that took into consideration variations in health system performance, geographic location, population served, and other forms of diversity. This selection process is detailed elsewhere [36]. Within each district/province, we replicated the Demographic and Health Survey methodology [37] to define primary sampling units within each jurisdiction and randomly selected 20 units. All facilities offering abortion services within each primary sampling unit were included. Study participants were drawn from healthcare providers on the payroll in participating health facilities who belonged to professional cadres legally authorized to provide abortion. The managers of participating facilities provided lists of eligible providers in that facility. In Argentina, this included obstetricians/gynecologists (OB/GYNs) and general practice physicians employed as sexual and reproductive health providers. In Ghana, all OB/GYNs, general practice physicians, and midwives were eligible to participate. In India, OB/GYNs and general practice physicians with abortion certification were eligible. Participants were considered eligible if they were authorized to provide abortion care, were currently working in a participating center, and provided consent to participate. Exclusion criteria included providers on extended sick leave or those unable or unwilling to provide consent. To address the first validation question, we extracted data from the most recent country profiles in Countdown (8 August 2020) and GAPD (last updated 15 June 2021 for Argentina; 7 May 2017 for Ghana; and 15 June 2021 for India). Countdown metadata only included data on legal grounds for abortion, while GAPD included data on legal grounds for abortion and details on additional requirements to access abortion. We then conducted a comprehensive desk review of national (and, as relevant, subnational) policy through October 2021 in Argentina, July 2021 in Ghana, and July 2021 in India. In Argentina, we systematically searched two electronic legal databases [InfoLEG (http://www.infoleg.gob.ar) and Sistema Argentino de Información Jurídica (http://www.saij.gob.ar)] using keywords related to abortion and reviewed the reference lists of peer-reviewed publications on Argentina’s legal landscape regarding abortion. We also manually searched relevant ministerial documents and consulted with subject matter experts to request additional resources and ensure no documents were omitted. The documents reviewed were the National Penal Code (Arts. 85–88); National Civil and Commercial Code (Arts. 22–24, 26); Convention on the Rights of Persons with Disabilities; Law 25,673; Law 26,529; Law 26,485; Law 26,657; Law 23,179; Law 23,313; Law 24,632; Fallo F.A.L. decision; 2019 National Protocol on Care for Persons with the Right to a Legal Abortion; National Essential Medicines List; and relevant Ministerial declarations regarding the use of misoprostol (“ANMAT aclara acerca de producto con misoprostol” [ANMAT clarification regarding products with misoprostol] and “Sobre la autorización de los productos con ingrediente farmacéutico activo Misoprostol” [Regarding the authorization of products with the active pharmaceutical ingredient misoprostol]). In Ghana, we searched the websites of the Ghana Health Service, Nurses and Midwifery Council, and the Ministry of Health using keywords related to abortion for documents on the legal status of abortion. We also consulted with subject matter experts from the Family Health Division of the Ghana Health Service and the Ministry of Health to ensure all related documents were compiled. The documents ultimately included were Ghana’s 1992 Constitution Act 29, Comprehensive Abortion Care Protocols, and National Reproductive Health standards. In India, we searched all government and allied portals for legal documents and guidelines using keywords related to abortion. The included reference sources were: Medical Termination of Pregnancy Act of 1971, along with its several amendments (2002, 2003, 2020, 2021); Article 24 of the Constitution; Act Number 45 of the Indian Penal Code of 1860; Pre-Conception and Pre-Natal Diagnostic Techniques Act of 1994; Protection of Children from Sexual Offences Act of 2012; National List of Essential Medicines of India; 1945 Drugs and Cosmetics Rule (amended in 2013); FOGSI & ICOG Good Clinical Practice Recommendation of Medical Termination of Pregnancy; and Government of India’s Comprehensive Abortion Care-Training and Service Delivery Guidelines 2018. We also consulted subject matter experts to request any additional resources and materials to ensure comprehensive review. To facilitate consistent data collection across countries, we developed a standardized data extraction form with fields for each GAPD-reported criterion: legal grounds for abortion, additional requirements needed to obtain an abortion, and aspects of clinical care. Definitions for each term were based on WHO policy guidance [2]. Each legal ground was coded as either explicitly permitted, prohibited, or not specified in the reviewed documents. Each additional requirement was coded as either explicitly required, explicitly not required, or not specified in the reviewed documents. All relevant legal documents identified were reviewed and coded independently by two study team members, who resolved discrepancies by consensus. A third team member helped resolve disagreements as needed. Local experts in abortion policy were consulted to verify the local interpretation and identify relevant additional documents, including jurisprudence. To address the second validation question, we surveyed healthcare providers legally authorized to provide abortions. The surveys sought to: 1) capture respondents’ knowledge of the legal grounds for abortion and any restrictions on abortion in their jurisdictions; and 2) explore providers’ practice patterns to identify possible provider-level variations in the provision of legal abortion. Surveys were conducted July–October 2021 in Argentina, April 2021 in Ghana, and September–December 2020 in India. Recruitment and data collection procedures varied by country. In Argentina, meetings were held in each participating health facility to explain the project to eligible health providers. The facility data collector than collected email addresses of eligible providers to contact regarding participation. Eligible providers were emailed a link to a secure portal with detailed descriptions of the survey purpose and procedures. Providers who responded to the consent electronically were emailed a secure electronic link to access the survey. Those who elected to respond via a paper-based survey were provided a paper form and asked to complete it in a private room within the facility where they practice. Completed paper-based surveys were sealed in envelopes and transferred to the data center. In Ghana, data were collected via in-person interviews. Due to the sensitivity of abortion data, entries were made directly into the secure online platform by field researchers. Personal identifiers were kept separately in hard copies that were securely stored in a locker dedicated to the study with access restricted to only core study team members. Personal identifiers were not linked to electronic information collected. In India, contact numbers of abortion service providers were obtained from the district health department. Providers were contacted by the field team to obtain consent and schedule a telephone interview. Interviews were conducted in a local language or in English as per healthcare workers’ preference, with most conducted in English. Hard copies of the forms filled by interviewers during the telephone interviews were stored in a secured locker with access restricted to project personnel. Survey responses were de-identified, entered, and stored in a dedicated, secure web-based study platform with validation checks. All countries used the same password-protected secure web-based study platform (REDCap version 11.2.2). For the first validation question, we conducted comparative analysis of domestic legal frameworks (hereafter, “validation data”; considered the gold standard) and information reported in the global monitoring mechanisms (Countdown and GAPD country profiles) regarding legal grounds, requirements, and restrictions for each country. We drew from legal mapping and policy surveillance methodologies successfully used to identify variation in sources and abortion regulatory requirements [32,38–42]. Rather than using distinct political entities (e.g., states or subnational units) as the unit of analysis, we compared differences across three data sources for the same political/governance unit. To address the second validation question, we conducted descriptive analyses of the survey data, stratified by country. First, we calculated descriptive statistics for respondents in each country. Next, we tabulated the proportion of providers who: a) correctly identified whether a given ground for abortion was legal in their country; b) incorrectly believed that the provision of abortion on each ground was conditioned upon specific restrictions/additional requirements that were not stipulated by law; c) indicated they would personally perform an abortion on each of the grounds they indicated are legal; and d) reported having personal practices that imposed other barriers to abortion beyond those required by law. Finally, we compiled the responses for reasons for not performing an abortion for each legal ground. Surveys with missing data for some fields were included in the analysis; those returned blank were excluded. Data from the desk review of policy documents (national and, where relevant, subnational legal frameworks) served as the gold standard for comparison. Analysis was conducted using Stata version 15.1 (StataCorp, College Station, TX, USA). The study and informed consent process and forms were approved by the Office of Human Research Administration at Harvard University (IRB19-1086) and local institutional review boards [Argentina: Comité de Ética de la Investigación de la Provincia de Jujuy (approval ID not applicable), Comisión Provincial de Investigaciones Biomédicas de la Provincia de Salta (approval ID 321-284616/2019), Consejo Provincial de Bioética de la Provincia de La Pampa (approval ID not applicable), Comité de Ética Central de la Provincia de Buenos Aires (approval ID 2919-2056-2019); India: Sigma-IRB (IRB number: 10052/IRB/19-20); Ghana: Ghana Health Service Ethical Review Committee (approval number GHS-ERC022/08/19)]. All participants provided written informed consent. During the recruitment and informed consent processes, particular emphasis was put on the voluntary nature of participation, precautions taken to secure and de-identify data, the respondent’s ability to withdraw at any time, and the data protection procedures. Potential participants were encouraged to ask questions regarding the survey and given opportunities to discuss any concerns with local study coordinators. We carefully protected anonymity and confidentiality of the data throughout the entire data cycle (collection, entry, analysis). The recruitment process minimized the possibility that colleagues and supervisors would know whether participants were in the study. None of the study team members involved in recruitment or data analysis could see which providers decided to participate in the study or could access identifiable data. The data manager overseeing survey administration did not have access to the list of provider names or the content of respondent surveys. Data entry personnel did not have access to the list of provider names or any identifying information. To reduce the risk of deductive disclosure, data were aggregated such that the individual province/district, facility, or provider could not be identified. We blinded provinces/districts and reported them by assigning random numbers (1–4).