Development of a phone survey tool to measure respectful maternity care during pregnancy and childbirth in India: Study protocol

Background: Respectful maternity care (RMC) is a key barometer of the underlying quality of care women receive during pregnancy and childbirth. Efforts to measure RMC have largely been qualitative, although validated quantitative tools are emerging. Available tools have been limited to the measurement of RMC during childbirth and confined to observational and face-to-face survey modes. Phone surveys are less invasive, low cost, and rapid alternatives to traditional face-to-face methods, yet little is known about their validity and reliability. Objective: The primary objective of this study was to develop validated face-to-face and phone survey tools for measuring RMC during pregnancy and childbirth for use in India and other low resource settings. The secondary objective was to optimize strategies for improving the delivery of phone surveys for use in measuring RMC. Methods: To develop face-to-face and phone surveys for measuring RMC, we describe procedures for assessing content, criterion, and construct validity as well as reliability analyses. To optimize the delivery of phone surveys, we outline plans for substudies, which aim to assess the effect of survey modality, and content on survey response, completion, and attrition rates. Results: Data collection will be carried out in 4 districts of Madhya Pradesh, India, from July 2018 to March 2019. Conclusions: To our knowledge, this is the first RMC phone survey tool developed for India, which may provide an opportunity for the rapid, routine collection of data essential for improving the quality of care during pregnancy and childbirth. Elsewhere, phone survey tools are emerging; however, efforts to develop these surveys are often not inclusive of rigorous pretesting activities essential for ensuring quality data, including cognitive, reliability, and validity testing. In the absence of these activities, emerging data could overestimate or underestimate the burden of disease and health care practices under assessment. In the context of RMC, poor quality data could have adverse consequences including the naming and shaming of providers. By outlining a blueprint of the minimum activities required to generate reliable and valid survey tools, we hope to improve efforts to develop and deploy face-to-face and phone surveys in the health sector. Data collection is part of the impact evaluation of Kilkari; an IVR-based maternal messaging program that aims to empower women through improved access to essential health information. Led by the Ministry of Health and Family Welfare (MOHFW) and implemented by BBC Media Action with support from the Bill & Melinda Gates Foundation, United States Agency for International Development, and the Barr Foundation, Kilkari provides weekly stage-based audio messages on topics including birth preparedness, family planning, and maternal and child nutrition directly to the mobile phones of pregnant and postpartum women up to 1 year postpartum. With implementation currently underway in 13 states across India, Kilkari has delivered prerecorded audio content to 8.3 million users in 33 months [19]. Data collection is occurring in 4 districts (Mandsaur, Hoshangabad, Rewa, and Rajgarh) of MP. MP is located in the geographic heart of India and is home to a population of over 75 million. Among women, an estimated 59% are literate (as compared with 82% of men), 64% have ever attended school, and 29% have access to a mobile phone [3]. In 2015, 53% of pregnant women attended ANC in the first trimester, 36% received the recommended 4 ANC visits, 81% delivered in a health facility, 78% had births attended by a skilled provider, and 18% received a postnatal health check within 2 days following birth [3]. Data on differentials in health outcomes and/or utilization of health services among those with and without access to mobile phones are not available. Across all 4 districts in MP, data collection is occurring among a subsample of pregnant and postpartum women identified as part of a household listing exercise. During the household listing, all women of reproductive age with access to a mobile phone are identified. Women who are 4 to 7 months pregnant as well as those with a reported pregnancy outcome in preceding 1 to 4 months are then interviewed as part of the pregnant and postpartum women’s surveys. Freedman and Kruk define disrespect and abuse during childbirth as “interactions or facility conditions that local consensus deems to be humiliating or undignified, and those interactions or conditions that are experienced as or intended to be humiliating or undignified” [20]. Building off of this definition and a 2010 landscape analysis by Bowser and Hill [21], Bohren et al outlined 7 categories of disrespectful and abusive care during childbirth: (1) physical abuse; (2) sexual abuse; (3) verbal abuse; (4) stigma and discrimination; (5) failure to meet professional standards of care; (6) poor rapport between women and providers; and (7) health system conditions and constraints [6]. These categories were subsequently conceptualized in 2 dimensions: (1) intentional use of violence, including physical abuse, verbal abuse, and negligent withholding of care and (2) structural disrespect, which stems from “deviations from accepted standards for infrastructure, staffing, equipment availability, and supplies needed to deliver care, as well as in unnecessary interventions, demands for illegal payments, and the detainment of people in facilities until they have paid their bills” [22]. In this protocol study, we focus on the measurement of each of these major typologies of disrespect and abuse along with the underlying contextual factors that underpin them. Figure 1 outlines a conceptual framework for measuring RMC during pregnancy and intrapartum care, which brings together traditional approaches to measuring quality of care [9,23-27] with frameworks for assessing mistreatment of care during childbirth [28]. Viewing mistreatment through the lens of one perspective (eg, intrapartum women) at a single time point (eg, childbirth), although important, may nevertheless provide a limited view of the larger context within which treatment occurs and the risk factors underpinning it. This framework aims to illustrate that maternal health outcomes stem from the interaction of beneficiaries with providers in a complex and evolving community and health systems context through multiple points of contact in different facilities starting with ANC in primary health centers. We posit that women’s interactions with the larger health systems’ environment help to formulate their care experience and expectations, and ultimately outcomes, including utilization of services and perceptions of quality and satisfaction. Conceptual framework for measuring respectful maternity care (RMC). Multimedia Appendix 1 summarizes questions by RMC typology and domain for the proposed measurement of RMC during childbirth. Table 1 summarizes the number of questions by RMC domain for each of the survey planned and compares these against alternatives in the literature. In contrast to approaches in Kenya, Bihar, and Ethiopia, we distinguish questions in MP according to whether they aim to estimate the prevalence of a particular domain or rather users’ satisfaction with an aspect of care received. This distinction is important given its implications on the response options required (eg, Likert scales versus binary or categorical) and their associated implications for analyses. Measurement of RMC will occur through 2 modalities: (1) face-to-face survey and (2) phone surveys. Face-to-face surveys will be carried out on 2 populations as part of a larger baseline evaluation of Kilkari: (1) women who are 5 to 7 months pregnant and (2) women with a birth outcome in the preceding 1 to 4 months. In addition to RMC, face-to-face surveys include modules on mobile access and literacy, socioeconomic and demographic characteristics, birth history, and experiences with care during pregnancy or childbirth. Face-to-face surveys will be modified following analyses to yield the following phone survey tools: (1) RMC during pregnancy; (2) RMC during childbirth; and (3) essential newborn care and infant feeding. Comparison and summary of total number of questions by respectful maternity care domain for Madhya Pradesh, India, and other respectful maternity care studies identified in the literature. aQuestion not included. Figure 2 outlines proposed processes for validity and reliability testing, whereas Table 2 and Multimedia Appendix 2 summarize survey substudies and validity/reliability tests, respectively. Building off of a strong foundation of existing validated instruments [12], project activities will commence with a literature review from which survey tools will be developed for RMC measurement during pregnancy and childbirth, including scales for measuring satisfaction and prevalence [11]. Item generation for each scale was drawn from concurrent activities underway in Bihar by Rao et al [29] to measure RMC during childbirth through direct observations, exit interviews, and follow-up household interviews during the postpartum period. Indicators from the above listed and other validated survey tools elsewhere in the literature [12] were used in the MP survey tools to allow for cross-site comparison. Once consensus was achieved, items were translated into Hindi and checked by BBC Media Action and MOHFW personnel in Delhi for accessibility, appropriateness of language, tone, and engagingness. Cognitive testing followed in study districts in MP to ensure that survey questions are understandable, appropriate in language and tone, and the words interpreted as intended by varying respondent types. Processes for reliability testing. Summary description of survey substudies. aRMC: respectful maternity care. bCATI: computer-assisted telephone interview. Table 3 summarizes the sample size requirements for each substudy. Face-to-face surveys will be conducted to refine the scale and determine the prevalence of different typologies of disrespect and abuse in 4 districts of rural MP. Among pregnant women, a module on RMC during ANC will be integrated into a planned household survey among 5000 women who are 5 to 7 months pregnant. This is sufficient to measure the RMC indicator of reported verbal abuse (assumed 5% prevalence) during pregnancy with 80% power, alpha of .05, and precision of 1%. To measure RMC during childbirth, a total of 880 women with a birth outcome in the preceding 1 to 4 months will be interviewed. This sample size was designed to accommodate survey mode testing described in Phase 3 and is sufficient to additionally measure the prevalence of the RMC indicator of reported verbal abuse (assumed 10% prevalence) during childbirth with 80% power, alpha of .05, and precision of 2%. The number of participants needed by substudy. aThe total sample size reflects the sum of the sample across all study arms. bANC: antenatal care. cRMC: respectful maternity care. Once data are collected, analyses will principally aim to determine the validity of the scale using psychometric analyses (Figure 2). Criterion-related validity will be assessed by testing the hypothesis that scale is correlated to measures of reported satisfaction additionally collected as part of the face-to-face survey tool [12,13]. We propose testing this by regressing the main RMC scale and subscales on women’s ratings of their satisfaction with the services and whether they would deliver in the same facility if they were to have another baby [12]. Construct validity measures how well the items represent the underlying conceptual structure [13] and will be assessed using factor analysis and the Pearson correlation coefficient between the components. Reliability analyses will aim to determine the stability and consistency of results [13]. A Cronbach alpha of .7 or higher is proposed as the cutoff for determining sufficient evidence of reliability [13]. Additional analyses related to the internal consistency of the scale as well as the presence of floor and ceiling effects will be conducted and overall findings on validity and reliability summarized [13]. To assess reproducibility, a random subsample of pregnant and postpartum women interviewed as part of substudy 1 will be administered a repeat face-to-face survey between 1 and 2 weeks after the initial survey. This substudy will be conducted to determine the degree to which repeated measurements in women interviewed (test-retest) provide similar answers. Assuming a kappa of 0.80, a margin of error of 0.05, an alpha of .05, and the proportion of positive responses of 0.35 for rater 1 and 0.40 for rater 2, 146 participants who have completed the survey are required. Adjusting for a 15% loss to follow-up/refusal between the first and second women’s surveys will require a sample size of 168 women to be interviewed twice. Data will be analyzed for agreement between survey rounds and reliability will be tested using Cohen kappa. The kappa will be adjusted for prevalence and bias, providing Prevalence and Bias Adjusted Kappa. Phone survey mode testing will aim to determine the intermodal reliability of face-to-face versus CATI surveys for both the RMC pregnancy and childbirth surveys. Assuming a kappa of 0.80, a margin of error of 0.05, an alpha of .05, and the proportion of positive responses of 0.35 for rater 1 and 0.40 for rater 2, 146 participants who have completed each survey are required. Adjusting for loss to follow-up between the face-to-face women’s survey and the following mobile phone survey, 880 women with a birth outcome in the preceding 1 to 4 months will be interviewed face to face. Within 4 weeks of the initial interview, a random sample of those completing the face-to-face interview who consent to be called for the follow-up phone survey will be contacted. Assuming a 20% response rate, 880 women will be contacted as part of the phone survey to yield the 146 completed face-to-face and phone survey interviews. Only women with access to a mobile phone, aged 18 years or older, and who have had a birth outcome in the preceding 1 to 4 months and are identified in the study districts will be interviewed. Survey content refers to 2 components of the phone survey: (1) topical area covered and (2) response options and question framing. We will assess the effects on survey content and length (number of questions) of response, completion, and attrition rates using Kaplan-Meier curves to plot survey attrition by time spent for each survey implemented across key populations. This will include comparisons across RMC surveys administered to pregnant and postpartum women. Assuming a baseline survey completion percentage of 20% to detect an absolute 10% difference in survey completion between 2 study arms at an alpha of .05 and power of 80%, it is calculated that 294 individuals who have completed the survey will be needed per study arm. With a completion percentage of 20%, we estimate that 1470 participants would be required. To attain this sample size, the phone survey tool validated in substudy 3 will be applied to the population of 4500 women enrolled in the Kilkari impact evaluation in 4 districts of MP. This subanalysis aims to compare the demographic characteristics of respondents in the larger sampling frame versus those who complete, partially complete, and do not respond to phone surveys. Additional data points, including caste, education, and socioeconomic status, collected during the face-to-face household listing and baseline survey will be juxtaposed against CATI phone survey data. All data collected will remain in India and will be managed by the India-based research partner. Tablets used for data collection will be password protected. Any adverse events mentioned to the research team during data collection will be brought to the immediate attention of senior project investigators and Institutional Review Boards at Johns Hopkins School of Public Health and in India at Sigma Research and Consulting in New Delhi. Once collected, all data will be deidentified following the merging of data sets as required reliability analyses. Ethical approval for research activities in India has been obtained from Johns Hopkins School of Public Health’s Institutional Review Board in Baltimore, Maryland, United States, and from Sigma Research and Consulting in New Delhi, India.