Menu
Menu
Menu
Menu
Background To institutionalise respectful maternity care, frequent data on the experience of childbirth care is needed by health facility staff and managers. Telephone interviews have been proposed as a low-cost alternative to derive timely and actionable maternal self-reports of experience of care. However, evidence on the validity of telephone interviews for this purpose is limited. Methods Eight indicators of positive maternity care experience and 18 indicators of negative maternity care experience were investigated. We compared the responses from exit interviews with women about their childbirth care experience (reference standard) to follow-up telephone interviews with the same women 14 months after childbirth. We calculated individual-level validity metrics including, agreement, sensitivity, specificity, area under the receiver operating characteristic curve (AUC). We compared the characteristics of women included in the telephone follow-up interviews to those from the exit interviews. Results Demographic characteristics were similar between the original exit interview group (n=388) and those subsequently reached for telephone interview (n=294). Seven of the eight positive maternity care experience indicators had reported prevalence higher than 50% at both exit and telephone interviews. For these indicators, agreement between the exit and the telephone interviews ranged between 50% and 92%; seven positive indicators met the criteria for validation analysis, but all had an AUC below 0.6. Reported prevalence for 15 of the 18 negative maternity care experience indicators was lower than 5% at exit and telephone interviews. For these 15 indicators, agreement between exit and telephone interview was high at over 80%. Just three negative indicators met the criteria for validation analysis, and all had an AUC below 0.6. Conclusions The telephone interviews conducted 14 months after childbirth did not yield results that were consistent with exit interviews conducted at the time of facility discharge. Women’s reports of experience of childbirth care may be influenced by the location of reporting or changes in the recall of experiences of care over time.
Gombe State, the study setting, is one of the 36 states of the Federal Republic of Nigeria, located in the country’s North-East region. Gombe State has an estimated population of 2.6 million, based on population projections from the 2006 national census. About 75% of the state is rural, with a high fertility rate of 7.0 live births per 1000 females aged 15–49. Service utilisation for maternal and newborn health services is low: for example, only 44% of pregnant women sought 4 or more antenatal care visits in 2019, only 28% had a facility-based childbirth and only 21% of the deliveries were conducted by a skilled birth assistant.21–23 We collected data on 26 experience of maternity care indicators focusing on 8 positive maternity care experiences and 18 negative maternity care experiences. The negative maternity care experience indicators were drawn from the typology of mistreatment, which included domains of physical abuse, verbal abuse, sexual abuse, stigma and discrimination, failure to meet professional standards of care, poor rapport and communication between women and providers, and health systems conditions and constraints.12 We referred to the literature on improving quality of maternal and newborn care in health facilities and the earlier literature assessing experience of childbirth care to identify the eight positive maternity care experience indicators (ie, practices that recognise women’s preferences and needs).7 13 24–26 The research team agreed on the final list of indicators described in online supplemental table S1 through discussion and consensus. bmjgh-2021-008017supp001.pdf The study was nested within a programme of work aimed at understanding the quality of maternal and newborn care in Gombe State, Nigeria.27 We collected exit interview data from mothers in 10 primary healthcare (PHC) facilities, in Gombe State, in August–September 2019. Mothers were eligible and invited for the exit interviews if they were discharged (usually within 24 hours of childbirth) with a live baby following facility-based childbirth and provided informed consent to participate in the study. The exit interviews were conducted in Hausa. The exit interview instrument covered demographic information of study participants, the content of care provided to the mother and the newborn, and experiences of facility-based childbirth care. Women were also asked about their access to mobile phones and, for those with access, permission to make a follow-up call in the future was solicited. In October–November 2020, we conducted telephone interviews with the same mothers surveyed during exit survey. Only mothers that participated in the exit interviews, provided telephone numbers and consent were included in the follow-up telephone interviews. In both exit interviews and telephone interviews, mothers were asked the same questions about their experience of facility-based childbirth care (online supplemental table S1), with responses to questions dichotomised as ‘experienced an event’ (yes) and ‘not experienced an event’ (no).28 All interviewers for both exit and telephone interviews were from Gombe State and were trained in-house for 5 days to familiarise themselves with the questionnaires and data collection procedures, followed by a full pilot and refinement of the study tool. To ensure confidentiality, all the exit interviews were conducted in an area reserved for the interviews or in a separate room within the health facilities. For the telephone interviews, women were encouraged to find a quiet place at home conducive for the telephone interview. The exit interview data were collected in 10 facilities, with 2 trained data collectors and a supervisor working in shifts covering day and night deliveries, 7 days a week for approximately 4 weeks. The telephone interviews were completed in 2 weeks by three data collectors conducting approximately 10 telephone interviews per day. In both the exit and telephone interviews, women were assured that any information collected about them would be kept private and that all data including name, phone number their contact details and interview answers would be fully anonymised. A minimum sample size of 294 women interviewed at exit and at follow-up telephone interviews was estimated to be adequate to estimate sensitivity, specificity and AUC as an overall index of accuracy. This estimate was based on 50% prevalence of indicators from exit interviews (reference standard) and a set sensitivity of 75%±7% precision, specificity of 75%±7% precision, type 1 error of 0.05, assuming a normal approximation to a binomial distribution.29 Exit survey and the telephone interviews were matched by unique participant id. All analyses were conducted using STATA V.16 (www.stata.com). For the validation analysis, exit survey measures of positive and negative maternity care experiences were used as the reference standard and compared with telephone interview responses with the same mothers. We tabulated the mother’s characteristics at exit survey (all women interviewed without a mobile phone) and follow-up telephone interview to compare demographics and childbirth environment characteristics. We determined the prevalence of positive and negative maternity care experiences for each indicator by the measurement method. Exit interview and telephone interview responses were cross tabulated to construct two-by-two tables, excluding any do not know responses. We calculated per cent agreement between the exit and the telephone interviews. We calculated the sensitivity (true positive rate) and specificity (true negative rate) for each indicator. We quantified the area under the receiver operating characteristic curve (AUC) and estimated 95% CI assuming a binomial distribution. Because this study population included a large number of women with no formal education, we explored the association of educational status (not educated/ educated) of mothers with their reporting consistency for positive maternity care experience measures using the rocreg command in STATA.30 Consistent with the recommendation by Munos et al,31 indicators with very low or very high prevalence, that resulted in fewer than five counts per cell in the two-by-two tables, were included in tabulations for transparency but cannot be interpreted with confidence. An AUC value of 0.5 reflects a random guess while 1.0 reflects perfect accuracy.31 We presented findings below in line with Strengthening the Reporting of Observational Studies in Epidemiology statement.32 A preliminary consultation with a different set of women was conducted prior to the main telephone interviews to pretest the telephone interview protocol for appropriateness and understanding. We asked the respondents for feedback about the telephone interview procedures including perceived difficulty, compatibility and clarity of instructions. We used respondent’s inputs to refine the telephone interview protocol.
N/A
