Monitoring child survival in ‘real time’ using routine health facility records: Results from Malawi

Objectives: Few developing countries have the accurate civil registration systems needed to track progress in child survival. However, the health information systems in most of these countries do record facility births and deaths, at least in principle. We used data from two districts of Malawi to test a method for monitoring child mortality based on adjusting health facility records for incomplete coverage. Methods: Trained researchers collected reports of monthly births and deaths among children younger than 5 years from all health facilities in Balaka and Salima districts of Malawi in 2010-2011. We estimated the proportion of births and deaths occurring in health facilities, respectively, from the 2010 Demographic and Health Survey and a household mortality survey conducted between October 2011 and February 2012. We used these proportions to adjust the health facility data to estimate the actual numbers of births and deaths. The survey also provided ‘gold-standard’ measures of under-five mortality. Results: Annual under-five mortality rates generated by adjusting health facility data were between 35% and 65% of those estimated by the gold-standard survey in Balaka, and 46% and 50% in Salima for four overlapping 12-month periods in 2010-2011. The ratios of adjusted health facility rates to gold-standard rates increased sharply over the four periods in Balaka, but remained relatively stable in Salima. Conclusions: Even in Malawi, where high proportions of births and deaths occur in health facilities compared with other countries in sub-Saharan Africa, routine Health Management Information Systems data on births and deaths cannot be used at present to estimate annual trends in under-five mortality. © 2013 John Wiley & Sons Ltd. We selected two of the 28 districts in Malawi for the test of RMM approaches based on the criteria of high under-five mortality, high fertility, easy access for the study team, full coverage of community health workers deployed and average population size based on the distribution of district population size across the country (Appendix S1). Table 1 shows selected demographic and health system characteristics of the two districts – Balaka in the southern region and Salima in the central region. According to the 2008 Malawi Population Census, Balaka had a population of 316 748 and Salima 340 327. Both districts have high mortality among children under 5 years of age and high fertility (Malawi National Statistical Office (NSO) 2008; National Statistical Office (NSO) … ICF Macro 2011). Selected demographic and health system characteristics of Balaka and Salima districts, Malawi Before rollout, the RMM project was presented and discussed with stakeholders at national level and in the selected districts. The national-level stakeholders included MOH representatives and other partners involved in maternal, newborn and child health programmes in the country. At district level, the district health Office, the district assembly and some traditional authorities participated in orientation and discussion sessions. A small advisory group was established to provide guidance and ensure that study procedures were consistent with standard operating procedures and not duplicative or burdensome to district staff. In preparation for the study, the research team and district HMIS officers reviewed the HMIS database of births and deaths and visited all public and private health facilities in each of the two RMM districts to inspect available records of births and deaths. Current HMIS procedures call for recording of all births and deaths that occur in health facilities, including private facilities. Tallies of deliveries and births are collected every quarter from all health facilities with a maternity ward by the HMIS officers and compiled at district level before being sent to the national level. Cause of death information is recorded only for inpatient deaths. Deaths are not recorded systematically in health centres with no inpatient wards. HMIS forms (Appendix S2) do not allow breakdown of deaths by age. We developed a short form (Appendix S3) and trained the two district HMIS officers and facility staff to record deaths by age, disaggregated by neonatal, infant and child deaths. There was one district-level HMIS officer in each district, who works with the health centre data clerks or incharges. They were given one-day training on how to fill out the form and transmit the data to the National Statistical Office. They were then provided with monthly incentives of about US$30 as motivation for the extra requirement of disaggregating the deaths by age. Given that our interest was in assessing the level of reporting of births and deaths within the HMIS system, we did not attempt to modify the existing HMIS recording system for births and deaths. The HMIS officers visited each health facility every month to extract these data from the health facility records and transfer them to the research team at the National Statistical Office. Data collection began in January 2010 and continued through December 2011. The basis of this RMM method is the tautology that the true number of events (births or under-five deaths) in a period is equal to the number of events recorded divided by the proportion of all events that were reported. The number of events recorded is known, but the proportion is not. In the case of births, we estimate this proportion as the proportion of births in the past 2 years preceding the survey reported as occurring in a health facility for each district in the 2010 Demographic and Health Survey. However, the 2010 DHS did not record place of death. To apply the method, a question on place of death was included in the full birth history module of a mortality survey conducted in the two districts in late 2011 and early 2012. The objectives of this survey were twofold: to provide the needed proportion of deaths occurring in facilities and to provide ‘gold-standard’ estimates of child mortality against which to assess the performance of this and other RMM methods tested in the two districts. The ‘gold-standard’ survey sampled 12 000 households in each of the two RMM districts. Data were collected between 24 October 2011 and 17 February 2012. We used the 2008 population census frame to select the primary sampling units or enumeration areas (EA) for the survey, with probability proportional to size. Households were selected at a second sampling stage after a complete update of the list of households in each selected EA was conducted. We stratified the sample by district and applied sampling weights during analysis to ensure the representativeness of the results. Interviews were conducted with all women aged 15–49 to obtain a full birth history, that is, the date of birth, survival status, and for children who had died, age at death for each live birth the woman had ever had in her lifetime. We used these data to develop estimates of under-five mortality by dividing under-five deaths by births for four overlapping 12-month periods beginning in January, April, July and October 2010. We computed corresponding sampling errors using the jackknife resampling method and derived 95% confidence intervals (Lohr 1999). Interviewers also asked each woman who reported a child death where the death occurred, with response options of ‘home’, ‘health facility’ or ‘other’. The category ‘other’ included events that occur outside the home and a health facility, for example when a child died outside the home while being sent to a health facility. Two clerks entered the data independently; discrepancies were reconciled through reference to the original survey forms. We used CSPro 4.1 for data entry and STATA 12.1 for further cleaning and analysis. Full details of the survey methods and quality control mechanisms are included in Appendix S4. We applied the average proportions of births and deaths reported in the surveys to have occurred in health facilities in the years 2009 and 2010 for births and in the years 2010 and 2011 for deaths to the health facility data on births and deaths to estimate the annual number of births and under-five deaths in each district. We used the adjusted numbers of events to compute under-five mortality rates by dividing the total estimated number of under-five deaths in a 12-month period by the total estimated number of births in the same period. These rates were then compared with the direct rates calculated from the gold-standard household mortality survey by calculating the ratios of the two rates. Ethical clearance for the study, including the gold-standard mortality survey, was obtained from the Johns Hopkins School of Public Health’s Institutional Review Board and the Malawi National Health and Science Research Committee.