Monitoring child mortality through community health worker reporting of births and deaths in Malawi: Validation against a household mortality survey

N/A

Background: The rate of decline in child mortality is too slow in most African countries to achieve the Millennium Development Goal of reducing under-five mortality by two-thirds between 1990 and 2015. Effective strategies to monitor child mortality are needed where accurate vital registration data are lacking to help governments assess and report on progress in child survival. We present results from a test of a mortality monitoring approach based on recording of births and deaths by specially trained community health workers (CHWs) in Malawi. Methods and Findings: Government-employed community health workers in Malawi are responsible for maintaining a Village Health Register, in which they record births and deaths that occur in their catchment area. We expanded on this system to provide additional training, supervision and incentives. We tested the equivalence between child mortality rates obtained from data on births and deaths collected by 160 randomly-selected and trained CHWs over twenty months in two districts to those computed through a standard household mortality survey. CHW reports produced an under-five mortality rate that was 84% (95%CI: [0.71,1.00]) of the household survey mortality rate and statistically equivalent to it. However, CHW data consistently underestimated under-five mortality, with levels of under-estimation increasing over time. Under-five deaths were more likely to be missed than births. Neonatal and infant deaths were more likely to be missed than older deaths. Conclusion: This first test of the accuracy and completeness of vital events data reported by CHWs in Malawi as a strategy for monitoring child mortality shows promising results but underestimated child mortality and was not stable over the four periods assessed. Given the Malawi government’s commitment to strengthen its vital registration system, we are working with the Ministry of Health to implement a revised version of the approach that provides increased support to CHWs. © 2014 Amouzou et al.

Ethical clearance for the project was obtained in the United States from the Johns Hopkins Bloomberg School of Public Health (JHSPH)’s Institutional Review Board, and in Malawi from the National Health Sciences Research Committee. For the household survey, oral informed consent was obtained from each participant. The consent forms were translated into the local language, “Chichewa”. The IRB at JHSPH waived the need for written consent from the study participants given the low literacy of the population under study. Approval letters are available upon request. Malawi is a poor country in southern Africa with a population of 13.1 million according to the 2008 population census[13]. It is among countries with the poorest health indicators in the region. The 2010 Demographic and Health Survey reported an under-five mortality rate of 112, a maternal mortality ratio of 675 per 100,000 live births, and high fertility with an average number of children per woman of 5.7[14]. Malawi has one of the highest rates of HIV/AIDS prevalence in Africa, estimated in 2010 to be 10.6% among adults. Despite this high burden, Malawi has made good progress in child survival in recent years, and is one of few countries in sub-Saharan Africa reported to be on track to achieving MDG 4, with an annual rate of reduction in under-five mortality of 5.6%[1], [3]. The country has made a commitment to improving health indicators through the adoption of policies enabling implementation of high-impact interventions in maternal, newborn and child health and HIV/AIDS. In 2004, Malawi adopted a Health Sector Wide Approach (SWAp), with a five-year joint Program of Work (2004–2009) for delivery of an Essential Health Package supported by multiple donors. This resulted in the development of a Road Map for Accelerating Attainment of the MDGs on Maternal and Newborn Health in 2005 and the adoption of the Integrated Management of Childhood Illness Strategy (IMCI) for Accelerated Child Survival and Development (ACSD) in 2006. In addition to the IMCI strategy, Malawi has also adopted an integrated community case management (CCM) strategy to increase access to correct treatment for pneumonia, malaria and diarrhea. A key element of this strategy is the Health Surveillance Assistant (HSA), a level of personnel with about 10 years of education and 10 weeks of basic health training. HSAs are allocated to defined geographic areas of approximately 1,000 population each. CCM is delivered by CCM-trained HSAs who also conduct preventive care and community monitoring[15]. With support from the Global Fund, in 2008 the country doubled the number of HSAs from about 5,500 to 11,000, with the goal of reaching a ratio of one CHW per 1,000 population nationally[15]. The expansion of the HSA cadre provided a unique opportunity to test an RMM approach relying on these community health workers for reporting of pregnancies, births and deaths for monitoring of maternal and child mortality. Almost all districts in the country are covered by HSAs, and if this cadre is maintained in a sustainable way, it could represent an important asset for the development of a viable vital registration system in the country. We tested the approach in the districts of Balaka and Salima. District selection criteria included high under-five mortality, high fertility, easy access for the study team, full coverage of HSAs deployed, and average population size based on the distribution of district population size across the country. Balaka is located in the southern region of the country, with a population of 316,748 according to the 2008 population census. Salima is in the central region, with a population of 340,327. According to the 2010 DHS, Balaka and Salima have high under-five mortality at 125 and 150 deaths per 1000 live births, and high fertility at 6.0 and 6.6 children per woman, respectively[14]. In 2010, a total of 280 HSAs were deployed and working in Balaka, and 344 in Salima. HSAs are recruited after completion of 10 years of education (junior certificate level). They receive basic health service training of ten weeks’ duration, and are then assigned to a specific catchment area in a district[15]–[17]. HSAs are not necessarily drawn from among community members, although they are required to reside in their catchment area after deployment. About 60% of HSAs in 2008 were reported by the Ministry of Health to be male. HSAs report to the nearest health center and are supervised by Environmental Health Officers (EHO), Community Health Nurses (CHN) and Senior HSAs (HSAs who have spent at least two years at their post). HSAs monitor an average population of 1000 people, and are responsible for completing government-provided Village Health Registers (VHRs). They are supposed to complete the VHR by first conducting a complete enumeration of their catchment area population, and then routinely recording pregnancies, births, deaths, and other information including antenatal care visits, children’s immunization status, growth monitoring, and the status of water and sanitation facilities. The HSAs are supposed to develop a weekly work plan that includes household visits, inspection of public facilities, and the conduct of outreach clinics and village feedback meetings. The RMM project strengthened the performance of the monitoring tasks by providing supplemental training to the HSAs, along with incentives and increased supervision. We started the RMM work by conducting a formative research study using qualitative methods in June, 2009 in the districts of Kasungu and Mangochi. The study had two objectives: 1) to provide the information needed to develop clear and effective procedures for HSAs’ recording of vital events, including initial training, supervision and support; and 2) to define possible alternative strategies for collecting data on vital events in the local context. The formative research allowed us to learn more about HSAs, their duties in terms of identification and reporting of pregnancies, births and deaths, the reporting barriers and challenges, supervision issues and other alternative ways to identify vital events within the community. The formative research showed clearly that use of the VHRs was an appropriate community approach to RMM, and one that was likely to be sustainable and cost-effective in the Malawi context. These conclusions were confirmed in a meeting of Ministry of Health and other stakeholders in October 2009. We then organized and conducted one-day sensitization meetings in Balaka and Salima before the start of actual implementation. Participants included District Health Officers (DHOs), District Environmental Health Officers (DEHO), Traditional Authorities (TA), Members of the District Assembly (DA), representatives of the national MOH and representatives of local NGOs. The objectives of the sensitization meetings were to inform district officials and leaders about the plans for implementation of an activity related to the monitoring of mortality among children under the age of five, and to generate support for the project among district officials and stakeholders. The project received support and encouragement from the district officials and participants at the sensitization meetings. Using an equivalence test[18], we estimated that a sample of at least 80 HSA catchment areas, in each of the two selected RMM districts, was required to reject the hypothesis of non-equivalence between the mortality rate computed from the HSAs’ data and the rate generated from the validation survey with 24,000 households, using 80% power and a margin of error of 20% of the survey mortality rate. To select the catchment areas, we conducted a complete mapping of all HSA catchment areas in each of the two districts, and developed a sampling frame from which 80 catchment areas were randomly selected using a simple random sampling procedure. HSAs assigned to these catchment areas and their supervisors were then identified, trained and supported to complete their VHRs correctly and to extract data on pregnancies, births and deaths from the VHRs every month onto simple summary forms provided to them by the RMM project staff. The training required one day in each district and was led by the regular district HSA trainers. Following the training and before the start of data collection, each HSA was provided with at least one new VHR, a backpack, and a cell phone and airtime. In each district, about 15 HSA supervisors were identified. These individuals were the regular government-mandated HSA supervisors who serve as Environmental Health Officers (EHO) or Assistant Environmental Health Officers under the overall supervision of the District Environmental Health Officer (DEHO). The HSA supervisors also participated in a one-day training on supervision procedures, data recording and extraction from VHRs, and data flow processes. Additionally, each supervisor was required to conduct one-on-one refresher training of each HSA under his/her supervision. A district coordinator was appointed to manage RMM activities. RMM HSAs and their supervisors received quarterly allowances of cellphone airtime. A transportation allowance was provided to each supervisor to facilitate the supervision visits. Data extraction from VHRs started in January 2010 and has continued to date; in this analysis we focus on the vital events recorded through September 2011 before the mortality survey was carried out. As indicated above, HSAs are expected to identify pregnancies, births, and deaths within their catchment areas and to record the information in their VHR as a part of their routine duties. The performance of HSAs participating in the RMM project was monitored and reinforced via increased training, supervision, quarterly review meetings and incentives. The HSAs extract the information from the registers every month using a standard RMM extraction form (Web Annex S1). To familiarize themselves with their community, HSAs began their RMM activities by listing members of each household within their catchment area in the VHR. Each household and family member was assigned a code, represented by a sequence of 11 digits with the zone (1 digit), the district (1 digit), the traditional authority (2 digits), the Group Village Head (2 digits), the household (3 digits), and the household member (2 digits). HSAs provided the completed extraction form to the responsible supervisor, who checked the data and in turn provided the form to the district RMM coordinator. A photocopy was made and sent to NSO. NSO collected the RMM extraction forms every month from the district and maintained a spreadsheet to monitor HSA reporting. A dedicated data editor at NSO checked each form, edited it, and followed up on any errors either by calling the HSA or sending a copy of the form back for correction. Data entry was completed soon after editing using CSPro[19]. Double independent data-entry was performed to minimize errors, and any differences were reconciled through review of the original data. Data were then transferred into STATA 12.0 for analysis[20]. The district RMM coordinator assigned each HSA participating in RMM to a specific supervisor at the health center. The supervisor checked the HSA’s performance in completing the extraction forms and recording the births and deaths in the VHR, and made corrections as needed. The supervisor also provided immediate feedback to the HSA and conducted on-the-spot retraining in RMM procedures. Data review meetings were held regularly during the study period in both districts, with participation by the RMM HSAs, Ministry of Health officials, the District Health Officer, the HMIS officer, and other partners such UNICEF and WHO. During the review meeting, reports were made on progress in HSA reporting of pregnancies, births and deaths, and feedback was provided. HSAs and supervisors were given the opportunity to discuss challenges, issues, and to suggest solutions. NSO used the data review meetings as an opportunity to provide refresher training on RMM procedures and the data flow process, and to reinforce the importance of ensuring data quality in terms of accuracy, reliability, and completeness. Ideally, gold-standard measures of child mortality would rely on vital registration data that provide real-time information on dates of births and deaths. However, accurate and complete vital registration data are not available in Malawi, and it was not feasible to establish such for the purposes of this study. We therefore used direct mortality measurement through a high-quality household survey as our reference standard. The survey collects complete birth history data from women aged (15–49), including the date of birth of each child ever born alive, and the date or age at death for children deceased. In absence of accurate vital statistics, household surveys with full birth histories are the most internationally accepted and widely used approach for collecting required data for direct mortality measurement. We conducted a mortality survey in Balaka and Salima districts from October 2011 through February 2012 to validate the RMM approach. The survey was carried out in 12,000 households in each district, reflecting the sample size required to reject the hypothesis of non-equivalence between the mortality rates derived from the RMM approach and the mortality survey, with 80% power. RMM mortality rates falling within 20% of the validation mortality rates were assumed to be acceptable. The survey used a stratified two-stage cluster sampling procedure, with stratification by district and clusters based on 2008 population census enumeration areas (EAs). In the 2008 census frame, there were 293 EAs in Balaka and 435 EAs in Salima. Given a sample size of 12,000 household in each district, a total of 343 clusters was needed in each district, with 35 randomly-selected households in each cluster. The first stage involved a systematic random sampling of clusters of households, the clusters being EAs in Salima, but EAs or subdivided EAs in Balaka. In Balaka, the total number of EAs in the district was lower than the total required, so we subdivided large EAs, and all clusters were sampled. A full listing of all households in each selected EA was completed to generate an updated sampling frame of households. At the second stage, thirty five households were randomly selected to participate in the interview. A short household questionnaire was used to interview the head of each sampled household to obtain the list of all members of the household and their demographic characteristics such as age and sex. This list allowed identification of women 15–49 eligible for interview. Full birth histories were collected from all women aged 15–49 in each selected household. The quality of the data was assessed thoroughly before the mortality analysis was conducted. Web Annex S2 provides full details on the survey procedures and the results of the data quality assessment, which indicated that the data were of good quality and could support the analysis. Careful training and field supervision of interviewers were used to prevent the types of measurement errors known to occur during the collection of full birth history data. These errors include potential recall bias, errors in age and date declaration and omission of births and deaths. The period of mortality assessment for the current study covers the eighteen months before the survey, and is therefore recent enough to minimize recall errors in event reporting. Data on births and under-five deaths reported by HSAs in the period from January 2010 to September 2011 were included in the analysis. We calculated the number of births and neonatal, infant and under-five deaths for rolling periods of twelve months beginning in January, April, July and September 2010. The number of births and deaths were adjusted for missing reports in specific months by imputing the average number of such events (births or deaths) reported by HSAs across months. We computed neonatal, infant and under-five mortality rates by dividing the number of such deaths reported in a specific period by the total number of births for the same period. These measures correspond to conventional calculations of neonatal and infant mortality, but tend to under-estimate under-five mortality slightly compared to an approach based on life table procedures. The validation analysis was conducted in two steps. We first estimated the level of completeness of the births and deaths reported by HSAs. We calculated the expected numbers of births and under-five deaths for each annual period defined above. Using the full birth history data from the validation survey, we computed crude birth rates for each annual period. Each rate was multiplied by the total population of the RMM HSAs’ catchment areas to estimate the total number of births expected to be reported by HSAs. The expected number of under-five deaths was then estimated by multiplying the expected number of births by the under-five mortality rate computed from the validation survey. The completeness of reporting of births and deaths by HSAs was then calculated as the ratio of the total number of births and under-five deaths reported by HSAs to the expected number based on the validation survey. In the second step, we compared the estimates of mortality rates from the HSA data directly to the rates obtained from the mortality survey. To ensure comparability, mortality rates from the mortality survey were computed using the same procedure used for the HSA data. We computed standard errors of the mortality rates from both datasets using the Jackknife repeated replications procedure[21]. The equivalence of the mortality rate from the HSA data and that of the mortality survey was assessed by computing the ratio of both rates and its standard error using the Delta method[18]. We then computed the 95% confidence interval of the mortality ratio. Based on a tolerance margin of 20% of the survey mortality rate, we rejected the hypothesis of equivalence between the two mortality rates if the upper bound of the 95% confidence interval of the ratio was less than 0.80 (indicating significant under-estimation) or the lower bound was greater than 1.20 (indicating significant overestimation)[18]. All analyses drawing on the mortality survey data were adjusted for sampling weights to take into account the sampling design of the survey. The analysis did not account for possible migration effects on the HSA reporting of the events or any recall error in the household survey.

• amouzou, agbessi
• banda, benjamin
• kachaka, willie
• joos, olga
• kanyuka, mercy
• hill, kenneth
• bryce, jennifer