Verbal autopsy interpretation: A comparative analysis of the InterVA model versus physician review in determining causes of death in the Nairobi DSS

Background: Developing countries generally lack complete vital registration systems that can produce cause of death information for health planning in their populations. As an alternative, verbal autopsy (VA) – the process of interviewing family members or caregivers on the circumstances leading to death – is often used by Demographic Surveillance Systems to generate cause of death data. Physician review (PR) is the most common method of interpreting VA, but this method is a time- and resource-intensive process and is liable to produce inconsistent results. The aim of this paper is to explore how a computer-based probabilistic model, InterVA, performs in comparison with PR in interpreting VA data in the Nairobi Urban Health and Demographic Surveillance System (NUHDSS).Methods: Between August 2002 and December 2008, a total of 1,823 VA interviews were reviewed by physicians in the NUHDSS. Data on these interviews were entered into the InterVA model for interpretation. Cause-specific mortality fractions were then derived from the cause of death data generated by the physicians and by the model. We then estimated the level of agreement between both methods using Kappa statistics.Results: The level of agreement between individual causes of death assigned by both methods was only 35% (κ = 0.27, 95% CI: 0.25 – 0.30). However, the patterns of mortality as determined by both methods showed a high burden of infectious diseases, including HIV/AIDS, tuberculosis, and pneumonia, in the study population. These mortality patterns are consistent with existing knowledge on the burden of disease in underdeveloped communities in Africa.Conclusions: The InterVA model showed promising results as a community-level tool for generating cause of death data from VAs. We recommend further refinement to the model, its adaptation to suit local contexts, and its continued validation with more extensive data from different settings. © 2010 Oti and Kyobutungi; licensee BioMed Central Ltd. Since 2002, the African Population & Health Research Center (APHRC) has been operating the NUHDSS. The NUHDSS covers the two urban informal settlements (slums) of Korogocho and Viwandani, both located about 5 to 10 kilometers from Nairobi, the capital city of Kenya. Viwandani and Korogocho each occupy an area of 0.45 and 0.52 km2, respectively, and are inhabited by about 60,000 people from more than 15 ethnic groups. The population in Viwandani is mainly comprised of labor migrants working in the neighboring industrial area, while that of Korogocho is mainly comprised of long-term settlers engaged in the informal sector. These slum settlements, like most others in Nairobi, are characterized by relatively high crime rates, drug and alcohol abuse, risky sexual behaviors, high unemployment rates, poor access to health facilities, low school participation, and extreme poverty compared to other urban residents as well as their rural counterparts [31]. Data on individual and household core demographic events (birth, death, in-migration, and out-migration) in the two slums are collected at four-month intervals – also known as data collection rounds. In addition to the routine data collection rounds, the DSS also integrates the VA process for COD ascertainment. Deaths are usually identified during the DSS data collection rounds by trained field interviewers who complete a one-page death registration form (DRF) and then inform their field supervisors about the deaths. Supervisors are experienced field interviewers who have a minimum qualification of a bachelor’s degree. They conduct the VA interviews using a VA questionnaire developed in conjunction with other INDEPTH sites. This questionnaire has two formats: one for deaths of children less than 5 years of age and the other for deaths of persons 5 years and older. The latter has an additional section on maternal deaths. The questionnaire covers the background characteristics of the deceased and the respondent as well as structured filter questions on specific signs and symptoms experienced by the deceased up to the point of death. There is also an open section that allows for recording of a narrative account of the events leading to the death. On average, it takes about 30-45 minutes to administer the VA questionnaire. Before a VA interview is conducted, the field supervisor visits the household in his/her zone where a death has occurred as soon as he/she learns of the event and consoles the bereaved family. He then assesses the situation and decides whether the timing is appropriate to conduct the interview. If it is not appropriate, he/she makes an appointment with the family to return at an agreed later date – usually three to four weeks later. However, VA interviews may be conducted as long as six months after death due to operational reasons. At the first visit, the field supervisor identifies a “credible respondent” – usually a spouse or relative – who will participate in the interview. If the deceased is a child, the preferred credible respondent is usually a parent. Several revisits may be made to the household until a credible respondent is identified. After five such visits or if it is established that the remaining household members are no longer residents of the area, a credible neighbor is interviewed if he/she is willing. Otherwise, the verbal autopsy is coded as missing, and no cause of death is assigned to such cases. All completed VA questionnaires are collated and sent to three local physicians for interpretation. At least one of the three physicians is a full-time researcher employed with APHRC, while the other two are consultants or medical officers in public or private practice who review VA data on a contractual basis. Each physician independently reviews all the VA questionnaires and assigns a single COD based on ICD-10. The complete ICD-10 list has 12,420 unique codes for diseases, signs, symptoms, abnormal findings, complaints, social circumstances, and external causes of injury [32]. Hence, for practical purposes, the physicians use an abridged version of the ICD-10 list modified in such a way that uncommon causes of death in the study area are collapsed into broader categories. This modified list has 60 codes for possible COD (see additional file 1). If two of the assigned COD for each VA questionnaire are identical, this is taken as the final COD for the deceased. However, if all three of the assigned COD are different, the physicians hold a consensus meeting and review the case. In cases where consensus is not reached at these meetings, the COD is classified as “indeterminate.” The InterVA model is a probabilistic model based on Bayes’ theorem that seeks to define the probability of a cause (C) given the presence of a particular indicator (I), represented as P(C|I). This probability can be stated as: where P(!C) is the probability of not (C). Therefore, for a set of VA symptom-level data or indicators (I1…In) and for each possible cause of death resulting from these indicators (C1…Cm), there is an associated indicator Ij and cause Ck, whose probability of occurrence at population level can be determined. For each case, therefore, the probability of Ck is initially the value found among all deaths in total, which gives the cause-specific mortality fraction. For each case and each applicable indicator, however, the above theorem can modify the probability of Ck. Thus, the VA model adjusts the probability of each likely cause according to a matrix of P((I1…In)|(C1…Cm)) and then produces a summary listing of as many as three possible causes and their corresponding likelihood values [28]. Full details of the InterVA model and how it was developed based on the above theorem have been described in previous studies [27-29]. The model is run using computer software -Visual FoxPro – that provides a user interface into which a set of 100 indicators must be entered for each VA case in order for the model to generate a COD. These indicators are basically specific information comprising reported symptoms, signs, and medical history that need to be extracted from completed VA questionnaires. Some examples of the required indicators include: “Any difficulty in breathing?” “Any weight loss?” “Any coughing with blood?” Thus, when the model is run on these indicators, it automatically generates a listing of any of 30 probable COD for each verbal autopsy case (see additional file 1 for COD listing). A maximum of three probable causes of death and their corresponding likelihoods (in percentages) are presented in the list. Additionally, the InterVA model has a built-in facility to adjust for the prevalence of malaria and HIV/AIDS in any setting such that before running the model, the prevalence of HIV/AIDS and malaria in the study population can be set as high or low. This was introduced during a process of refining the original model to address underlying conceptual issues of VA data collection and interpretation. The Delphi technique using a panel of experts was utilized to develop consensus on key conceptual issues of cause of death classification and VA usage, including adjustment for large variations in the prevalence of malaria and HIV/AIDS at the population level between regions. This adjustment significantly improved the performance of the model and increased the model’s potential to be applied in different settings [27,28]. Details of the methods and how the built-in facility was developed are beyond the scope of this paper. For our study, we set the prevalence of malaria to be low and that of HIV/AIDS to be high. Previous research in our study population has demonstrated that the prevalence of malaria within this population is less than 0.5% [33], while HIV/AIDS prevalence is as high as 12.4% [APHRC 2008, unpublished data]. Between August 2002 to December 2008, 1,823 VA questionnaires were reviewed by physicians who assigned a COD to each case. The required indicators from each of these questionnaires were extracted and entered into the InterVA model to automatically generate COD. There are a total of 60 possible causes of death assigned by physicians, while the InterVA model only assigns 27 causes (see additional file 1). Therefore, to allow for meaningful comparison between physicians and the model, we re-categorized all causes in both methods into 14 main groups of causes for two reasons. First, we took this step to have comparable cause of death categories between both methods being analyzed. Where possible, we retained the categories common to both methods. For instance, malaria and meningitis, which are common to both physician review and InterVA, were retained as stand-alone causes. In cases where there were no direct correlates, we had to collapse and/or re-categorize the causes of death into cause groups to match each other in a broad sense. For instance, the InterVA model has only one broad category of maternity-related deaths representing all types of pregnancy-related deaths. However, the physicians coded causes such as eclampsia and ante-partum and post-partum hemorrhage. Such causes were therefore recoded into one broad category of maternity-related deaths so we could compare with the corresponding InterVA category. Frequently occurring conditions, such as pulmonary tuberculosis, HIV/AIDS, and pneumonia, were left as stand-alone causes. Second, it was more important to us that the model and the physicians arrived at broad agreement in identifying cause of death groups with the greatest public health importance at population level, rather than individual-level causes. Hence, causes such as kidney disease and cancers were recoded as chronic diseases, while causes such as rabies, tetanus, and typhoid were grouped into other acute/infectious diseases. We then determined the cause-specific mortality fractions (CSMF) of using the InterVA model and physician review. We conducted our analysis for the general population and by two main age groups: children aged less than 5 years and for adults aged 18 years and older. While it is possible to conduct the analysis across various age categories, we decided to focus on the under-5 and adult deaths due to high levels of mortality in these age groups from preventable conditions such as diarrheal disease and HIV/AIDS, respectively [34]. Such preventable conditions are of great public health significance, especially in developing countries. Thereafter, we estimated the level of agreement between InterVA and physician-assigned COD using Kappa statistics. All analyses were carried out using STATA version 10 statistical software. In all our analyses, we only considered the most probable COD assigned by the model rather than all three possible causes. This is because the COD assigned by the physicians included only a single cause of death.