A critical assessment of the WHO responsiveness tool: Lessons from voluntary HIV testing and counselling services in Kenya

Background. Health, fair financing and responsiveness to the user’s needs and expectations are seen as the essential objectives of health systems. Efforts have been made to conceptualise and measure responsiveness as a basis for evaluating the non-health aspects of health systems performance. This study assesses the applicability of the responsiveness tool developed by WHO when applied in the context of voluntary HIV counselling and testing services (VCT) at a district level in Kenya. Methods. A mixed method study was conducted employing a combination of quantitative and qualitative research methods concurrently. The questionnaire proposed by WHO was administered to 328 VCT users and 36 VCT counsellors (health providers). In addition to the questionnaire, qualitative interviews were carried out among a total of 300 participants. Observational field notes were also written. Results. A majority of the health providers and users indicated that the responsiveness elements were very important, e.g. confidentiality and autonomy were regarded by most users and health providers as very important and were also reported as being highly observed in the VCT room. However, the qualitative findings revealed other important aspects related to confidentiality, autonomy and other responsiveness elements that were not captured by the WHO tool. Striking examples were inappropriate location of the VCT centre, limited information provided, language problems, and concern about the quality of counselling. Conclusion. The results indicate that the WHO developed responsiveness elements are relevant and important in measuring the performance of voluntary HIV counselling and testing. However, the tool needs substantial revision in order to capture other important dimensions or perspectives. The findings also confirm the importance of careful assessment and recognition of locally specific aspects when conducting comparative studies on responsiveness of HIV testing services. © 2009 Njeru et al; licensee BioMed Central Ltd. The study was initially intended to use quantitative methods, but the pilot study indicated the need for mixed methods so as to explore other elements of responsiveness that were not captured by the closed questions within the quantitative survey. Mixing quantitative and qualitative methods can be used to add insights likely to be missed when only a single method is used and to increase generalizability to the results [25]. A concurrent nested study design [26] was adapted to enable the researchers to gain broader perspectives of responsiveness, by adding a qualitative open-ended question to the quantitative questionnaire. This type of design enables the collection of quantitative and qualitative data during one phase and it involves interviewing the same persons using different techniques, which in turn could help to identify measurement and methodological problems [26,27]. The study was conducted from October to November 2007 in Malindi district, in Kenya where the EU-funded five year intervention study “REsponse to ACcountable priority setting for Trust in health systems” (REACT) is being conducted [28]. The intervention being applied is an explicit ethical framework for legitimate and fair priority setting, accountability for reasonableness (AFR). The values being focused in the evaluation of the intervention are quality, equity and trust [28]. In addition to Kenya, the REACT research project is ongoing in Tanzania and Zambia. The present paper is a result of a study within the frame of REACT. Malindi district was chosen due to the relative similarity to the other two districts within the REACT project in terms of disease burden, health system and population [28]. HIV control programmes were defined as one of the several evaluation domains of the project due to the high HIV prevalence in the project countries. The adult (15-49) HIV prevalence is between 15%-17% in Malindi [29]. The World health organization developed and validated a questionnaire to measure responsiveness that incorporates the 7 elements indicated in Table ​Table1,1, with varying number of questions related to each element [30]. The present study applied the tool that had been implemented in a previous study (among key informants in 35 countries) [3], but tailored it to address 6 of the 7 elements to fit the study setting. The element ‘access to social support networks during care’ was omitted because the questions within this section were not deemed applicable for VCT. These questions were relevant in the context of inpatient care [4,7]. Elements as defined in the WHO responsiveness concept [4,7] To ensure equivalence of the original version, a bilingual English to Swahili translator with medicine, epidemiology and public health background (who also has an understanding of the local language in Malindi area) was asked to perform a back-translation after the English WHO questionnaire had been translated to Swahili by a professional. Where differences were noted, the issues were discussed among the 2 translators, as well as with Swahili speakers at the study area. Social demographic questions that were included in the questionnaire for the VCT user’s i.e. individuals who have utilized VCT services and for the health-care providers whom we refer to as ‘providers’, captured information on the type of VCT visited, geographical location, sex and age. Marital status was also mapped in the questionnaire for the VCT users. The health-care providers’ questionnaire rated the sub-elements on a 4 point scale ranging from ‘never’ (1) to ‘always’ (4) or ‘very poor’ (1) to ‘very good’ (4). To measure the perceived importance of the elements of responsiveness, the study participants (both users and providers), were asked to indicate how important they felt the WHO elements or aspects were on a scale from 0 (not at all important) to 10 (very important). The questionnaire was administered as an exit questionnaire to the users of VCT to generate responses based on their immediate experience with the facilities. A total of 328 VCT users and 36 health-care providers were interviewed. The qualitative part of the study consisted of an open-ended question that was added to the quantitative questionnaire as well as the use of observations. The open-ended question was added to allow informants to respond in their own words which in turn permit understanding of responsiveness as seen by the informants. The question sought to bring up potential issues of relevance for responsiveness that were not captured by the existing responsiveness tool. Observation was used in this study to add to our informant’s responses. The open question was phrased thus, “in your view are there any other characteristics (other that the ones we have discussed) that you think should be included in a responsive VCT?” A number of probe questions were added to generate more in-depth information on this topic. For example we asked: “What should be done to make VCT services more responsive and to increase its utilization? Probe: Why do you think so? Who should be responsible?” Further probes were formulated depending on the initial responses given. The responses from the users were recorded through hand writing. As the informants felt more comfortable when their responses were not electronically recorded, 4 of the 36 provider’s responses were tape recorded. The survey was carried out among all the VCT counsellors or health-care providers who were available as well as among all the users of VCT services in the 15 VCT existing in the district at the time of study (October – November 2007). Most of the VCT facilities in Malindi are integrated or situated within health facilities where there are many other points of provider-initiated HIV testing and counselling such as maternal and child health clinics, tuberculosis clinics and general outpatient services among others. HIV testing and counselling at the VCT facility is mainly client initiated. Anonymity – as documented in the use of code numbers and mother’s names was practiced in the Malindi VCT facilities to ensure confidentiality. Those who tested HIV positive were provided with further referral following the ministry of health recommended procedure in order to maintain confidentiality. A total of 331 VCT users were asked to participate, out of which 328 accepted to be interviewed for the quantitative part. All the 36 counsellors approached agreed to participate. The study participants were recruited after being informed about the study focus, the voluntary nature of the study and after assurance of confidentiality and anonymity. The inclusion criteria for the ‘providers’ was VCT staff; while the users were individuals above 18 years who had utilized existing VCT service. The open ended question was addressed to a total of 264 out of the 328 users. Of the remaining 64 participants, 39 of them declined to be interviewed further because they were satisfied with the existing VCT services and 25 did not wish to respond further. All the interviews were carried out face-to-face by the first author [MKN] with assistance from trained and experienced local interviewers. Data was obtained through exit interviews which were seen as the best option for this study in order to avoid recall and recognition bias around perceptions, attitudes and experiences encountered by the VCT users. The quantitative section of the interviews lasted between 30 minutes and one hour, while the qualitative interviews (consisting of the open ended question), took another 30 minutes or more. Among the health providers, English was the primary language of communication, with Swahili words occasionally employed where necessary, while English, Swahili and some Giriama (local language) words were used among the VCT users. All interviews were conducted in a private area or a room provided within the VCT vicinities. The collected information was kept in a locked cupboard to ensure confidentiality. In addition to the interviews, individual observational field notes were written on a daily basis. These consisted of:exact locations of the VCT, observed dynamics amongst people using VCT, reception of clients by providers, gestures by users, the VCT infrastructure, sitting arrangements before a VCT session, type of health provider and information, education and communication (IEC) material available at the site. Quantitative data was analyzed using SPSS version 15 for Microsoft Windows. Graphs and tables were produced using Microsoft Excel and Microsoft Word. In accordance with the WHO approach in a previous study [3], performance outcomes were dichotomised into good performance (combining responses very good and good or always and usually) and poor performance (combining responses poor and very poor or never and sometimes). For the user’s data set, ‘yes’ responses were classified as good performance and ‘no’ as poor. The importance question was grouped into very important (combining responses 9 and 10), important (5-8) and less important (1-4). The open-ended responses were mainly translated to English and thematic analysis was employed. Thematic analysis has the following stages that were adapted in the analysis; familiarization with the material, identification of a thematic framework, indexing or coding, mapping and interpretation [31]. The coding process was conducted so as to identify specific pieces of data which were relevant to the responsiveness elements in order to add information. Scientific and ethical approval was obtained from the Kenya Medical Research Institute (KEMRI) and the National Ethical Review Committee (NERC) of Kenya prior to conducting the study (KEMRI SSC number 1273). Written informed consent was obtained from all participants prior to the interviews.