A quasi-experimental assessment of the effectiveness of the Community Health Strategy on health outcomes in Kenya

Background: Despite focused health policies and reform agenda, Kenya has challenges in improving households’ situation in poverty and ill health; interventions to address the Millennium Development Goals in maternal and child health, such as focused antenatal care and immunization of children, are yet to achieve success. Research has shown that addressing the demand side is critical in improving health outcomes. This paper presents a model for health systems performance improvement using a strategy that bridges the interface between the community and the health system. Methods. The study employed quasi-experimental design, using pre- and post-intervention surveys in intervention and control sites. The intervention was the implementation of all components of the Kenyan Community Health Strategy, guided by policy. The two year intervention (2011 and 2012) saw the strategy introduced to selected district health management teams, service providers, and communities through a series of three-day training workshops that were held three times during the intervention period. Baseline and endline surveys were conducted in intervention and control sites where community unit assessment was undertaken to determine the status of health service utilization before and after the intervention. A community health unit consists of 1000 households, a population of about 5000, served by trained community health workers, each supporting about 20 to 50 households. Data was organized and analyzed using Excel, SPSS, Epi info, Stata Cal, and SAS. Results: A number of health indicators, such as health facility delivery, antenatal care, water treatment, latrine use, and insecticide treated nets, improved in the intervention sites compared to non-interventions sites. The difference between intervention and control sites was statistically significant (p<0.0001) for antenatal care, health facility delivery, water treatment, latrine use, use of insecticide treated nets, presence of clinic card, and measles vaccination. Degree of improvement across the various indicators measured differed by socio-demographic contexts. The changes were greatest in the rural agrarian sites, compared to peri-urban and nomadic sites. Conclusion: The study showed that most of the components of the strategy were implemented and sustained in different socio-demographic contexts, while participatory community planning based on household information drives improvement of health indicators. © 2014 Olayo et al; licensee BioMed Central Ltd. The study was a quasi-experimental design, using pre- and post-intervention surveys, in intervention and control sites. The intervention in this study was based on implementation of all components of the Community Health Strategy as per the policy implementation guidelines. The study engaged the consumers, policy makers, community and the health system managers in the processes of research to improve health service utilization. A stakeholder analysis was done to ensure the participation of all stakeholders and have a consensus on the study framework and methodology. It involved the participation of health management teams both at the province and district levels, the community and local government administration. The study sites were identified and selected during the stakeholder analysis process. The selection of the intervention sites was based on the willingness of the communities to participate. In comparison, a control matched site was identified for assessments. For the control sites, factors such as geographical location, service coverage, including level of health facility, was considered. Baseline surveys were conducted to confirm statistical congruence between intervention and control sites in terms of socio-demographic factors. A total of eight community health units and eight health dispensaries were involved as intervention sites. The rural agrarian study site consisted of four community health units and four health facilities, while peri-urban had two community health units and two health facilities, and two community units and health facilities for the nomadic site. A community health unit was defined by the population coverage of 5000 population and with about 1000 households. An initial pre-intervention assessment was conducted in early 2010 while post-intervention observation took place two years later, in 2012. The study took two years and was introduced to the selected health management teams at the district level, service providers, and the community. For consistency in the implementation process, three day trainings were held in three phases during the intervention period. The first phase involved building the capacity of the implementers with the necessary skills to implement the intervention. The research team members participated in all trainings together with the district health team members to ensure consistency in implementation. The conceptual framework in Figure ​Figure11 demonstrates the linkage between the community and the health system in governance, management, and service provision leading to health outcomes. The key elements of the intervention package included (see Figure ​Figure11): Logical framework of the complex intervention Adapted from Kaseje et al, 2011 • The formation of committees at the community and health facility levels as governance/linkage structures. • The identification and training of community health workers to support households in health improvement initiatives, as well as to maintain the village register, and facilitate dialogue at the household level. • The community health workers are lay volunteer health workers covering 20 to 50 households within their own villages where they reside. They volunteer their services to work within their neighborhoods. The key elements of the intervention also involved the identification, training and deployment of community health extension workers (CHEWs) for each community unit as the facilitator of dialogue at the community level and supporter of CHWs. They were also the maintainer of the Community Based Health Information System. The CHEW is responsible for supervision of the CHWs within a sub-location. CHEWs are professional health workers employed by the health system, living and working in the community. The process also involved the establishment of village registers of all households to provide community based information, including all health status aspects targeted for improvement. The information collected in the household registers was updated every six months by the community health worker to monitor change in health seeking behavior among the household members. The information from the registers was analyzed and displayed on chalk boards within the sub-location. This would lead to timeliness of analysis, dissemination, and utilization of Health Management Information System (HMIS) data. Once collected at the sub-location level, reports were submitted to the district level for electronic processing. The intervention facilitated manual analysis of relevant health facility data for posting on chalk boards at the sub-location level. Client satisfaction interviews using questionnaires were conducted and analyzed every six months to gauge the level of service satisfaction. Dialogue sessions were held based on data from the community and health facilities depicting the current situation regarding elements targeted for improvement. The dialogue sessions were held on a monthly and quarterly basis at household and community levels, respectively, and every six months at a health facility and the sub-district levels. The dialogue sessions were facilitated by CHWs during home visits and by CHEWs at general community meetings, while the health facility staff facilitated dialogue at the management committee meetings and the District Health Management Teams (DHMTs) facilitated at sub-district health stakeholder forums. At this level timing was based on the district reporting cycle. The dialogue process was attended by managers, service providers, and community representatives from community units representing defined constituencies in their community. The dialogue process involved displaying the data from the health facilities and from the community chalkboard to clearly depict the current situation in the community. This was then followed by discussion towards consensus building regarding the data presented and what was not acceptable and what needed improvement. Action towards improvement was agreed on and a plan of action developed, with targets to be achieved before the next dialogue session. Since the sessions at the community and sub-district levels were as large as 50 people or more, the action planning stage of the process was undertaken in groups of 8 to 12 participants (usually the community health committee). Depending on the level, the timeframe for the dialogue session varied from about an hour at the community level and much longer at the sub-district level. The dialogue process took place at household, community unit, health centre, and sub-district levels, based on issues emerging from the data gathered and analyzed at every level. The data utilized for dialogue came from household registers interpreted by CHWs, community unit summaries prepared by CHEWs, and quarterly reports. The time frame was flexible based on level and issues for discussion. The participation also varied with the level. The participants increased with the level, from household to sub-district (Figure ​(Figure22). Dialogue to Drive Planning, Action Adapted from: F et al, 2011 Ethical review was obtained at four levels. The study protocol was first presented to the Great Lakes University of Kisumu Ethics Review Board, which is the lead research institute, and later to Moi University Ethics Review Board. Both Ethics Review Boards granted the clearance for the study to be undertaken. Further permission and clearance was obtained from the Kenya Ministry of Health (then MOPHS) through the DHMTs within the study districts of Butere, Mumias, Kisumu, and Garissa. Informed consent was obtained from individuals who participated in the face-to-face interviews after having been appropriately informed of the purpose of the study. Participation in the study was voluntary and there were no incentives granted in return, except to the extent that the process led to improvement in health indicators within intervention sites. Confidentiality of data was maintained in several ways. Firstly, data collection was done by research assistants who had been trained on code of conduct and confidentiality of data. The research assistants signed a code of conduct and terms of reference stipulating the standards expected. Data processing into electronic versions was also done by trained data clerks who had signed terms of reference contracts with clear stipulation of expected standards of confidentiality. Both electronic and manual data were kept under lock and key by a senior research officer and supervision of the Principal Investigator (PI). The PI has the sole responsibility of approving access to the data so stored. The study was undertaken in four units in rural agrarian, two in peri-urban, and two in nomadic sites, which were purposively selected from three different contexts (social, economic, and ecological) in Kenya. The four rural agrarian Community Health Units (CHUs) included in the study were situated in Butere district, Western Kenya while peri-urban and nomadic districts were situated in Kisumu and Garissa, respectively. All selected CHUs were part of the Community Health Strategy scale-up partnership districts between the Ministry of Public Health and Sanitation and the Great Lakes University of Kisumu (GLUK). They comprised CHUs that were in the process of being formed and required to go through the complete cycle of establishment, operation, and sustainability. In the Kenyan Community Health Strategy, local populations of 5 000 were served by a community health unit served by 30 CHWs, each providing health services to 20 to 50 households. However, CHWs in the peri-urban area served up to 100 households due to high density of households. Study sites were purposively selected based on readiness to launch the Community Health Strategy. The control sites were selected from neighboring districts, which were matched by geographical location, ethnicity, means of earning livelihoods and economic status. Four, two, and two health facilities were selected in Butere, Nyalenda, and Garissa, respectively. The same numbers of health facilities were selected in the respective control sites. All CHUs served by these health facilities were included, as were all households within the selected units. For baseline and endline surveys, a 20% sample was randomly selected for the surveys in both control and intervention sites. Data for assessment was collected on the implementation level of elements of the Community Health Strategy at the unit and health service utilization level. This was done on a biannual basis and comprised collection of data on: 1. Functionality of community health committee, based on number of training days, frequency of meetings verified by a copy of minutes, representation on link health facility management committee; 2. Functionality of the community health workers, based on number of training days, frequency of CHW meetings, village representation, attendance at CHW meetings, and frequency of household visits. 3. CHEW support to the CHU, measured by the number of supportive visits to CHWs, and number of refresher trainings for CHWs. 4. Community based health information system functionality, measured by the coverage of households registered in the unit, frequency of household register updates, and number of monthly chalk board updates. 5. Frequency of community dialogue days held, presence of reports, and attendance level at the dialogue days. 6. Frequency of community action days conducted by the unit, presence of reports, and attendance level at the action days. Health services utilization levels were assessed at the beginning and end of the study through community surveys. The community health unit assessment was undertaken by trained research assistants only in the intervention sites every six months. Data on functionality of community health units was captured manually through interviews carried out by trained research assistants. Baseline and endline surveys were conducted before and after the intervention to determine the status of health service utilization. These surveys were conducted in both the study and control sites. A baseline tool was developed to cover household demographic characteristics, individual health services utilization, morbidity among household members, and mortality in the household over the six months preceding the survey. Indicators measured included health facility delivery, antenatal care, presence of clinic cards, immunization coverage, vitamin A supplement, use of modern family planning methods, use of treated nets, water treatment, latrine use, and food availability. The tool was pre-tested to ensure validity and reliability of the data that would be collected. It was administered by trained research assistants who were supervised by senior researchers. Face-to-face administration of the tool to household heads took on average 20 minutes. Analysis was undertaken using five software (Excel, SPSS, Epi info, Stata Cal, SAS). The first level of data analysis was undertaken of baseline and endline data collected in October 2010 and December 2012, respectively, to describe the intervention and outcome variables for each of the study sites. The comparison analysis was done both for individual CHUs and aggregated by districts, then a district by district comparison was done. Combined data from all sites was also analyzed descriptively. Bivariate analysis was undertaken to determine relationships between the Community Health Strategy intervention and health outcomes by comparing baseline and endline outcome variables in intervention and control sites. Chi square test was used to determine the significance of differences in dependent outcome proportions (pre-post, and intervention versus control), using the Community Health Strategy elements as independent variables. A p-value of <0.05 was considered statistically significant. A binary logistic regression analysis was used to assess the effect of Community Health Strategy on health outcomes. In the model, we created binary variables; study arm (intervention=1, control =0), post (endline survey=1, baseline survey=0), and an interaction factor coded as 1 (if study arm=1 and endline=1) and 0 otherwise. These were added in the model as predictor variables while the binary health outcome variables included: attending antenatal care four times or more, health facility delivery, and availability of immunization card, presence of a latrine, use of insecticide treated nets, use of treated water, pentavalent 1, pentavalent 3, and availability of food in the household. Each outcome had its own regression model. In order to interpret the results from this regression model, we considered that we were testing the null hypothesis that differences in proportions of health outcomes in the baseline and endline surveys in control sites was equal to differences in the baseline and endline surveys in the intervention sites. The interaction factor helped us to control for those households that participated in both baseline and endline surveys. To test if the intervention worked, we tested the coefficient of interaction factor in zero (which is the odds ratio of the co-efficient of the interaction factors of those who were in the study arm and in the post test survey). If the odds ratio and 95% confidence interval (CI) of this factor was >1 then the intervention worked, if OR<1 then control did better than intervention sites, if the interval was 0 then there was no effect. The comparative pooled analysis method was used to combine data from the six sites. A Poisson regression model was used to assess the effect of Community Health Unit components on the health service utilization outcomes. The Poisson regression model was used because the outcome of interest was considered as count data and the data was analyzed on population level rather than household level.