Relationships between structure, process and outcome to assess quality of integrated chronic disease management in a rural South African setting: applying a structural equation model

Background: South Africa faces a complex dual burden of chronic communicable and non-communicable diseases (NCDs). In response, the Integrated Chronic Disease Management (ICDM) model was initiated in primary health care (PHC) facilities in 2011 to leverage the HIV/ART programme to scale-up services for NCDs, achieve optimal patient health outcomes and improve the quality of medical care. However, little is known about the quality of care in the ICDM model. The objectives of this study were to: i) assess patients’ and operational managers’ satisfaction with the dimensions of ICDM services; and ii) evaluate the quality of care in the ICDM model using Avedis Donabedian’s theory of relationships between structure (resources), process (clinical activities) and outcome (desired result of healthcare) constructs as a measure of quality of care. Methods: A cross-sectional study was conducted in 2013 in seven PHC facilities in the Bushbuckridge municipality of Mpumalanga Province, north-east South Africa – an area underpinned by a robust Health and Demographic Surveillance System (HDSS). The patient satisfaction questionnaire (PSQ-18), with measures reflecting structure/process/outcome (SPO) constructs, was adapted and administered to 435 chronic disease patients and the operational managers of all seven PHC facilities. The adapted questionnaire contained 17 dimensions of care, including eight dimensions identified as priority areas in the ICDM model – critical drugs, equipment, referral, defaulter tracing, prepacking of medicines, clinic appointments, waiting time, and coherence. A structural equation model was fit to operationalise Donabedian’s theory, using unidirectional, mediation, and reciprocal pathways. Results: The mediation pathway showed that the relationships between structure, process and outcome represented quality systems in the ICDM model. Structure correlated with process (0.40) and outcome (0.75). Given structure, process correlated with outcome (0.88). Of the 17 dimensions of care in the ICDM model, three structure (equipment, critical drugs, accessibility), three process (professionalism, friendliness and attendance to patients) and three outcome (competence, confidence and coherence) dimensions reflected their intended constructs. Conclusion: Of the priority dimensions, referrals, defaulter tracing, prepacking of medicines, appointments, and patient waiting time did not reflect their intended constructs. Donabedian’s theoretical framework can be used to provide evidence of quality systems in the ICDM model. This study was conducted in PHC facilities in the rural Agincourt sub-district situated in the Bushbuckridge municipality, Mpumalanga province, northeast South Africa. At the time this study was conducted, the ICDM model was being implemented in 17 of the 38 PHC facilities in the sub-district. Seven of these 17 health facilities implementing the ICDM model are situated in Agincourt sub-district which covers an area of about 420 km2. The sub-district underpinned by a robust Health and Demographic Surveillance System (HDSS) which has been monitoring the population in these villages for two decades. The population under surveillance in the HDSS as at 1st July 2011 was 115,000 people in 20,000 households in 27 villages [19]. Three referral hospitals are situated 25 km to 45 km from the study setting. The pilot of the ICDM model was commenced in these facilities in June 2011 (field diary of interviews with the operational managers and the sub-district health manager in July 2013), but preceded by two months of pre-implementation preparedness which started in April 2011 [9]. Tsonga is the most widely spoken language in the study area. Having immigrated into South Africa mainly as war refugees in the early- and mid-1980s, one-third of the population in the study site are Mozambicans [19]. In the South African PHC model, the professional nurse is the service provider at the PHC facilities, which is the first point of entry into the public health system. Services provided by the nurses include: maternal and child care, immunization, family planning, treatment of sexually transmitted infections, minor trauma, care for chronic diseases and referrals. Medical doctors visit the PHC facilities at intervals to offer support to the nurses [20]. This was a cross-sectional survey conducted between August and November 2013. It was part of a broader four-year longitudinal study (January 2011 and December 2014), with qualitative and quantitative components, designed to contribute to understanding the effectiveness of the ICDM model in improving the quality of healthcare and technical health outcomes of chronic disease patients. The study population consisted of patients 18 years and above receiving treatment for chronic diseases in the sub-district health facilities. Other study participants included the operational managers (professional nurses-in-charge) of the selected seven PHC facilities in the sub-district. The ICDM model addresses the following chronic diseases: HIV/AIDS, tuberculosis, hypertension, diabetes, chronic obstructive pulmonary disease, asthma, epilepsy and mental health illnesses that are to be managed at the PHC level [9]. Considering the burden of chronic diseases in the study area, patients with markers of chronic diseases for HIV, hypertension and diabetes in the health facilities were included in the study, while those with other chronic diseases were excluded. Patients who had their chronic condition(s) managed five months before the initiation of the ICDM model until the time the study commenced in August 2013 were identified for recruitment. The reason for including patients receiving treatment five months before the ICDM model was implemented was to assess the levels of satisfaction of patients who had received treatment before the implementation of the ICDM model and continued to receive treatment during its implementation in efforts to gauge possible changes in the quality of chronic disease care attributable to the ICDM model. Minors less than 18 years were excluded from the study because they were below the age of autonomy (≥18 years) for judging satisfaction with the quality of services provided in the health facilities. The elderly with reduced capacity for comprehension during informed consent were also excluded from the study. Diminished capacity for comprehension was determined by the inability of prospective patients to comprehend or respond to the information verbally provided by the interviewer during informed consent. Using the subjects-to-variables ratio (minimum of 10 subjects per variable in the study instrument) for estimating sample size for studies utilising factor analysis [21, 22], a sample size of 390 patient respondents was calculated (17 subjects per each of the 23 variables in the study instrument). The minimum sample size of approximately 435 (390/0.9) patients was reached after adjusting for 10% non-response. All the seven operational managers of the PHC facilities, the maximum number possible, were selected because they offered clinical services to the patients and the authors perceived their role as managers of the health facilities critically important to understanding the quality of the ICDM model more than other professional nurses. The study participants were identified through a three-step process (Additional file 1). First, the number of patients recruited at each of the seven health facilities was determined by proportionate sampling. The sampling fraction of 435/3602 (435 represents the desired sample size out of a total of 3602 HIV, hypertension, and diabetes registered patients) was multiplied by the number of these chronic disease patients in each health facility to determine the number of patients to be recruited per facility. Secondly, the patients in each health facility were stratified by HIV, hypertension, and diabetes status in order to get a representative sample of the patients with markers of chronic diseases using a health facility-specific sampling frame. Finally, the numbers of patients specified in step two were recruited for a daily interview until the desired sample size in each clinic was achieved. In this study, we used the multi-scale patient satisfaction questionnaire (PSQ-18) which was developed by Ware et al. [23]. The PSQ-18 comprises 18 items derived from the full-length version (50-item) PSQ-III counterpart [23]. The PSQ-18 assesses multiple dimensions of patient satisfaction and includes general satisfaction; technical quality; interpersonal relations; communication; financial aspect; time spent with health provider; and accessibility and convenience (Additional file 2). The PSQ-18 sub-scales show acceptable reliability and correlate with the sub-scales in the PSQ-III [24]. Furthermore, PSQ-18 is appropriate for use in situations where there is need for brevity [24], as was the case in this study where it was administered to patients leaving the health facility after consultations with the nurses (patient exit interviews). The PSQ-18 instrument is reflective of Donabedian’s SPO constructs and succinctly measures patient satisfaction with dimensions of care for which SPO constructs are intended. The authors are not aware of any study that has used the PSQ-18 as a study instrument to operationalise Avedis Donabedian’s SPO theoretical framework in SSA. Mahomed et al. described the innovative approaches in the HIV programme leveraged for NCDs by the NDoH [25]. From these, the study team consulted with the health facility managers and officers of the Mpumalanga Province Department of Health in selecting eight dimensions of care that patients are able to respond to as a result of their lived experiences with healthcare services in the PHC health facilities. The rationale for this selection was because some aspects of these innovative approaches were functions performed by nurses, laboratory staff and health policy implementers which patients were not privy to. This study compared self-reported satisfaction of the patients and self-reported satisfaction of the operational managers with the dimensions of care listed in the ICDM model using the multi-scale PSQ-18. This is in view of literature depicting views of health care providers differing from users regarding the quality of health care [26]. Responses to statements were scored on a five-point Likert scale ranging from 4 (strongly agree) to 0 (strongly disagree) for positively-phrased statements, and from 4 (strongly disagree) to 0 (strongly agree) for negatively-phrased statements for the purpose of undertaking confirmatory factor analysis and structural equation modeling. Similar to another study in which the PSQ tool was adapted to measure patient satisfaction with pharmacy services [27], this study adapted the PSQ-18 by altering a number of statements to fit the ICDM model. For example, the structure-related statement, “I have easy access to the medical specialists I need,” was changed to the ICDM-process-related dimension, “Health care providers usually refer me to the doctor/hospital when there is need for the doctor to review me – P5” (Additional files 2, 3 and 4). One structure-related (supply of critical medicines) and two process-related (defaulter tracing of patients and prepacking of medicines) variables were included in the adapted questionnaire. One process-related statement in the PSQ-18 was changed from “health care providers act too business-like and impersonal toward me” to “Health care providers are professional in the conduct of their clinical duties”. Regarding the types of outcome constructs (technical and interpersonal) specified by Donabedian, the focus of this study was on the subjective interpersonal outcome. Two outcome statements on “satisfaction with perfect health care” and “dissatisfaction with some care” in the PSQ-18 were changed to the dimension on “satisfaction with coherent integrated chronic disease care” and “dissatisfaction with coherent integrated chronic disease care”, respectively. Two statements around the financial costs of health care (D1 and D2) were dropped during the adaptation of the PSQ-18 (Additional file 3). This is because the government of the Republic of South Africa implements a pro-equity policy, a component of free health care for everyone using the public primary health system [28]. However, transport-related costs were not considered in this study because it is not the responsibility of South Africa’ Department of Health to provide transport for the implementation of the ICDM model. The 17 dimensions of care in the adapted questionnaire are shown in Fig. 1, and details of the adapted PSQ tool used in the current study for patients and operational managers are shown in Additional files 3 and 4, respectively. The 17 dimensions of care for which the structure, process and outcome constructs were intended. *The dimensions in red colour indicate the priority areas in the ICDM model Eight dimensions of care were identified by experts on quality of care in the study team as priority areas for enhancing service efficiency and quality of care: supply of critical medicines, equipment, hospital referral, defaulter tracing, prepacking of medicines, clinic appointments, patient waiting time, and coherence of integrated chronic disease care (Additional files 5 and 6) [9]. This is because these priority areas are components of the tools and systems used in the successful HIV programme which is being leveraged to support or scale-up services for improving the quality of care for NCDs and patients interfaced directly with these areas in the health facilities (Fig. 1). The adapted PSQ tool for the patients was forward translated to Tsonga (the local language) and back-translated to English by two experienced field workers who were blinded to each other. An experienced quantitative field worker was trained on how to administer the adapted PSQ tool. A pilot study was conducted in Cork clinic, a PHC facility situated outside the study site, to assure understanding and correct use of the PSQ tool. Only a few statements had to be rephrased after the pilot study. An important characteristic of the original PSQ-18, which was considered in the adaptation of the study instrument, is the control for Acquiescent Response Set (ARS) – a tendency to agree with statements of opinion regardless of their content [29]. Acquiescent response set is a measurement error, specifically information bias, inherent in surveys assessing satisfaction with medical care. According to Ware et al. [29], there is a need to minimise information bias by assessing ARS in satisfaction surveys. Six variables were phrased in opposite directions, bringing to 23 the total number of variables in the adapted questionnaire (Additional files 3 and 4). These measures are beneficial in detecting skewness toward satisfaction [29] and identifying specific programme areas that respondents are satisfied or dissatisfied with. Having consulted with the professional nurses and received their medicines, the prospective study participants were invited to a (consultation) room designated for patient interviews. Only the interviewer had access to this consultation room. Patients were invited to take part in the satisfaction survey after explaining the purpose of the study. They were assured that there will be no penalty or loss of benefits to which they were entitled to if they chose to not participate in this study or decide to discontinue participation in this study. Written informed consent was obtained from the patients who were willing to participate in the study and interviews were conducted with the patients. The adapted PSQ contained measures reflective of SPO constructs and was used to assess satisfaction of patients and operational managers with the dimensions of integrated chronic disease services. There was no clear division of the statements in the adapted PSQ tool into the respective constructs. However, these statements have been categorised under these constructs in Additional files 3 and 4 for clarification. In order to minimise bias that may result from assessing acquiescent response set, the positive and negative statements did not follow each other in the questionnaire as shown in Additional files 3 and 4. The respondents were judged to be satisfied with the dimensions of care if the total relative frequency was ≥ 50% for “strongly agree” and “agree” responses to positively-phrased statements. Similarly, the respondents were judged to be satisfied with the dimensions of quality of care if the total relative frequency was ≥ 50% for “strongly disagree” and “disagree” responses to negatively-phrased statements. A satisfaction score of at least 50% was considered an average score using a scale of 0% to 100%. The patients and operational managers were scored comparatively on their (dis)satisfaction with the dimensions of care in the ICDM model to measure the first objective of the study. Determining the quality of care in the ICDM model was the second objective of this study which was measured by conducting structural equation modelling (SEM) using the data on patients’ (dis)satisfaction with the dimensions of quality of care in the ICDM model. However, SEM could not be performed with the data collected from the operational managers because of the very small sample size (seven operational managers). The following linear pathways were specified in the SEM: (1) the unidirectional pathway which states that good structure promotes good process and good process in turn promotes good outcome, (2) the mediation pathway which posits states that good structure directly promotes good outcome, good structure promotes good process and good process in turn promotes good outcome; and (3) the reciprocal pathway which hypothesises that good structure promotes good process, good process promotes good outcome and good outcome in turn promotes good process. The last two pathways were examined in this study to explore other linear relationships between SPO constructs other than the unidirectional pathway originally postulated by Donabedian (Fig. 2). Pathways for operationalising Donabedian’s theory in the ICDM model of care in South Africa. a Unidirectional path: Good structure should promote good process and good process in turn should promote good outcome. b Mediation path: Good structure directly promotes good outcome, good structure promotes good process and good process in turn promotes good outcome. c Non-recursive (reciprocal) path: Good stucture promotes good process, good process promotes good outcome and good outcome in turn promotes good process Fitting of the proposed pathways involved a four-step systematic process using patient data. First, a priori identification of the variables for which the SPO constructs were intended was performed by the experts on quality of care on the study team in order to assess the validity of the adapted questionnaire (Additional files 3 and 4). This method was adopted by Kunkel et al. in which a panel of experts categorised variables in a questionnaire into SPO constructs [17]. Secondly, Cronbach’s alpha (range: 0–1), which is a measure of internal consistency, was used to quantify the reliability of the multi-item variables in the adapted PSQ in measuring the SPO constructs. Cronbach’s alpha coefficient of reliability was categorised as excellent (α ≥ 0.9), good (0.7 ≤ α < 0.9), acceptable (0.6 ≤ α < 0.7), poor (0.5 ≤ α < 0.6) and unacceptable (α < 0.5) [30]. Next, the negative statements in the pair of statements phrased in opposite directions were dropped if there was no evidence of ARS. The fit of each construct and its individual items were assessed to remove any of the remaining variables with low coefficient of determination (CD < 0.2). Variables with low CD contribute high levels of error in the structural equation modelling [31]. Thereafter, Confirmatory Factor Analysis (CFA) was conducted to identify and remove the variables that did not load significantly (factor loading < 0.300) onto their intended constructs. The following step used structural equation modelling (SEM) to assess the specified pathways, as used elsewhere [32], in order to determine the relationships between the SPO constructs (Fig. 2). Selection of the final path model was based on the variables that reflected their intended factors (factor loading ≥ 0.300). The Maximum Likelihood for Missing Values (MLMV) technique was used to impute for S5, P1 and P11 variables with 0.5%, 0.25% and 0.25% missing observations, respectively. The MLMV is a technique that handles missing data by estimating a set of parameters that maximise the probability of getting the data that was observed. It is a more superior and preferable method for handling missing data than the more popular multiple imputation [33], which is a simulation-based method that predicts missing values as close as possible to the true ones by replacing missing data with probable values based on other available information [34]. Assessment of the fit of the pathways using MLMV approach was based on two or more of the following fit indices [35]: (i) Relative/normed Chi-squared test statistic is an absolute fit index that assesses the discrepancy between observed and expected covariance matrices. It minimizes the impact of sample size on the model and is derived by dividing the Chi square value by the degrees of freedom (χ 2/df). Although there is no concensus regarding the acceptable ratio for this statistic, values ranging from 2 to 5 are recommended as good fit indices. [31]; (ii) Root Mean Squared Error of Approximation (RMSEA) is another absolute fit index that measures how well a model with optimally chosen parameter estimates fit the population’s covariance matrix - RMSEA value ≤ 0.06 is a good fit; (iii) Comparative Fit Index (CFI) is an incremental fit index that assesses the improvement in fit of the hypothesised model compared with a baseline (null) model, when population covariance is assumed to be zero - (CFI ≥ 0.90 is a good fit); (iv) Tucker-Lewis Index (TLI) is also an incremental fit index that corrects for model complexity by favouring parsimonious models over more complex ones - (TLI ≥ 0.90 is a good fit); and (v) Coefficient of determination (CD) indicates how well data fit a statistical model. We used CD to decide the model that explained the most variability. CD value of 1.00 is a perfect fit. The higher the number of criteria used, the better the fit of the model with the data [31]. Data were entered into Access 2010 and imported into Stata 12.0 (College Station, TX, USA) for statistical analysis. Relative frequencies were used to quantify satisfaction of the patient and operational managers with the dimensions of integrated chronic disease services. At p-value ≤ 0.05, CFA and SEM were used to fit the specified structural path models in order to determine the quality of care in the ICDM model from the patient perspective.