Integrated Child and Family Hub models for detecting and responding to family adversity: protocol for a mixed-methods evaluation in two sites

Introduction Integrated community healthcare Hubs may offer a € one stop shop’ for service users with complex health and social needs, and more efficiently use service resources. Various policy imperatives exist to implement Hub models of care, however, there is a dearth of research specifically evaluating Hubs targeted at families experiencing adversity. To contribute to building this evidence, we propose to co-design, test and evaluate integrated Hub models of care in two Australian community health services in low socioeconomic areas that serve families experiencing adversity: Wyndham Vale in Victoria and Marrickville in New South Wales. Methods and analysis This multisite convergent mixed-methods study will run over three phases to (1) develop the initial Hub programme theory through formative research; (2) test and, then, (3) refine the Hub theory using empirical data. Phase 1 involves co-design of each Hub with caregivers, community members and practitioners. Phase 2 uses caregiver and Hub practitioner surveys at baseline, and 6 and 12 months after Hub implementation, and in-depth interviews at 12 months. Two stakeholder groups will be recruited: caregivers (n=100-200 per site) and Hub practitioners (n=20-30 per site). The intervention is a co-located Hub providing health, social, legal and community services with no comparator. The primary outcomes are caregiver-reported: (i) identification of, (ii) interventions received and/or (iii) referrals received for adversity from Hub practitioners. The study also assesses child, caregiver, practitioner and system outcomes including mental health, parenting, quality of life, care experience and service linkages. Primary and secondary outcomes will be assessed by examining change in proportions/means from baseline to 6 months, from 6 to 12 months and from baseline to 12 months. Service linkages will be analysed using social network analysis. Costs of Hub implementation and a health economics analysis of unmet need will be conducted. Thematic analysis will be employed to analyse qualitative data. Ethics and dissemination Royal Children’s Hospital and Sydney Local Health District ethics committees have approved the study (HREC/62866/RCHM-2020). Participants and stakeholders will receive results through meetings, presentations and publications. Trial registration number ISRCTN55495932. This study is a multisite implementation and evaluation project that employs a convergent mixed-methods design conducted over three research phases displayed in figure 1 below. Quantitative and qualitative data will be collected in parallel and integrated during data analysis and interpretation.36 Evaluation processes and phases. CIMO, context-intervention-mechanism-outcome. The study is underpinned by an interdisciplinary conceptual framework drawn from implementation science, critical realism, improvement science and collaborative co-design paradigms.37–44 Interdisciplinary research strengthens empirical robustness through synthesis of approaches from many disciplines, and has been shown to increase the uptake of research into policy and practice.45 The approach and rationale for the interdisciplinary framework in order of importance for this study are: Figure 1 displays the three study phases based on Pawson and Tilley’s46 realist evaluation framework: (1) Development of the initial Hub programme theory through formative research; (2) Testing of the initial Hub theory using empirical data; and (3) Refinement of the initial Hub theory. The initial Hub programme theory was developed through an intensive formative research phase during which we collected contextual information and co-designed the Hub models with intersectoral (eg, health, social, legal and education) practitioners and caregivers from each local community. This phase occurred from October 2020 to June 2021 in Victoria and January to August 2021 in NSW. First, a historical and current needs assessment of the health, social, legal and community contexts of each site was conducted using qualitative data from online group discussions, individual interviews and meetings with key stakeholders including intersectoral practitioners (ie, from health, social, legal and community sectors) and caregivers. We used CFIR42 to capture and synthesise these data into barriers and enablers across the outer setting, inner setting, individual level, process and intervention components. Second, we reviewed the HHAN programme theories and other relevant realist literature focused on integrated care for children and families experiencing adversity23 30 56–59 to identify key CIMO configurations relevant to the Hubs. HHAN provided a useful starting point because our study builds on the HHAN model of care. For pragmatic reasons, we focus the process and outcome evaluation in this study on three key mechanisms and associated CIMO configurations in the initial programme theory. Namely, (1) trust between caregivers and the service/practitioners, and between practitioners, (2) knowledge acquisition and motivation of practitioners and (3) perceived benefits of collaboration by practitioners. Figure 2 displays the initial realist-informed programme theory. We then formulated the logic model displayed in figure 3 proposing how the Hub models will be investigated and are theorised to contribute to outcomes. The logic model includes how immediate outcomes, that is, increased detection and response to adversity are hypothesised to lead to our intended outcomes, including increased uptake of services and improved child and caregiver mental health and caregiver quality of life. Initial Hub programme theory from a realist perspective. Logic model proposing how the Hub models will be investigated. PDSA, Plan-Do-Study-Act. Third, the Hub models were then developed and refined through a collaborative co-design process with intersectoral practitioners, caregivers and children at each site. The co-design process focused on the client journey through each proposed Hub and the workforce capacities and infrastructure needed to support implementation of the Hub. In Wyndham Vale, the Hub model was co-designed through an intensive 10-week series of co-design workshops and consultations with local caregivers and practitioners using human-centred design processes.37–39 In Marrickville, the Hub model will be co-designed through focus group discussions and workshops employing the Nominal Group Technique consensus method60 to prioritise and develop implementation strategies. Different approaches to co-design are adopted that reflect the capacity, capabilities and preferences for engagement of the research team and co-design partners at each site. The theorised Hub models comprise: We will empirically test the initial programme theory for the Hubs through a mixed-methods process with 6 and 12 months outcomes evaluation. The evaluation aims to assess for whom, how and why the Hub models had an impact (if any) in the two different contexts of Wyndham Vale and Marrickville across the child, caregiver, practitioner and system level outcomes specified in figures 2 and 3, table 1 and ‘Outcomes Assessment’ below. Primary and secondary outcomes *Caregivers with more than one child will respond to questions pertaining to one child in their family based on the child they are most concerned about. †The ASQ-SE is limited by its design as a screening tool that may not be a sensitive outcome measure. The measure is used in this study because it more directly measures mental health and well-being than the ASQ. The study is a mixed-methods repeated measures evaluation that uses caregiver and Hub practitioner surveys at baseline, 6 and 12 months after Hub implementation begins and in-depth interviews at 12 months. We aim to recruit two stakeholder groups: caregivers (n=100–200 per site) and Hub practitioners (n=20–30 per site). Potential caregiver participants are those who: (i) care for a child aged 0–8 years, including women who are pregnant, (ii) access any of the universal or specialist services provided in the Hub, such as general practitioners (GPs), paediatricians, lawyers and (iii) can understand written or spoken English language. The English language criterion is a limitation of our study; however, funding is not available to provide interpreters. Furthermore, culturally and linguistically diverse persons participated in an informed and meaningful way in phone-based or in-person surveys for HHAN (personal communication Professor Eastwood). Given the preventative focus of this study, caregivers will not be screened for adverse experiences as an inclusion criterion. Instead, caregivers will report the frequency of 15 adversity types in the baseline survey. These adversity types are derived from the Parent Engagement Resource61 and include challenges with: social support, finances, housing, employment, family physical health, parent mental health, parenting, child neglect, alcohol and substance use, family relationships, family violence, child abuse, visa and immigration issues, crime issues and discrimination. Potential practitioner participants are those who: (i) work in any of the intersectoral services provided as part of each Hub, including GPs, maternal and child health nurses, paediatricians, allied health professionals, social workers, lawyers, and (ii) can understand written or spoken English language. As this study is testing the Hub models using a non-experimental design, we have not conducted a formal sample size calculation. However, the sample size of 200–400 caregivers is likely to provide rich data on the primary and secondary outcomes. The study is designed to accommodate a range of participants, many of whom may have complex life circumstances.62 Figure 4 presents the caregiver participant study flow. We will recruit caregivers from the waiting rooms of each Hub and through Hub practitioners who will ask their clients’ permission for the research team to contact them. Researchers will approach potential caregiver participants (in the waiting room or on the telephone), show and/or email them an informational video and study information pack (including a written Caregiver Participant Information Consent Form, PICF) and invite them to take part in the study. Participants can provide informed consent verbally on the phone or in-person, or in writing by clicking through the online consent form preceding the survey. Caregivers will be provided with a AUD$25 honorarium for each survey and/or interview they complete. Caregiver participant study flow. PICF, Participant Information Consent Form. We will invite practitioners working in each Hub who have been identified and recruited by the study team or health centre managers. We will send an invitation email to potential practitioner participants with the Practitioner PICF attached before they attend the workforce training. Practitioner participants will provide written (online or hard copy) informed consent prior to completing the baseline survey. The study will assess the child, caregiver, practitioner and systems outcomes summarised in table 1. The Caregiver and Hub practitioner baseline and outcome surveys consisting of the measures outlined in table 1 and available in online supplemental file 1 will be built in Research Electronic Data Capture (REDCap)63 hosted by Murdoch Children’s Research Institute. After completing an informed consent process, caregivers can complete the baseline survey: (i) in paper form, (ii) over the telephone with a researcher or (iii) online. Practitioners will complete the practitioner baseline survey in paper form or online via REDCap. The practitioner survey will be sent towards the end of baseline data collection to minimise the potential impact of the survey on clinical practice prior to the Hub testing period. Caregiver and practitioner participants will be contacted by telephone, email or text message by the research team 6 and 12 months after the Hub implementation begins and invited to complete the follow-up survey. bmjopen-2021-055431supp001.pdf Twelve months after Hub implementation begins, we will conduct realist-informed semi-structured interviews with 18–24 caregivers and 5–10 Hub practitioners per site.64 We will purposively sample participants to capture caregivers from diverse cultural backgrounds, experiences of adversities and varying levels of engagement with intersectoral services. These realist-informed interviews will enhance the testing of the initial programme theory to triangulate with the findings from the quantitative measures. The interviews aim to uncover (i) the mechanisms and contextual elements relevant to the Hub models (to inform theory development), (ii) acceptability and feasibility of the Hub, (iii) their experience of being asked about adversity, the care offered to them and the process of referrals including barriers and enablers to uptake (caregivers only) and (iv) confidence and competence in detecting and supporting families experiencing adversity (practitioners only). Statistical analyses will be conducted using Stata or R statistical software packages.65 66 The baseline characteristics of the caregivers, children and practitioners will be described using the mean, median and IQR for continuous data and proportions for categorical data. Primary and secondary outcomes will be assessed by examining change in proportions/means from baseline to 6 months, from 6 to 12 months and from baseline to 12 months. We will also conduct linear regression analyses to assess the association between each caregiver-reported primary outcome at 6 and 12 months with the outcomes of child and caregiver mental health and caregiver quality of life, at 6 and 12 months, respectively. Quantitative measures of the strength and structure of service linkages between Hub practitioners will be analysed using social network analysis (SNA) software package UCINET.67 SNA is a complex systems discipline and quantitative methodology widely used to measure networked relationships between organisations and individual actors in health service and policy settings.68–71 UCINET67 will calculate indicators (ie, density, degree, centrality and betweenness) for the network and each Hub practitioner based on: (i) contact, (ii) referrals to and from and (iii) quality of the relationship. These indicators and analytical approach are recommended for descriptive social network analysis.68–71 Costs of the Hubs will be identified, including practitioner training, administration for referrals, personnel to oversee clinical implementation and other relevant costs. Downstream costs of services and potential cost-offsets will be collected and analysed to obtain the cost per additional child/or caregiver attending the Hub. Quality of life will be measured using the carer-specific measure (EQ-HWB-S72). An analysis of the value of unmet need will be conducted by determining whether (i) the child and family’s needs were met, (ii) no needs or (iii) unmet needs using the Ages & Stages Questionnaire or Strengths and Difficulties Questionnaire (SDQ) to determine perceived need compared with the level of service use. Individual interviews will be transcribed verbatim by a transcription service and imported into NVivo Release 1.073 to assist in the process of analysis. Experienced qualitative researchers will employ Braun and Clarke’s six-stage reflexive thematic analysis74 to analyse the qualitative interview data. In line with the critical realist philosophy underpinning this study, we will employ inductive coding and deductive (a priori) coding based on the CIMO configurations for the Hub. We will conduct a process evaluation using multiple data sources, including routinely collected data (eg, attendance rates at training and visits to the Well-being Coordinator) and observation field notes of Hub practitioner clinical practice, the training sessions and the case-based discussions. The process evaluation examines the acceptability, feasibility and fidelity of implementation for each Hub component shown in the initial Hub programme theory in figure 2. The process evaluation will also include realist analytical methods in which observed outcomes are explained by looking into the mechanisms and contextual elements contributing to the focus CIMOs for this study. We will also conduct improvement cycles guided by the Model for Improvement.43 Small improvement teams will be formed at each site who will establish their specific improvement aims and measurements. Once the aims are agreed, the team will undertake short learning cycles via Plan-Do-Study-Act (PDSA) cycles.43 Each PDSA cycle begins by articulating: the change and recording predictions about what we expect to happen (Plan); attempting to make the change and documenting what happened (Do); comparing the results to the predictions (Study); and then deciding on what to do next (Act). By the end of the project, learning cycles will be embedded as part of routine Hub service delivery, making the Hub more likely to be sustained and used.41 To maximise rigour, we will triangulate data collected through the process evaluation and PDSA cycles with outcome measures collected through the 6 and 12 month surveys.75 We will use the process and outcome evaluation data to refine the initial Hub programme theory. Further testing will be undertaken by explicitly seeking to confirm or contradict theories throughout the PDSA cycles, in-depth interviews and knowledge translation activities (eg, forums, workshops and webinars). A key strength of our study is the assessment of outcomes across multiple levels of the Hub intervention (ie, child, family, practitioner, service and system levels). The triangulation of multiple data sources to evaluate the implementation of the Hub model serves to maximise the study’s rigour and confidence in our findings.75 76 Caregivers and practitioners were actively involved in the co-design of the Hub models and are members of local advisory groups at each site which oversee the study design, recruitment, piloting of instruments, interpretation and dissemination of findings. Their engagement is aimed at ensuring the research is responsive to each context and to facilitate the translation of findings into practice.77