Using theory of change in child health service interventions: A scoping review protocol

Background: The objective of this scoping review is to map the evidence of how child health service interventions use their theory of change. A theory of change is a hypothesis of how and why an intervention is intended to bring about change. It can be used as a program design, implementation, and evaluation tool. This scoping review will provide an overview of the evidence base for, and identify the way in which, theories of change in child health service interventions are defined, rationalised, developed, presented, and refined. Methods: The inclusion criteria for this scoping review is any child health service intervention globally, that describes their theory of change or theory of change development process. Relevant exclusions include: logic models or logic frameworks that do not meet this review’s definition of theory of change, systematic reviews, behavioural change interventions that target patient’s behaviour, school-based interventions, and maternal health interventions not related to child health outcomes. This scoping review will follow the Joanna Briggs Institute Reviewer’s manual. Relevant publications will be first searched on selected electronic databases and grey literature. A search strategy will be developed. The search will be limited to articles written in the English language. Results of the search will be curated using Endnote and duplicates removed. Results will be imported to Rayyan. The inclusion criteria will be applied during the process of title and abstract screening, by two independent reviewers and disagreements resolved by a third independent reviewer. Full-texts will have the inclusion criteria applied via the same reviewer process. Data relevant to the research sub-questions will be extracted, analysed, charted and discussed. Ethics and dissemination: Ethical approval is not required for this review as we will make use of already published data. We aim to publish the findings of our review in a peer-reviewed journal.

The proposed scoping review will be conducted in accordance with the JBI methodology for scoping reviews ( Joanna Briggs Institute, 2015; Peters et al., 2020) and was also used for the development of this protocol. Participants. Health service interventions targeted at children and youth aged 0–19 years will be included. Health service interventions will be defined as interventions targeting the access to, and the use, costs, quality, delivery, organisation, financing, and outcomes of health care services. The term ‘intervention’ will be defined as any organised activity, program, project, or initiative, that is supported by resources and established with the purpose of enacting change. This scoping review will exclude studies in which the main target of the intervention was individual patients rather than the health service more broadly. For example, an intervention using behavioural change theory aiming to influence a patient’s cognition and behaviour would be excluded. However, if an intervention used behavioural change theory aiming to change the behaviour of staff members in order to more effectively deliver a healthcare service, this will be included. Settings outside healthcare facilities such as schools, will be excluded. Obstetric health service interventions that aim to improve maternal health outcomes rather than newborn health outcomes will also be excluded. Some health service interventions that overlap with public health interventions such as vaccination programs will be included in this scoping review. Concept. Interventions that discuss their ToC will be included. Specifically, papers will be included if they a) describe how a ToC was utilised throughout any stage of a child health service intervention such as design, implementation, or evaluation. Or b) describe the development process for a ToC planned to be used in a child health service intervention. A ToC will be defined as a theory or hypothesis of how and why an intervention is intended to bring about change. As highlighted by Breuer et al. (2016), it may be difficult to ascertain if a study has discussed a ToC as per this definition purely from the title and abstract alone. Therefore, during title and abstract screening a study will be included for full-text screening if it refers to the development of their own ToC, theory-based evaluation, theory informed evaluation, program theory, outcomes hierarchy, results chain logic model, or logic framework or, if the title/abstract include comments on the steps of how, or mechanisms behind why, an intervention worked. Then, during full-text screening, when a more thorough analysis of how these studies used the various terms above can be conducted, the specific definition of ToC above will be applied. The included studies should include a consideration of at least one of the following ToC components; assumptions, activities, mechanisms, measurement indicators, outcomes, and context, or the linkages between these various components. Sources. This scoping review will consider a number of different study designs including both quantitative, qualitative and mixed-methods study designs. From the grey literature, organisational e.g., private organisations and NGO’s as well as government ToC documents will be included. Systematic reviews will not be included. There will be no other limits of date, study design, or type of publication. The search strategy will aim to locate both published and unpublished studies. An initial limited search of MEDLINE was conducted to identify articles on the topic. The text words contained in the titles and abstracts of relevant articles, and the index terms used to describe the articles were used to develop full search strategies for each database (see Extended data). The search strategy, including all identified keywords and index terms, will be adapted for each included database and/or information source. Broadly, the database search will combine terms from the two themes: childhood age range and ToC. The reference list of all included sources of evidence will be screened for additional studies. Studies published in English will be included. This is due to resourcing constraints within the research team. No publication year restrictions will be used in the search in order to gain an understanding of how different terminology has been used over time. The databases to be searched include, MEDLINE, EMBASE, Global Health, WHO Global Index Medicus, CINAHL and SCOPUS. Sources of unpublished studies/ grey literature to be searched include the first 10 pages of a specific google search ‘Child* “theory of change” filetype:pdf’. The titles and then full-texts of these pdfs will then have the same criteria applied as the literature. The most recent search of these sources in formulating the search strategy was completed on 14 th December 2021. This will be an iterative process whereby the literature will be searched and the search strategy refined. Following the search, all identified citations will be collated and uploaded into Endnote V20 and duplicates removed before being uploaded to Rayyan, an online review management software. Following a pilot test, titles and abstracts will then be screened by two or more independent reviewers for assessment against the inclusion criteria for the review. The inclusion and exclusion criteria were refined and agreed by all three researchers. As outlined in the introduction, there are a number of terms used to describe diagrams that are similar to, but often lack the depth of, a complete ToC. This presents a challenge during particularly the title and abstract screening stage, where a paper may not explicitly acknowledge an articulation of a ToC and label it as such. Therefore, careful analysis of the abstracts to identify if comments are made on the ‘how’ and ‘why’ an intervention has worked, will be critical when there is an absence of specific phrases such as; ToC, theory based evaluation, theory informed evaluation, logic model, logic framework, or program theory. This approach to study selection diverges from that of the review conducted by Breuer et al. (2016) who limited the criteria to needing the specific term ‘theory of change’ to be explicitly mentioned in the title or abstract order for the paper to meet the inclusion criteria. Following this, potentially relevant sources will be retrieved in full. The full text of selected citations will be assessed in detail against the inclusion criteria by two or more independent reviewers. Reasons for exclusion of sources of evidence at full text that do not meet the inclusion criteria will be recorded and reported in the scoping review. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion. The results of the search and the study inclusion process will be reported in full, in the final scoping review and presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram. Data will be extracted from papers included in the scoping review by two or more independent reviewers using a data extraction tool developed by all reviewers, to aid with consistency of which data will be extracted ( see Extended data). The data extracted will include specific details about the participants, concept, context, study methods and key findings relevant to the review questions. A draft extraction form is provided (see Extended data). The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included evidence source. Modifications will be detailed in the scoping review. Any disagreements that arise between the reviewers will be resolved through discussion, or with an additional reviewer(s). If appropriate, authors of papers will be contacted to request missing or additional data, where required. Data analysis and presentation. Data will be presented in table form. Furthermore, a narrative summary will accompany the tabulated information and will describe how the results relate. Data will be analysed using both qualitative and quantitative methods. Qualitative data analysis will involve all reviewers engaging in an in-depth discussion about the ToC data and describing the major concepts arising. Literature will be analysed to study location, type of health service intervention, and ToC definition (using a checklist of key ToC components such as assumptions, activities, mechanisms, measurement indicators, outcomes, context, and linkages), ToC presentation. Data extracted from the data extraction tool (see Extended data) will then be charted as per the JBI. Firstly, in the data charting process, a calibration exercise with the full team will be implemented. This will involve a random sample of 20 citations for title and abstract screening. A roundtable discussion will be had to clarify any issues. After the initial calibration exercise, one author (BJ) will chart the data independently and two authors (ME and SN) will verify the data for accuracy. Inconsistencies and disagreements will be resolved through discussion. An iterative approach will be taken with the data charting process and major revisions with rationale will be included in the final report. Critical appraisal of individual sources of evidence will not be completed as it is out of the scope of this review. This scoping review has a number of potential impacts. Firstly, it will further elucidate to implementation researchers how ToC are described in the literature, what is meant by the term ToC, the value that ToCs bring to interventions, what is missing from these descriptions, and encourage consideration of how these descriptions could be improved. Secondly, for health service practitioners working within the field of child health service interventions, it will map the available evidence on how these interventions may use a ToC. Thirdly, this scoping review may encourage those in fields outside of child health service interventions to consider using a ToC as a program design, implementation, and evaluation tool. This scoping review will be conducted in early 2022.

N/A

• jones, ben
• nagraj, shobhana
• english, mike