Systematic review of indoor residual spray efficacy and effectiveness against Plasmodium falciparum in Africa

Indoor residual spraying (IRS) is an important part of malaria control. There is a growing list of insecticide classes; pyrethroids remain the principal insecticide used in bednets but recently, novel non-pyrethroid IRS products, with contrasting impacts, have been introduced. There is an urgent need to better assess product efficacy to help decision makers choose effective and relevant tools for mosquito control. Here we use experimental hut trial data to characterise the entomological efficacy of widely-used, novel IRS insecticides. We quantify their impact against pyrethroid-resistant mosquitoes and use a Plasmodium falciparum transmission model to predict the public health impact of different IRS insecticides. We report that long-lasting IRS formulations substantially reduce malaria, though their benefit over cheaper, shorter-lived formulations depends on local factors including bednet use, seasonality, endemicity and pyrethroid resistance status of local mosquito populations. We provide a framework to help decision makers evaluate IRS product effectiveness. A meta-analysis of IRS experimental hut trials is used to summarise measures of IRS efficacy. Whilst experimental hut trials cannot account for all of the effects of IRS alone58 they provide a relatively standardised method to assess IRS efficacy and are considered the entomological equivalent of a Phase II trial31. They are also a pivotal part of the testing of new products and are required by WHO Prequalification29 which enables products to be bought by international procurers for low-income countries. The meta-analysis was conducted based on the PRISMA guidelines which highlight how best to perform systematic reviews for clinical trial data. Here, we are interested in count data for mosquitoes in Phase II studies over a time series of multiple months. Four search engines were used (Web of Knowledge, PubMed, JSTOR and Google Scholar) to identify relevant data resources. Policy teams and author’s regularly conducting these studies were also contacted to access unpublished resources. A schematic of the process (Supplementary Fig. 1) and table noting the reasons for excluding studies are included in Supplementary Table 1. To the author’s knowledge, there has been no previous published systematic meta-analysis on IRS compounds tested in experimental hut trials. Studies are limited to trials conducted in Africa (where the biggest burden of falciparum malaria is found) and to mosquito species belonging to the Anopheles family (vectors of the disease). Experimental hut studies typically report 24-h product-induced: mortality, blood-feeding inhibition, exophily and deterrence31. Here we present absolute values of mortality, blood-feeding and exophily as measured in the treated huts. This is to allow the results of different studies to be appropriately statistically combined, though each are presented individually, and insecticide-induced estimates can be calculated from Supplementary Data 1, analysis 1. There is relatively little variation in the level of mortality, blood-feeding and exophily observed in the control (unsprayed) huts in the studies examined here and this method is consistent with previous modelling efforts20. Summaries of each are described below. (i) Mortality: The number of female mosquitoes found in the hut which are dead on collection or die within the next 24-h is denoted D. In the following equations, the subscript denotes whether the number dead (or other characteristic) was measured in the control (unsprayed hut = C) or the sprayed hut (T). If N is the total number of female mosquitoes that were found in the hut or exit traps then, (ii) Exophily: Exophily is the propensity for mosquitoes to rest outdoors after feeding which can diminish the impact of IRS. It is calculated as the number of female mosquitoes in exit traps (E) compared to the sum of the number collected in the hut and exit traps (N) as, (iii) Blood feeding: The number of mosquitoes that are blood fed which were collected in the hut and exit traps is denoted B so the percentage blood fed in a sprayed hut is given by, (iv) Deterrence: Deterrence induced by IRS is defined as the reduction in the entry rate of mosquitoes into experimental huts with or without IRS, The first analysis summarises and compares the initial impact of different IRS products. Data were restricted to initial timepoints collected within 2 months of IRS application as the active ingredient decays with time, so that averaging across the whole dataset may mis-represent the initial potency of IRS as studies had different durations. Statistical models were fit to generate overall estimates of the efficacy of the chemical class. These explanatory factors included the mosquito vectors (classified at the species complex level and species level where possible, i.e. A. arabiensis, A. funestus s.l. and A. gambiae s.l.), experimental hut type (West or East African design) and hut wall substrate (cement or mud) alongside the chemical class used for the IRS (carbamate, clothianidin, organophosphate and pyrethroid). Preliminary data exploration revealed that there were too few data to perform an extensive statistical test on all covariates. To overcome this a subset of the full database was generated by removing Ifakara hut studies, wall substrates that were not mud or cement and chemistries other than pyrethroids, organophosphates, carbamates or neonicotinoids. Binomial logistic regression models were fitted to the remaining count data (N = 78) to estimate the number of mosquitoes that were dead in 24-h, had exited, blood-fed or been deterred by the IRS product. The predicted value for the proportion of mosquitoes being killed, exiting, blood-fed or deterred is calculated as: where πi is the estimated proportion for the ith data (e.g. the proportion of mosquitoes killed), β0 is the intercept, the subscript h denotes the covariate of interest (taking number of 1 to H) and Xh is a matrix of explanatory factors (mosquito species, hut type, substrate and chemistry sprayed) with coefficients βh59. Bayesian models were fitted using Hamiltonian Monte Carlo sampling methods60,61. Four chains were initialised to assess the convergence of 2000 iterations, the first 1000 of each were discarded as burn in. The posterior distributions of parameters (4000 iterations) and 90% Bayesian credible intervals were estimated, posterior checks were performed using ShinyStan (version 1.0.0)62 and visually confirmed to fit the data (Supplementary Fig. 2–5). Four insecticide active ingredients, pyrethroids (including deltamethrin, lambda-cyhalothrin and alpha-cypermethrin), pirimiphos methyl, bendiocarb and clothianidin were further characterised from data identified in the meta-analysis. These four groups of active ingredients were chosen as they are likely to be the main insecticides used by NMCPs for IRS in the next few years (prior to 2020) where sufficient published and unpublished data were available (Table 2). For simplicity insecticides containing the appropriate concentration of pirimiphos methyl and clothianidin are subsequently referred to by their product names Actellic®300CS and SumiShield®50WG, respectively. The impact of IRS depends on its initial efficacy and how this changes over time. Studies with 3 or more experimental hut trial time-points were considered sufficient to characterise temporal changes. Reasons for excluding studies are noted in Supplementary Table 1. Altogether 8 published and 1 unpublished studies (providing 21 time series) were found that reported experimental hut trials on pyrethroid-IRS17,25,63–68 (Table 2). Three published studies and a further unpublished dataset were identified for bendiocarb39,63,66. Four published and 1 unpublished studies provided 6 time series data for Actellic®300CS25,26,63,65. In total, 2 published and 2 unpublished datasets were available for SumiShield®50WG67,68. This new formulation was tested at different concentrations and we include concentrations of 300 g m−2 and above in the presented analyses. The mode of action of the neonicotinoid insecticide clothianidin has been shown to act over multiple days on the insect’s nervous system so the 24-h mosquito mortality measured in a SumiShield®50WG experimental hut trial is unlikely to fully represent the efficacy of this chemistry69. To generate comparable parameterisation of products with different modes of action a simple conversion is used to convert 72-h experimental hut trial mortality rates into 24-h mortality rates that can be used in the transmission dynamics model. This is possible if it is assumed that SumiShield®50WG exposed mosquitoes have no epidemiological impact between 24 and 72-h following exposure, which, given the frequency of blood-feeding, appears the most parsimonious assumption. If mosquitoes caught in the other arms of the trial (untreated huts and those with fast acting chemistries) die at a constant rate between 24 and 72-h, then the background mosquito death rate for a mosquito in captivity following a hut trial can be estimated using the exponential function. If lSCt denotes the proportion of mosquitoes that are dead in captivity t days after the start of the hut trial, then the background mortality rate (μB) can be estimated using, Fitting this function to all datasets where 72-h (t = 3 days) mortality were recorded gave μB = 0.035. This value was then used to adjust the mortality observed 72-h after the start of the SumiShield®50WG trials to generate estimates of 24-h mortality comparable to the other insecticides. Data were not always disaggregated by the mosquitoes that had fed and survived or fed and died. Therefore, it was not possible to directly infer which mosquitoes were successfully feeding. Instead, before fitting the time series data, we adjusted the number of mosquitoes that were blood feeding (Nfed) to provide an estimate for the successful blood-feeding mosquitoes (Nsuccessfully_fed), those that feed and survive, as follows: Ntotal and Ndead denote the total number of mosquitoes sampled and the total number recorded as dead for each time series. Logistic binomial models were fitted to the count data to determine the relationship between the probable outcome of a mosquito feeding attempt (the mosquito is deterred, killed, successfully feeds or exits without feeding) and how these change over time. Briefly, to determine, for example, the relationship for the proportion of mosquitoes that are killed (ls) in the presence of an IRS product over time t, we fit: The proportion of mosquitoes dying following entering a hut is denoted lS and is dependent on a parameter that determines initial efficacy (lSϑ) and how this changes over time, denoted by the depreciation parameter (lSγ). The mosquitoes that are successfully feeding or deterred are modelled in the same way (Supplementary Methods). These different probable outcomes of a feeding attempt are then translated into the probability of a mosquito being killed, successfully feeding or being repelled as detailed in Supplementary Methods. To determine uncertainty, the maximum and minimum data for each unique time point were fitted in the same way. The ranges for the probability of mosquitoes successfully feeding, exiting or being killed at each feeding attempt in the presence of each IRS product could then be distinguished (Supplementary Fig. 8). As previously, Bayesian models were fitted using Hamiltonian Monte Carlo sampling methods60,61. The fits were visually confirmed to fit the data (Fig. 2). A widely used transmission dynamics model of malaria2,70–72 is used to investigate the public health impact of different IRS compounds. In this model, people are born susceptible to Plasmodium falciparum infection and are exposed to infectious mosquito bites at a rate dependent on local mosquito density and infectivity. Maternal immunity is acquired for new born infants and this decays in the initial 6 months of life. Individuals are susceptible to clinical and severe disease and death after exposure71,72. The risk of developing infection declines with age due to naturally acquired immunity following continual exposure. Mosquito dynamics capture the effects of mosquito control and the resulting decline in egg laying70. A small number of minor changes (see Supplementary Methods) are adopted to the IRS component of the model to reflect the varying impact of the new chemistries and how these change over time. These changes unify the way LLINs and IRS are represented in the model (and are parameterised with experimental hut trials) and provide greater flexibility to capture the impact of different insecticides. The transmission model is used to simulate across the parameter ranges (Supplementary Table 3) to explore the minimal and maximal entomological impact of a given product and the knock-on predicted impact on cases (Supplementary Data 2). Discriminating dose bioassays (WHO tube assay, WHO cone assay, CDC bottle assay) are a practical option for control programmes to assess the proportion of the mosquito population that are killed by a standard dose. The assumption is made that the inverse of this proportion, i.e. those mosquitoes surviving in the presence of the standard dose of insecticide, is representative of the level of insecticide resistance in the mosquito population. Although the simple bioassay has its limitations20,37 it provides a useful measure to link the severity of mosquito insecticide resistance estimated in the field to the results of experimental hut trials evaluating new products20,34. The concentration of insecticide used in the discriminatory dose bioassay varies with the type of pyrethroid insecticides used. There were 18 data points identified in the meta-analysis where pyrethroid bioassays were conducted on the same mosquito population as the experimental hut studies using a pyrethroid IRS (Fig. 3a). There were a further 21 datasets with time series data, but not bioassay mortality data, so that the initial (time t = 1 day) mosquito mortality, successful feeding and exiting probabilities and how the impact of pyrethroid IRS on mosquito behaviour changes over time could be estimated. This is insufficient data to differentiate between different types of pyrethroid so all pyrethroid data are pooled together, recent work suggests this may be a reasonable assumption73. These two datasets are used to associate 24-h mortality using a discriminatory dose bioassay (our proxy for the level of pyrethroid resistance in the mosquito population) and the parameters influencing IRS efficacy (see Supplementary Methods). These changes are demonstrated in Supplementary Fig. 9. The code for analyses 1–3 are provided in Supplementary Methods. RCT are the gold standard for assessing intervention efficacy and effectiveness in the field. Results from two RCTs testing the additional benefit of Actellic®300CS8 or bendiocarb38 IRS in combination with standard LLINs over standard LLINs alone were compared to model predictions to determine whether the IRS parameterisations satisfactorily match observed data. The location-specific seasonality profile and historic bednet use were taken from1,40,74, whilst study-specific parameters such as epidemiological information (for example cohort age), intervention type (for example decay of net use) and mosquito information (for example the ratio of different mosquito species present and the level of pyrethroid resistance) were taken from the relevant publications and discussions with study authors (see Table 3 for a summary of input parameters). Predictions were made for all RCT data combined without differentiating between clusters. Absolute mosquito abundance is varied to ensure model predictions at baseline match the average malaria prevalence for the age cohort examined. Future predictions are then made using the model and compared visually to observed RCT prevalence measured during cross-sectional surveys. The parameter sets fitted to the temporal data (Supplementary Data 1, analysis 2) that describe the mean impact, as well as the maximum and minimum impact (Supplementary Methods for parameter estimates), of the respective IRS products on mosquito feeding outcomes were used to predict the public health impact. The parameter sets for each individual study (Supplementary Fig. 6) were also overlaid to provide some indication of the potential uncertainty in the experimental hut data. For the Actellic®300CS trial8, the best-matched experimental hut data were from Rowland et al.25 which took place in Benin and used West African huts with both cement and mud walls. The principal mosquito in both localities (the Benin experimental hut trial and the Kagera RCT) was A. gambiae s.s. and houses in Kagera also have, most commonly, cement or mud walls. Site-specific factors used to parameterise the transmission dynamics model Site-specific factors used to parameterise the transmission dynamics model and investigate its ability to predict the impact of IRS in the two randomised control trial (RCT). The efficacy of standard LLINs and pyrethroid IRS is adjusted according to the mosquito mortality in the discriminating dose bioassay using the adjustments noted in Supplementary Methods and methods presented in Churcher et al.20 for standard LLINs. All other parameters are consistent with Griffin et al.2, White et al.70, Griffin et al.71, and Winskill et al.40 (for seasonality profiles, historical intervention coverage, drug treatment information). There is insufficient data to characterise whether the mosquito species distribution or the level of pyrethroid resistance changed over time, so these are assumed to have remained constant throughout in all intervention arms. Systematic non-compliance is assumed in arms where both LLINs and IRS were distributed aDenotes parameters which are kept constant throughout the simulations for all intervention arms bThe same parameter estimates are used for A. arabiensis IRS and LLINs are used concurrently (i.e. the same people receive IRS and LLIN) in many malaria endemic communities75. The efficacy of IRS on top of LLINs will depend on LLIN coverage, the level of pyrethroid resistance and the seasonality of malaria transmission. To illustrate how these factors influence disease control, the transmission model is parameterised for a theoretical perennial and a highly seasonal setting. For simplicity all simulations are initially run in an area with moderate transmission (slide prevalence of 30% without intervention or treatment) in an area with no history of malaria control. At the start of year 0, LLINs are distributed at a pre-determined coverage level (the percentage of people who use them) ranging from 0 to 100%. The net usage remains at this level for the whole simulation. Pyrethroid resistance is simulated to arrive overnight at a defined level (as described by the percentage of mosquitoes surviving a 24-h discriminating dose bioassay test, 0–100%) with the introduction of nets at year 0. At the start of year 3, IRS is introduced, be it a long-lasting product (e.g. Actellic® or SumiShield®), a short-lasting product (e.g. bendiocarb) or a pyrethroid performing at the defined level of resistance. SumiShield® produced broadly similar results to Actellic® and is not represented in Fig. 5. Eighty percent of the people are protected by the IRS. The households are sprayed just prior to the peak in the transmission season each year. A 3-yearly reporting cycle is adopted to coincide with the redistribution of LLINs (generally every 3 years). The clinical cases averted per 1000 people per year by the respective IRS chemistries are calculated between years 3 and 6 relative to a scenario for the same level of pyrethroid resistance where no IRS is implemented. Every area of Africa has a different malaria seasonality and history of control interventions so the impact of adding different types of IRS will vary locally. To facilitate assessment of the public health and economic benefit of adding different IRS options their impact is simulated at each administration unit 1 across sub-Saharan Africa and at increasing levels of pyrethroid resistance. Location-specific seasonal profiles74 and historic scale-up of IRS and LLIN interventions from 2000 to 2015 are used (Malaria Atlas Project, MAP1) as per40. The mosquito density is adjusted for each location to capture the underlying transmission intensity and ensure model predictions match MAP estimates for P. falciparum prevalence in 2–10-year olds. Mosquito densities are then scaled up for each country so that the total cases estimated is equal to the WHO estimate for 2015 for that location whilst also capturing administration-level heterogeneity in transmission40. Pyrethroid resistance is switched on overnight in 2018, whilst maintaining 2015 net coverage levels. A 3-yearly reporting cycle is once again adopted to coincide with the redistribution of LLINs (generally every 3 years). IRS is introduced at 80% coverage in 2021 using either long-lasting (Actellic® or SumiShield®), short-lasting (bendicarb, with annual or biannual application) or pyrethroid-IRS for distinct levels of pyrethroid resistance (ranging from 0 to 100%) (Supplementary Data 2). The IRS product parameterisations are the mean fits for the temporal data (Supplementary Table 3), with the exception of pyrethroid IRS, which is complicated by the presence of resistance in local mosquito populations. To provide some indication of the uncertainty in these product impacts, the transmission model is also used to simulate the predicted maximum and minimum impact of non-pyrethroid IRS products as estimated from the temporal analysis (Supplementary Data 1, analysis 2). Bendiocarb was modelled annually and biannually because it is usually sprayed twice a year if used for IRS programmes. The mean number of cases averted per 1000 people per year across the following 3 years, 2021–2024, was then calculated relative to a scenario where no IRS was used.