Jump to comment:

• Example of a modeller’s temptation to seek alternative trial results
  Sanne M Thysen, Ane B Fisker and Christine S Benn
  Published on: 17 November 2018

• Published on: 17 November 2018

  Example of a modeller’s temptation to seek alternative trial results

  • Sanne M Thysen, PhD student Bandim Health Project, Guinea-Bissau
  • Other Contributors:
    • Ane B Fisker, Associate Professor
    • Christine S Benn, Professor

  Recently, Colbourn et al questioned the use of modelling to seek alternative trial results1. They cited a radio intervention study from Burkina Faso that based on mathematical modelling suggested that the radio intervention was associated with a 7.1% reduction in under-5 mortality, whereas the actual trial results suggested no effect (Rate ratio: 1.00 (95% CI: 0.82-1.22))1. Colbourn and colleagues raised the important point that modelled estimates should not take precedence over empirical mortality data.
  We would like to support the point raised by Colbourn and colleagues with an example from the field of vaccinology.
  The phase 3 trial of the RTS,S/AS01 malaria vaccine found a vaccine efficacy of 18-36% against clinical malaria2. The study was not powered to assess mortality endpoints, but the results suggested that RTS,S/AS01 was associated with 24% (95% CI: -3 – 58%) higher all-cause mortality3. This was obviously not what was expected; a vaccine that reduces clinical malaria would be expected to reduce all-cause mortality. However, based on our experience, it could indicate that the vaccine, like other non-live vaccines, could have negative non-specific effects4. If that was the case, we predicted that the negative effect would be strongest in females as seen for the other non-live vaccines3. Subsequent analyses indeed revealed that RTS,S/AS01 was associated with higher mortality in girls (Relative Risk of death for RTS,S/AS01 compared with control (RR): 1.9...

  Show More

  Recently, Colbourn et al questioned the use of modelling to seek alternative trial results1. They cited a radio intervention study from Burkina Faso that based on mathematical modelling suggested that the radio intervention was associated with a 7.1% reduction in under-5 mortality, whereas the actual trial results suggested no effect (Rate ratio: 1.00 (95% CI: 0.82-1.22))1. Colbourn and colleagues raised the important point that modelled estimates should not take precedence over empirical mortality data.
  We would like to support the point raised by Colbourn and colleagues with an example from the field of vaccinology.
  The phase 3 trial of the RTS,S/AS01 malaria vaccine found a vaccine efficacy of 18-36% against clinical malaria2. The study was not powered to assess mortality endpoints, but the results suggested that RTS,S/AS01 was associated with 24% (95% CI: -3 – 58%) higher all-cause mortality3. This was obviously not what was expected; a vaccine that reduces clinical malaria would be expected to reduce all-cause mortality. However, based on our experience, it could indicate that the vaccine, like other non-live vaccines, could have negative non-specific effects4. If that was the case, we predicted that the negative effect would be strongest in females as seen for the other non-live vaccines3. Subsequent analyses indeed revealed that RTS,S/AS01 was associated with higher mortality in girls (Relative Risk of death for RTS,S/AS01 compared with control (RR): 1.91 (1.30-2.79)), but not in boys (RR 0.84 (0.61-1.17))5.
  WHO established a collaboration where four groups (the Institute for Disease Modelling, GSK Vaccines, Imperial College London and the Swiss Tropical and Public Health Institute) modelled the expected public health impact of introducing the RTS,S/AS01 vaccine. Based on the measured vaccine efficacy against clinical malaria, the models estimated that per 100,000 fully vaccinated children in a four-dose schedule, 484 deaths would be averted6. However, if the RTS,S/AS01 mortality results5 were representative, fully vaccinating 100,000 children in a four dose schedule would cause 464 additional deaths: Stratified by sex, 100,000 fully vaccinated boys would result in 252 fewer deaths, whereas 100,000 fully vaccinated girls would result in 1182 additional deaths.
  The modelled results are in line with the general perception that vaccines only have one effect: prevention of the target disease. However, increasing evidence suggest that non-live vaccines may have negative non-specific effects despite protection against the target disease. The RTS,S/AS01 results fits the pattern previously observed perfectly: the negative non-specific effects of non-live vaccines are particularly pronounced in females.
  Nonetheless, the RTS,S/AS01 vaccine is now being introduced as a four-dose schedule in Ghana, Kenya and Malawi as part of the WHO malaria vaccine pilot programme targeting 720,000 children. Policymakers may feel confident about this decision given the effect on clinical malaria and the modelled mortality data. However, based on the empirical mortality data, and the fact that they fit into an already established pattern of mortality effects on non-live vaccines, we fear that this introduction of RTS,S/AS01 may lead to unnecessary female deaths. Discrepancies between modelled and real-life data must be taken seriously. No model is better than the data and assumptions used. Therefore, empirical data should rank higher.

  References
  1. Colbourn T, Prost A, Seward N. Making the world a simpler place: the modeller’s temptation to seek alternative trial results. BMJ Global Health 2018;3(5) doi: 10.1136/bmjgh-2018-001194
  2. RTS.S Clinical Trials Partnership. Efficacy and safety of RTS,S/AS01 malaria vaccine with or without a booster dose in infants and children in Africa: final results of a phase 3, individually randomised, controlled trial. Lancet 2015;386(9988):31-45. doi: 10.1016/S0140-6736(15)60721-8
  3. Aaby P, Rodrigues A, Kofoed P-E, et al. RTS,S/AS01 malaria vaccine and child mortality. The Lancet 2015;386(10005):1735-36. doi: 10.1016/S0140-6736(15)00693-5
  4. Benn CS, Netea MG, Selin LK, et al. A small jab - a big effect: nonspecific immunomodulation by vaccines. Trends in immunology 2013;34(9):431-9. doi: 10.1016/j.it.2013.04.004
  5. Klein SL, Shann F, Moss WJ, et al. RTS,S Malaria Vaccine and Increased Mortality in Girls. mBio 2016;7(2) doi: 10.1128/mBio.00514-16
  6. Penny MA, Verity R, Bever CA, et al. Public health impact and cost-effectiveness of the RTS,S/AS01 malaria vaccine: a systematic comparison of predictions from four mathematical models. The Lancet 2016;387(10016):367-75. doi: 10.1016/s0140-6736(15)00725-4

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top