The CARAMAL study

This policy reverse is because of a large, ‘real world’, sequential observational study (CARAMAL),13–15 which evaluated the implementation of RAS in the Democratic Republic of Congo (DRC), Uganda and Nigeria. The study described shortcomings in patient referral to health facilities after they had received RAS, and incomplete follow-up treatment with oral artemisinin combination treatments. CARAMAL provides useful guidance for improving the deployment of RAS. However, the study also associated the deployment and use of rectal artesunate with an apparent increase in mortality.13 14 This unexpected finding was attributed to a failure by parents or carers to refer the treated children for further care. In other words, it was hypothesised that provision of rectal artesunate reassured the parents or carers, and that the consequent delay in further antimalarial treatment proved fatal. We think that this explanation is incorrect, and that the subsequent decision by WHO’s Global Malaria Programme to recommend withholding further deployment of rectal artesunate12 is flawed.

The main concerns

First, the sequential observational nature of the CARAMAL study makes any causal estimates of the effects of RAS roll-out unreliable.16 There is a potentially large bias (ie, confounding) by indication: CHWs naturally allocate potentially life-saving medicine to those who they think need it the most. These are the children who are most severely ill, and most likely to die. Substantial temporal confounding is also likely in the CARAMAL study because the periods compared and referral processes were different. The pre roll-out phase in Nigeria, where mortality was substantially higher than in DRC or Uganda, covered only half the rainy season (second half of 2018) and recruited predominantly via CHWs, whereas the major increase in mortality after roll-out occurred during the COVID-19 pandemic in patients recruited at primary health centres (PHCs). Parents may have been more reluctant to bring their sick children to healthcare facilities when there was a perceived risk of catching COVID-19. In the DRC, we have observed substantial changes in health seeking behaviour during the COVID-19 pandemic.

The CARAMAL study results were extremely heterogeneous13; mortalities exceeded 10% in Nigeria but were less than 1% in Uganda. Nigeria was the only country with a significant increase in mortality after RAS roll-out (a fourfold increase: 16.1% vs 4.2%) and where, after roll-out, children receiving RAS were much more likely to die than those who did not receive RAS (2.5-fold difference: 19.7% vs 7.7%). Yet in Nigeria RAS roll-out was incomplete; only half of the Nigerian patients actually received RAS. In the 48 hours following enrolment, mortality in Nigerian patients who did receive RAS was over twice that in those who did not (10% vs 4%). This very large difference in acute mortality (within a single malaria parasite asexual life-cycle) cannot be explained by lower rates of referral and delayed administration of parenteral treatment (RAS reduces parasite numbers by 10 000-fold over one 48 hours life-cycle).3 17 Bias in RAS allocation seems a much more likely explanation, that is, healthcare providers were more likely to administer RAS to the sicker children.

Second, the authors did not write a statistical analysis plan before analysing the data. This is unusual for a large preregistered multi-country study today, and it is particularly important given that the results have changed WHO policy. Prespecifying the statistical analysis is critical to avoid data dependent analyses.18 There are three possible exposure contrasts (pre roll-out vs post roll-out, pre roll-out vs post roll-out and receiving RAS, and receiving vs not receiving RAS post roll-out), three different countries, and two enrolment types (CHW vs PHC), so there are at least 18 subdivisions of the data. Although the primary outcome was clearly prespecified, the reported associations were not. The lack of prespecification is illustrated by comparing the analysis of referral completion with the analysis of mortality: the set of covariates chosen for adjustment are completely different.13 No rationale for either set of choices was provided. This lack of prespecification means that the main reported results of the CARAMAL study, as currently presented, should be regarded as post hoc subgroup analyses.

Third, some of the key published results are misleading. They do not reflect the findings of the study. For example, in the published report on referral completion the authors state: ‘In DRC and Uganda, RAS users were less likely to complete referral than RAS non-users in the pre roll-out phase (adjusted OR (aOR)=0.48, 95% CI 0.30 to 0.77 and aOR=0.72, 95% CI 0.58 to 0.88, respectively)’.13 This suggests that RAS was associated with referral failure, but it misrepresents the results of actual analysis. In fact, all referral completion was lower after RAS roll-out, and it was not different in RAS non-users (indeed in DRC, patients receiving RAS were slightly more likely to complete referral than those who did not receive RAS).

Fourth, there is the pharmacological implausibility of attributing an increased proportion of rapid death to receipt of 10 mg/kg rectal artesunate. Some patients with strictly defined severe malaria do die rapidly after receiving artesunate, but over 90% survive whereas, without any treatment, nearly all die.2 The majority of the life-saving effect of artesunate results from the first dose.2 17 For the post-treatment burden of parasites to return to lethal levels, more than two asexual cycles (>4 days) are required. Blood stage parasites cannot reproduce more rapidly. Yet CARAMAL has been interpreted as showing that the failure to receive a second dose resulted in a substantially increased malaria-attributable mortality within 48 hours, compared with no prereferral treatment. A large post-RAS increase in acute malaria attributable mortality (ie, the differential) simply could not have occurred so rapidly.

Finally, we should consider the accuracy of the diagnosis of severe malaria by CHWs and PHCs. Sepsis is the major cause of preventable death in children living in malaria endemic areas. Even within prospective studies in hospital-based research centres, the diagnosis of severe malaria is incorrect in about one third of cases.19 The misdiagnosed children commonly have sepsis, and they have a higher case specific mortality. Twenty years ago, when the ineffective chloroquine was still the main antimalarial drug in Africa, malaria was a much more important cause of childhood death. Today, in a context of widespread insecticide treated bed-net use, access to oral artemisinin combination treatments, and increasing deployment of seasonal malaria chemoprevention, sepsis is a much more likely cause of death than malaria. Sepsis would certainly explain why delays in referral and receiving parenteral antibiotics (whether or not RAS was administered) were associated with increased mortality.