The impact of response to the results of diagnostic tests for malaria: cost-benefit analysis
BMJ 2008; 336 doi: https://doi.org/10.1136/bmj.39395.696065.47 (Published 24 January 2008) Cite this as: BMJ 2008;336:202
- Yoel Lubell, health economist1,
- Hugh Reyburn, clinical senior lecturer2,
- Hilda Mbakilwa, clinical officer3,
- Rose Mwangi, social scientist3,
- Semkini Chonya, data manager3,
- Christopher J M Whitty, professor of international health2,
- Anne Mills, professor of health economics1
- 1Health Economics and Financing Programme, Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, London WC1E 7HT
- 2Department of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine
- 3Joint Malaria Programme, Kilimanjaro Christian Medical Centre, PO Box 2228, Moshi, Tanzania
- Correspondence to: Y Lubell yoel.lubell{at}lshtm.ac.uk
- Accepted 25 October 2007
Abstract
Objective Rapid diagnostic tests for malaria seem cost effective in standard analyses, but these do not take account of clinicians’ response to test results. This study tested the impact of clinicians’ response to rapid diagnostic test or microscopy results on the costs and benefits of testing at different levels of malaria transmission and in different age groups.
Design Cost-benefit analysis using a decision tree model and clinical data on the effectiveness of diagnostic tests for malaria, their costs, and clinicians’ response to test results.
Setting Tanzania.
Methods Data were obtained from a clinical trial of 2425 patients carried out in three settings of varying transmission.
Results At moderate and low levels of malaria transmission, rapid diagnostic tests were more cost beneficial than microscopy, and both more so than presumptive treatment, but only where response was consistent with test results. At the levels of prescription of antimalarial drugs to patients with negative tests that have been found in observational studies and trials, neither test methodis likely to be cost beneficial, incurring costs 10-250% higher, depending on transmission rate, than would have been the case with fully consistent responses to all test results. Microscopy becomes more cost beneficial than rapid diagnostic tests when its sensitivity under operational conditions approaches that of rapid diagnostic tests.
Conclusions Improving diagnostic methods, including rapid diagnostic tests, can reduce costs and enhance the benefits of effective antimalarial drugs, but only if the consistency of response to test results is also improved. Investing in methods to improve rational response to tests is essential. Economic evaluations of diagnostic tests should take into account whether clinicians’ response is consistent with test results.
Introduction
Various diagnostic tests are being made available for some of the major diseases in sub-Saharan Africa, including HIV, sexually transmitted infections, and malaria. Although these tests could improve the targeting of drugs to those patients most in need, their use could consume scarce funding with little effect if clinicians’ response to test results is not taken into account. Most economic evaluations assume that clinicians will prescribe an antimicrobial agent to a patient with positive test results and not to a patient with negative results.
Improved targeting of antimalarial drugs in Africa is a priority as new artemisinin combination therapies, which cost much more than previous and now generally ineffective antimalarials, come into use. The widespread practice of treating any non-specific febrile illness as malaria threatens the sustainability of artemisinin combination therapy; in many settings most antimalarial drugs are prescribed for patients who have no malaria parasites.123 Other potentially serious causes of illness are ignored because of the assumption that the cause of fever is malaria. Overprescription of antimalarials has a great impact on the cost effectiveness of artemisinin combination therapies.4 If artemisinin combination therapies are to be afforded by the poorest people, they will need to be subsidised, and subsidies are also threatened by overprescription on the current scale.
Reflecting these concerns, malaria treatment guidelines now recommend that treatment for non-severe malaria should, at least for older children and non-pregnant adults, be restricted to people with positive results on a parasitological test for malaria. In many settings this is difficult to achieve with blood slide testing, which is of variable accuracy and available only in larger health facilities, where a minority of patients seek care. Recent improvements in rapid immunochromatographic diagnostic tests for malaria potentially solvemany of these problems; also, the testscost less than most courses of artemisinin combination treatment.5 6 Studies that have explored the economic consequences of using rapid diagnostic tests alongside artemisinin combination therapies have shown these tests are cost effective, but these studies assume that prescribers respond to negative test results by not prescribing an artemisinin combination therapy.5 789
The assumption in these models—that prescribers will respond in a way that is fully consistent with test results—is unsafe. In Zambia, where rapid diagnostic tests are used routinely in settings where microscopy is not available, antimalarials continued to be prescribed to over a third of patients who had negative test results.2 In Tanzania, a recent randomised trial of rapid diagnostic tests compared with blood slide testing found that in low transmission areas, over 90% of all antimalarials prescribed were given to patients with a negative test result, irrespective of the test method used.3 These levels of overprescribing extend also to patients with severe illness.10
Using data from a trial of clinicians’ behaviour, this paper examines the relation between the level of response consistent with results of parasitological tests and the total costs resulting from the use of rapid diagnostic tests and microscopy. Both diagnostic methods are evaluated, as microscopy is likely to continue to have an important role in the management of febrile patients. We consider the costs of diagnosis and treatment and the cost of life years lost through incorrect diagnosis and inappropriate treatment.
Methods
We obtained data from a randomised controlled trial carried out in three hospitals in northeast Tanzania, which compared clinicians’ response to microscopy results against those for rapid diagnostic tests.3 The relevant data for this analysis are prevalence of parasitaemia among febrile patients, clinicians’ response to test results, the accuracy (sensitivity and specificity) of the tests, and the cost of diagnosis and treatment.
Prevalence is defined as the proportion of febrile patients presenting with parasitaemia, and varies with the rates of transmission of malaria.11 Prevalence of under 10% is classified as likely when transmission is low, 10-50% when moderate, and above 50% in high transmission settings. Consistent response rate is defined as the percentage of negative test results for which clinicians do not prescribe antimalarials: at a consistent response rate of 100%, no patient with a negative test receives an antimalarial; at consistent response rate of 0%, all patients with a negative test receive an antimalarial. We did not examine inconsistent response to positive test results as this was a rare occurrence (<1%). Prescribing antibiotics to patients with negative test results was set at just over 50%, independently of provision of an antimalarial, as documented in the trial. (Whether or not patients with negative test results received an antimalarial, just over half had received an antibiotic in the trial.3) We derived the estimates for the accuracy of the rapid diagnostic tests and microscopy by combining data from all three sites in the trial.
We calculated the costs for the management of a simulated population of 1000 patients, including costs of microscopy or rapid diagnostic tests, antimalarials, antibiotics, care for patients with severe illness, and a cost representing the value of life years lost owing to incorrect diagnosis. The analysis primarily compares the varying costs of microscopy and rapid diagnostic tests with respect to action consistent with their results, and also the cost of presumptively treating the population, to indicate the circumstances in which use of a test becomes inefficient.
The analysis uses a cost-benefit framework, taking a societal perspective accounting for both provider costs and the monetary value of years of life lost owing to incorrect diagnosis and inappropriate treatment. This framework allows for a more coherent analysis in directly observing the impact of inconsistent response with test results on total costs, without the need for cost effectiveness ratios.
We chose a decision tree as the best model to synthesise the data, represent alternative options, and indicate the most efficient outcomes (fig 1)⇓. A number of variables were made to interact—for instance, the probability of developing severe malaria responded to prevalence and age, as did treatment costs for artemisinin combination therapies. In addition to the influence of age on probability of transmission and case fatality rates, the responsiveness of the model to patients’ age was important for policy considerations, since in some settings presumptive treatment is recommended for younger patients.
Fig 1 Decision trees in the model. The probabilities for developing severe illness and case fatality rates differ with respect to age, transmission intensity, and status as either false negative or untreated bacterial illness (see table 1)
Patients for whom the test provided an incorrect result that the clinician acted on were classified as false positives or false negatives. Their costs were estimated with the use of a simple flow chart as shown in the lower panel of figure 1⇑, following the model used by Goodman et al and Coleman et al.12 13 The probability of an episode of malaria being self limiting was determined according to the patient’s age and the transmission rate, with varying case fatality rates according to whether the patients were admitted as inpatients (tables 1⇓ and 2⇓).
Variables in assessing response to malarial testing
Transition probabilities and case fatality rates by age and transmission intensity
On the death of a patient the number of years of life lost depends on age and was related to relevant life expectancy tables14 and discounted at 3%.15 For the primary analysis, a year of life lost was assigned a cost of $150, reflecting the WHO benchmark for an “attractive” intervention in terms of cost effectiveness.16 All other costs and outcomes accrue instantaneously, with no further discounting required. The outcomes of the different branches summarise the total costs of treating 1000 patients with each strategy. The benefits of diagnosis are integrated in this value (as the averted cost of life years lost), so the intervention with the lowest total cost is the most attractive.
We tested the results for sensitivity to variation in all measures and produced a tornado graph to identify the measure with highest influence. The sensitivity analysis also presents results when the value for microscopy sensitivity is set at 95%, WHO’s minimum requirement for a malaria diagnostic test. This was included as the documented sensitivity for microscopy in the trial was very low—under 75%.
Results
The model output shows the importance of response to test results to the cost-benefit of diagnostic tests for malaria and indicates some situations where even modest compromises in consistent response rate led to higher expenditure than that incurred with presumptive treatment. This depends critically on transmission rates and age of the patient.
In a low transmission setting (prevalence of malaria 10% in those with febrile illness) and for a 15 year old, using either test incurred higher costs than presumptive treatment when consistent response rate was below 20%, and testing became increasingly attractive as response consistency improved (fig 2)⇓. For instance, at a consistent response rate of 50%, either test was less costly than presumptive treatment (as both tests were below the presumptive treatment threshold), about $5200 (£2500; €3500) for 1000 patients. Above this level of consistent response rate, rapid diagnostic tests became marginally more attractive, but despite their higher sensitivity they did not have a great advantage over microscopy in low transmission settings.
Fig 2 Total costs for rapid diagnostic tests and microscopy, using $150 per year of life lost and the profile of a 15 year old patient
In a high transmission setting (prevalence of 60%) and for a 15 year old, both tests incurred higher costs than a strategy of presumptive treatment if consistent response rate was below approximately 65% (fig 2⇑). The graph shows that the two tests followed conflicting trends as response improved further. Although consistent response to results of rapid diagnostic tests led to substantial cost savings, the cost of microscopy continued to rise as consistent response rate improved, mainly because of its low sensitivity, which resulted in considerable numbers of false negative results.
Figure 3⇓ compares each of the tests directly with presumptive treatment across all levels of prevalence and consistent response rate, showing the proportional change in cost of rapid diagnostic tests and microscopy relative to presumptive treatment. The upper left corners in both charts indicate that in low prevalence settings, use of either test with consistent response to results led to cost savings of between 50% and 100% as compared with presumptive treatment. As prevalence increases to the medium-high range, however, consistent response rate must increase more than proportionately if use of the tests is to remain attractive. At very high levels of prevalence, both tests appeared more costly, irrespective of response by clinicians, primarily because of imperfect test sensitivities, and presumptive treatment remained the more efficient option.
Fig 3 Cost difference of rapid diagnostic tests and microscopy with presumptive treatment as a baseline
To illustrate these findings, the levels of consistent response rate seen in the trial (about 50%) imply that rapid diagnostic tests would increase costs by 250% in low transmission settings and 35% in high transmission settings in comparison to settings where no patients with negative test results were prescribed antimalarials. Increases were similar for microscopy at low transmission, but in a moderate to high transmission setting the cost increase was only 10%. At higher prevalence, microscopy would become less attractive than presumptive treatment.
Figure 4 ⇓ explores the most attractive strategy at all prevalences and levels of consistent response rate, using profiles for patients aged 3, 7, and 25 years. They indicate that using either parasitological test for younger patients was unattractivein settings of high transmission, even if response to tests was fully consistent.
Fig 4 Most attractive strategy stratified by patient’s age. Shading indicates which strategy is preferred at specific combinations of prevalence of malaria and consistent response rate
Sensitivity analysis
Results were most sensitive to the cost assigned to a year of life lost. Higher values for this variable led to scenarios that were more costly and effective being considered more attractive. If the cost of years of life lost is ignored, rapid diagnostic tests would never be the most efficient option, while microscopy was still attractive although decreasingly so as prevalence increased and response fell.
Figure 5 ⇓ shows the circumstances under which each of the strategies is most attractive, stratified by cost per year of life lost averted. At a value of $25 (the value where WHO considers interventions to be “very attractive”), rapid diagnostic tests gained some advantage in the mid-range of prevalence as long as response was consistent with results, and microscopy remained the preferred option for low prevalence areas even at low levels of consistent response rate. Using $150 per year of life lost, rapid diagnostic tests became the preferred option up to a prevalence of about 70%, when presumptive treatment became the more efficient option. At a value of $680—twice the Tanzanian gross national income per capita for the year 2005,17 an alternative rule of thumb18—rapid diagnostic tests dominated across all but the lowest levels of consistent response rate and highest levels of prevalence.
Fig 5 Most efficient strategy by response to test results and prevalence of malaria with respect to value of years of life lost
Lastly, we ran the model using a value for microscopy sensitivity (95%) higher than that seen in the trial. At this level, microscopy was the preferred option for most levels of consistent response rate and when prevalence was up to 90% (fig 6)⇓.
Fig 6 Most efficient strategy with sensitivity of microscopy set at 95%
Discussion
A major barrier to the use of artemisinin combination therapy is considered to be its cost. Targeting drugs to people in greatest need is essential if programmes are to be sustainable. Diagnostic tests are costly and have to be cost beneficial if they are to be used. This study shows that clinicians’ response to test results has a major impact on the total cost of both microscopy and rapid diagnostic tests, and that this varies with rate of transmission of malaria. In low transmission settings, testing remains attractive even when consistent response rate is relatively poor, while at higher prevalences consistent response rate must increase more than proportionately for tests to remain attractive. At very high prevalences, presumptive treatment remains attractive given the imperfect sensitivity of tests (particularly microscopy) under field conditions. The relatively low accuracy of microscopy under field conditions explains its poor performance relative to rapid diagnostic tests.
Previous economic analyses of malaria diagnostic tests for febrile patients have ranged from those based on simple formulas (comparing the cost of treatment to the cost of the test plus the cost of treatment across differing levels of prevalence) to more advanced analyses seeking to incorporate test accuracy and the cost of false results.8 19202122 A recent study evaluating the effectiveness of introducing new diagnostic tests for malaria included several novel elements, such as the infrastructure required for the tests and subsequent accessibility, and an estimate of “harm of treatment” associated with antimalarials.8 The cost implications of inconsistent response to test results have, however, largely been ignored, despite the overwhelming evidence that this is a problem throughout Africa. This study shows the importance of allowing for compromised consistency in response to test results, both in terms of implications for expenditure on diagnostic tests and treatment and for potential health outcomes. These implications are likely to hold for economic assessments of diagnostic tests other than those for malaria.
Limitations
The study assumed that only febrile patients were tested, although in reality almost 20% of antimalarials prescribed were given to patients who did not have a history of fever, of whom only 1.7% were parasitaemic. This runs counter to guidelines such as those of WHO and the Tanzanian Ministry of Health and will increase the waste of resources.23 24 In addition, the study evaluated the impact of testing on prescribing antimalarials only, although tests may influence clinicians’ behaviour in prescribing additional drugs, especially antibiotics.
Three further potential limitations of the methods should be highlighted. Some of the data within the model had to be estimated from indirect sources because locally relevant data are not available. These points are highlighted as “expert opinion” in the tables. Secondly, the model does not differentiate between patients with mild symptoms but varying levels of parasitaemia, largely because the evidence that level of parasitaemiainfluences outcome is mixed. Thirdly, the methods of estimating rate of transmission from hospital data are inevitably not precise. Characterising transmission is prone to numerous difficulties. In this study we used prevalence of parasitaemia among febrile patients, although this method is subject to several limitations. An alternative method would be the use of serological markers as evidence of longer term exposure,25 although the impact of this method on results obtained in our model would be small. Data for case fatality rates for untreated malaria, for instance, cannot be measured precisely, and the interaction between case fatality ratesand rates of transmission and age adds an additional level of uncertainty. Future models could allow users to enter their own estimates to observe the impact of their variation on final results.
Policy implications
This study has confirmed findings that show the potential cost savings from use of a diagnostic test before prescribing antimalarial treatment, if test results are used to guide the prescribing of antimalarials. It also shows, however, that in some circumstances diagnostic tests rapidly become less attractive when prescribers even only partially ignore negative results. The primary implication is therefore that response to test results must be taken into account in designing public health programmes, as investing in rapid diagnostic tests without changing prescribing behaviour is unlikely to be cost beneficial.
Currently there are only preliminary data as to how this behaviour change can be achieved, and change is unlikely to be easy. Both the current levels of response to treatment guidelines and the methods to improve these are under-researched.26 Influencing prescribers’ behaviour might require changing the whole approach to treating febrile illnesses, and the reasons for current (irrational) practice need to be understood. If prescribers’ behaviour is driven by a perception that patients expect to be treated for malaria, then either patients’ expectations or prescribers’ perceptions will need to be changed. Unfortunately there is little evidence on how best to do this in Africa; the relevance of lessons learnt in developed countries (for example, reducing overprescription of antibiotics) is uncertain. Effective methods to support behaviour change need to be identified and pursued to ensure that the use of rapid diagnostic tests and artemisinin combination therapies is cost beneficial.
Investment in improving the quality of field microscopy is also a priority. In this analysis, because of the low levels of sensitivity of microscopy seen in practice, its use currently results in high total costs in most circumstances, and with high prevalence of malaria the total costs increased further as response to test results improved. Where microscopy is used for routine diagnosis, measures to ensure higher accuracy as well as response consistent with test results are necessary to maximise cost effectiveness. Changing prescribers’ behaviour will have costs, but these can be offset by the savings gained with better response to test results.
What is already known on this topic
Introducing new antimalarials requires reconsidering presumptive treatment policies
Use of rapid diagnostic tests before prescription of artemisinin combination treatments can be a cost effective measure in the management of febrile disease
Clinicians often respond to negative results of tests for malaria by ignoring them
What this study adds
At some of the currently observed levels of response to test results, the use of diagnostic tests will not be cost beneficial
Response to negative test results should be monitored and where necessary consistency should be enhanced before rapid diagnostic tests are widely disseminated
Response to test results should be taken into account in economic evaluations of diagnostic methods
Footnotes
The trial was sponsored by the Gates Malaria Partnership with funds from the Bill & Melinda Gates Foundation. It is part of the Joint Malaria Programme of NE Tanzania, a collaboration between the National Institute for Medical Research in Tanzania, Kilimanjaro Christian Medical Centre, London School of Hygiene and Tropical Medicine, and the University of Copenhagen. Alan Minja, Kereja Mlay, and Rajabu Malahiyo were key hospital staff in each site. Anna Mtei, Emmanuel Mwakasungula, Lilian Ngowi, Boniface Njau, Yustina Mushi, Happiness Manaso, Mary Urio, and Nico Funga collected the outpatient data. Magdalena Massawe, James Kalabashanga, and Hatibu Athumani read research blood films. Thanks to all the patients and staff in study sites for their support and participation.
Contributors: HR, CJMW, HM and RM facilitated the trial and data collection; Raimos Olomi contributed to the study design; SC was responsible for data management and primary analysis; YL and AM conducted the economic analysis; YL, HR, CJMW and AM wrote the initial draft; all authors contributed to the writing of the final manuscript. YL and AM are guarantors.
Competing interests: None declared.
Ethical approval: Ethics committees of the National Institute for Medical Research, Tanzania, and the London School of Hygiene and Tropical Medicine.
Provenance and peer review: Not commissioned; externally peer reviewed.