Article Text
PDF
Abstract
Background Medication errors are known to be a widespread problem affecting patient safety and treatment efficacy. We were approached by a Guatemalan clinic interested in piloting an app to aid their junior clinicians in correctly calculating medication dosages.
Methods We programmed a medication-dosing app using CommCare. In a prospective study among junior physicians at a small high-throughput clinic, we primarily assessed the app's dosing accuracy and efficiency. Secondarily, we measured the app's usability and effect on patient-centredness.
Results Six clinicians aged 21–24 tested the app. Among 366 prescriptions, dosages were 40% more likely to be correct when calculated using the app (relative risk: 1.39; 95% CI 1.16 to 1.68; p=0.0005). Accuracy improved from 64.7% (N=156) to 92.4% (N=210). Using the app in a time-constrained context improved clinician efficiency by over 20% with a decrease in average consultation time of 1.5 min (p<0.0001) to 5.23 min on average (N=178). However, questionnaires revealed most clinicians did not believe the app improved efficiency, and none thought its recommendations were ‘always accurate’. No change in patient-centredness was observed (N=167).
Conclusions The app was shown to be safe and efficient. Making this app available to junior physicians may significantly improve patient safety by enhancing dosing accuracy. This study demonstrates that dosing apps can be an efficacious means of decreasing medication errors in developing countries. We found that different strategies to introduce novel apps to providers might improve providers’ trust in the technologies and thereby make apps more efficacious.
- mHealth
- Medical Apps
- Global Health
Statistics from Altmetric.com
Introduction
In 2014, we were asked by Primeros Pasos, a non-profit clinic located in Quatzeltenango, Guatemala, to design and pilot a medication-dosing app. Their rotating junior clinicians had persistent difficulty in correctly calculating medication dosages. Worldwide, errors made while prescribing medications are a preventable, dangerous and strikingly prevalent problem.
Prescribing errors occur in 50% of hospital admissions in the UK1 and account for 70% of medication-related errors that can result in adverse effects.2 Adverse drug events caused by a prescription error can cause serious harm by doubling a patient's risk of mortality and doubling the length of hospitalisation.3 ,4 A meta-analysis found that junior physicians make prescription errors in 4–82% of their prescriptions.5 Incorrect dosing—which accounts for more than half of all medication errors4—is the type of prescription error most associated with preventable adverse drug events.6 The difficulty of calculating paediatric dosages puts paediatric patients at an even greater risk for experiencing an adverse drug event.7 ,8 Medication errors are not confined to medically sophisticated countries, though only limited data are available on the scale of the problem in developing countries.9
In developing countries, paper-based medication dosing support tools are readily available. However, these are limited by their paper format and often do not incorporate information pertinent for proper dosing. In the USA, current best practices for dosing medications necessitates that clinicians know multiple relevant dosing rules and have the time and focus necessary to perform all needed calculations. Errors are frequently made when providers are unaware of relevant dosing rules (eg, not knowing to alter a drug dose for pregnant patients).10–13 Automated systems are a recommended means of reducing prescription errors.14 ,15 Thus far, strategies to improve dosing practices in developing countries remain largely unexplored.
In the USA, 90% of physicians use mHealth apps to reference drug information.16 However, there are extremely limited/no data available on the efficacy of even commonly used medical reference apps.17 mHealth tools have exciting potential for improving multiple aspects of healthcare delivery worldwide.18 ,19 A systematic literature review of the mHealth tools used to train healthcare professionals in developing countries concluded that mHealth tools have great potential but, overall, lack demonstrated success.19
We developed and then studied an app that utilises the standard-of-care in medication dosing to efficiently provide clinicians with precalculated drug dosages. An earlier version of this software was run on ‘candy-bar’ style phones and was shown to be preferred to paper-based dosing guidelines among community health workers in rural Mexico.20 JBS and JBA travelled to Quetzaltenango, Guatemala, to study the app's usage among junior physicians at a rural clinic. To the best of our knowledge, this is the first study looking at the impact of an original mHealth dosing tool on the safety and efficiency of drug dosing in a resource-poor setting.
Methods
Study design and participants
In this prospective study, we evaluated an mHealth dosing tool's effect on medication dosing accuracy and efficiency among junior clinicians in Guatemala. The study was carried out in the Palajunoj Valley of Quetzaltenango, Guatemala, where our partner organisation, Primeros Pasos, operates. The organisation sees 91% of the children in a mostly-indigenous patient population of approximately 15 000.21 Throughout the year, six Guatemalan externos—equivalent to interns in the USA—work at Primeros Pasos on 3-month rotations. These providers were each given an identical Android-based smartphone with the preloaded dosing app and were allowed to bring home and practice using this phone prior to the study.
Provider accuracy and efficiency were assessed on multiple days over 7 weeks, during which the externos were tasked with seeing as many paediatric patients as possible in 3–4 h. During these ‘mobile clinics’, three providers travelled to a rural community, where each saw 20–40 patients in a repurposed school classroom. They treated diseases common in tropical areas, such as diarrhoea, and those caused by parasites and amoebas. Because the externos staffed a full-time clinic in addition to the ‘mobile clinics’, a maximum of three of the six externos were simultaneously present at the study site at any given time. The app was incrementally rolled out where an additional provider received the phone after an entire class of students passed through the clinic, until all clinicians had received the app. This process was to ensure that possible dosing errors caused by unfamiliarity with the technology did not go undetected by JBS and JBA.
We obtained written consent from clinicians prior to the study (see online supplementary appendix 1). All users received individual training on the mHealth tool at no cost, and were given the option of receiving training but not taking part in this study. The IRB at Harvard Medical School reviewed this study protocol and exempted the study.
Overview of the mHealth tool
The mHealth tool's clinical content was authored by DP and previously piloted on ‘candy-bar’ style phones in a classroom setting with community health workers.20 For this study, JBA and JBS reprogrammed the tool's interface using CommCare, an open source platform developed by the mHealth company DiMagi Inc. It was specifically designed to run on Android-based smartphones and has full functionality without the requirement of cellular connectivity.
The mHealth tool has a user interface that guides users through the medication dosing process in a stepwise fashion (see online supplementary appendix 2). Once a specific medication is chosen, the programme navigates novel decision trees designed by JBA to ensure that each user-prompt omits irrelevant variables. Hence users are asked only for the information necessary to achieve the correct dose. Once all clinically relevant information has been entered, the clinician is given the exact numerical output for medication dose in addition to formulation-specific dosages in whole, half and quarter-pill increments (for tablets and pills) or teaspoons (for liquids) (see online supplementary appendix 3).
For this study, a physician at our partner organisation limited the drug database supported by the app to 10 medications, based on the clinic's drug usage (see online supplementary appendix 4). The user interface was translated to Spanish by a bilingual medical student. The dosing guidelines utilised in the app were compiled by DP using common medical references such as UpToDate and Epocrates.
Procedures
Content accuracy was extensively checked prior to clinical deployment and validated with 5% LQAS sampling.22 Clinicians testing the electronic reference tool were trained in one-on-one sessions and given time to practice independently prior to deploying the app in clinical contexts. Whenever medications were dosed using the app—but before patients left the clinic—JBS and JBA verified prescriptions against current clinical guidelines. When JBS and JBA detected a dosing error, the error was recorded, and if the dosing discrepancy was not made intentionally on the basis on the physician's clinical judgment, the prescription was corrected before the patient left the clinic. User efficiency was measured simultaneously. JBS and JBA only recorded efficiency data for clinical encounters in which 1 of the 10 medications supported by the app was prescribed (see online supplementary appendix 4). Later, JBS and JBA reviewed pharmacy records to confirm the prescriptions given out by each provider.
Outcomes
The primary outcomes we measured were dosing accuracy and user efficiency. JBS and JBA assessed accuracy by measuring the incidence of medication dosing errors. Dosages in this study were counted as incorrect when prescribed medication dosages deviated outside the range of current clinical guidelines.6 DP created a list of clinical standards, which Primeros Pasos clinicians approved.
Efficiency was assessed with the average time of the providers’ clinical encounters. These were measured as the time interval between when a patient sat down with a provider to when their interaction was finished and the patient stood up to leave.
The secondary outcomes we measured were the usability of the app and its effect on patient-centredness. To assess usability, the externos were asked quantitative and qualitative questions at the end of the study, covering four themes associated with technology adoption in resource poor settings: (1) comfort (ie, how easy was the tool to use?), (2) acceptability (ie, how appropriate was the tool for dosing medication?), (3) preference (ie, was the tool better than prior practices?) and (4) accuracy (ie, did providers trust that the tool gave clinically correct information?).20 JBS and JBA sent online surveys to the externos after the study had concluded and the externos were informed that their individual answers would not be shared with Primeros Pasos.
To assess patient centredness, all adult patients at the Primeros Pasos clinic were asked whether they believed their provider had resolved the issue that they had come in for, and whether they believed their provider cared about them. A pharmacist asked all questions in Spanish at the end of each patient's visit. The pharmacist was blinded to whether the patient's medications had been dosed manually or with the electronic reference tool.
Statistical analysis
We calculated descriptive statistics for 366 medication prescriptions evaluated for dosing accuracy and 276 clinical encounters assessed for efficiency. To test for changes in accuracy and efficiency preimplementation and postimplementation of the app, we conducted analyses using generalised estimating equations with an exchangeable correlation structure and robust SEs, allowing us to account for clustering among providers. Efficiency (time in minutes) was assessed as a continuous variable and modelled using the identity link and normal distribution. Accuracy was treated as a dichotomous outcome and modelled using a log link and a binary distribution.23
Results
All six externos at the clinic consented to take part in the study. The externos ranged in age from 21 to 24. All externos owned their own personal smartphones prior to the study.
The first primary outcome we assessed was the impact of the app on dosing accuracy. Dosing accuracy improved from 64.7% (among 156 prescriptions) to 92.4% (among 210 prescriptions) when providers used the app. Relative to dosages prescribed prior to app implementation, dosages prescribed after implementation were 40% more likely to be correct (relative risk: 1.39; 95% CI 1.16 to 1.68; p=0.0005). This overall increase in dosing accuracy translated to improvements at the individual provider level: all providers appeared to dose medications more accurately after the intervention (figure 1).
Dosing accuracy by provider. Changes in dosing accuracy varied by provider. Changes in dosing accuracy were: (1) 4.97% (n=32 before, n=34 after), (2) 28.8% (n=29 before, n=45 after), (3) 66.5% (n=21 before, n=29 after), (4) 116% (n=34 before, n=23 after), (5) 16.6% (n=7 before, n=66 after), (6) 32.4% (n=33 before n=13 after).
The second primary outcome we assessed was the app's impact on provider efficiency. Under pressure to maximise the number of patient visits, the 98 patients treated prior to the intervention were seen in 7.15 min on average compared to 5.23 min among the 178 patients seen after introducing the app. Overall, the app yielded consults were, on average, 1.5 min shorter than consults without the app (−1.53; 95% CI −2.07 to −0.99; p<0.0001). All users had shorter clinical encounters when using the application (figure 2).
Efficiency by provider. Though clinicians varied in the average time of clinical encounters, all clinicians had a shorter time of average clinical encounters when using the application. Improvement by respective providers was (1) 14.4% (n=53); (2) 18.1% (n=44); (3) 33.3% (n=31); (4) 19.7% (n=35); (5) 19.6% (n=68); and (6) 24% (n=45). Overall, the app yielded consults that were, on average, 1.5 min shorter than consults without the app (−1.53; 95% CI −2.07 to −0.99; p<0.0001).
Our secondary outcomes were the usability of the app and the impact of the app on patient centredness. All clinicians responding to our survey indicated that the app was ‘easy’ or ‘very easy’ to use (n=5; 1 lost to follow-up). During training sessions, all the externos expressed comfort using the mHealth application.
Two weeks after the study, 3/5 of the respondents continued to use the application with the same frequency as during the study, and 2/5 continued to use it but with less frequency. One externo who used the application less frequently indicated that this was because the phone did not carry enough medications. In free-response questions about what users did not like, 4/5 of respondents noted that they did not like the fact that the phone only supported a limited number of medications. Another externo who did not continue to use the application after the study believed it was faster to use their cellphone calculator to come up with a prescription dose. One of five of respondents thought that using the application was faster than not using it, 2/5 thought it was the same speed, and 2/5 believed it was slower to dose a prescription using the application.
In discussing what they liked about the phone, one externo wrote that they liked ‘the ease of calculating drug doses’. Another discussed how ‘It has several formulations of each medication, and you can use the conversation between millilitres and tablespoons so you can explain it better to patients’. Three of five of the clinicians thought the doses provided by the phone were, ‘frequently correct’, though 2/5 thought they were ‘sometimes incorrect’. None of the respondents indicated that they believed the doses provided by the phone were ‘always correct’. The two externos who used the phone with less frequency indicated after the study ended that they believed the doses provided were ‘sometimes incorrect’.
Among 167 patients queried, no change in patient-centredness was observed. Before and after the intervention, 100% of patients reported that their provider had resolved the issue they had come for, and that they believed their provider cared about them.
Conclusions
Medication errors are a widespread and well-recognised problem. In response to a request from a clinic in Guatemala, we developed and then tested an app intended to help clinicians correctly dose medications. Our study determined that the application is more accurate and more efficient than current practices. By reducing the chances of an adverse drug event, the app had a positive effect on patient safety. In developing countries, mHealth tools are a promising yet largely unproven means of improving healthcare delivery.19 This was the first study we are aware of to measure the effect of a novel medication-dosing app on the safety and efficiency of drug dosing in a resource poor setting. Our results demonstrate that dosing apps can be an efficacious means of decreasing medication errors in developing countries.
A limitation of this study is that the app may have affected non-dosing medication errors in a few ways we did not assess. First, because the app's dosing recommendations were split into instructions for providers and instructions for patients, by giving patients simple directions for example, ‘take one tablespoon’, instead of ‘take 15 mL’—medication errors made by patients at home may have potentially decreased through improved instruction. Second, to ensure that patients received enough medication to complete a drug regimen, the app's directions to providers included instructions about how many bottles or pills to give a patient. However, medication errors from improperly truncated treatment duration were not assessed by our study design. Finally, the app provides instructions to clinicians with easy-to-understand directions to read to a patient. We did not assess changes in medication errors caused by provider miscommunication. In the future, phrasing of patient instructions can be tailored to further improve patient communication in culturally-specific contexts. The CommCare platform that we used allows for these directions to be customised in future endeavours as well.
While the application significantly improved dosing accuracy to near perfect, clinicians using the app still made occasional dosing errors. It is not clear how many errors were made by users intentionally ignoring the recommendation from the application or providers improperly using the application (eg, inputting the wrong weight). One limitation of our study was that our criteria for a medical error did not assess clinical judgment—the former type of error—when providers knowingly prescribed against current dosing guidelines. For example, we observed six situations in which a patient technically needed ¾ of a pill for a dose of metronidazole, but if the provider thought the patient would not be able to break pills into quarters without pulverising them, the provider instructed the patient to take ½ of a pill per dose instead. These situations were counted as errors in our analysis and accounted for 37.5% of postintervention dosing errors. Prior research has elucidated that there is a discrepancy between prescriber and researcher definitions of medication error.1 If these intentionally erroneous prescriptions were not counted as errors, the overall dosing accuracy using the app would have improved more than we report here.
Surprisingly, some providers reported in the user survey that the app's dosing recommendations were ‘sometimes incorrect’. Similarly, some providers thought that using the application was ‘slower’ or ‘the same speed’ as dosing medications without the app. Yet our programme incorporated standard-of-care drug dosing rules, and we measured that using the app improved clinical efficiency for every user. Providers who believed the app was slow or inaccurate were less likely to continue using it after the study ended. These results indicate to us that provider trust in our app affects its efficacy and adoption. Thus, when designing novel apps to meet the needs of community partners, how the technology is introduced needs to be a primary consideration. New strategies to improve provider trust might include the introduction of the technology by established clinicians rather than medical students. Alternatively, apps could be targeted to patients—empowering them to confirm that the care they receive meets the standard-of-care. Such strategies that improve trust would in turn make novel apps even more effective.
Acknowledgments
The authors thank the Scholars in the Medicine Office at Harvard Medical School for funding this study. The authors gratefully acknowledge our partner Primeros Pasos and, in particular, Jamie Pet, for making this study possible. The authors thank Mary-Ellen Garner from Rollins College and Talia Gerstle from the University of Michigan, for helping collect data on application efficiency. The authors also thank Lucy Martinez from the David Geffen School of Medicine at UCLA for translating the prototype into Spanish. The authors thank the Boston University School of Public Health SPH IH 804 RapiDose team for volunteering consulting work on the prototype CommCare application. Finally, the authors thank the six Guatemalan externos for their time, feedback and commitment to improving patient safety.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online appendix
Footnotes
JBS and JBA contributed equally.
Contributors JBS and JBA contributed equally. JBS and DP conceptualised and designed the study. JBS and JBA carried out the study in Guatemala. JBA was the lead programmer. JBA and DP developed the app content. JBS analysed preliminary data and drafted the report, and, together, JBS, JBA and DP interpreted the results. MFF performed the statistical analysis. All authors reviewed, edited and approved the final draft of the manuscript.
Funding This study was funded by the Scholars in the Medicine Office at Harvard Medical.
Competing interests None declared.
Provenance and peer review Not commissioned; internally peer reviewed.