Reminder function

DATs provide reminders to patients, addressing forgetfulness, which is a common barrier to adherence.8 Forgetfulness is commonly thought of as a cognitive barrier to adherence, but it also reflects psychosocial and structural barriers faced by patients, such as forgetting doses due to alcohol use or working multiple jobs. Reminders may promote habit formation in pill-taking behaviour.15 32 For most DATs, reminders take the form of SMS texts. Digital pillboxes have embedded audiovisual reminders (eg, glowing light and ringing sound), which have the benefit of prompting patients to the site where medications are stored.

Digital observation of dose-taking

Most DATs digitally ‘observe’ or record dose-taking, which is especially relevant in TB given the historical reliance on DOT. VDOT mimics DOT by allowing HCPs to view patients swallow pills. As with DOT, VDOT may raise concerns about patient autonomy—as patients may feel that being watched taking every dose is an invasion of privacy—although new technologies using automated facial recognition and pill identification could obviate the need for HCPs to watch every video.35 36 For some DATs, such as two-way SMS-based strategies, 99DOTS and VDOT, an extra step is needed to report dose-taking that requires effort by the patient herself, such as responding to an SMS text, placing a phone call or getting on a video call. For these technologies, dosing histories are compiled based on patient responses, and inaccuracies in the dosing history may be introduced because patients could send SMS responses or phone calls without taking doses (ie, over-reporting) or take doses without sending SMS responses or phone calls (ie, under-reporting).

Digital pillboxes may minimise patient effort and the risk of HCPs accidentally mis-recording information on paper records because opening and closing the pillbox is digitally recorded as a ‘dose taken’. However, there are other potential limitations to their accuracy—for example, if a patient removes medication blister packs from the pillbox, allowing doses to be taken without opening it. Ingestible sensors record pill-taking with relatively high sensitivity and specificity37 38; however, removal of the adhesive monitor that records information transmitted by the ingested sensors would result in under-reporting of adherence. For all of these strategies, inaccuracy in patient reporting may be reduced by patient education about the purpose and appropriate use of these devices.

Compilation of dosing histories

DATs compile patient dosing histories, which allow for ‘real-time’ or clinic visit–based adherence monitoring. In real-time approaches, doses are recorded right after the patient engages the technology (eg, opening the pillbox or sending an SMS response). HCPs remotely access these histories on a web-based interface, allowing them to identify non-adherence before the patient’s next medication refill visit (figure 3). In clinic visit–based monitoring, HCPs access dosing histories during patient visits, for example by uploading the record to a computer from the patient’s digital pillbox. HCPs can then counsel patients using dosing information compiled since the last clinic visit. For example, in a study from China, at each clinic visit, the doctor reviewed the patient’s dosing history for the prior month on a computer, discussed the reason for missed doses and switched patients who missed numerous doses to more intensive management strategies.39

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Example of how digital adherence technologies involving daily reporting of dose-taking could potentially facilitate earlier identification and intervention to address medication non-adherence. The 99DOTS model is used for illustrative purposes. Each box represents a calendar day on the dashboard viewed by healthcare providers. Green boxes represent doses that were ‘called in’ on a given day and red boxes represent doses that were not called in. SMS, short messaging service.

Triage and provision of differentiated care

In contrast to DOT, in which patients are treated using a uniform approach, DATs can facilitate triage of patients based on different levels of adherence. Triage may be performed by HCPs during routine reviews of dosing histories. Alternatively, computer algorithms can be used to triage and alert HCPs about non-adherent patients to prompt early intervention, saving time for HCPs and potentially preventing patient loss to follow-up (figure 3). Triage can then facilitate escalation in the intensity of care for patients with a high level of non-adherence. For example, in a randomised trial of DATs in China, patients who missed three to six medication doses were switched to ‘intensive management’ consisting of weekly HCP visits to the patient’s home. Those who missed >7 doses were switched to DOT.39

While DATs may help identify poor adherence, a careful clinical evaluation is needed to address its causes, which are often complex and may include psychosocial, treatment-related and health system–related barriers. As such, identification of non-adherence using DATs should ideally be viewed as the starting point for more intensive face-to-face engagement with patients to understand their specific reasons for non-adherence, so that individualised packages of care can be provided (ie, ‘differentiated care’). By triaging patients, HCPs may be able to spend more time on these high-risk patients to address their needs. Triage and differentiated care have the potential to focus limited resources on higher-risk patients, which may improve the efficiency of care delivery.

Acceptability and ethical questions

Acceptability may differ based on the type of DAT and the context in which it is deployed. Development of DATs should ideally follow a participatory approach that iteratively incorporates views of users and stakeholders—including patients with TB, caregivers and HCPs—in order to improve acceptability.40 41 Models that have been shown to predict acceptance of health technologies, such as the Unified Theory of Acceptance and Use of Technology and the Technology Acceptance Model, can guide evaluations of DAT acceptance.42–44 In general, these models evaluate ‘ease of use’ (ie, how easily users are able to learn the DAT), ‘perceived usefulness’ (ie, whether users think it is valuable for TB care) and the availability of infrastructure to support their use. Frameworks have also been proposed for evaluating ethical aspects of DATs.45 46 These frameworks emphasise evaluation of patient autonomy (including concerns about ‘surveillance’),47 privacy and confidentiality, stigma and intrusiveness, and trust between patients and HCPs.45 46

Patient literacy—with regard to language (for two-way SMS), using a feature phone (for two-way SMS or 99DOTS) or using a smartphone (for VDOT or ingestible sensors)—should be assessed in any evaluation of acceptability. In populations with low language literacy, providing reminders in local languages or voice messages (rather than SMS texts) may expand the reach of some DATs. Researchers should also assess optimal programming of a DAT’s reminder function. Reminders that are programmed for the wrong times—for example when patients are sleeping or at work—may make it difficult to respond in a timely manner. Overly frequent reminders may result in patients opening pillboxes just to shut off reminders or discarding SMS texts before they are read.39

Maintaining privacy of a patient’s TB diagnosis is an important aspect of acceptability45 46 because stigma can result in discrimination or negatively affect a patient’s coping capacity.6 48 Unintentional disclosure of disease status could occur if other individuals see a patient’s SMS texts,49 medication envelopes (for 99DOTS), digital pillbox,46 video observation session or adhesive monitor (for ingestible sensors). This may especially be a problem in settings where there are high levels of shared cellphone use within families. Confidentiality may be breached if unauthorised persons access digital adherence data, which may particularly be a problem in settings where regulations for electronic health data are lacking or not enforced. Understanding how cultural characteristics shape patient tolerance for loss of privacy and confidentiality may enable user-centred design. For example, use of a password allowing SMS texts to be read only by the patient may be an option to protect privacy in some settings.50

Studies of SMS texting reminders (conducted in Peru, Argentina, Uganda, Pakistan and China),51 ^– 52 VDOT (conducted in the USA, Mexico and Kenya),53–56 and a digital pillbox (conducted in China)57 have generally shown high acceptability of these technologies by patients in surveys and qualitative interviews. Notably, however, one study from the USA suggested low acceptability (33%) of SMS dose reminders, and SMS response rates in actual implementation are often lower (eg, 60%–80% at best) than indicated in acceptability surveys.52 58

Feasibility

Despite widespread cellular ownership in many low-income and middle-income countries (LMICs), there is still considerable variability in levels of cellphone access and cellular network coverage by country as well as potentially within specific subpopulations within countries. As such, feasibility challenges remain in some settings, including the following for cellphone-based strategies: limited cellphone ownership, use of shared cellphones, low cellphone literacy, poor audio or video quality,59 poor cellular network connexions, technical failures preventing receipt of SMS texts, electricity outages and changing phone numbers.49 60 Feasibility challenges for digital pillboxes may include battery failure, device malfunction (leading to data losses) and problems related to cellular networks. In general, there have been ongoing improvements in the feasibility of DATs. For example, the battery life of digital pillboxes has improved to as long as 6 months. Some devices temporarily store data in flash memory or use GPRS (General Packet Radio Service) to maintain data in transit until acknowledgement of receipt by the web server, reducing data losses due to power failures or poor cellular connectivity.61

Surveys conducted in Argentina and China suggested feasibility of SMS-based strategies due to high cellphone ownership and literacy.62 ,52 However, a recent study from Peru highlights that, although cellphone access may be high in the general population, access may be considerably lower in patients who suffer from TB and in particular for patients with TB with poor treatment outcomes.63 Implementation studies from high-income and middle-income countries (USA, Canada, Belarus, Mexico) have suggested that VDOT has high feasibility,53 64–66 though some patients had to be shifted back to in-person DOT53 and low video or audio quality sometimes made dose observation difficult.65 67 Ingestible sensors had high feasibility in studies conducted in the USA and Mexico, with >95% of sensor signals detected after ingestion.37 38

In some settings, a more fundamental barrier to implementation of DATs may be health system resource constraints, such as lack of computers in clinics to view dosing histories and shortages of HCPs who could act on this information to address non-adherence in high-risk patients.

Accuracy of digital observation

Each DAT has limitations outside of technical failures that may reduce its accuracy for verifying true medication adherence, resulting in over-reporting (ie, false-positive signals in the dosing history) or under-reporting (ie, false-negative signals). Strategies relying on self-reporting via SMS texts or phone calls are at particular risk for under-reporting of adherence if patient engagement wanes due to ‘technology fatigue’, as illustrated in a two-way SMS intervention in Pakistan (figure 4).68 Digital pillboxes may under-report adherence if medication blister packs are taken out of the pillbox so that doses can be taken without opening the device or from device non-use due to patient travel or stigma. Ingestible sensors could under-report true adherence if the adhesive monitor used to record signals from the ingestible sensors is removed. Alternatively, over-reporting may occur if patients indicate adherence via SMS texts, phone calls or pillbox openings without taking the doses, though such behaviour often wanes over time.69

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Example of ‘technology fatigue’. Patient response rates to short messaging service (SMS) texts to indicate dose ingestion declined throughout the course of tuberculosis therapy in a study of a two-way SMS intervention in Pakistan, reducing the accuracy of this monitoring approach. Source: Mohammed et al.68 RR, response rate.

As such, evaluation of accuracy in research studies is critical to ensure that a DAT provides a reasonable proxy of true adherence. When conducting such research evaluations, careful use of nomenclature is important because it is easy to conflate the quality of a patient’s medication adherence with the quality of her engagement with the DAT, even though these two are not the same. Building on nomenclature recommendations for describing medication adherence,70 we suggest parallel nomenclature for describing a patient’s engagement with DATs (table 2).

Evaluations of accuracy generally compare DAT-compiled dosing histories to an alternative adherence metric to determine the DAT’s sensitivity, specificity, positive predictive value and negative predictive value for true adherence. These alternative metrics include biological tests of drug ingestion (eg, urine testing for isoniazid content), pill counts, refill data or concurrent monitoring with another DAT, such as use of a ‘silent’ digital pillbox (ie, reminder function disabled) in patients using feature phone-based strategies (online supplementary appendix 1 for details). There is no ‘gold standard’ metric since all comparators, including biological tests, have their own inaccuracies; examples include interindividual and intraindividual variation in drug absorption and metabolism.71–73

We recommend using multiple comparators to gain the fullest understanding of a DAT’s accuracy. For example, urine isoniazid testing provides a ‘snapshot’ of dose-taking that can be compared with DAT dosing histories over the prior 24–72 hours, while medication refills provide data on persistence with therapy that can be compared with longer-term dosing histories. Collecting biological test or pill count data during unannounced home visits (ie, without prior notice) may help to minimise the ‘Hawthorne effect’—that is, short-term changes in patient adherence or DAT engagement in anticipation of clinic visits.74 In patients concurrently taking medications for other comorbid conditions, such as HIV or diabetes, it is also important to assess the impact of a DAT on medication adherence for all conditions, if possible.

Few studies have rigorously evaluated the accuracy of DATs. One study in China found high correlation between dosing histories from a digital pillbox and urine rifampin test results.75 A small pilot study in South Africa found high correlation between dosing histories from digital pillboxes and pill counts conducted for patients taking therapy for MDR TB and HIV.76 Studies have suggested that VDOT and ingestible sensors may be more accurate than in-person DOT since they are able to ‘observe’ a greater proportion of doses, especially on weekends and holidays.38 67

Adherence and treatment outcomes

Health outcomes—including medication adherence, treatment success and post-treatment recurrence free-survival—are the most important indicators of DATs’ impact on TB care (table 3, online supplementary appendix 1). When assessing health outcomes, especially in randomised trials or quasi-experimental studies, it is important to identify the appropriate comparator representing the baseline standard of care against which a DAT-based care model will be compared. In many settings, DAT-based models should be compared against DOT, recognising that DOT models may vary from setting to setting, including facility-based DOT, in-person DOT, and community or family DOT (table 1). In other settings, SAT may be the standard of care, though SAT models may also vary based on the frequency of patient–provider contact or medication refills (eg, biweekly, monthly etc). When DAT-based care models are compared against SAT models, we recommend that the SAT model should at minimum include a protocol for patient outreach and engagement if a patient misses a medication refill date since missing a refill may be suggestive of non-adherence. Comparing DAT-based models to SAT models that follow up on missed refills may reveal whether providing HCPs with more granular day-to-day real-time information from DATs (a proposed ‘value-add’ of some of these technologies) actually leads to better outcomes than a more crude but simple approach of following up promptly on patients who miss their medication refill dates.

While a variety health outcomes may be assessed, improvements in surrogate endpoints that DATs may affect directly (eg, medication adherence) may not always translate into impacts on longer-term outcomes (eg, treatment success or recurrence-free survival) because improvements in longer-term outcomes often require addressing multiple aspects of quality of care, such as ensuring early diagnosis, drug-susceptibility testing, appropriate medication dosing and so on.77

As such, we recommend that researchers initially focus on assessing whether DATs improve TB medication adherence. In settings where DATs transform care considerably (eg, shifting away from facility-based DOT), it is also important to routinely monitor treatment success rates to ensure that these outcomes remain comparable, at minimum. When evaluating treatment outcomes, it is important that the entire ‘package of care’ involving the DAT be well defined—including approaches for triaging patients based on dosing histories and the interventions that will be delivered to non-adherent patients.

The study design used to evaluate health outcomes depends on the study goal and resources available. Cohort studies can evaluate whether health outcomes are achieving minimal standards recommended by TB programmes; however, this design is not optimal for understanding whether DATs themselves have contributed to improvements or deterioration in outcomes compared with prior standards of care. Historical programmatic data may be vulnerable to inaccuracy and provide a poor baseline for understanding the relative benefits of DATs.

Quasi-experimental designs that evaluate outcomes prior to and after rollout of a DAT-based intervention may provide more helpful evidence regarding changes in outcomes; however, these findings may be confounded by other concurrent changes to TB care. Trials using randomisation of patients or larger units of care (eg, health facilities) to DAT-based interventions provide the most rigorous evidence of impact on health outcomes.

As highlighted in a recent systematic review,24 evidence regarding the impact of DATs on TB outcomes remains limited. Randomised trials of SMS strategies have not shown improvements in adherence or treatment success,39 52 68 78 with the exception of a Kenyan study that included prompt engagement by HCPs for patients who did not report pill-taking via SMS.79 One South African study of digital pillboxes found improved TB cure rates compared with historical controls, though cure rates in the controls were poor compared with international standards.80 A more rigorous cluster-randomised trial in China found that patients in study arms monitored with a digital pillbox had reduction in medication non-adherence (ie, patient-months with >20% of doses missed) compared with the standard of care.39 Cohort studies of patients monitored by VDOT in the USA and Australia have shown comparable outcomes to routine in-person DOT.59 67

Costs and cost-effectiveness

In settings with high baseline rates of treatment success, the primary benefit of DATs may be in reduction of costs and patient and HCP burdens associated with existing DOT or SAT models.9 26 28 The ‘costs’ of implementation should be defined broadly from a societal perspective to include material components, communication costs, new personnel, changes in time use by existing HCPs, and changes in costs and other burdens borne by patients (table 4). Micro-costing techniques (eg, time and motion studies) may allow estimation of costs resulting from changes in time and work use by HCPs.81

Estimating an incremental cost-effectiveness ratio requires data on the difference in costs between the existing care model and a DAT-based model, as well as the difference in health outcomes between the two models, ideally represented as disability-adjusted life years. As such, costing studies should collect data on the costs of existing care models (eg, DOT or SAT), and concurrent studies estimating health outcomes for the different models would be required for a formal cost-effectiveness analysis. Modelling may be required to estimate the cost-effectiveness of DATs for TB programmes a national level.21 82 Cost-effectiveness of different DATs may vary between high-income countries (where personnel costs are high) and LMICs (where personnel costs are relatively low).

A US study of VDOT factoring in HCP and patient time and travel costs estimated an average cost savings of US$2248 per patient per TB treatment course as compared with home-based DOT83; however, an Australian study found that VDOT would be more cost-effective than in-person DOT only with scale-up or decreased technology costs.59 A South African study of digital pillboxes estimated a return on investment of 23% over 5 years, largely from cost savings due to improved treatment outcomes; however, this study assumed very poor treatment success rates seen with historical controls.80

Triage strategies and provision of differentiated care

Most DATs aim to identify non-adherent patients who may require additional support to improve adherence or prevent loss to follow-up. Poorly designed interventions for addressing non-adherence may therefore attenuate the beneficial impacts of DATs. HCPs might find themselves in a ‘data glut’, without the guidance or capacity to act productively on the rich real-time data compiled by DATs. A critical question hovering over DAT-based strategies is: how can the data-rich dosing histories compiled by DATs be best leveraged to provide effective individualised (or ‘differentiated’) care that will improve adherence?

Addressing this question will require researchers and HCPs to think beyond the technologies to understand the medical, psychosocial, cultural, structural and health system barriers that contribute to non-adherence.7 Such research could inform the development (by iterative testing) of intervention packages targeted towards patients with different levels of risk for non-adherence. Examples of adherence intervention packages that leverage DAT dosing histories exist for other diseases, such as HIV.21 These packages of care could screen for and address common causes of non-adherence in each setting, which may include poor treatment literacy,84 TB medication toxicities,5 depression,5 85 86 substance use disorders,4 financial burdens7 or difficulties travelling to health facilities.7 By triaging patients, DATs may allow HCPs to focus more time and attention on a smaller group of high-risk patients, and ancillary personnel, such as dedicated counsellors or psychologists, could potentially help address their more intensive needs. To date, research evaluating triage and differentiated care strategies has been limited.