Skip to main content

Translating evidence to patient care through caregivers: a systematic review of caregiver-mediated interventions

Abstract

Background

Caregivers may promote the uptake of science into patient care and the practice of evidence-informed medicine. The purpose of this study was to determine whether caregiver-mediated (non-clinical caregiver-delivered) interventions are effective in improving patient, caregiver, provider, or health system outcomes.

Methods

We searched the MEDLINE, Embase, PsycINFO, Cumulative Index of Nursing and Allied Health, and Scopus databases from inception to February 27, 2017. Interventions (with a comparison group) reporting on a quality improvement intervention mediated by a caregiver and directed to a patient, in all ages and patient-care settings, were selected for inclusion. A three-category framework was developed to characterize caregiver-mediated interventions: inform (e.g., provide knowledge), activate (e.g., prompt action), and collaborate (e.g., lead to interaction between caregivers and other groups [e.g., care providers]).

Results

Fifty-six studies met the inclusion criteria, and 64% were randomized controlled trials (RCTs). The most commonly assessed outcomes were patient- (n = 40) and caregiver-oriented (n = 33); few health system- (n = 10) and provider-oriented (n = 2) outcomes were reported. Patient outcomes (e.g., satisfaction) were most improved by caregiver-mediated interventions that provided condition and treatment education (e.g., symptom management information) and practical condition-management support (e.g., practicing medication protocol). Caregiver outcomes (e.g., stress-related/psychiatric outcomes) were most improved by interventions that activated caregiver roles (e.g., monitoring blood glucose) and provided information related to that action (e.g., why and how to monitor). The risk of bias was generally high, and the overall quality of the evidence was low-moderate, based on Grading of Recommendations Assessment Development and Evaluation ratings.

Conclusions

There is a large body of research, including many RCTs, to support the use of caregiver-mediated interventions that inform and activate caregivers to improve patient and caregiver outcomes. Select caregiver-mediated interventions improve patient (inform-activate) and caregiver (inform-activate-collaborate) outcomes and should be considered by all researchers implementing patient- and family-oriented research.

Systematic review

PROSPERO, CRD42016052509.

Peer Review reports

Background

Patient- and family-centered care (PFCC) research, where patients and families partner with providers in the planning and delivery of healthcare, represents a shift from solely studying provider-identified priorities [1, 2]. PFCC represents a cultural change in healthcare that acknowledges that patients and their caregivers are central figures in decision-making and the delivery of care [1, 2]. Perceptions of PFCC and its benefits are high among patients and family members, though there are barriers to its implementation, including a purported lack of provider knowledge of PFCC, system barriers (e.g., lack of personnel, visiting hours), and a perceived increase in provider burden [3,4,5]. The uptake of PFCC interventions is limited by these barriers, and the science of PFCC is lagging behind scientific, clinical, operational, and public interest and desire.

PFCC may provide a mechanism for quality improvement initiatives designed to implement evidence-informed practices into patient care. Patients and caregivers can act as agents of change, making evidence more accessible and providing a method for implementing science into patient care, regardless of the clinical setting [2]. Due to the nature of their illness and the use of invasive therapies, some patients may not be able to act as active agents of change [6]; children, and a growing number of adults with complex care needs, represent populations who require a caregiver. This presents a unique role for caregivers to be engaged in PFCC, though the evidence supporting the delivery and effectiveness of caregiver-mediated interventions is limited. Because of the growing role of patients and caregivers in care, there is an opportunity to engage and empower them to promote the uptake of science into patient care and the practice of evidence-informed medicine.

In order to effectively carry out PFCC, we must create, implement, and measure the effect of PFCC interventions. A recent systematic review of patient-mediated interventions suggested benefit [7]. There is a large and diverse literature on caregiver-mediated interventions, though no systematic review has been conducted. We therefore asked the question: Are caregiver-mediated interventions (interventions delivered in some part by the caregiver) effective in improving patient, caregiver, provider, or health system outcomes?

Methods

This systematic review was conducted and reported per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria [8] (see Additional file 1 for the checklist). An a priori protocol was published online through PROSPERO (registration number CRD42016052509; Additional file 2). Ethical approval was not required, as previously published data were used.

Patient and stakeholder involvement

Our methods were informed by our stakeholders: a 2-day stakeholder (patients, families, researchers, care providers) engagement meeting was held (November 17–18, 2016, Calgary, Canada) to determine relevant, patient-oriented priorities. Attendees included three former intensive care unit (ICU) patients, four family members of former ICU patients, 19 researchers, and 10 care providers. Patients and family members were asked to identify relevant and important patient-oriented priorities in critical care medicine; the discussion was open and guided by an experienced facilitator. Study findings will be presented to these patients and family members at the next stakeholder engagement meeting planned for the Fall of 2018.

Outcome measures

Based on feedback from stakeholders, the primary research question was: Are caregiver-mediated interventions effective in improving patient, caregiver, provider, or health system outcomes? Secondary research questions were the following: (1) What types of interventions have been deemed effective? (2) Does effectiveness vary according to the intervention type, patient population, caregiver group, contextual factors, or implementation strategy? (3) How have these interventions been evaluated?

Populations, interventions, comparators, settings, and study designs

Inclusion criteria for studies were as follows: (1) interventional study design with a comparison group; (2) reporting on an intervention mediated by caregivers and directed to a patient; (3) conducted in a patient-care setting of any age group; and (4) in any language, and published at any time. Studies were excluded if they were not interventional in design, reported on interventions directed at the caregiver, or were not conducted in the context of patient care (e.g., took place at home). For the purposes of this review, we established definitions for key concepts and defined: (1) a caregiver as any non-clinical person who regularly provides support to the patient and is in some way directly implicated in the patient’s care or directly affected by the patient’s health problem; (2) caregiver-mediated as the caregiver being directly involved in delivering the intervention; (3) a patient as someone who has a health condition, is at risk for developing a health condition, or is attending a patient-care setting to receive care; and (4) a patient-care setting as any setting where a patient may receive care for an existing health condition or preventative care from a healthcare professional. We distinguish caregiver-mediated interventions from caregiver-directed interventions—where the caregiver is the target of the intervention and is not involved in delivering the intervention him/herself. Quality improvement (or implementation science or knowledge translation) was defined as the synthesis, dissemination, exchange, and application of health-related knowledge and research [9]. Studies were excluded if they were only abstracts or were not primary research.

Data sources and searches

The search strategy (Additional file 3) was developed with the assistance of a medical librarian (H.G.) and in consultation with experts in quality improvement in healthcare (H.T.S., J.P.L.). The MEDLINE (OVID), Embase (OVID), PsycINFO (OVID), Cumulative Index of Nursing and Allied Health (CINAHL [EBSCO]), and Scopus (Elsevier) bibliographic databases were searched from inception to February 27, 2017. Using subject headings, keywords, and synonyms, we searched the following domains: (1) caregivers; (2) intervention; (3) mediated; (4) quality improvement/knowledge translation; (5) clinical care. The reference lists of included papers were reviewed to identify potential studies missed in the search.

Study selection

All titles and abstracts were reviewed independently in duplicate by two reviewers (K.M.F., C.J.M.); any study selected by either reviewer at this stage was selected to move on to the next stage. The full text of all articles was reviewed independently in duplicate by two of three reviewers (K.M.F., C.J.M., D.D.); all articles selected by both reviewers at this stage were included in the final review. Disagreements were resolved by discussion or the involvement of a third reviewer when necessary. References were managed in EndNote X7 (Clarivate Analytics, Philadelphia, PA, USA).

Data extraction and quality assessment

Two reviewers (C.J.M., D.D.) abstracted data independently, in duplicate, using a data collection sheet developed and piloted by the study team. Information on study design, study characteristics (e.g., age, setting), patient population (e.g., infants, older adults; as defined by the individual papers), caregiver group (e.g., spouses, parents, family caregivers [which may include parents when undefined]), intervention, and control group was abstracted, and risk of bias and strength of the evidence (see below) assessed. Translation was sought for all non-English articles (n = 7; 2 French, 2 Cantonese, 1 Spanish, 1 Turkish, 1 Serbo-Croatian).

The Cochrane Collaboration’s risk of bias tool was used to evaluate the internal validity of all included studies [10]. To be assigned an overall low risk of bias, all domains needed to be rated as low. The presence of a single high or unclear item resulted in the overall risk of bias being rated as high or unclear, respectively. The strength of the evidence was described as low, moderate, or high according to the Grading of Recommendations Assessment Development and Evaluation (GRADE) working group approach [11].

Data synthesis and analysis

Findings were summarized using descriptive statistics ( median, frequencies) and their accompanying measures of variability ( interquartile range [IQR]) where appropriate using STATA 14.1. Meta-analysis was not possible due to heterogeneity in study design, type of intervention, and outcomes reported. Using a framework of engagement described in previous research on patient-mediated interventions [7] and adapted from a previous meta-review [12], we categorized interventions into three engagement strategies: inform ( interventions that provide knowledge), activate ( interventions that prompt action), and collaborate ( interventions that lead to interaction between the caregiver and other groups such as providers or other caregivers) (Additional file 1). Within these three strategies were 13 types of engagement support, including lifestyle advice, psychological strategies, and information on available resources (examples in Additional file 1). Interventions were further classified as single-component ( involving a single component/engagement support) or complex interventions ( involving more than one component/engagement support) [13]. Study outcomes were classified as patient-oriented, caregiver-oriented, provider-oriented, or health system-oriented.

Role of the funding source

The study funder had no role in the study design; in the collection, analysis, or interpretation of data; in the writing of the report; or in the decision to submit the article for publication. The researchers are independent from the study funders.

Results

Results of the search

We screened 4273 unique abstracts and reviewed 231 full-text articles; 175 full-text articles were excluded, the most common reasons being that the intervention was not mediated by a caregiver 110/175) or that the study did not include an intervention (37/175) (Fig. 1). Hand searching resulted in the inclusion of 18 additional studies. Characteristics of the 56 studies meeting all inclusion criteria are described in Additional file 2.

Fig. 1
figure 1

Study flow diagram

Description of included studies

The 56 intervention studies [1, 2, 14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67] were published between 1984 and 2017 and conducted in the USA (n = 32, 57.1%), Canada (n = 3, 5.4%), and 14 other countries (n = 1 or 2 studies each) (Additional file 2). Most studies (n = 36, 64.3%) were randomized controlled trials [14, 15, 18, 20, 24, 26,27,28,29,30,31, 33,34,35,36, 38, 39, 41,42,43,44,45,46,47, 50, 51, 54,55,56,57,58,59, 62, 64,65,66] and conducted in outpatient settings (including primary care and the community) (n = 33). Caregivers were most commonly parents (n = 25, 44.6%) [14,15,16,17,18, 20, 25, 28, 30, 34, 36, 37, 39,40,41, 4648, 49, 53, 58, 60, 64,65,66] and family caregivers more broadly—family caregivers could include parents, but the term was broadly defined—(n = 17, 30.4%) [1, 2, 22, 23, 31,32,33, 35, 42, 44, 45, 54, 56, 59, 61, 63, 67], with the most common populations receiving care being premature/low birth weight infants (n = 8, 14.3%) [14, 17, 18, 40, 47, 49, 65, 66], children with asthma (n = 6, 10.7%) [26, 27, 34, 41, 46, 56], and stroke survivors (n = 4, 7.0%) [33, 45, 51, 55]. More than half of the interventions (n = 30, 53.6%) were directed towards children, with the interventions mediated by parents or family caregivers [14,15,16,17,18, 20, 22, 24,25,26,27,28, 30, 34, 36, 37, 39,40,41, 46,47,48,49, 53, 56, 58, 60, 64,65,66].

Interventions and assessments

Caregiver-mediated interventions were categorized by the type of engagement strategy and support provided (Additional file 1). Figure 2 provides a conceptual framework of the engagement strategy and supports, caregiver-mediated interventions employed, and reported outcomes. All interventions were considered complex, with a median of 4 (IQR 2–6) engagement supports. The most common type of engagement strategy was inform (n = 55, 98.2%), followed by activate (n = 54, 96.4%), and collaborate (n = 27, 48.2%). The most common engagement support used as part of the inform strategies was condition/treatment education (n = 47, 83.9%), for the activate strategies the most common engagement support was practical management activities (n = 40, 71.4%), and for the collaborate strategies the most common engagement support was providing a safety net to caregivers (n = 11, 19.6%). All inform engagement strategies were accompanied by either activate and/or collaborate strategies. The most common pairing of engagement strategies was inform-activate (n = 28, 50.0%), and among those, educational engagement supports combined with practical management strategies (n = 8). For example, one inform-activate engagement strategy included family caregiver education and involvement in the basic care of patients admitted to the ICU; specifically, caregivers were provided with information on the advantages of their involvement in the basic care of the patient and details on the activities they could carry out, including general hygiene, mouth hygiene, mobilization, and feeding [42]. Similarly, two inform-collaborate engagement strategies included an education program aimed to improve knowledge about stroke and stroke prevention (inform), described services available in hospital and the community (collaborate), and provided an opportunity to ask questions and receive group support (collaborate) [51].

Fig. 2
figure 2

Conceptual framework derived from synthesis of data

Outcomes

Patient- and caregiver-oriented outcomes were most commonly reported (n = 40 and n = 33, respectively) (Table 1). Health system-oriented outcomes were reported in 10 studies and provider-oriented outcomes in two studies. Patient-oriented outcomes included cognitive development, symptom control, and communication. Caregiver-oriented outcomes included anxiety, depression, and satisfaction. Provider-oriented outcomes included improved communication with caregivers and satisfaction with the intervention. Health system-oriented outcomes included length of hospital stay, number of hospital visits, and costs. Interventions with condition and treatment education (inform) and practical management (activate) engagement supports were generally effective in improving patient-reported outcomes. For caregiver-reported outcomes, engagement supports that included action plans (activate), practical management (activate), or lifestyle monitoring (activate), and condition and treatment education (inform) or lifestyle advice (inform) engagement supports were generally effective in improving outcomes. Provider outcomes were improved by providing caregivers engagement supports related to activities of daily living (inform) and notice of available resources (collaborate). Interventions with social support and safety net engagement supports (collaborate) tended to improve health system outcomes. No studies reported whether harmful effects were associated with the interventions.

Table 1 Type and effectiveness of interventions for primary study outcomes

Location of patient care

Interventions conducted in inpatient (or combined inpatient and outpatient) settings (n = 34) had more statistically significant positive effects on patient and caregiver outcomes than no/mixed effects (Table 2). In outpatient settings (including primary care and the community) (n = 39), interventions examining caregiver outcomes had more statistically significant positive effects than no/mixed effects. The same trend was observed for patient outcomes. Most inpatient interventions (n = 15, 44.1%) were inform-activate, while most outpatient interventions (n = 18, 46.2%) were inform-activate-collaborate.

Table 2 Type and effectiveness of interventions for select subgroup analyses

Patient population

The majority of interventions were directed towards children (newborn–18 years) (n = 30, 53.6%), followed by adults (greater than 18 years) (n = 11, 19.6%) and older adults (greater than 55 years) (n = 15, 26.8%). Ten pediatric studies focused on newborn, low birth weight, or premature infants, while the remainder (n = 20) reported on children overall. In studies of infants, mothers (n = 6) delivered the intervention more frequently than both parents (n = 4), while in studies of children, both parents (n = 15) or a caregiver (n = 5) delivered the intervention. In studies of adults, the most commonly studied populations were persons with stroke (n = 3), persons with comorbid substance misuse, and persons in the ICU (n = 2). Among adult populations, the persons providing care were caregivers broadly (not specifically defined) (n = 10) or spouses (n = 1). Engagement strategies for adults were either inform-activate (n = 6) or inform-activate-collaborate (n = 5). Interventions directed at older adults were most commonly conducted in persons with dementia or Alzheimer’s disease (n = 8) and utilized an inform-activate-collaborate engagement strategy (n = 11). Caregiver outcomes were reported in more studies of older adults (n = 13) than patient (n = 8), health system (n = 4), and provider (n = 1) outcomes. For both patient and caregiver outcomes in infants, children, or adolescents, there was a similar number of positive effects and mixed effects (Table 2). In adults, there were considerably more positive effects observed for both patient and caregiver outcomes. In older adults, caregiver outcomes were more frequently improved, though this was not observed for patient outcomes.

Acceptability and feasibility

The acceptability and feasibility of caregiver-mediated interventions were inconsistently reported in the included studies, with less than half (n = 26, 46.4%) reporting response rates or process evaluations.

Risk of bias assessment and strength of evidence

None of the 56 included studies had a low overall risk of bias; risk of bias was high in 47 studies (83.9%) and unclear in nine studies (16.1%) (Additional file 4). The overall strength of the evidence based on GRADE criteria was low-moderate (Additional file 5). A lack of blinding of participants (i.e., caregivers and patients) and outcome assessors (i.e., patients not blinded to patient reported outcomes) was the main factor that limited the strength of the evidence.

When the analyses were restricted to studies with moderate or greater strength of evidence (n = 33, 58.9%), the results were similar, with most statistically significant effects observed for inform-activate engagement strategies for patient outcomes (Additional file 6).

Discussion

We utilized systematic review methodology to synthesize the literature on the effectiveness of caregiver-mediated interventions. There is a large and recent body of literature (including many randomized controlled trials [RCTs]) suggesting that complex interventions comprised of inform-activate and inform-activate-collaborate engagement strategies are associated with improved patient and caregiver outcomes. In contrast, there is very little literature regarding provider and health system outcomes, though all interventions involved care providers and/or the health system in some way. The potential for unintended consequences or harm from caregiver-mediated interventions was not explored in the included studies, and much of the literature focused on the parent-child relationship.

That interventions with an inform engagement strategy always included parts of activate and/or collaborate engagement strategies is noteworthy, as research suggests that education-only interventions (inform in this language) in various settings and populations are some of the weakest [68,69,70,71]. Education with prompts for action or education with active collaboration may circumvent the limitations of education alone [68,69,70,71]. Most interventions were mediated by parents or family caregivers (any family member, though not defined specifically), though there was heterogeneity in both the type of caregiver included and in how caregiver was defined. It is important to recognize that one caregiver is not the same as another; they may differ in their commitment, readiness, and capacity to fulfill the role. Validated tools to assess caregiver readiness (Willingness to Care Scale [72], Caregiver Self Expectations Inventory [73]) should be employed alongside caregiver-mediated interventions in this context [74]. Interventions to support caregivers once they are already in a caregiving role dominate the literature [75,76,77], with little or no information supporting the use of interventions to prepare the caregiver prior to entering the caregiving role. Nearly all studies of adults found improvement in both patient and caregiver outcomes, though in pediatric studies only half found improvement in patient and caregiver outcomes. Improved outcomes may be dependent on whether the caregiver-patient relationship is parent-child (i.e., parents providing support to children) versus child-parent (i.e., children providing support to parents).

All included interventions were complex (including multiple components). Thus, we are uncertain of the ”active ingredients” of the interventions and how these components affect outcomes. The knowledge translation literature suggests that complex interventions may be no better than single-component interventions [68, 69]. If this holds true for caregiver-mediated interventions, then single-component interventions may represent a resource-efficient means of improving patient, caregiver, provider, and health system outcomes. We observed variation in reported outcomes according to the types of engagement pairings, suggesting that different combinations of interventions might have differing effects: the previously observed effect of simple-component versus complex interventions may not hold true for caregiver-mediated interventions. Inform-activate engagement strategies provide the strongest evidence for improving patient outcomes through caregiver-mediated interventions. Based on the results of the current study, we recommend interventions that provide both condition-specific information and prompts or tools to support action for managing a condition to the caregiver. For example, an intervention to encourage early mobilization of patients recovering from a serious illness might include both information about how the illness affects the patient’s conditioning (inform) paired with instructions and prompts for caregiver-directed bedside physical therapy (activate). It is unclear how these interventions will apply in other contexts, though they appear to work in children and adults with any family caregiver in an inpatient setting. There was a trend towards more positive than equivocal effects for improved caregiver outcomes in an outpatient setting. Inform-activate-collaborate engagement strategies are supported by the strongest evidence to improve caregiver outcomes. We recommend utilizing this type of strategy for infants or children in outpatient or community settings with any family caregiver. For example, there could be a three-component (inform-activate-collaborate) telephone coaching intervention for parents of children with asthma (activate) combined with lifestyle advice on how to avoid asthma triggers (inform) and the provision of contact information for extra information or clinical supports (collaborate). These interventions have the potential to improve caregiver outcomes. At this time, there have been few evaluations of interventions using inform-collaborate (e.g., providing information to support activities of daily living and a list of available resources) or activate-collaborate (e.g., creating action plans specific to a medical condition and the provision of contact information for extra support) engagement strategies. For interventions conducted in the community (either partially or fully), it appears that collaborate engagement supports are necessary to improve outcomes.

Despite the large body of literature, there are a number of key gaps that must be addressed: provider and health system outcomes, the role of caregiver, the potential harms of caregiver-mediated interventions; and the economic impact of these interventions. Only two studies explored provider outcomes of caregiver-mediated interventions. Providers represent an important mechanism for implementing PFCC research [4, 78, 79]; it is essential to measure the impact that caregiver-mediated interventions have on this population. Ten studies reported on health system outcomes, which included hospital and emergency room visits and associated costs. As all caregiver-mediated interventions engage with the healthcare system, it is important to measure the impact they have on these limited resources. Within our aging population, investigations should explore whether the inverse effect of the parent-child dyad applies to caregiver-mediated interventions—that is, when children become the caregiver, are the same outcomes observed? The evidence from parent-child relationships may not inform the child-parent caregiver relationship in this context. No included studies reported on potential harms of caregiver-mediated interventions (e.g., falls among patients being mobilized by caregivers, caregiver feelings of guilt related to intervention adherence, conflict with providers); unintended effects of these interventions are an important area that requires further study. Caregiver-mediated interventions represent an opportunity for healthcare cost savings by transferring select care responsibilities from the clinical team to the caregiver. Future studies should assess the financial impact of employing caregiver-mediated interventions.

This study has a number of strengths, including the use of rigorous methodology defined by an a priori published protocol and adherence to recommended reporting criteria. We utilized previously validated tools to assess both the quality and strength of the evidence presented. The search was executed in five bibliographic databases, it was not restricted by language, and it was intentionally broad to ensure that caregiver-mediated interventions across all areas of healthcare were included. By focusing on healthcare settings, home-based interventions were excluded, including many studies of caregiver-mediated interventions in persons with dementia. In spite of these strengths, there are limitations to note. Our findings are limited by the methodological quality of the included studies; the risk of bias was generally high, and the overall quality of the evidence was low-moderate. When restricted to studies of at least moderate quality, there was a trend towards observing more positive effects. Risk of bias tends to be high in studies where blinding does not occur, as is the case in many of these interventions, and does not necessarily mean they are methodologically poor [80]. The literature is rich with RCTs, the quality of which is limited by methodological factors inherent in this type of research (e.g., lack of blinding) that are unlikely to be overcome in the future. We did not search the gray literature and could have missed studies, though our comprehensive search strategy included databases indexing both North American and European journals and full-text hand-searching was completed. We noted a lack of scientific abstracts, limited use of applicable keywords, and variable terminology to describe caregiver-mediated interventions in the included studies. This highlights the importance of full-text hand-searching, as these papers may not be found through standard database searching. All included papers reported on complex interventions, and thus it is not possible to determine which parts of the interventions were effective. The lack of effect in some studies may be a failure of implementation (few process evaluations conducted to inform) or a failure of measurement (i.e., use of the wrong measure) and not the intervention itself.

Conclusions

There is a large body of research, including many RCTs, to support the use of caregiver-mediated interventions that inform and activate caregivers to improve patient and caregiver outcomes. Additional study is needed to evaluate provider and health system outcomes, interventions for relationships outside of the parent-child role, potential harms, and economic implications. We recommend consideration of inform-activate and inform-activate-collaborate interventions when implementing patient and family-oriented research.

Abbreviations

GRADE:

Grading of Recommendations Assessment Development and Evaluation

IQR:

Interquartile range

PFCC:

Patient and family-centered care

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RCT:

Randomized controlled trial

References

  1. Li H, Melnyk BM, McCann R, Chatcheydang J, Koulouglioti C, Nichols LW, Lee MD, Ghassemi A. Creating avenues for relative empowerment (CARE): a pilot test of an intervention to improve outcomes of hospitalized elders and family caregivers. Res Nurs Health. 2003;26(4):284–99.

    Article  PubMed  Google Scholar 

  2. Mitchell M, Chaboyer W, Burmeister E, Foster M. Positive effects of a nursing intervention on family-centered care in adult critical care. Am J Crit Care. 2009;18(6):543–52.

    Article  PubMed  Google Scholar 

  3. Gasparini R, Champagne M, Stephany A, Hudson J, Fuchs MA. Policy to practice: increased family presence and the impact on patient- and family-centered care adoption. J Nurs Adm. 2015;45(1):28–34.

    Article  PubMed  Google Scholar 

  4. Bauman G, Watling CJ, Hahn K. Oncologists’ attitudes towards patient and family-centered care. J Clin Oncol. 2015;33(29_suppl):111.

    Article  Google Scholar 

  5. Baird J, Davies B, Hinds PS, Baggott C, Rehm RS. What impact do hospital and unit-based rules have upon patient and family-centered care in the pediatric intensive care unit? J Pediatr Nurs. 2015;30(1):133–42.

    Article  PubMed  Google Scholar 

  6. Tonelli MR, Misak CJ. Compromised autonomy and the seriously ill patient. Chest. 2010;137(4):926–31.

    Article  PubMed  Google Scholar 

  7. Gagliardi AR, Legare F, Brouwers MC, Webster F, Badley E, Straus S. Patient-mediated knowledge translation (PKT) interventions for clinical encounters: a systematic review. Implement Sci. 2016;11:26.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Knowledge Translation. http://www.cihr-irsc.gc.ca/e/29418.html. Accessed 2 Feb 2018.

  10. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Taylor SJC, Pinnock H, Epiphaniou E, Pearce G, Parke HL, Schwappach A, Purushotham N, Jacob S, Griffiths CJ, Greenhalgh T, et al. A rapid synthesis of the evidence on interventions supporting self-management for people with long-term conditions: PRISMS - Practical systematic Review of Self-Management Support for long-term conditions. Southampton: University of Southampton; 2014.

    Google Scholar 

  13. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council G. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Achenbach TMH, T C, Aoki MF, Rauh VA. Nine-year outcome of the Vermont intervention program for low birth weight infants. Pediatrics. 1993;91(1):45–55.

    PubMed  CAS  Google Scholar 

  15. Anderson BJ, Brackett J, Ho J, Laffel LM. An office-based intervention to maintain parent-adolescent teamwork in diabetes management: impact on parent involvement, family conflict, and subsequent glycemic control. Diabetes Care. 1999;22(5):713–21.

    Article  PubMed  CAS  Google Scholar 

  16. Bagner DM. Father’s role in parent training for children with developmental delay. J Fam Psychol. 2013;27(4):650–7.

    Article  PubMed  Google Scholar 

  17. Bao XL, National Cooperative Research Group for Lowering Incidence of Cerebral Palsy of Premature Infants through Early Intervention. Lowering incidence of cerebral palsy of premature infants through early intervention. Zhonghua Er Ke Za Zhi. 2005;43(4):244–7.

  18. Barrera ME, Rosenbaum PL, Cunningham CE. Early home intervention with low-birth-weight infants and their parents. Child Dev. 1986;57(1):20–33.

    Article  PubMed  CAS  Google Scholar 

  19. Bass DM, Judge KS, Snow AL, Wilson NL, Morgan R, Looman WJ, McCarthy CA, Maslow K, Moye JA, Randazzo R, Garcia-Maldonado M, Elbein R, Odenheimer G, Kunik ME. Caregiver outcomes of partners in dementia care: effect of a care coordination program for veterans with dementia and their family members and friends. J Am Geriatr Soc. 2013;61(8):1377–86.

    Article  PubMed  Google Scholar 

  20. Beeghly MB, Brazelton TB, Flannery KA, Nugent JK, Barrett DE, Tronick EZ. Specificity of preventative pediatric intervention effects in early infancy. J Dev Behav Pediatr. 1995;16(3):158–66.

    Article  PubMed  CAS  Google Scholar 

  21. Berthelsen CB, Kristensson J. The SICAM-trial: evaluating the effect of spouses’ involvement through case management in older patients’ fast-track programmes during and after total hip replacement. J Adv Nurs. 2016;4:4.

    Google Scholar 

  22. Blauw-Hospers CH, Dirks T, Hulshof LJ, Bos AF, Hadders-Algra M. Pediatric physical therapy in infancy: from nightmare to dream? A two-arm randomized trial. Phys Ther. 2011;91(9):1323–38.

    Article  PubMed  Google Scholar 

  23. Boltz M, Resnick B, Chippendale T, Galvin J. Testing a family-centered intervention to promote functional and cognitive recovery in hospitalized older adults. J Am Geriatr Soc. 2014;62(12):2398–407.

    PubMed  PubMed Central  Google Scholar 

  24. Arauz Boudreau AD, Kurowski DS, Gonzalez WI, Dimond MA, Oreskovic NM. Latino families, primary care, and childhood obesity: a randomized controlled trial. Am J Prev Med. 2013;44(3 SUPPL. 3):S247–57.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Brown JAW, Juliann J. Effects of a triadic parent-implemented home-based communication intervention for toddlers. J Early Interv. 2015;37(1):44–68.

    Article  Google Scholar 

  26. Butz A, Kub J, Donithan M, James NT, Thompson RE, Bellin M, Tsoukleris M, Bollinger ME. Influence of caregiver and provider communication on symptom days and medication use for inner-city children with asthma. J Asthma. 2010;47(4):478–85.

    Article  PubMed  Google Scholar 

  27. Chen SH,H, L J, Yeh KW, Tsai YF. Interactive support interventions for caregivers of asthmatic children. J Asthma. 2013;50(6):649–57.

    Article  PubMed  Google Scholar 

  28. Dawson AM, Brown DA, Cox A, Williams SM, Treacy L, Haszard J, Meredith-Jones K, Hargreaves E, Taylor BJ, Ross J, Taylor RW. Using motivational interviewing for weight feedback to parents of young children. J Paediatr Child Health. 2014;50(6):461–70.

    Article  PubMed  Google Scholar 

  29. Eloniemi-Sulkava U, Saarenheimo M, Laakkonen ML, Pietila M, Savikko N, Kautiainen H, Tilvis RS, Pitkala KH. Family care as collaboration: effectiveness of a multicomponent support program for elderly couples with dementia. Randomized controlled intervention study. J Am Geriatr Soc. 2009;57(12):2200–8.

    Article  PubMed  Google Scholar 

  30. Fiks AG,M, L S, Karavite DJ, Suh A, O'Hara R, Localio AR, Ross M, Grundmeier RW. Parent-reported outcomes of a shared decision-making portal in asthma: a practice-based RCT. Pediatrics. 2015;135(4):e965–73.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Fortinsky RH, Kulldorf M, Kleppinger A, Kenyon-Pesce L. Dementia care consultation for family caregivers: collaborative model linking an Alzheimer's association chapter with primary care physicians. Aging Ment Health. 2009;13(2):162–70.

    Article  PubMed  Google Scholar 

  32. Fortinsky RHU, Christine G, Garcia RI. Helping family caregivers by linking primary care physicians with community-based dementia care services: the Alzheimer's Service Coordination Program. Dementia. 2002;1(2):227–40.

    Article  Google Scholar 

  33. Galvin R, Cusack T, O’Grady E, Murphy B, Stokes E. Family mediated exercise intervention [fame]: evaluation of a novel form of exercise delivery after stroke. Physiotherapy (United Kingdom). 2011;97:eS387–8.

    Google Scholar 

  34. Garbutt JM, Banister C, Highstein G, Sterkel R, Epstein J, Bruns J, Swerczek L, Wells S, Waterman B, Strunk RC, et al. Telephone coaching for parents of children with asthma: impact and lessons learned. Arch Pediatr Adolesc Med. 2010;164(7):625–30.

    Article  PubMed  Google Scholar 

  35. Garcia-Huidobro D, Bittner M, Brahm P, Puschel K. Family intervention to control type 2 diabetes: a controlled clinical trial. Fam Pract. 2011;28(1):4–11.

    Article  PubMed  Google Scholar 

  36. Golley RK, Magarey AM, Baur LA, Steinbeck KS, Daniels LA. Twelve-month effectiveness of a parent-led, family-focused weight-management program for prepubertal children: a randomized, controlled trial. Pediatrics. 2007;119(3):517–25.

    Article  PubMed  Google Scholar 

  37. Gomez DB, Ana J, Andrews AR III, Cavell TA, Pastrana FA, Gregus SJ, Ojeda CA. Delivering parent management training in an integrated primary care setting: description and preliminary outcome data. Cogn Behav Pract. 2014;21(3):296–309.

    Article  Google Scholar 

  38. Grover M, Naumann U, Mohammad-Dar L, Glennon D, Ringwood S, Eisler I, Williams C, Treasure J, Schmidt U. A randomized controlled trial of an internet-based cognitive-behavioural skills package for carers of people with anorexia nervosa. Psychol Med. 2011;41(12):2581–91.

    Article  PubMed  CAS  Google Scholar 

  39. Helgadottir HL, Wilson ME. A randomized controlled trial of the effectiveness of educating parents about distraction to decrease postoperative pain in children at home after tonsillectomy. Pain Manag Nurs. 2014;15(3):632–40.

    Article  PubMed  Google Scholar 

  40. Holditch-Davis D, White-Traut RC, Levy JA, O’Shea TM, Geraldo V, David RJ. Maternally administered interventions for preterm infants in the NICU: Effects on maternal psychological distress and mother-infant relationship. Infant Behav Dev. 2014;37(4):695–710.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Horn IB, Mitchell SJ, Gillespie CW, Burke KM, Godoy L, Teach SJ. Randomized trial of a health communication intervention for parents of children with asthma. J Asthma. 2014;51(9):989–95.

    Article  PubMed  Google Scholar 

  42. Martínez RM del C, Morilla FR, del Pino AR, Almenara MIM, Bannik JT, Caballero LJF, Macías GC, Bernet LA, González CA. Family involvement in the critically ill patient basic care. Enfermería Intensiva. 2003;14(3):96–108.

    Article  Google Scholar 

  43. Mittelman MS, Ferris SH, Shulman E, Steinberg G, Levin B. A family intervention to delay nursing home placement of patients with Alzheimer disease. A randomized controlled trial. JAMA. 1996;276(21):1725–31.

    Article  PubMed  CAS  Google Scholar 

  44. Mueser KT, Glynn SM, Cather C, Xie H, Zarate R, Smith LF, Clark RE, Gottlieb JD, Wolfe R, Feldman J. A randomized controlled trial of family intervention for co-occurring substance use and severe psychiatric disorders. Schizophr Bull. 2013;39(3):658–72.

    Article  PubMed  Google Scholar 

  45. Nayeri ND, Mohammadi S, Razi SP, Kazemnejad A. Investigating the effects of a family-centered care program on stroke patients’ adherence to their therapeutic regimens. Contemporary Nurse. 2014;47(1/2):88–96.

    Article  PubMed  Google Scholar 

  46. Nelson KA, Highstein GR, Garbutt J, Trinkaus K, Fisher EB, Smith SR, Strunk RC. A randomized controlled trial of parental asthma coaching to improve outcomes among urban minority children. Arch Pediatr Adolesc Med. 2011;165(6):520–6.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Nurcombe B, Howell DC, Rauh VA, Teti DM, Ruoff P, Brennan J. An intervention program for mothers of low-birthweight infants: preliminary results. J Am Acad Child Psychiatry. 1984;23(3):319–25.

    Article  PubMed  CAS  Google Scholar 

  48. Ozonoff S, Cathcart K. Effectiveness of a home program intervention for young children with autism. J Autism Dev Disord. 1998;28(1):25–32.

    Article  PubMed  CAS  Google Scholar 

  49. Pridham KA, Krolikowski MM, Limbo RK, Paradowski J, Rudd N, Meurer JR, Uttech A, Henriques JB. Guiding mothers’ management of health problems of very low birth-weight infants. Public Health Nurs. 2006;23(3):205–15.

    Article  PubMed  Google Scholar 

  50. Robinson JK, Turrisi R, Stapleton J. Examination of mediating variables in a partner assistance intervention designed to increase performance of skin self-examination. J Am Acad Dermatol. 2007;56(3):391–7.

    Article  PubMed  Google Scholar 

  51. Rodgers H, Atkinson C, Bond S, Suddes M, Dobson R, Curless R. Randomized controlled trial of a comprehensive stroke education program for patients and caregivers. Stroke. 1999;30(12):2585–91.

    Article  PubMed  CAS  Google Scholar 

  52. Rodríguez-Gonzalo A, García-Martí C, Ocaña-Colorado A, Baquera-De Micheo MJ, Morel-Fernández S. Efficiency of an intensive educational program for informal caregivers of hospitalized, dependent patients: cluster randomized trial. BMC Nurs. 2015;14(1):1–12.

    Article  Google Scholar 

  53. Sénéchal C, Larivée S, Thermidor G. Parents as co-therapists: a winning solution for treating autistic children. Annales Medico-Psychologiques. 2013;171(9):603–9.

    Article  Google Scholar 

  54. Tantirangsee N, Assanangkornchai S, Marsden J. Effects of a brief intervention for substance use on tobacco smoking and family relationship functioning in schizophrenia and related psychoses: a randomised controlled trial. J Subst Abus Treat. 2015;51:30–7.

    Article  Google Scholar 

  55. Wang T-C, Tsai AC, Wang J-Y, Lin Y-T, Lin K-L, Chen JJ, Lin BY, Lin TC. Caregiver-mediated intervention can improve physical functional recovery of patients with chronic stroke: a randomized controlled trial. Neurorehabil Neural Repair. 2015;29(1):3–12.

    Article  PubMed  CAS  Google Scholar 

  56. Watson WT, Gillespie C, Thomas N, Filuk SE, McColm J, Piwniuk MP, Becker AB. Small-group, interactive education and the effect on asthma control by children and their families. CMAJ. 2009;181(5):257–63.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Mavandadi S, Wright EM, Graydon MM, Oslin DW, Wray LO. A randomized pilot trial of a telephone-based collaborative care management program for caregivers of individuals with dementia. Psychol Serv. 2017;14(1):102–11.

    Article  PubMed  Google Scholar 

  58. Sun G-C, Hsu M-C. Effects of nurse-led child- and parent-focused violence intervention on mentally ill adult patients and victimized parents: a randomized controlled trial. Int J Nurs Stud. 2016;60:79–90.

    Article  PubMed  Google Scholar 

  59. Toye C, Parsons R, Slatyer S, Aoun SM, Moorin R, Osseiran-Moisson R, Hill KD. Outcomes for family carers of a nurse-delivered hospital discharge intervention for older people (the Further Enabling Care at Home Program): single blind randomised controlled trial. Int J Nurs Stud. 2016;64:32–41.

    Article  PubMed  Google Scholar 

  60. Tyler DO, Horner SD. A primary care intervention to improve weight in obese children: a feasibility study. J Am Assoc Nurse Pract. 2016;28(2):98–106.

    PubMed  Google Scholar 

  61. Souaibi L, Choueifati D, Kerbage H, Richa S. Therapeutic education of families of patients with schizophrenia: the Lebanese Hôtel-Dieu de France model. Annales Medico-Psychologiques. 2016;174(8):677–82.

    Article  Google Scholar 

  62. Callahan CM, Boustani MA, Unverzagt FW, Austrom MG, Damush TM, Perkins AJ, Fultz BA, Hui SL, Counsell SR, Hendrie HC. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA. 2006;295(18):2148–57.

    Article  PubMed  CAS  Google Scholar 

  63. Gitlin LN, Reever K, Dennis MP, Mathieu E, Hauck WW. Enhancing quality of life of families who use adult day services: short- and long-term effects of the adult day services plus program. Gerontologist. 2006;46(5):630–9.

    Article  PubMed  Google Scholar 

  64. Melnyk BM, Alpert-Gillis L, Feinstein NF, Crean HF, Johnson J, Fairbanks E, Small L, Rubenstein J, Slota M, Corbo-Richert B. Creating opportunities for parent empowerment: program effects on the mental health/coping outcomes of critically ill young children and their mothers. Pediatrics. 2004;113(6):e597–607.

    Article  PubMed  Google Scholar 

  65. Melnyk BM, Alpert-Gillis L, Feinstein NF, Fairbanks E, Schultz-Czarniak J, Hust D, Sherman L, LeMoine C, Moldenhauer Z, Small L, et al. Improving cognitive development of low-birth-weight premature infants with the COPE program: a pilot study of the benefit of early NICU intervention with mothers. Res Nurs Health. 2001;24(5):373–89.

    Article  PubMed  CAS  Google Scholar 

  66. Melnyk BM, Feinstein NF, Alpert-Gillis L, Fairbanks E, Crean HF, Sinkin RA, Stone PW, Small L, Tu X, Gross SJ. Reducing premature infants’ length of stay and improving parents’ mental health outcomes with the Creating Opportunities for Parent Empowerment (COPE) neonatal intensive care unit program: a randomized, controlled trial. Pediatrics. 2006;118(5):e1414–27.

    Article  PubMed  Google Scholar 

  67. Skoog M, Milner KA, Gatti-Petito J, Dintyala K. The impact of family engagement on anxiety levels in a cardiothoracic intensive care unit. Crit Care Nurse. 2016;36(2):84–9.

    Article  PubMed  Google Scholar 

  68. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153(10):1423–31.

    PubMed  PubMed Central  CAS  Google Scholar 

  69. Wensing M, van der Weijden T, Grol R. Implementing guidelines and innovations in general practice: which interventions are effective? Br J Gen Pract. 1998;48(427):991–7.

    PubMed  PubMed Central  CAS  Google Scholar 

  70. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342(8883):1317–22.

    Article  PubMed  CAS  Google Scholar 

  71. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–30.

    Article  PubMed  Google Scholar 

  72. Abell N. Assessing willingness to care for persons with AIDS: validation of a new measure. Res Soc Work Pract. 2001;11:118–30.

    Article  Google Scholar 

  73. Y-IL S. Development and testing of the Family Caregiving Factors Inventory for home health assessment in Taiwan. J Adv Nurs. 2000;32:226–34.

    Article  Google Scholar 

  74. Alliance FC. Selected caregiver assessment measures: a resource inventory for practitioners. 2nd ed; 2012.

    Google Scholar 

  75. Stockle HS, Haarmann-Doetkotte S, Bausewein C, Fegg MJ. The feasibility and acceptability of short-term, individual existential behavioural therapy for informal caregivers of patients recruited in a specialist palliative care unit. BMC Palliat Care. 2016;15(1):88.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Rexhaj S, Leclerc C, Bonsack C, Golay P, Favrod J. Feasibility and accessibility of a tailored intervention for informal caregivers of people with severe psychiatric disorders: a pilot study. Frontiers of Psychiatry. 2017;8:178.

    Article  Google Scholar 

  77. Dam AEH, van Boxtel MPJ, Rozendaal N, Verhey FRJ, de Vugt ME. Development and feasibility of Inlife: a pilot study of an online social support intervention for informal caregivers of people with dementia. PLoS One. 2017;12(9):e0183386.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  78. Philibert I, Patow C, Cichon J. Incorporating patient- and family-centered care into resident education: approaches, benefits, and challenges. J Grad Med Educ. 2011;3(2):272–8.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Ayub EM, Sampayo EM, Shah MI, Doughty CB. Prehospital Providers’ perceptions on providing patient and family centered care. Prehosp Emerg Care. 2017;21(2):233–41.

    Article  PubMed  Google Scholar 

  80. Boutron I, Guittet L, Estellat C, Moher D, Hrobjartsson A, Ravaud P. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med. 2007;4(2):e61.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge Ms. Heather Ganshorn (University of Calgary) for the development of the search strategies and the patients and family members who participated in the stakeholder engagement meeting. We would also like to acknowledge Dr. Khara Sauro (University of Calgary) and Dr. Daniel Niven (University of Calgary) for providing valuable feedback on the manuscript.

Funding

Canadian Institutes of Health Research. The funding body had no role in the design of the study or in the collection, analysis, or interpretation of data, or in writing the manuscript.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its additional files.

Author information

Authors and Affiliations

Authors

Contributions

KMF, JPL, and HTS conceived the study. KMF, CJM, and DD analyzed the data. All authors interpreted the data, and KMF wrote the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kirsten M. Fiest.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Characteristics of Caregiver-Mediated Interventions. Description of the engagement strategies and engagement supports and examples of each utilized in the study. (DOCX 17 kb)

Additional file 2:

Characteristics of Included Studies. Table of the studies included in the systematic review with information including setting, participant population, intervention, and outcomes. (DOCX 120 kb)

Additional file 3:

PRISMA Checklist. A completed PRISMA checklist for study transparency. (DOCX 24 kb)

Additional file 4:

Risk of Bias Assessment. Risk of bias ratings for each included study. (DOCX 20 kb)

Additional file 5:

GRADE Ratings. GRADE ratings for each included study. (DOCX 19 kb)

Additional file 6:

Type and Effectiveness of Interventions for Primary Study Outcomes in Studies with Moderate or High Strength Evidence According to the GRADE Criteria. Main results restricted to those studies with a moderate or high strength of evidence according to the GRADE criteria. (DOCX 13 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fiest, K.M., McIntosh, C.J., Demiantschuk, D. et al. Translating evidence to patient care through caregivers: a systematic review of caregiver-mediated interventions. BMC Med 16, 105 (2018). https://doi.org/10.1186/s12916-018-1097-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12916-018-1097-4

Keywords