You are here

Article Text

Article menu

Download PDFPDF

XML
Research methods
Research
Process evaluations of primary care interventions addressing chronic disease: a systematic review
  1. Hueiming Liu1,2,3,
  2. Alim Mohammed4,
  3. Janani Shanthosh2,3,
  4. Madeline News2,3,
  5. Tracey-Lea Laba2,3,5,
  6. Maree L Hackett1,2,3,
  7. David Peiris2,3,
  8. Stephen Jan1,2,3
  1. 1University of Sydney, Sydney, New South Wales, Australia
  2. 2The George Institute for Global Health, Sydney, NSW, Australia
  3. 3University of New South Wales, Sydney, NSW, Australia
  4. 4The George Institute for Global Health, Hyderabad, India
  5. 5Menzies Centre for Health Policy, University of Sydney, Sydney, New South Wales, Australia
  1. Correspondence to Dr Hueiming Liu; hliu{at}georgeinstitute.org.au

Abstract

Objective Process evaluations (PEs) alongside randomised controlled trials of complex interventions are valuable because they address questions of for whom, how and why interventions had an impact. We synthesised the methods used in PEs of primary care interventions, and their main findings on implementation barriers and facilitators.

Design Systematic review using the UK Medical Research Council guidance for PE as a guide.

Data sources Academic databases (MEDLINE, SCOPUS, PsycINFO, Cumulative Index to Nursing and Allied Health Literature, EMBASE and Global Health) were searched from 1998 until June 2018.

Eligibility criteria We included PE alongside randomised controlled trials of primary care interventions which aimed to improve outcomes for patients with non-communicable diseases.

Data extraction and synthesis Two independent reviewers screened and conducted the data extraction and synthesis, with a third reviewer checking a sample for quality assurance.

Results 69 studies were included. There was an overall lack of consistency in how PEs were conducted and reported. The main weakness is that only 30 studies were underpinned by a clear intervention theory often facilitated by the use of existing theoretical frameworks. The main strengths were robust sampling strategies, and the triangulation of qualitative and quantitative data to understand an intervention’s mechanisms. Findings were synthesised into three key themes: (1) a fundamental mismatch between what the intervention was designed to achieve and local needs; (2) the required roles and responsibilities of key actors were often not clearly understood; and (3) the health system context—factors such as governance, financing structures and workforce—if unanticipated could adversely impact implementation.

Conclusion Greater consistency is needed in the reporting and the methods of PEs, in particular greater use of theoretical frameworks to inform intervention theory. More emphasis on formative research in designing interventions is needed to align the intervention with the needs of local stakeholders, and to minimise unanticipated consequences due to context-specific barriers.

PROSPERO registration number CRD42016035572.

  • process evaluations
  • systematic review
  • non-communicable disease
  • qualitative research
  • primary care

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2018-025127

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is a reflexive thematic synthesis and interpretation of the papers by a multidisciplinary team.

    • A bespoke quality assessment tool based on the Medical Research Council guidance for process evaluations was used.

    • Given variations in lexicon, our search strategy may not have exhaustively captured all process evaluations of complex primary care interventions.

    • Use of the framework and quality assessment tool to synthesise the findings has not been tested previously.

    Introduction

    An accessible, effective and affordable primary healthcare (PHC) system is needed to equitably reduce the rising non-communicable disease (NCD) burden.1–3 Complex interventions which comprise ‘multiple interacting components (although additional dimensions of complexity include the difficulty in their implementation and the number of organisational levels they target)’ are often used to reduce this burden.4 These interventions often require individual and organisational behaviour change within dynamic health system contexts.5 6 Randomised controlled trials (RCTs) of complex primary care interventions have been conducted, but there is often ambiguity as to what was implemented on the ground.4 7 8 Process evaluations (PEs) are conducted alongside trials to examine if a complex intervention was implemented as intended, and to explore if, for whom, how and why the intervention had an impact.4

    A process evaluation is defined by the UK Medical Research Council (MRC) as a study to ‘understand the functioning of an intervention, by examining implementation, mechanisms of impact, and contextual factors’.4 The MRC PE framework and guidance published in 2015 is based on the synthesis of influential frameworks and theories in public health research, and informed by the authors’ PEs.4 The key points are briefly summarised and elaborated on in box 1.

    In the guidance, concepts of reach, fidelity and adoption were highlighted as key to examining implementation quality. For example, assessing ‘fidelity’ would help determine whether the research was conducted as per protocol. The MRC guidance also highlighted the value of describing the intervention theory more explicitly, that is, providing the hypothesis relating to how the complex intervention could potentially interact with contextual factors to produce variation in outcomes.9 10 Ideally, the intervention theory would determine the process (qualitative and quantitative) data to be collected and analysed before the RCT outcomes are known. The PE findings could potentially help explain variation in RCT outcome, refine the intervention theory and inform future research priorities. Recognising the need to facilitate implementation of evidence into practice and policy, the MRC guidance also expands on the importance for PEs to be conducted across all stages of research, that is, feasibility/piloting, evaluation of effectiveness and post-evaluation stages. While the guidance was well received, outstanding questions remain in this developing field. For example, what is the role of other theories and frameworks for PEs? What methods can be used and how?11–13 Synthesising the collective ‘experience’ described in published PEs may answer some of these questions.

    This review has two primary objectives. First is to review the methods used in published PEs and their alignment with the MRC guidance, and second to identify the key implementation barriers and facilitators reported in these PEs. We used the three key points (as summarised in box 1) from the MRC guidance as a lens to evaluate the published literature, and employed it as a framework for synthesising our findings.

    Box 1

    Summary points of the Medical Research Council guidance for process evaluations

    1. Expanded on the functions of the process evaluation.

    Implementation: ‘What is implemented and how?’

    • Implementation process: What did the research team do?

    • Reach: Did you recruit participants that your intervention was intended to have an impact on?

    • Fidelity: Did you do what you planned to do as per protocol?

    • Adaptation: What changes were made in the delivery of the intervention to the local context?

    • Dose: Is the frequency of the intervention delivered as planned?

    Mechanisms of impact: ‘How does the delivered intervention produce change?’

    • Participants’ experiences of the intervention.

    • Mediators.

    • Unexpected consequences and pathways.

    Context: ‘How does the context affect implementation and outcomes?’

    • Contextual factors which shape the theory of the intervention.

    • Contextual factors that could affect and be affected by implementation, mechanisms and outcomes.

    • Causal mechanisms within the context that could potentiate the effects.

    2. Provided a systematic approach to the design and conduct of process evaluations.

    • Planning: relationships with stakeholders and degree of separation between evaluators and research team.

    • Design and conduct: describe intervention and causal assumptions, identify the key process questions, and inform data collection.

    • Analysis: use of reporting guidelines for methods; transparently report whether process data are analysed and reported prior or after knowing outcomes.

    • Reporting: intervention theory and how it informed data collection; protocol to link multiple outputs.

    3. Described and expanded on the function of process evaluations across the different stages of development, evaluation and implementation.

    • Development: feasibility and acceptability of implementation strategies and optimising design and evaluation of intervention.

    • Effectiveness: fidelity of intervention, mechanisms of action and contextual factors.

    • Post-evaluation implementation: implementation of intervention into usual practice, long-term surveillance.

    Methods

    The systematic review protocol has been prespecified and published in detail as a separate protocol.14 A summary of the methods is presented here according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (see online supplementary appendix 1 for the PRISMA checklist).15

    Eligibility criteria for the RCTs with the included PE

    • Population: patients with NCDs (diabetes, depression, cardiovascular disease, chronic obstructive pulmonary disease, chronic kidney disease, type 2 diabetes mellitus) and their primary care providers.5

    • Intervention: complex interventions which comprise ‘multiple interacting components although additional dimensions of complexity include the difficulty of their implementation and the number of organisational levels they target’ within PHC.4

    • Comparator: the control condition may include treatment as usual, active control or placebo control.

    • Outcomes for this systematic review: (1) strengths and limitations of each PE using the MRC guidance as a reference point; and (2) identification of implementation barriers and facilitators for the complex interventions.

    • Timing: published data from 1998 until June 2018.

    • Design: PE of the included RCTs as defined by the MRC as ‘a study which aims to understand the functioning of an intervention, by examining implementation, mechanisms of impact, and contextual factors’.4 Given that PEs are often not explicitly labelled as such,10 we included studies with comparable aims.

    • Exclusion criteria: not a journal article, not a report based on empirical research, not reported in English, reviews and not human research.

    • Search strategy and data extraction: databases reporting academic publications (MEDLINE, SCOPUS, PsycINFO, Cumulative Index to Nursing and Allied Health Literature, EMBASE and Global Health) were searched. Standard systematic review methods were followed for searching, screening and extracting data from eligible studies.14 Our search strategy is included as online supplementary appendix 2. Two reviewers (HL, AM) conducted most of the data extraction, with a third reviewer (MN) assisting in data extraction with some papers and as part of quality assurance, and checked on the data extraction for a 10% sample of the identified papers. Given that a key aim of this study was about PE methodology, we deviated from the published systematic review protocol, by including our interpretation in addition to the study’s strengths and limitations posited by the authors of those papers.

    Data analysis and synthesis

    Descriptive items (eg, number of positive RCTs) were tallied and synthesised into three tables: (1) Overall characteristics: presenting the studies grouped into different diseases and ordered by year of publication (online supplementary appendix 3); (2) methods table: grouping studies by the stages of the PE (ie, feasibility/piloting, effectiveness, post-evaluation) (online supplementary appendix 4); and (3) quality assessment: findings based on a bespoke assessment tool which was designed based on some of the key recommendations from the MRC guidance and two other papers which had synthesised PE literature before (online supplementary appendix 5).

    View this table:
    Table 1

    Summary of the characteristics of the included studies

    Extracted qualitative data were coded by HL, and grouped into categories of context, mechanisms and implementation. Inductive derivation of the key themes was done through constant comparison between the findings from the papers within each category and examining the relationships between them. Online supplementary appendix 6 provides illustrative quotations. Using a modified MRC framework, we mapped our methodological and implementation findings to triangulate and synthesise our findings.

    Patient and public involvement

    While patient and public perspectives were synthesised from published papers, no public and patients were directly involved in this study.

    Findings

    Characteristics of included studies

    We identified 69 studies. The PRISMA flow chart is presented in figure 1. In summary, 66 studies were conducted in high-income countries, 1 study in Zambia, 1 in Malaysia and 1 in India. Cardiovascular disease, diabetes mellitus and depression were the conditions most often investigated, with only six studies on chronic obstructive pulmonary disease and one study on chronic kidney disease. Overall, the complex primary care interventions fit within the general categories of facilitating patient self-management (13 studies), organisational change to include collaborative care (15 studies), facilitating better case management using clinical information systems (eg, telehealth) (15 studies), and the use of decision support and guideline implementation (eg, referral systems) (22 studies). In addition, five studies explored the challenges when conducting trials in primary care, for example, the recruitment of patients.

    Figure 1
    Figure 1

    PRISMA figure. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT, randomised controlled trial.

    Only 22 studies were labelled clearly as PEs, although this was more common in recent years. Twenty studies were conducted at the feasibility stage with 5 labelled as PEs, 43 studies at the effectiveness stage with 17 labelled as PEs, and 6 studies at the post-evaluation stage with none labelled as PEs. (The methods used at these different stages are described in greater detail in the next section.) In 35 studies the degree of separation between the process and outcome evaluation researchers was explicit. The cost considerations for the system and stakeholders were mentioned in 10 papers (see table 1 for more details). In figure 2, the context of the studies and an overview of the main methodological and implementation findings are diagrammatically presented in an adapted PE framework.

    Figure 2
    Figure 2

    Medical Research Council process evaluation framework with tallies of studies, methods and synthesised findings. N/A, not applicable; RCT, randomised controlled trial.

    PEs’ strengths and limitations

    Description of intervention theory: clear intervention description and clarification of causal assumptions

    Thirty papers were assessed as having clear intervention descriptions and clarification of causal assumptions, and in 16 it was unclear because despite clear intervention descriptions the causal assumptions were not described explicitly. An example of a paper that explicitly describes intervention theory is Grant et al, who uses the Template for Intervention Description and Replication (TIDieR) checklist to clearly describe researchers’ assumptions of the intervention’s mechanism as compared with stakeholders’ perspectives.16

    Use of existing theories and frameworks

    A strength of 22 studies was the use of existing theoretical frameworks to inform their intervention development and/or evaluation (see table 2). Theories and frameworks used are grouped according to Nilsen’s proposed categories.17 This is depicted in box 2, with illustrative examples from the identified studies. In essence, 11 studies used classic theory to inform the development of the intervention theory. In eight studies determinant, implementation theories and evaluation frameworks were used to assist in the synthesis and analysis of qualitative data. The authors of two studies also used their findings and implementation theories to iteratively inform their implementation strategies. The evaluation frameworks were used by study authors to comprehensively evaluate and synthesise their PE data. The MRC framework for complex interventions was used to inform the approach to intervention development in three studies.

    Box 2

    Illustrative examples of the use of theories and frameworks

    Classic theories.

    • Theory of planned behaviour. ‘Using the theory of planned behaviour (TPB), we hypothesised that changes in thiazide prescribing would be reflected in changes in intention, consistent with changes in attitude and subjective norm, with no change to their perceived behavioural control (PBC), and tested this alongside the RCT…A strength of this study is its use of a well-tested theory of behaviour operationalized according to best recommended practice to investigate the underlying mechanisms of an implementation intervention’ (Presseau). This theory informed their process evaluation to explore if their intervention of printed educational materials increased practitioners’ intention to prescribe according to recommendations in the guidelines.

    • Self-determination theory. ‘self-determination theory which proposes that real shifts in behaviour arise through heightened autonomy or personal ownership of behavioural success’ (Chalder). This theory informed their theoretical model underpinning their intervention to improve physical activity for the management of depression.

    • Grounded theory. ‘This qualitative study was conducted with the objective of better understanding the PP (Prevention practitioner) intervention in the BETTER Trial described above, including the development of the PP (prevention practitioner) role, perceived barriers, facilitators, benefits and disadvantages, and of exploring the feasibility and sustainability of this approach for CDPS (Chronic disease prevention and screening)’ (Manca). This study used grounded theory to better understand their intervention as implemented and to retrospectively describe their intervention theory.

    • Diffusions of innovation. ‘Key principles, which derive from diffusion of innovations theory, include working initially with practices and clinicians that not only have an interest in the innovation and view it as compatible with their needs, values, and resources, but also have the ability to try it with minimal investment and observe its impact’ (Dietrich). The theory was used to inform their practice change strategy for the sustainability of a chronic care model for depression proven effective in an RCT.

    Determinant frameworks.

    • PARIHS as an implementation model. ‘We used the Promoting Action on Research in Health Services (PARIHS) framework as an ‘Implementation model’ to assist clinical partners in adopting the health-coaching intervention. The PARIHS framework posits three interrelated elements that influence successful implementation of evidence-based practices: the (i) perceived strength of the ‘evidence’, (ii) ‘context’ of the environment and (iii) ‘facilitation’ support created for implementation of the intervention…Using a codebook developed a priori from sub elements of the PARIHS framework’ (Naik). This study used PARIHS to inform their participatory approach between research team and primary healthcare teams, and also used it in evaluation of the qualitative data in assessing the building of the partnership to test and implement a health-coaching intervention.

    Implementation theories.

    • Normalisation process theory. ‘as part of mixed-methods process evaluation, semi-structured interviews were conducted by phone with 27 providers participating in the study. Interviews were audio-taped and transcribed. Thematic content analysis was used to identify themes. Themes were categorized according to the four domains of Normalization Process Theory (NPT)’ (Vest). The authors discuss how the findings are informing their ongoing implementation strategies, for example, clinical mentors for the general practitioners who described a discomfort in their lack of expertise in screening and managing early chronic kidney disease (other papers include Burridge, Coupe, Gask, Hanley, Vest).

    Evaluation frameworks.

    • Medical Research Council’s (MRC) framework for complex interventions. ‘The MRC framework provided a useful structure through which to examine our theoretical hypothesis and analyse the feasibility evidence’ (Sturt). ‘Guided self-help intervention was developed following a modelling phase which involved a systematic review, meta synthesis and a consensus process…’ (Lovell). The authors used the MRC framework for intervention development. Similarly Byrne et al also used the MRC framework for intervention development of literature review, focus group discussion and modelling and then interviews to refine the intervention.

    • RE-AIM. ‘The process evaluation followed the RE-AIM (Reach, Efficacy/effectiveness, Adoption, Implementation and Maintenance) framework. Data were collected on attendance and attrition for classroom-based CBT and attention control PSHE (Personal, Social and Health Education) by programme facilitators. An independent observer attended 5% of classroom based CBT sessions to assess treatment fidelity. Feedback was gathered from teachers, young people and facilitators using questionnaires and qualitative interviews’ (Stallard) (other papers include Stallard, Wozniak, Lakeverld).

    • Realist evaluation. ‘All data assigned to codes relating to the polypill strategy in CVD management were analysed…and the Realist framework of context–mechanism–outcomes utilized to develop the themes’ (Liu). The framework was used to guide the analysis of the qualitative data.

    View this table:
    Table 2

    Summary of the methodology used and quality assessment of the studies

    The use of theoretical frameworks seems to enable an indepth investigation of stakeholders’ perspectives of the perceived mechanisms of the intervention, by in a sense providing a checklist of actions and behaviours to be examined.18–28 An illustrative example is the PE of a trial in improving primary care referrals of patients with diabetic retinopathy to specialists through the use of educational printed materials.28 A behavioural theory was used to inform the design and use of a questionnaire to explore the mechanism of the intervention. It was found that the primary care providers’ intention to refer patients was the same before and after the trial, and this may have explained their negative trial results. The authors highlighted that the use of existing behavioural theory enhanced the ‘generalisability and replicability’ of their methods.

    Interaction with contextual factors

    In 14 papers the interaction of the intervention and contextual factors was explicitly explored. As mentioned above, theoretical frameworks often facilitated a closer and systematic way to consider context. For example, the authors examined if there was ‘contextual integration’, that is, organisational changes necessary to integrate a collaborative model of care for depression into routine practice.29 Otherwise, contextual factors (eg, impact of the introduction of a new policy30) were reported retrospectively in some papers in a more ad hoc manner, for instance being reported as implementation facilitators and barriers, or discussed as possible influences on the outcomes.

    Methods used

    Assessing the studies based on our bespoke assessment tool, we found that most authors (64/69) clearly justified their choice of methods and stated the studies’ purpose, and most of the qualitative studies (42/50) were of a reasonable quality, with 24 of 50 studies covering all three domains of the Consolidated criteria for Reporting Qualitative research (COREQ) and 18 of 50 studies covering two domains. The methods could be categorised as qualitative studies (eg, interviews, focus group discussions, documentary analysis), quantitative (eg, processes of care, baseline demographics, secondary outcomes), and studies which presented the synthesis of qualitative and quantitative data sources to indicate implementation, acceptability, fidelity and reach (see table 2 and online supplementary appendix 4 for more details).

    A strength of some studies was the triangulation of quantitative indicators with the qualitative findings of the acceptability and implementation of the intervention to determine intervention fidelity (ie, whether the intervention was delivered per protocol) (see online supplementary appendix 3 for more details).31 32 The data sources indicating intervention fidelity included routine administrative data, trial/study management logs21 and trial secondary outcomes.33–35 Innovative indicators of e-health interventions included recording process measures such as time logged on by participants.36 Another method to determine intervention fidelity across multiple sites was having independent expert assessors reviewing intervention delivery using standardised forms. Three studies investigated ‘for whom’ an intervention had an impact on with the use of logistical regression of baseline demographics to identify relationships of participants’ characteristics with the primary or secondary outcomes.37

    Sampling limitations in the qualitative studies were described as potentially introducing bias in the findings about intervention acceptability/mechanisms.18 19 23–25 28 38–42 For example, authors highlighted that respondents who having agreed to be interviewed may have a more favourable opinion of the intervention.18 38 43–45 Maximum variation sampling (types of participants, sociodemographics, by ‘negative’ baseline of outcome characteristics’), comparing the characteristics of participants who did not partake in the interviews/surveys with the participants who did, and triangulation with other data sources may increase the robustness of such findings.19 20 22 28 38 39 46

    PE findings under mechanisms, implementation and context

    Does the intervention fit local needs?

    Stakeholders were generally motivated to adopt/implement the complex intervention if it addressed the contextual gap in care, that is, intervention fit. For example, a nurse-led secondary prevention clinic was implemented effectively when the health providers perceived it as improving teamwork, care continuity and providing a ‘safety net’ for the patients. In contrast, at other sites, this intervention was poorly implemented by the healthcare providers who viewed it as duplicating the existing model of care.47 As another example, general practitioners reported that training them to manage acute and discrete episodes of depression did not improve their management of depression. This was because this training did not upskill them for the chronic and relapsing nature of depression associated with personality and social problems increasingly seen in primary care.41 48 Similarly, patients’ health literacy about their chronic disease (eg, effectiveness of lifestyle modification for diabetes) was crucial as it affected engagement with the PHC services and their uptake of the intervention.22 25 49–51

    Do key actors believe in and adopt their ‘assigned’ roles and responsibilities?

    The extent to which key actors believed in and adopted their ‘assigned’ roles and responsibilities as part of implementing the complex intervention was a key theme under the heading ‘Implementation’.21 26 27 42 43 48 52 For example, in a study which used telemonitoring to improve management of patients with chronic obstructive pulmonary disease in the community, there were differing views of the role of the patient. Some health providers described concerns that telemonitoring would reinforce the ‘sick role’ of the patient, and an over-reliance on technology and practitioner support. Therefore, they were less willing to implement this model of care. On the other hand, some patients described that telemonitoring was empowering as it provided knowledge and increased access to health practitioners who could provide reassurance in the management of the disease, and were thereby keen to continue this model of care.22

    Facilitators to improve key actors’ uptake of the interventions included the provision of intense training over a transition period prior to the start of the trial, significant research support, and ongoing communication with the researchers to help identify key actors’ concerns and tailor implementation strategies to address them. For example, ensuring adequate communication between nurse practitioners and general practitioners was essential in task-shifting models of care. This facilitated greater trust between nurse practitioners and doctors, which was needed to effectively deliver collaborative services.42 52 Such strategies were especially relevant for collaborative care interventions where new tasks were introduced within established hierarchical systems and interaction between different stakeholders was necessary for effective implementation.

    Is the context of the intervention conducive?

    Health system structures such as governance, health financing structures and workforce were often mentioned as impacting on intervention implementation. Governance structures were pivotal to the successful adoption of the intervention.23 33 For example, an intervention to enhance referrals to mental health services was implemented well at a site when the intervention was perceived as ‘service delivery’ and directly supported by the mental health trust. In comparison, uptake of the intervention was limited when the intervention was not viewed as ‘service delivery’ and was considered ‘primary research’.33 Similarly, cultivating a strong partnership between researchers and clinicians through the formation of clinical advisory teams facilitated intervention implementation in bureaucratic and geographically complex environments.23 Limited workforce, equipment shortages and inadequate funding structures were reported by several authors as barriers to the intervention adoption. For example, health providers stated that the lack of government reimbursement for allied health services reduced the acceptability of the telehealth model of care for ongoing monitoring of diabetes at home.39 General practitioners reported that time constraints in their busy practices prevented them from using the skills they learnt through an educational intervention to better manage depression.41 Likewise, macro-level context, such as medication being out of stock in rural Zambia, was a barrier to better outcomes, in spite of the provision of an evidence-based intervention to improve clinical assessment and management.32

    Importantly, an iterative collaborative approach was described as a facilitator of intervention fit.18 36 45 50 51 53 54 For example, study authors described how early stakeholder involvement identified the key characteristics of the lay worker needed (ie, female, with visibility in the community) for their intervention to improve mental healthcare in India. This preparatory phase in the development of their model of care led to a definitive RCT with positive outcomes. In their PE of the RCT, they found that the provision of a lay worker was not relevant for the primary care practitioners in private practice who had established therapeutic relationships with their patients, but more so for the health providers in the public system who were time-poor. These findings would then inform future scale-up of the intervention within the right context (ie, public health system) for the intervention.53 55

    Discussion

    Statement of principal findings

    To our knowledge this is the first systematic appraisal using MRC guidance on PEs of primary care interventions. Of 69 studies, 66 were conducted in high-income countries, while cardiovascular disease, diabetes and depression were the most frequently studied conditions. There was an overall lack of consistency in the way PEs were conducted and reported. Indeed there was a lack of consistency in nomenclature, with only 47 of the 69 studies identifying as ‘process evaluations’, although their purpose was essentially as such. Few studies (n=30) were underpinned by an intervention theory, description of hypothesised intervention mechanisms of action within local contextual factors. Most studies used robust sampling strategies and frequently triangulated qualitative and quantitative data to better understand the mechanisms of implementation. The MRC PE guidance with its focus on the interaction/configuration between context, implementation and mechanisms of intervention provided a useful framework for the synthesis of the findings. Mapping our findings on the framework also gives our findings and assessment tools greater validity. The findings of these studies can be synthesised into a number of key messages: (1) that often there was a fundamental mismatch between what the intervention was designed to achieve and local needs; (2) that the roles and responsibilities of key actors required to implement the intervention were often not clearly understood; and (3) that health system context—factors such as governance, financing structures and workforce—were often critical to implementation, and as a consequence there were a number of studies where the unanticipated influence of these adversely impacted on implementation.

    Comparison with other literature and implications

    A key finding is identifying the breadth of literature which fits the MRC definition of PE. This highlights the growing scope in this field to potentially address the evidence to practice gap through greater understanding of the interactions between intervention mechanisms, context and implementation.12 56 57 However, greater consistency is needed in the reporting of PEs—as this would facilitate evidence synthesis, prevent research duplication and enhance transferability of interventions to other settings.58 We note that the consistency in reporting seems to have increased since the publication of the MRC guidance.

    An important finding is that theoretical frameworks helped guide a more indepth development of intervention theory, design and implementation.12 59 The MRC PE guidance suggests that PE can help to explain the outcomes variations, and by doing so help refine the intervention theory.17 We note that given the growing focus on self-management for chronic diseases, the theories around behavioural change (eg, empowerment) were most commonly used. Second, the focus on organisational change and the adoption of guidelines in NCDs meant that implementation theories such as the normalisation process theory (NPT) were particularly relevant. Therefore, there should be more consistent use of theoretical frameworks, recognising that different frameworks will be applicable to different settings. In addition, the use of checklists such as the TIDieR or the Standards for Reporting Implementation Studies (StaRI) will ensure consistency in the reporting of intervention theory and implementation, thus reducing research waste.56 60 61 We also note that there were only six ‘post–evaluation’ studies identified. This is not surprising as implementation and translational research is a more recent phenomena, and there may have been less funding for such work and greater difficulties in publishing such findings, a point likewise raised in the MRC guidance report in 2015. We anticipate, however, that more implementation research will be published as this emerging field matures.

    We found that the intervention interaction with dynamic contextual factors was often inconsistently reported or reported retrospectively in an ad hoc manner. This gap has been similarly reported in the literature.62 These findings emphasise MRC PE guidance’s value in explicitly appraising context through examining factors ‘that shape theories of how the intervention works’, and affect implementation, and act to ‘sustain the status quo, or potentiate effects’.4 However, this guidance is relatively broad and non-specific, and the question remains as to what should be explored a priori and how best to report such findings. For example, the Context and Implementation of Complex Interventions framework highlights seven domains of context (‘geographical, epidemiological, socio-cultural, socio-economic, ethical, legal and political context’) that could be examined.56 Similarly, the StaRI checklist has context as an item in the methods (ie, ‘consider social, economic, policy, healthcare, organisational barriers and facilitators that might influence implementation elsewhere’) and in the results (‘contextual changes (if any) which may have affected outcomes’).60 These domains are comprehensive, and as a consequence if a study is to examine only a subset of these factors, it is better that this is prespecified in full acknowledgement of the evaluation as a whole. This should be consistently reported and linked through a full report or reference to a protocol.4 As a baseline, a standardised PHC template informed by the questions ‘Does the intervention fit local needs? Do the key actors believe in and adopt their ‘assigned’ roles and responsibilities? Is the health system context (looking specifically at health workforce, governance, health financing structures and availability of medications) conducive?’ and relevant implementation theories (eg, NPT) could be presented for testing in a systematic way. This could be done by PHC researchers engaging with stakeholders at various time points, and iteratively added to.63–65 Such an approach could potentially facilitate a greater shared understanding between stakeholders and greater consistency in the reporting of context.62 64 66–68

    Most of these studies were conducted in high-income countries with established PHC systems and universal health coverage (eg, National Health Service in the UK). Therefore, some primary care interventions (eg, improving referrals in collaborative care) may be of limited relevance to low-income and middle-income countries (LMIC) PHC systems given the different context especially with regard to health system structures.69 70 This reinforces the need for more formative research with local stakeholders when developing evidence-based interventions so as to address local needs, and minimise the unanticipated consequences of health system factors.71 72

    Strengths and limitations of this study

    We were unable to conduct a subgroup analysis of implementation findings by country context (ie, of high-income countries as compared with LMIC) as we identified studies conducted mainly in high-income countries. Some studies conducted in LMIC initially identified in the search were excluded because they did not meet our criteria (eg, not RCTs, not on NCDs), and as such a review with different inclusion criteria may be better suited for this secondary objective. A second limitation is that despite our extended search terms to include components of chronic care model, and similar terms as PEs, we may not have exhaustively captured all PEs of complex primary care interventions. We note, however, that we reached thematic saturation in our analysis across the spectrum of studies, which provides greater validity to our findings. Another limitation is that we appraised the studies using a tool which we developed based on the MRC guidance,4 which has not been tested elsewhere. This was challenging given the heterogeneous studies that were included. For example, we only assessed qualitative methods with COREQ, and did not appraise the quality of statistical methods such as modelling. A strength of this review is having a multidisciplinary team of authors with vast experience in clinical trials and PEs to enable a reflexive thematic synthesis and interpretation of the papers.73

    Conclusion

    Greater consistency is needed in the reporting of, and the methods used in, PEs. In particular there should be more consistent use of theoretical frameworks to inform intervention theory, and the triangulation of qualitative and quantitative data. Greater emphasis on formative research in designing primary care interventions is needed so that they are clearly aligned with the needs of local stakeholders, that the roles and responsibilities of key actors are better understood, and that unanticipated consequences arising from context-specific barriers to implementation are minimised. We hope this review will inform future PEs and facilitate the sustainability of evidence-based interventions.

    References

    1. 1.↵
      1. Christopher Dye TB,
      2. Evans D,
      3. Harries A, et al
      . The world health report 2013: research for universal health coverage: World Health Organisation, 2013.
    2. 2.↵
      1. Byass P
      . Universal health coverage is needed to deliver NCD control. Lancet2018;391:738.doi:10.1016/S0140-6736(18)30244-7
    3. 3.↵
      1. Evans TG,
      2. Kieny MP
      . Systems science for universal health coverage. Bull World Health Organ2017;95:484.doi:10.2471/BLT.17.192542
      OpenUrl
    4. 4.↵
      1. Moore GF,
      2. Audrey S,
      3. Barker M, et al
      . Process evaluation of complex interventions: Medical Research Council guidance. BMJ2015;350:h1258.doi:10.1136/bmj.h1258
      OpenUrlFREE Full Text
    5. 5.↵
      1. Davy C,
      2. Bleasel J,
      3. Liu H, et al
      . Effectiveness of chronic care models: opportunities for improving healthcare practice and health outcomes: a systematic review. BMC Health Serv Res2015;15:194.doi:10.1186/s12913-015-0854-8
      OpenUrl
    6. 6.↵
      1. Campbell M,
      2. Fitzpatrick R,
      3. Haines A, et al
      . Framework for design and evaluation of complex interventions to improve health. BMJ2000;321:694–6.doi:10.1136/bmj.321.7262.694
      OpenUrlFREE Full Text
    7. 7.↵
      1. Glasziou P,
      2. Altman DG,
      3. Bossuyt P, et al
      . Reducing waste from incomplete or unusable reports of biomedical research. Lancet2014;383:267–76.doi:10.1016/S0140-6736(13)62228-X
      OpenUrlCrossRefPubMedWeb of Science
    8. 8.↵
      1. Van Belle S,
      2. Wong G,
      3. Westhorp G, et al
      . Can "realist" randomised controlled trials be genuinely realist?Trials2016;17:313.doi:10.1186/s13063-016-1407-0
    9. 9.↵
      1. Lewin S,
      2. Glenton C,
      3. Oxman AD
      . Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ2009;339:b3496.doi:10.1136/bmj.b3496
      OpenUrlAbstract/FREE Full Text
    10. 10.↵
      1. Grant A,
      2. Treweek S,
      3. Dreischulte T, et al
      . Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials2013;14:15.doi:10.1186/1745-6215-14-15
    11. 11.↵
      1. Marchal B,
      2. Westhorp G,
      3. Wong G, et al
      . Realist RCTs of complex interventions - an oxymoron. Soc Sci Med2013;94:124–8.doi:10.1016/j.socscimed.2013.06.025
      OpenUrlCrossRefPubMed
    12. 12.↵
      1. Geng EH,
      2. Peiris D,
      3. Kruk ME
      . Implementation science: relevance in the real world without sacrificing rigor. PLoS Med2017;14:e1002288.doi:10.1371/journal.pmed.1002288
      OpenUrl
    13. 13.↵
      IReSP. Process evaluation of population health intervention research: a complement or an alternative contribution to randomised controlled trial (Workshop), 2016.
    14. 14.↵
      1. Liu H,
      2. Muhunthan J,
      3. Hayek A, et al
      . Examining the use of process evaluations of randomised controlled trials of complex interventions addressing chronic disease in primary health care-a systematic review protocol. Syst Rev2016;5:138.doi:10.1186/s13643-016-0314-5
      OpenUrl
    15. 15.↵
      1. Shamseer L,
      2. Moher D,
      3. Clarke M, et al
      . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ2015;349:g7647.doi:10.1136/bmj.g7647
      OpenUrlAbstract/FREE Full Text
    16. 16.↵
      1. Grant A,
      2. Dreischulte T,
      3. Guthrie B
      . Process evaluation of the data-driven quality improvement in primary care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement Sci2017;12:4.doi:10.1186/s13012-016-0531-2
      OpenUrl
    17. 17.↵
      1. Nilsen P
      . Making sense of implementation theories, models and frameworks. Implement Sci2015;10:53.doi:10.1186/s13012-015-0242-0
      OpenUrlCrossRefPubMed
    18. 18.↵
      1. Lovell K,
      2. Bower P,
      3. Richards D, et al
      . Developing guided self-help for depression using the Medical Research Council complex interventions framework: a description of the modelling phase and results of an exploratory randomised controlled trial. BMC Psychiatry2008;8:91.doi:10.1186/1471-244X-8-91
      OpenUrlCrossRefPubMed
    19. 19.↵
      1. Chalder M,
      2. Wiles NJ,
      3. Campbell J, et al
      . A pragmatic randomised controlled trial to evaluate the cost-effectiveness of a physical activity intervention as a treatment for depression: the treating depression with physical activity (TREAD) trial. Health Technol Assess2012;16:1–164.doi:10.3310/hta16100
      OpenUrlCrossRefPubMed
    20. 20.↵
      1. Coupe N,
      2. Anderson E,
      3. Gask L, et al
      . Facilitating professional liaison in collaborative care for depression in UK primary care; a qualitative study utilising normalisation process theory. BMC Fam Pract2014;15:78.doi:10.1186/1471-2296-15-78
      OpenUrl
    21. 21.↵
      1. Dietrich AJ,
      2. Oxman TE,
      3. Williams JW, et al
      . Going to scale: re-engineering systems for primary care treatment of depression. Ann Fam Med2004;2:301–4.doi:10.1370/afm.102
      OpenUrlAbstract/FREE Full Text
    22. 22.↵
      1. Fairbrother P,
      2. Pinnock H,
      3. Hanley J, et al
      . Exploring telemonitoring and self-management by patients with chronic obstructive pulmonary disease: a qualitative study embedded in a randomized controlled trial. Patient Educ Couns2013;93:403–10.doi:10.1016/j.pec.2013.04.003
      OpenUrlCrossRefPubMed
    23. 23.↵
      1. Naik AD,
      2. Lawrence B,
      3. Kiefer L, et al
      . Building a primary care/research partnership: lessons learned from a telehealth intervention for diabetes and depression. Fam Pract2015;32:216–23.doi:10.1093/fampra/cmu084
      OpenUrlCrossRefPubMed
    24. 24.↵
      1. Stallard P,
      2. Phillips R,
      3. Montgomery AA, et al
      . A cluster randomised controlled trial to determine the clinical effectiveness and cost-effectiveness of classroom-based cognitive-behavioural therapy (CBT) in reducing symptoms of depression in high-risk adolescents. Health Technol Assess2013;17:1–109.doi:10.3310/hta17470
      OpenUrlCrossRefPubMedWeb of Science
    25. 25.↵
      1. Vest BM,
      2. York TR,
      3. Sand J, et al
      . Chronic kidney disease guideline implementation in primary care: a qualitative report from the TRANSLATE CKD Study. J Am Board Fam Med2015;28:624–31.doi:10.3122/jabfm.2015.05.150070
      OpenUrlAbstract/FREE Full Text
    26. 26.↵
      1. Burridge LH,
      2. Foster MM,
      3. Donald M, et al
      . Making sense of change: patients' views of diabetes and GP-led integrated diabetes care. Health Expect2016;19:74–86.doi:10.1111/hex.12331
      OpenUrl
    27. 27.↵
      1. Manca DP,
      2. Greiver M,
      3. Carroll JC, et al
      . Finding a BETTER way: a qualitative study exploring the prevention practitioner intervention to improve chronic disease prevention and screening in family practice. BMC Fam Pract2014;15:66.doi:10.1186/1471-2296-15-66
      OpenUrlCrossRef
    28. 28.↵
      1. Grimshaw JM,
      2. Presseau J,
      3. Tetroe J, et al
      . Looking inside the black box: results of a theory-based process evaluation exploring the results of a randomized controlled trial of printed educational messages to increase primary care physicians' diabetic retinopathy referrals [Trial registration number ISRCTN72772651]. Implement Sci2014;9:86.doi:10.1186/1748-5908-9-86
      OpenUrl
    29. 29.↵
      1. Gask L,
      2. Bower P,
      3. Lovell K, et al
      . What work has to be done to implement collaborative care for depression? Process evaluation of a trial utilizing the Normalization Process Model. Implement Sci2010;5:15.doi:10.1186/1748-5908-5-15
      OpenUrlCrossRefPubMed
    30. 30.↵
      1. Wells S,
      2. Rafter N,
      3. Kenealy T, et al
      . The impact of a point-of-care testing device on CVD risk assessment completion in New Zealand primary-care practice: A cluster randomised controlled trial and qualitative investigation. PLoS One2017;12:e0174504.doi:10.1371/journal.pone.0174504
    31. 31.↵
      1. Grant A,
      2. Dreischulte T,
      3. Guthrie B
      . Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing. BMJ Open2017;7:e015281.doi:10.1136/bmjopen-2016-015281
    32. 32.↵
      1. Yan LD,
      2. Chirwa C,
      3. Chi BH, et al
      . Hypertension management in rural primary care facilities in Zambia: a mixed methods study. BMC Health Serv Res2017;17:111.doi:10.1186/s12913-017-2063-0
      OpenUrl
    33. 33.↵
      1. Slade M,
      2. Gask L,
      3. Leese M, et al
      . Failure to improve appropriateness of referrals to adult community mental health services--lessons from a multi-site cluster randomized controlled trial. Fam Pract2008;25:181–90.doi:10.1093/fampra/cmn025
      OpenUrlCrossRefPubMed
    34. 34.↵
      1. Smith SM,
      2. Paul G,
      3. Kelly A, et al
      . Peer support for patients with type 2 diabetes: cluster randomised controlled trial. BMJ2011;342:d715.doi:10.1136/bmj.d715
      OpenUrlAbstract/FREE Full Text
    35. 35.↵
      1. Hanley J,
      2. Ure J,
      3. Pagliari C, et al
      . Experiences of patients and professionals participating in the HITS home blood pressure telemonitoring trial: a qualitative study. BMJ Open2013;3:e002671.doi:10.1136/bmjopen-2013-002671
      OpenUrlAbstract/FREE Full Text
    36. 36.↵
      1. Hetlevik I,
      2. Holmen J,
      3. Krüger O, et al
      . Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess Health Care2000;16:210–27.
      OpenUrlCrossRefPubMedWeb of Science
    37. 37.↵
      1. Thornett AM,
      2. Mynors-Wallis LM
      . Credibility of problem-solving therapy and medication for the treatment of depression among primary care patients. Med Sci Monit2002;8:CR193–6.
      OpenUrlPubMed
    38. 38.↵
      1. Bennett M,
      2. Walters K,
      3. Drennan V, et al
      . Structured pro-active care for chronic depression by practice nurses in primary care: a qualitative evaluation. PLoS One2013;8:e75810.doi:10.1371/journal.pone.0075810
      OpenUrlCrossRefPubMed
    39. 39.↵
      1. Carlisle K,
      2. Warren R
      . A qualitative case study of telehealth for in-home monitoring to support the management of type 2 diabetes. J Telemed Telecare2013;19:372–5.doi:10.1177/1357633X13506512
      OpenUrlCrossRefPubMed
    40. 40.↵
      1. Casey D,
      2. Murphy K,
      3. Cooney A, et al
      . Developing a structured education programme for clients with COPD. Br J Community Nurs2011;16:231–7.doi:10.12968/bjcn.2011.16.5.231
      OpenUrlPubMed
    41. 41.↵
      1. Gask L,
      2. Dixon C,
      3. May C, et al
      . Qualitative study of an educational intervention for GPs in the assessment and management of depression. Br J Gen Pract2005;55:854-9.
      OpenUrl
    42. 42.↵
      1. Wozniak L,
      2. Soprovich A,
      3. Rees S, et al
      . Contextualizing the effectiveness of a collaborative care model for primary care patients with diabetes and depression (Teamcare): a qualitative assessment using RE-AIM. Can J Diabetes2015;39:S83–S91.doi:10.1016/j.jcjd.2015.05.004
      OpenUrl
    43. 43.↵
      1. Lee PW,
      2. Dietrich AJ,
      3. Oxman TE, et al
      . Sustainable impact of a primary care depression intervention. J Am Board Fam Med2007;20:427–33.doi:10.3122/jabfm.2007.05.070045
      OpenUrlAbstract/FREE Full Text
    44. 44.↵
      1. Kenealy TW,
      2. Parsons MJ,
      3. Rouse AP, et al
      . Telecare for diabetes, CHF or COPD: effect on quality of life, hospital use and costs. A randomised controlled trial and qualitative evaluation. PLoS One2015;10:e0116188.doi:10.1371/journal.pone.0116188
      OpenUrlCrossRefPubMed
    45. 45.↵
      1. van der Weegen S,
      2. Verwey R,
      3. Spreeuwenberg M, et al
      . The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: a user-centered design. JMIR Mhealth Uhealth2013;1:e8.doi:10.2196/mhealth.2526
      OpenUrl
    46. 46.↵
      1. Eborall HC,
      2. Dallosso HM,
      3. McNicol S, et al
      . Explaining engagement in self-monitoring among participants of the DESMOND Self-monitoring Trial: a qualitative interview study. Fam Pract2015;32:596–602.doi:10.1093/fampra/cmv060
      OpenUrlCrossRefPubMed
    47. 47.↵
      1. Murchie P,
      2. Campbell NC,
      3. Ritchie LD, et al
      . Running nurse-led secondary prevention clinics for coronary heart disease in primary care: qualitative study of health professionals' perspectives. Br J Gen Pract2005;55:522–8.
      OpenUrlAbstract/FREE Full Text
    48. 48.↵
      1. Gask L,
      2. Ludman E,
      3. Schaefer J
      . Qualitative study of an intervention for depression among patients with diabetes: how can we optimize patient-professional interaction?Chronic Illn2006;2:231–42.doi:10.1177/17423953060020030401
      OpenUrlCrossRefPubMed
    49. 49.↵
      1. Passey ME,
      2. Laws RA,
      3. Jayasinghe UW, et al
      . Predictors of primary care referrals to a vascular disease prevention lifestyle program among participants in a cluster randomised trial. BMC Health Serv Res2012;12:234.doi:10.1186/1472-6963-12-234
      OpenUrlCrossRefPubMed
    50. 50.↵
      1. Pylypchuk G,
      2. Vincent L,
      3. Wentworth J, et al
      . Diabetes risk evaluation and microalbuminuria (DREAM) studies: ten years of participatory research with a First Nation’s home and community model for type 2 diabetes care in Northern Saskatchewan. Int J Circumpolar Health2008;67:190–202.doi:10.3402/ijch.v67i2-3.18264
      OpenUrlPubMed
    51. 51.↵
      1. Fairbrother P,
      2. McCloughan L,
      3. Adam G, et al
      . Involving patients in clinical research: the Telescot Patient Panel. Health Expect2016;19:691–701.doi:10.1111/hex.12132
      OpenUrl
    52. 52.↵
      1. Oishi SM,
      2. Shoai R,
      3. Katon W, et al
      . Impacting late life depression: integrating a depression intervention into primary care. Psychiatr Q2003;74:75–89.doi:10.1023/A:1021197807029
      OpenUrlCrossRefPubMed
    53. 53.↵
      1. Chatterjee S,
      2. Chowdhary N,
      3. Pednekar S, et al
      . Integrating evidence-based treatments for common mental disorders in routine primary care: feasibility and acceptability of the MANAS intervention in Goa, India. World Psychiatry2008;7:39–46.doi:10.1002/j.2051-5545.2008.tb00151.x
      OpenUrlPubMedWeb of Science
    54. 54.↵
      1. Chew-Graham CA,
      2. Lovell K,
      3. Roberts C, et al
      . A randomised controlled trial to test the feasibility of a collaborative care model for the management of depression in older people. Br J Gen Pract2007;57:364–70.
      OpenUrlAbstract/FREE Full Text
    55. 55.↵
      1. Pereira B,
      2. Andrew G,
      3. Pednekar S, et al
      . The integration of the treatment for common mental disorders in primary care: experiences of health care providers in the MANAS trial in Goa, India. Int J Ment Health Syst2011;5:26.doi:10.1186/1752-4458-5-26
      OpenUrlPubMed
    56. 56.↵
      1. Pfadenhauer LM,
      2. Gerhardus A,
      3. Mozygemba K, et al
      . Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci2017;12:21.doi:10.1186/s13012-017-0552-5
      OpenUrlCrossRefPubMed
    57. 57.↵
      1. Hawe P
      . Lessons from complex interventions to improve health. Annu Rev Public Health2015;36:307–23.doi:10.1146/annurev-publhealth-031912-114421
      OpenUrlCrossRefPubMed
    58. 58.↵
      1. Campbell M,
      2. Katikireddi SV,
      3. Hoffmann T, et al
      . TIDieR-PHP: a reporting guideline for population health and policy interventions. BMJ2018;361:k1079.doi:10.1136/bmj.k1079
      OpenUrlFREE Full Text
    59. 59.↵
      1. Fletcher A,
      2. Jamal F,
      3. Moore G, et al
      . Realist complex intervention science: applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. Evaluation2016;22:286–303.doi:10.1177/1356389016652743
      OpenUrlCrossRef
    60. 60.↵
      1. Pinnock H,
      2. Barwick M,
      3. Carpenter CR, et al
      . Standards for Reporting Implementation Studies (StaRI) Statement. BMJ2017;356:i6795.doi:10.1136/bmj.i6795
      OpenUrlAbstract/FREE Full Text
    61. 61.↵
      1. Hoffmann TC,
      2. Glasziou PP,
      3. Boutron I, et al
      . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ2014;348:g1687.doi:10.1136/bmj.g1687
      OpenUrlAbstract/FREE Full Text
    62. 62.↵
      1. Lau R,
      2. Stevenson F,
      3. Ong BN, et al
      . Achieving change in primary care--causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci2016;11:40.doi:10.1186/s13012-016-0396-4
      OpenUrlPubMed
    63. 63.↵
      1. Damschroder LJ,
      2. Aron DC,
      3. Keith RE, et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci2009;4:50.doi:10.1186/1748-5908-4-50
      OpenUrlCrossRefPubMed
    64. 64.↵
      1. Keith RE,
      2. Crosson JC,
      3. O’Malley AS, et al
      . Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci2017;12:15.doi:10.1186/s13012-017-0550-7
      OpenUrl
    65. 65.↵
      World Health Organisation. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies, 2018.
    66. 66.↵
      1. Masterson-Algar P,
      2. Burton CR,
      3. Rycroft-Malone J
      . Process evaluations in neurological rehabilitation: a mixed-evidence systematic review and recommendations for future research. BMJ Open2016;6:e013002.doi:10.1136/bmjopen-2016-013002
    67. 67.↵
      1. Cooper Robbins SC,
      2. Ward K,
      3. Skinner SR
      . School-based vaccination: a systematic review of process evaluations. Vaccine2011;29:9588–99.doi:10.1016/j.vaccine.2011.10.033
      OpenUrlCrossRefPubMedWeb of Science
    68. 68.↵
      1. Munodawafa M,
      2. Mall S,
      3. Lund C, et al
      . Process evaluations of task sharing interventions for perinatal depression in low and middle income countries (LMIC): a systematic review and qualitative meta-synthesis. BMC Health Serv Res2018;18:205.doi:10.1186/s12913-018-3030-0
      OpenUrl
    69. 69.↵
      1. Lagomarsino G,
      2. Garabrant A,
      3. Adyas A, et al
      . Moving towards universal health coverage: health insurance reforms in nine developing countries in Africa and Asia. Lancet2012;380:933–43.doi:10.1016/S0140-6736(12)61147-7
      OpenUrlCrossRefPubMedWeb of Science
    70. 70.↵
      1. Jan S,
      2. Laba TL,
      3. Essue BM, et al
      . Action to address the household economic burden of non-communicable diseases. Lancet2018;391:2047–58.doi:10.1016/S0140-6736(18)30323-4
      OpenUrlCrossRefPubMed
    71. 71.↵
      1. Pandian JD,
      2. Liu H,
      3. Gandhi DB, et al
      . Clinical stroke research in resource limited settings: tips and hints. Int J Stroke2018;13:1747493017743798.doi:10.1177/1747493017743798
      OpenUrl
    72. 72.↵
      1. Luoto J,
      2. Shekelle PG,
      3. Maglione MA, et al
      . Reporting of context and implementation in studies of global health interventions: a pilot study. Implement Sci2014;9:57.doi:10.1186/1748-5908-9-57
      OpenUrl
    73. 73.↵
      1. Lewin S,
      2. Bohren M,
      3. Rashidian A, et al
      . Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table. Implement Sci2018;13(Suppl 1):10.doi:10.1186/s13012-017-0689-2
      OpenUrl

    Footnotes

    • Contributors HL and SJ conceived the idea for a systematic review of process evaluations. DP, SJ, TL and MH provided guidance to HL in the development of the protocol. AM, JS and MN assisted with the selection of papers, data extraction and analysis. TL assisted with the adjudication of the papers. HL drafted the manuscript, and all authors contributed to the revisions of the manuscript and approved the final manuscript.

    • Funding MLH was supported by an NHMRC Career Development Fellowship Level 2(APP1141328). SJ is a recipient of an NHMRC Principal Research Fellowship (2017-21). TL was supported by a NHMR ECF (Sidney Sax) APP 1110230. HL was supported by a NHMRC post graduate scholarship (APP1114897).

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement This systematic review selected published papers through database searches.

    • Author note This systematic review forms part of HL’s PhD thesis and is not externally funded or commissioned.

    • Patient consent for publication Not required.