Tools and frameworks for handling context, mapped to stages of a systamatic review process
Stage of the review | Tool or framework | Strengths | Limitations |
Throughout | Realist evaluation7/Realist synthesis34 | Offers a method for integrating quantitative and qualitative data. Focuses on context as one line of inquiry, specifically addressed when assessing programme outcomes.15 | Resource intensive; requires reviewers to prioritise which explanations would benefit from further exploration. |
Systems thinking4 | Offers holistic picture of complex phenomena and interactive elements including context. | No specific review or synthesis tools or methodologies have been designed for this perspective. | |
Evidence to Decision frameworks (eg, WHO-INTEGRATE10 or GRADE EtD38 framework) | Seek to identify elements of context throughout entire evidence production and guideline development process. | Comprehensive coverage makes it difficult to use entire tool within a tight systematic review time window. Reviewer must prioritise items relevant to specific review. | |
Logic models2 | Provide a graphical representation of context surrounding an intervention. | Offer simplistic perspective with a focus on inputs, processes, outputs and outcomes. Context is only identified as a backdrop to programme activities. | |
Planning the review | |||
Question formulation and defining eligibility criteria | Question structures:11 PICOC,41 CIMO,42 SPICE,43 PerSPE©TIF11 | Include important elements of context/setting when compared with intervention-based PICO formulation. | Adopt a simple linear framework for representing review question. |
Protocol development | PROSPERO (International prospective register of systematic reviews) template45 | Item #23 (context) requires summary details of the setting. | Focuses on context as it relates to study eligibility. |
Use of frameworks (see itemised list below) | Offer prompts to ensure appropriate issues are considered when conducting the review. | Few available frameworks for systematic consideration of context. | |
PRISMA-P44 | Reporting standard for all elements of a protocol. | Superficial approach to context (setting and time frame). | |
Conducting the review | |||
Data extraction and/or analysis | Framework for the design and reporting of cluster RCTs.46 | Requires careful description of context to help interpret findings. | Designed as a framework for primary research. Requires evaluation within a synthesis context. |
CFIR20 | Itemises 39 factors influencing implementation of interventions offering taxonomy for classifying factors.20 Terminology reported as clear and coherent.91 | Complex framework that may include too many factors to be practical for data extraction reflecting that it has been designed for primary studies.91 | |
CICI framework22 | Based on concept analysis therefore offers broad coverage of context issues. | Limited conceptual thinking on the inter-relationships of framework elements of context and implementation. | |
Diffusion of innovations40 | Considers temporal issues of context and mechanisms of sustainability and spread. | Does not offer practical framework for most aspects of context. | |
iCAT-SR33 | Requires review team to make judgements on the extent to which the effects of the intervention depend on the context or setting within which it is implemented. | Limited specific detail on context although supplemented by embedded items with contextual implications, for example, organisational level. | |
Normalisation process theory49 | Acknowledges wider societal and contextual barriers and organisational and social factors. Defines context at early stages of intervention; implementation and evaluation. | While recognising context as a dynamic environment, and as a process, not a place; challenging to operationalise within a synthesis context. | |
PARIHS47 | Context is core construct and interacts with other constructs. | Focuses on implementation setting, not wider social, political or economic environment. Originally designed for primary studies.47 | |
PROGRESS-Plus53 | Includes population-based, social and cultural factors to explore/explain variation in intervention feasibility, acceptability, meaningfulness and effectiveness. | Broad population focus at expense of less common but equally important factors resulting in individual variation. | |
RE-AIM framework48 | Uses stakeholders to prioritise aspects for particular focus. | Primary study focus. Extensive resource use when exploring all aspects within a synthesis. Focuses mainly on number individuals adopting evidence as main contextual factor.16 | |
Theoretical Domains Framework50 | Specifically designed for implementation contexts. Includes Environmental context and resources and Social influences. Extensive use in systematic reviews. | Focuses on organisational level context with passing acknowledgement of external factors in the environment that cause stress. | |
TIDieR55 | Includes a WHERE question to elicit different levels of contextual detail. | Focuses on intervention description; offers exemplar for how context might be developed. Focuses on spatial aspects of context. | |
Literature searching | Citations, Lead authors, Unpublished materials, (Google) Scholar, Theories, Early examples, Related projects (CLUSTER) technique58 | Identifies additional ‘sibling studies’ to expand detail of context. | Potentially time-consuming and dependent on linked publications. |
Web searches for supplementary data on context not present in published reports | Offers accompanying detail to complement published studies. | Requires opportunistic searching; challenging to document. | |
Data extraction | Context templates | Prompts review team to compile proforma assessments to standardise reporting of context. | Missing data leads to inconsistent reporting and impaired comparison. |
iCAT_SR tool33 | Includes context among 10 dimensions that examine interplay of factors contributing to complexity.2 | Context dimension is optional, not required.92 Other elements implicitly relate to context. | |
TIDieR55 | As discussed above. | As discussed above. | |
Making use of review findings | |||
Producing the report | Reporting Standards (eg, CONSORT;62 PRISMA;25 PRISMA-CI;26 ENTREQ63) | Prompts to identify and elicit missing data on context. | Context or setting typically constitutes a single (or limited) number of items. |
Grading of findings | GRADE65 | Global influence supported by established methods group. Recently addressed contextualisation for (1) guidelines and (2) HTAs/systematic reviews.93 | Lacks structured approach for incorporating non-epidemiological evidence on context and implementation.39 |
GRADE-CERQual66 | Includes relevance element and broadens beyond time and place. Work in progress. | Requires further development to extend beyond less salient aspects of context. | |
Research recommendations | EPICOT-dts84 | Optional elements include Disease burden or relevance and Time aspect of core EPICOT elements. | Little evidence of uptake and utilisation (excepting NICE and isolated WHO guidelines). |
Implementation | Oxford Implementation Index77 | Contextual factors is largest of four domains: intervention design, actual delivery by trial practitioners, uptake of intervention by participants.77 | Sheer number of contextual elements requires prioritisation, particularly within context of evidence synthesis. |
Case studies, profiles, vignettes | Designed for use within local implementation context and to permit local tailoring. | Only focus on isolated simplistic aspects of context. | |
Spreadsheets, calculators (eg, NICE Physical Activity Return on Investment Tool)75 | Enables user to evaluate a portfolio of interventions in their geographical area (eg, region, county or local authority) and model different payback returns. Interventions are mixed and matched to identify best 'value for money'. | Underpinning assumptions behind model may change. Requires periodic updating. | |
Adaptation | SUPPORT tools59 | Offer a comprehensive and systematic guide for considering local context throughout the decision-making process. | Limited number of examples. May oversimplify decision making process—lack a complex system perspective. |
ADAPTE and other adaptation guidelines94 | Offer alternatives to developing de novo guidelines (eg, ADAPTE covers organisational and cultural considerations). | Frameworks need to be evaluated for rigour, efficiency and transparency of process. |
CFIR, Consolidated Framework for Implementation Research; CICI, Context and Implementation of Complex Interventions; CIMO, Context-Intervention-Mechanisms-Outcome;CONSORT, Consolidated Standards of Reporting Trials; ENTREQ, Enhancingtransparency in reporting the synthesis of qualitative research; EPICOT,Evidence, Population, Intervention, Comparison, Outcomes, and Time Stamp; EtD, Evidenceto Decision; GRADE, Grading of Recommendations, Assessment, Development,and Evaluation; GRADE-CERQual, Grading of Recommendations, Assessment, Development,and Evaluation-Confidence in Evidence from Reviews of Qualitative research; HTA, Health Technology Assessment; iCAT-SR, Intervention Complexity Assessment Tool for Systematic Reviews; LMIC, NICE,National Institute for Health and Clinical Excellence; PARIHS, PromotingAction on Research Implementation in Health Services; PerSPE©TIF, Perspective-Setting-Phenomenon-Environment-Comparison-Timing-Findings; PICOC, Population-Intervention-Comparison-Outcome-Context; PRISMA-P, PreferredReporting Items for Systematic Reviews and Meta-Analyses for Protocols; PRISMA-CI,Preferred Reporting Items for Systematic Reviews and Meta-Analyses for ComplexIntervention; PROGRESS-Plus, Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital;RCT, randomised controlledtrial; RE-AIM, Reach, Effectiveness,Adoption, Implementation, and Maintenance; SPICE, Setting-Perspective-Interest,-Phenomenon-of-Comparison-Evaluation; TIDieR, Template for Intervention Descriptionand Replication.