Table 2

Tools and frameworks for handling context, mapped to stages of a systamatic review process

Stage of the review	Tool or framework	Strengths	Limitations
Throughout	Realist evaluation7/Realist synthesis34	Offers a method for integrating quantitative and qualitative data. Focuses on context as one line of inquiry, specifically addressed when assessing programme outcomes.15	Resource intensive; requires reviewers to prioritise which explanations would benefit from further exploration.
	Systems thinking4	Offers holistic picture of complex phenomena and interactive elements including context.	No specific review or synthesis tools or methodologies have been designed for this perspective.
	Evidence to Decision frameworks (eg, WHO-INTEGRATE10 or GRADE EtD38 framework)	Seek to identify elements of context throughout entire evidence production and guideline development process.	Comprehensive coverage makes it difficult to use entire tool within a tight systematic review time window. Reviewer must prioritise items relevant to specific review.
	Logic models2	Provide a graphical representation of context surrounding an intervention.	Offer simplistic perspective with a focus on inputs, processes, outputs and outcomes. Context is only identified as a backdrop to programme activities.
Planning the review
Question formulation and defining eligibility criteria	Question structures:11 PICOC,41 CIMO,42 SPICE,43 PerSPE©TIF11	Include important elements of context/setting when compared with intervention-based PICO formulation.	Adopt a simple linear framework for representing review question.
Protocol development	PROSPERO (International prospective register of systematic reviews) template45	Item #23 (context) requires summary details of the setting.	Focuses on context as it relates to study eligibility.
	Use of frameworks (see itemised list below)	Offer prompts to ensure appropriate issues are considered when conducting the review.	Few available frameworks for systematic consideration of context.
	PRISMA-P44	Reporting standard for all elements of a protocol.	Superficial approach to context (setting and time frame).
Conducting the review
Data extraction and/or analysis	Framework for the design and reporting of cluster RCTs.46	Requires careful description of context to help interpret findings.	Designed as a framework for primary research. Requires evaluation within a synthesis context.
	CFIR20	Itemises 39 factors influencing implementation of interventions offering taxonomy for classifying factors.20 Terminology reported as clear and coherent.91	Complex framework that may include too many factors to be practical for data extraction reflecting that it has been designed for primary studies.91
	CICI framework22	Based on concept analysis therefore offers broad coverage of context issues.	Limited conceptual thinking on the inter-relationships of framework elements of context and implementation.
	Diffusion of innovations40	Considers temporal issues of context and mechanisms of sustainability and spread.	Does not offer practical framework for most aspects of context.
	iCAT-SR33	Requires review team to make judgements on the extent to which the effects of the intervention depend on the context or setting within which it is implemented.	Limited specific detail on context although supplemented by embedded items with contextual implications, for example, organisational level.
	Normalisation process theory49	Acknowledges wider societal and contextual barriers and organisational and social factors. Defines context at early stages of intervention; implementation and evaluation.	While recognising context as a dynamic environment, and as a process, not a place; challenging to operationalise within a synthesis context.
	PARIHS47	Context is core construct and interacts with other constructs.	Focuses on implementation setting, not wider social, political or economic environment. Originally designed for primary studies.47
	PROGRESS-Plus53	Includes population-based, social and cultural factors to explore/explain variation in intervention feasibility, acceptability, meaningfulness and effectiveness.	Broad population focus at expense of less common but equally important factors resulting in individual variation.
	RE-AIM framework48	Uses stakeholders to prioritise aspects for particular focus.	Primary study focus. Extensive resource use when exploring all aspects within a synthesis. Focuses mainly on number individuals adopting evidence as main contextual factor.16
	Theoretical Domains Framework50	Specifically designed for implementation contexts. Includes Environmental context and resources and Social influences. Extensive use in systematic reviews.	Focuses on organisational level context with passing acknowledgement of external factors in the environment that cause stress.
	TIDieR55	Includes a WHERE question to elicit different levels of contextual detail.	Focuses on intervention description; offers exemplar for how context might be developed. Focuses on spatial aspects of context.
Literature searching	Citations, Lead authors, Unpublished materials, (Google) Scholar, Theories, Early examples, Related projects (CLUSTER) technique58	Identifies additional ‘sibling studies’ to expand detail of context.	Potentially time-consuming and dependent on linked publications.
Literature searching	Web searches for supplementary data on context not present in published reports	Offers accompanying detail to complement published studies.	Requires opportunistic searching; challenging to document.
Data extraction	Context templates	Prompts review team to compile proforma assessments to standardise reporting of context.	Missing data leads to inconsistent reporting and impaired comparison.
	iCAT_SR tool33	Includes context among 10 dimensions that examine interplay of factors contributing to complexity.2	Context dimension is optional, not required.92 Other elements implicitly relate to context.
	TIDieR55	As discussed above.	As discussed above.
Making use of review findings
Producing the report	Reporting Standards (eg, CONSORT;62 PRISMA;25 PRISMA-CI;26 ENTREQ63)	Prompts to identify and elicit missing data on context.	Context or setting typically constitutes a single (or limited) number of items.
Grading of findings	GRADE65	Global influence supported by established methods group. Recently addressed contextualisation for (1) guidelines and (2) HTAs/systematic reviews.93	Lacks structured approach for incorporating non-epidemiological evidence on context and implementation.39
	GRADE-CERQual66	Includes relevance element and broadens beyond time and place. Work in progress.	Requires further development to extend beyond less salient aspects of context.
Research recommendations	EPICOT-dts84	Optional elements include Disease burden or relevance and Time aspect of core EPICOT elements.	Little evidence of uptake and utilisation (excepting NICE and isolated WHO guidelines).
Implementation	Oxford Implementation Index77	Contextual factors is largest of four domains: intervention design, actual delivery by trial practitioners, uptake of intervention by participants.77	Sheer number of contextual elements requires prioritisation, particularly within context of evidence synthesis.
Implementation	Case studies, profiles, vignettes	Designed for use within local implementation context and to permit local tailoring.	Only focus on isolated simplistic aspects of context.
	Spreadsheets, calculators (eg, NICE Physical Activity Return on Investment Tool)75	Enables user to evaluate a portfolio of interventions in their geographical area (eg, region, county or local authority) and model different payback returns. Interventions are mixed and matched to identify best 'value for money'.	Underpinning assumptions behind model may change. Requires periodic updating.
Adaptation	SUPPORT tools59	Offer a comprehensive and systematic guide for considering local context throughout the decision-making process.	Limited number of examples. May oversimplify decision making process—lack a complex system perspective.
Adaptation	ADAPTE and other adaptation guidelines94	Offer alternatives to developing de novo guidelines (eg, ADAPTE covers organisational and cultural considerations).	Frameworks need to be evaluated for rigour, efficiency and transparency of process.

CFIR, Consolidated Framework for Implementation Research; CICI, Context and Implementation of Complex Interventions; CIMO, Context-Intervention-Mechanisms-Outcome;CONSORT, Consolidated Standards of Reporting Trials; ENTREQ, Enhancingtransparency in reporting the synthesis of qualitative research; EPICOT,Evidence, Population, Intervention, Comparison, Outcomes, and Time Stamp; EtD, Evidenceto Decision; GRADE, Grading of Recommendations, Assessment, Development,and Evaluation; GRADE-CERQual, Grading of Recommendations, Assessment, Development,and Evaluation-Confidence in Evidence from Reviews of Qualitative research; HTA, Health Technology Assessment; iCAT-SR, Intervention Complexity Assessment Tool for Systematic Reviews; LMIC, NICE,National Institute for Health and Clinical Excellence; PARIHS, PromotingAction on Research Implementation in Health Services; PerSPE©TIF, Perspective-Setting-Phenomenon-Environment-Comparison-Timing-Findings; PICOC, Population-Intervention-Comparison-Outcome-Context; PRISMA-P, PreferredReporting Items for Systematic Reviews and Meta-Analyses for Protocols; PRISMA-CI,Preferred Reporting Items for Systematic Reviews and Meta-Analyses for ComplexIntervention; PROGRESS-Plus, Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital;RCT, randomised controlledtrial; RE-AIM, Reach, Effectiveness,Adoption, Implementation, and Maintenance; SPICE, Setting-Perspective-Interest,-Phenomenon-of-Comparison-Evaluation; TIDieR, Template for Intervention Descriptionand Replication.