Table 2

Tools and frameworks for handling context, mapped to stages of a systamatic review process

Stage of the reviewTool or frameworkStrengthsLimitations
ThroughoutRealist evaluation7/Realist synthesis34 Offers a method for integrating quantitative and qualitative data. Focuses on context as one line of inquiry, specifically addressed when assessing programme outcomes.15 Resource intensive; requires reviewers to prioritise which explanations would benefit from further exploration.
Systems thinking4 Offers holistic picture of complex phenomena and interactive elements including context.No specific review or synthesis tools or methodologies have been designed for this perspective.
Evidence to Decision frameworks (eg, WHO-INTEGRATE10 or GRADE EtD38 framework)Seek to identify elements of context throughout entire evidence production and guideline development process.Comprehensive coverage makes it difficult to use entire tool within a tight systematic review time window. Reviewer must prioritise items relevant to specific review.
Logic models2 Provide a graphical representation of context surrounding an intervention.Offer simplistic perspective with a focus on inputs, processes, outputs and outcomes. Context is only identified as a backdrop to programme activities.
Planning the review
 Question formulation and defining eligibility criteriaQuestion structures:11 PICOC,41 CIMO,42 SPICE,43 PerSPE©TIF11 Include important elements of context/setting when compared with intervention-based PICO formulation.Adopt a simple linear framework for representing review question.
 Protocol developmentPROSPERO (International prospective register of systematic reviews) template45 Item #23 (context) requires summary details of the setting.Focuses on context as it relates to study eligibility.
Use of frameworks (see itemised list below)Offer prompts to ensure appropriate issues are considered when conducting the review.Few available frameworks for systematic consideration of context.
PRISMA-P44 Reporting standard for all elements of a protocol.Superficial approach to context (setting and time frame).
Conducting the review
 Data extraction and/or analysisFramework for the design and reporting of cluster RCTs.46 Requires careful description of context to help interpret findings.Designed as a framework for primary research. Requires evaluation within a synthesis context.
CFIR20 Itemises 39 factors influencing implementation of interventions offering taxonomy for classifying factors.20 Terminology reported as clear and coherent.91 Complex framework that may include too many factors to be practical for data extraction reflecting that it has been designed for primary studies.91
CICI framework22 Based on concept analysis therefore offers broad coverage of context issues.Limited conceptual thinking on the inter-relationships of framework elements of context and implementation.
Diffusion of innovations40 Considers temporal issues of context and mechanisms of sustainability and spread.Does not offer practical framework for most aspects of context.
iCAT-SR33 Requires review team to make judgements on the extent to which the effects of the intervention depend on the context or setting within which it is implemented.Limited specific detail on context although supplemented by embedded items with contextual implications, for example, organisational level.
Normalisation process theory49 Acknowledges wider societal and contextual barriers and organisational and social factors. Defines context at early stages of intervention; implementation and evaluation.While recognising context as a dynamic environment, and as a process, not a place; challenging to operationalise within a synthesis context.
PARIHS47 Context is core construct and interacts with other constructs.Focuses on implementation setting, not wider social, political or economic environment. Originally designed for primary studies.47
PROGRESS-Plus53 Includes population-based, social and cultural factors to explore/explain variation in intervention feasibility, acceptability, meaningfulness and effectiveness.Broad population focus at expense of less common but equally important factors resulting in individual variation.
RE-AIM framework48 Uses stakeholders to prioritise aspects for particular focus.Primary study focus. Extensive resource use when exploring all aspects within a synthesis. Focuses mainly on number individuals adopting evidence as main contextual factor.16
Theoretical Domains Framework50 Specifically designed for implementation contexts. Includes Environmental context and resources and Social influences. Extensive use in systematic reviews.Focuses on organisational level context with passing acknowledgement of external factors in the environment that cause stress.
TIDieR55 Includes a WHERE question to elicit different levels of contextual detail.Focuses on intervention description; offers exemplar for how context might be developed. Focuses on spatial aspects of context.
 Literature searchingCitations, Lead authors, Unpublished materials, (Google) Scholar, Theories, Early examples, Related projects (CLUSTER) technique58 Identifies additional ‘sibling studies’ to expand detail of context.Potentially time-consuming and dependent on linked publications.
Web searches for supplementary data on context not present in published reportsOffers accompanying detail to complement published studies.Requires opportunistic searching; challenging to document.
 Data extractionContext templatesPrompts review team to compile proforma assessments to standardise reporting of context.Missing data leads to inconsistent reporting and impaired comparison.
iCAT_SR tool33 Includes context among 10 dimensions that examine interplay of factors contributing to complexity.2 Context dimension is optional, not required.92 Other elements implicitly relate to context.
TIDieR55 As discussed above.As discussed above.
Making use of review findings
 Producing the reportReporting Standards (eg, CONSORT;62 PRISMA;25 PRISMA-CI;26 ENTREQ63)Prompts to identify and elicit missing data on context.Context or setting typically constitutes a single (or limited) number of items.
 Grading of findingsGRADE65 Global influence supported by established methods group. Recently addressed contextualisation for (1) guidelines and (2) HTAs/systematic reviews.93 Lacks structured approach for incorporating non-epidemiological evidence on context and implementation.39
GRADE-CERQual66 Includes relevance element and broadens beyond time and place. Work in progress.Requires further development to extend beyond less salient aspects of context.
 Research recommendationsEPICOT-dts84 Optional elements include Disease burden or relevance and Time aspect of core EPICOT elements.Little evidence of uptake and utilisation (excepting NICE and isolated WHO guidelines).
 ImplementationOxford Implementation Index77 Contextual factors is largest of four domains: intervention design, actual delivery by trial practitioners, uptake of intervention by participants.77Sheer number of contextual elements requires prioritisation, particularly within context of evidence synthesis.
Case studies, profiles, vignettesDesigned for use within local implementation context and to permit local tailoring.Only focus on isolated simplistic aspects of context.
Spreadsheets, calculators (eg, NICE Physical Activity Return on Investment Tool)75 Enables user to evaluate a portfolio of interventions in their geographical area (eg, region, county or local authority) and model different payback returns. Interventions are mixed and matched to identify best 'value for money'.Underpinning assumptions behind model may change. Requires periodic updating.
 AdaptationSUPPORT tools59 Offer a comprehensive and systematic guide for considering local context throughout the decision-making process.Limited number of examples. May oversimplify decision making process—lack a complex system perspective.
ADAPTE and other adaptation guidelines94 Offer alternatives to developing de novo guidelines (eg, ADAPTE covers organisational and cultural considerations).Frameworks need to be evaluated for rigour, efficiency and transparency of process.
  • CFIR, Consolidated Framework for Implementation Research; CICI, Context and Implementation of Complex Interventions; CIMO, Context-Intervention-Mechanisms-Outcome;CONSORT, Consolidated Standards of Reporting Trials; ENTREQ, Enhancingtransparency in reporting the synthesis of qualitative research; EPICOT,Evidence, Population, Intervention, Comparison, Outcomes, and Time Stamp; EtD, Evidenceto Decision; GRADE, Grading of Recommendations, Assessment, Development,and Evaluation; GRADE-CERQual, Grading of Recommendations, Assessment, Development,and Evaluation-Confidence in Evidence from Reviews of Qualitative research; HTA, Health Technology Assessment; iCAT-SR, Intervention Complexity Assessment Tool for Systematic Reviews; LMIC, NICE,National Institute for Health and Clinical Excellence; PARIHS, PromotingAction on Research Implementation in Health Services; PerSPE©TIF, Perspective-Setting-Phenomenon-Environment-Comparison-Timing-Findings; PICOC, Population-Intervention-Comparison-Outcome-Context; PRISMA-P, PreferredReporting Items for Systematic Reviews and Meta-Analyses for Protocols; PRISMA-CI,Preferred Reporting Items for Systematic Reviews and Meta-Analyses for ComplexIntervention; PROGRESS-Plus, Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital;RCT, randomised controlledtrial; RE-AIM, Reach, Effectiveness,Adoption, Implementation, and Maintenance; SPICE, Setting-Perspective-Interest,-Phenomenon-of-Comparison-Evaluation; TIDieR, Template for Intervention Descriptionand Replication.