A systematic approach to context-mapping to prepare for health interventions: development and validation of the SETTING-tool in four countries

Introduction

Evidence-based interventions consistently fail to be implemented into routine practice, resulting in a substantial waste of resource potential and undermining health outcomes.1 2 Implementation—the act of carrying an intervention into effect3—is a typically effortful, complex process.4 Context-driven implementation strategies help to avoid implementation failure.2 3 5–10 However, there is no evidence-based, practical guide on how to first understand and map contexts in order to design such strategies.

Context can be defined as a set of unique characteristics and circumstances surrounding the implementation effort.11 A critical dimension of context pertains to local attitudes and knowledge regarding an intervention.8 Health beliefs and risk perceptions shape behaviour, ranging from perceived causes of a disease to perceived barriers to change disease-related behaviour.12–14 For example, it would be unfeasible to motivate a community to instal kitchen chimneys to reduce indoor smoke, when local beliefs entail that ghosts enter homes via the chimneys. However, implementers are often unaware of such beliefs.15

Beliefs and behaviours are mostly not adequately mapped as it is resource-consuming and time-consuming.15 Moreover, there are no standardised practical methodologies available to guide context-mapping efforts.6 16 17 The identification, development and testing of practical implementation methodologies are now considered top priorities in implementation science.16 Especially clinical researchers without implementation science expertise would benefit from systematic guidance to enable them to reproduce evidence-based methodologies in practice.18

Paradoxically, implementation evidence is particularly limited for low-resource settings, where the global burden of disease is highest and resources are scarcest.9 19 Therefore, we aimed to develop and validate a practical, systematic context-mapping tool for assessing local beliefs and behaviours in low-resource settings. To validate the tool, it was tested during a research project on chronic respiratory diseases (CRDs). CRDs are a major burden to low-resource settings; 90% of chronic obstructive pulmonary disease (COPD)-related deaths and 80% of asthma-related deaths occur in low-income and middle-income countries (LMICs).20In this paper, we describe the development and validation of the Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies (SETTING-tool) in practice, and report on its feasibility, acceptability and effectiveness.

Development and validation of the context-mapping tool

With a multidisciplinary expert panel including professionals from four diverse low-resource countries, we employed a multistage and multimethod approach to develop a context-mapping tool (see online supplemental appendix 1). The approach was similar to the approach for the development of the ImpRes tool.18 We first generated components for the tool based on a pragmatic literature search and methodological principles considered state-of-the art in our fields (see online supplemental appendix 1). A principle would be, for example, that mixed-method designs generally provide better insights than qualitative or quantitative research alone). Together with the expert panel, we then detailed the components (eg, use a ‘Rapid Assessment Process (RAP) for qualitative data collection’) based on expected feasibility, acceptability and effectiveness. We discussed these components until consensus was reached, and consolidated these into a comprehensive step-by-step tool. The resulting context-mapping tool had a stepwise approach, and was named the SETTING-tool (figure 1). Next, a research team sequentially validated the SETTING-tool prospectively in six diverse low-resource settings across four countries (see online supplemental appendix 2) within the Free Respiratory Evaluation and Smoke-exposure reduction by primary Health cAre Integrated gRoups (FRESH AIR) project (Box 1). Feasibility and acceptability of the tool were evaluated using the modified Conceptual Framework for Implementation Fidelity.21Effectiveness of the tool was assessed by determining the degree to which its application resulted in context-driven adjustments in the implementation strategies of subsequent FRESH AIR lung health interventions. Results guided iterative fine-tuning of the tool for the next setting. Online supplemental appendix 1 provides a detailed description of the full methodology for the development and evaluation of the tool.

Request permission

Application of the SETTING-tool in the FRESH AIR project

The tool is discussed step by step, including the rationale for each step and its application in the FRESH AIR project (see also table 1).

STEP 1: Coset study priorities (figure 2)

Request permission

Figure 2

Step 1: collaboratively set the study priorities. Key to the treasure case (to successful completion of the step) is to define end-users of the study data, identify other stakeholders and actively engage all. Then explore the needs and coformulate the exact aim and scope of the context assessment, the setting and other priorities.

As a first step, we (the context-mapping researchers) collaboratively set the study priorities with our end-users and other influential stakeholders. We defined end-users of our study findings as (1) our FRESH AIR consortium colleagues, who would use the data to design context-driven implementation strategies for their planned interventions and (2) future implementers in similar settings, such as policy-makers, non-governmental organisations (NGOs) and health workers. The other influential stakeholders we identified, ranged from the Minister of Health to local community members (CMs). There were diverse in terms of age, sex and educational background. They were engaged in Stakeholder Engagement Groups (see online supplemental appendix 3 table E1). Stakeholders received no financial compensation for participation. We engaged stakeholders from the beginning to ensure the study would closely meet their needs; this can be crucial in fostering uptake of the findings into practice later.3 7 22–25

Together with the FRESH AIR consortium colleagues and the Stakeholder Engagement Groups, we collaboratively set the context-mapping study’s priorities. We formulated the exact study aim: ‘To explore CRD-related beliefs and behaviours in diverse low-resource settings.’ We used a theoretical framework (a combination of three well-stablished health-behaviour frameworks,12–14 online supplemental appendix 4) to further guide the specification of the aim and our methodological orientation.26–29 This framework consisted of factors leading to (intention to conduct) behaviour, such as the locally perceived identity of CRDs, perceived causes, and perceived barriers to change CRD-related behaviour. CRD-related behaviour included risk behaviour (tobacco smoking, cooking/heating in a way that generates household air pollution), help-seeking behaviour by CMs, and helping behaviour by healthcare professionals.

We also determined the study settings. Implementation research generally aims to produce knowledge applicable across various settings,24 and rural settings have a high prevalence of CRD and risk factors to CRD.30 31 Therefore, six rural settings across four countries were selected purposively to represent diversity, and conveniently based on established relationships with the settings (online supplemental appendix 2 further details the settings and the definition of rural). These settings were in Uganda, Kyrgyzstan (a highland and lowland setting), Vietnam and Greece (a Roma camp and a rural Greek setting).

STEP 2: A mixed-method study design: Combine a rapid assessment with a survey (figure 3)

Request permission

Figure 3

Step 2: a mixed-method study design. Key is to include (1) a qualitative rapid assessment process (RAP) in which data are collected by multiple methods, sources and researchers, and (2) a quantitative survey.

Second, we codesigned the context-mapping study with the end-users (the FRESH AIR consortium colleagues) and other stakeholders. Our researchers’-perspective enabled us to underpin the study with evidence-based methods, whereas the stakeholders ensured high compatibility of the selected methods with the local context. Codesigning the methods furthermore benefited the stakeholders’ engagement.3 7

We cocreated a population-based, cross-sectional, observational design. Mixed-method designs generally provide a better understanding of situations than qualitative or quantitative research alone.3 32 Therefore, we adopted a mixed-method approach enabling to (1) explore and describe qualitatively how people perceive CRD, and what their related behaviour is like, and then 2) quantify the findings. We selected an RAP for the qualitative data collection,33 as it allowed us to timely inform our FRESH AIR colleagues on the implementation strategies of their interventions. We chose to collect the quantitative data using a survey. This enabled us to triangulate our data (to compare data collected through multiple methods, sources, theories and/or researchers) to optimise the validity of our findings (figure 3).34

The qualitative RAP

The key principles of an RAP are (1) a system (community) perspective, (2) rapid, in-depth and iterative data collection and analysis and (3) triangulation of data. We immersed into the rural communities for a short period and collected data simultaneously in subgroups using multiple methods.35 We included semistructured interviews, focus groups, documents and observations of healthcare professionals (respiratory consultations) and CMs (smoking, cooking and heating behaviour). Online supplemental appendix 2 details our planning and field methods.

Informants were selected based on opportunity and diversity (in sex, age, background, etc), to obtain rich data.26 We distinguished three participant groups: healthcare workers (HPs), CMs and key informants (stakeholders with either in-depth knowledge or an overall overview, such as religious leaders) (see online supplemental appendix 3 for inclusion and exclusion criteria). The sample size relied on when data saturation was achieved (when additional research activities would no longer result in additional insights). Preliminary findings were collated in daily debriefings with the entire team, guided by the theoretical framework. New informants were identified, and emerging themes and data saturation discussed . Iterative adaptations to the data collection materials, strategy and planning were also made. Accordingly, data collection was highly flexible and driven by local needs.33

The quantitative survey

We conducted one survey for CMs (200 per setting) and one for HPs (40 per setting). The CMs were randomly selected using a three-stage sampling approach based on the WHO ‘Expanded programme for Immunisation’, because it was especially suitable for rural settings.36 We then invited all eligible HPs from the nearest healthcare facilities.

STEP 3: Create context-sensitive study materials with high validity (figure 4)

Request permission

Figure 4

Step 3: a toolbox for creating context-sensitive materials with high validity. Key to success is to use evidence-based components where available throughout every sub-step, in our case: use (A) a theoretical framework to guide development of the materials, (B) a syndromic approach for expected low awareness of the phenomenon studied, (C) a vignette to avoid stigmatisation or address sensitive topics, (D) validated questionnaires, (E) a careful translation process and (F) pilot-testing of the materials.

Step 3 concerns the development of context-sensitive research materials with high validity. For the field methods selected in step 2, we needed corresponding materials: topic lists for the interviews and focus groups, semistructured forms for the observations, and structured questionnaires for the survey. Our theoretical framework guided the development of the research materials (eg, the construct ‘perceived cause of CRD’) (step 3A, figure 4). We anticipated a low awareness of COPD, asthma and the implications.30 37 38 Hence, to enable informants to relate to the topic we adopted a syndromic (symptom-based) approach (step 3B). As informants could fear stigmatisation for tuberculosis they could be hesitant to speak freely about respiratory symptoms. Therefore, we introduced the topic using a vignette (3C) which detailed a story about someone with slowly progressive chronic respiratory symptoms typical for COPD.39 We then probed using questions such as: ‘Are there people with a similar situation living in your community?’ and ‘What, according to you, is the cause of the situation?’

The questionnaires for CMs and HPs were composed of validated questionnaires (3D). They included demographical questions, the revised version of the brief-Illness Perception Questionnaire,40 CM risk behaviour (solid fuel use for cooking/heating and tobacco use)41–43 and HP treatment behaviour respectively.44 Context-specific adjustments were made (eg, rainy/dry season instead of winter/summer).

The research materials were translated into each local language, back-translated and adjusted accordingly (3E). The qualitative materials were tested internally during our training sessions and iteratively improved throughout the study.27 The quantitative materials were pilot tested on feasibility and content validity among 10% or our targeted sample in each setting (3F). Resulting adaptations were, for example, replacing the Likert-scales (0–10) by multiple choice options, as scales turned out to be too abstract for local participants to comprehend. See online supplemental appendix 5 and online supplemental file 2 for the resulting research materials.

STEP 4: Collect data with a trained, diverse team including community researchers (figure 5)

Request permission

Figure 5

Step 4: collect data involving community researchers, who are trusted by their community and have a thorough understanding of local networks and local health beliefs and behaviours. Key to the treasure case is the combination of local researchers with an insiders’ perspective, with researchers with an external perspective.

In step 4, we set up a well-trained and diverse team to collect the data. It included researchers from the communities, such as local nurses. Their insiders’ perspectives offered deep understanding of local networks and prevailing themes,35 and they facilitated access to and openness of the participants. The combination with the ‘outsiders’ perspectives’ (team members from the country’s universities and Dutch researchers) helped to point out remarkable, typical local themes that were taken for granted by the community team members. Additional diversity in terms of background, age and gender helped to further enlighten the research topic from multiple perspectives and enrich the data.35 Notably, debriefing sessions generated additional data, for example, when community researchers and international researchers discussed explanations for certain findings from their own perspective. All team members received a thorough 1-day training from an experienced researcher, securing ethical standards during data collection and enhancing uniformity and high-quality data collection (online supplemental appendix 3).26 Further information on the data collection and training can be found in online supplemental appendix 3.

STEP 5: Analyse pragmatically and/or in-depth (figure 6)

Request permission

Figure 6

Step 5: decide to analyse the data pragmatically and/or in-depth, depending on the objective; key to success is timely informing the implementation design for related health interventions, which may have to be supplemented by more in-depth analyses for scientific purposes.

Step 5 emphasises the important difference between more pragmatic and in-depth analyses. The overall FRESH AIR project was bound to timelines that did not allow for time-consuming in-depth qualitative analyses of the context-assessment before the subsequent lung health interventions had to be implemented. Therefore, we pragmatically used our preliminary qualitative analyses of our debriefing sessions, which we communicated immediately after the RAP. We continued to update partners on the findings during the subsequent in-depth analyses.

For the in-depth analyses of qualitative data, we first transcribed the audio recordings verbatim.26 45 We coded the data both deductively and inductively, using thematic analyses to identify themes.46 Two researchers independently applied The Framework Method for data structuring and reduction (online supplemental appendix 6), compared results and discussed these until consensus was reached.45 We used appropriate qualitative software to facilitate data storing, analyses and sharing (online supplemental appendix 6). Next, data from the observations were charted into a table. These data together with field notes were used to complement and triangulate the data from the interviews and focus groups.

Our quantitative data were analysed using associative analyses and descriptive statistics, in which we calculated frequencies of prevalence.

STEP 6: Disseminate findings and promote data use (continuous step) (figure 7)

Request permission

Figure 7

Step 6: continuous communication between the research team, end-users, and other stakeholders by a tailored message and delivery method ensures that findings will be received and used.

As a continuous process, we disseminated the findings using a tailored message and delivery strategy, to optimise information use among the FRESH AIR colleagues and other stakeholders.7 23 23 47–49 We communicated our preliminary findings immediately after the RAP. Further dissemination of our in-depth findings occurred during international scientific conferences. Policy-makers, NGOs and health workers were reached via international and local health conferences in each country.

Validation outcomes

Testing of the SETTING-tool in the six settings demonstrated that feasibility of the tool was high. Generally, all steps were implemented as intended with reasonable effort, and within our budget and time planning (online supplemental appendix 7,8). However, in the Roma setting, we needed to invest a substantial amount of time for building trust with the community. Therefore, we could only test our methodology, but did not achieve data saturation. Acceptability of the tool was consistently high; both the users of the tool and to whom it was applied considered it appropriate, based on their collaborative attitude (online supplemental appendix 7). Lastly, effectiveness of the tool was considered high; the application resulted in findings that led to multiple major context-driven changes to the planning and implementation strategies of the subsequent FRESH AIR interventions (table 2).

Conclusion

We have developed and validated a systematic tool for context-mapping, enabling to design context-driven implementation strategies for (lung) health interventions in low-resource settings. Its application to the FRESH AIR project demonstrated that the SETTING-tool is highly feasible, acceptable, and effective in facilitating context-driven adaptations to implementation strategies in six diverse settings across the globe.

The SETTING-tool was cocreated by an international expert panel and local end-users. Development was both theory driven and practice driven. Concepts from a rich diversity of relevant fields were consolidated, ranging from implementation science to global health. Additionally, the methodology was prospectively validated in six diverse settings in four countries on three continents: from deprived Ugandan slums, to rapidly developing Vietnamese villages, to austerity-impacted rural Greece and highly risk-burdened Kyrgyzstan. Feasibility, acceptability and effectiveness of the tool were consistently high. This demonstrates its global applicability.

However, it is difficult to assess whether better alternatives would have been available to the components included in our tool. In our pragmatic literature searches, there were no studies comparing the effectiveness of different components and certainly not across contexts. In this practice paper, we also did not compare our context-mapping outcomes against those of a control group. Lastly, although application of the SETTING-tool was feasible, the overall feasibility of working with certain populations should remain of consideration. For example, the tool’s feasibility was also high in the Roma camp, yet working with the Roma population generally required more time than working in other settings.

Existing studies, frameworks and toolkits emphasised the need for context-driven implementation strategies and repetitive calls highlighted the need for evidence on how to design those.5 6 9 16 17 19 The SETTING-tool addresses these calls by guiding the researcher step-by-step through the context-mapping process. Other than existing approaches such as Community-Based Participatory Research,50 this tool is developed as a practical guide in the field. It covers the entire context-mapping process, including for example developing the research materials and promoting uptake of the findings. Each step is easily reproducible. Key elements for successful completion of the steps are highlighted. This structured guidance can particularly serve clinical researchers without implementation expertise.18

The SETTING-tool is particularly relevant as it was developed for low-resource settings, where the burden of disease is highest, while the means to combat disease and evidence on how to do so are scarcest.5 9 19 22 51 52 Furthermore, the tool addresses two important issues that contribute to the avoidable loss of 85% of investments in health and biomedical research (US$200 billion lost in 2010 alone): (1) failure to establish priorities based on stakeholders’ needs and (2) poorly designed research methods.53 54

The SETTING-tool can be used flexibly; it can be adapted according to local needs, as long as the core elements (the six steps) remain intact and the tool remains simple to use.7 55 Researchers should also ensure to continuously collaborate closely with the stakeholders (box 2). Flexible aspects are, for example, that not every health topic requires the use of a vignette to avoid stigmatisation. Use of the tool across diverse contexts, for other health topics and by researchers that were not involved in its development, could provide further insight on the generalisability of the tool. This would also allow for assessing the importance of individuals components depending on the setting or targeted disease. Although validated in LMICs, the tool might also be transferable to high-income settings.

Notably, the tool focuses on local beliefs and behaviours, and it is important to remain aware of other elements that shape context.8 Systematically assessing a context on multiple elements (beliefs and behaviour, physical environment, organisational structures, etc) would require a substantial amount of resources. We would then recommend to conduct the tool’s steps more pragmatically; particularly the in-depth analyses in step 5 as they are most time-consuming.

To conclude, the SETTING-tool can support researchers, policy-makers, health workers, NGOs and other implementers to apply evidence-based methods in context-mapping. This can facilitate them to design context-driven implementation strategies to increase implementation success of their health interventions, potentially avoiding the widescale waste of scarcely available resources. Ultimately, this could improve health outcomes.