Article Text
PDF
Abstract
Conversations regarding qualitative research and qualitative data analysis in global public health programming often emphasize the product of data collection (audio recordings, transcripts, codebooks and codes), while paying relatively less attention to the process of data collection. In qualitative research, however, the data collector’s skills determine the quality of the data, so understanding data collectors’ strengths and weaknesses as data are being collected allows researchers to enhance both the ability of data collectors and the utility of the data. This paper defines and discusses a process for systematic debriefings. Debriefings entail thorough, goal-oriented discussion of data immediately after it is collected. Debriefings take different forms and fulfill slightly different purposes as data collection progresses. Drawing from examples in our health systems research in Tanzania and Sierra Leone, we elucidate how debriefings have allowed us to: enhance the skills of data collectors; gain immediate insights into the content of data; correct course amid unforeseen changes and challenges in the local context; strengthen the quality and trustworthiness of data in real time; and quickly share emerging data with stakeholders in programmatic, policy and academic spheres. We hope this article provides guidance and stimulates discussion on approaches to qualitative data collection and mechanisms to further outline and refine debriefings in qualitative research.
- health services research
- public health
- qualitative study
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
The quality of data in qualitative global health research is stronger when researchers engage local interviewers in a systematic debriefing process.
Understanding interviewers’ strengths and weaknesses in real time allows researchers to enhance interviewer skills and thus data quality.
Through systematic debriefing, researchers can identify and address gaps in the data; capture nuances and other non-verbal information; enhance intellectual partnership within teams; triangulate data; and build theory.
Drawing from our research experiences in Tanzania and Sierra Leone, this paper outlines the process and value of debriefings.
Introduction
A major goal of qualitative research, as applied in global public health, is to give voice to those whose lives are affected by health policies and programs, but whose ability to be heard by those in power and to effectively change health systems and structures is limited. The archetypal means through which such data are collected is the interview, a one-on-one encounter where a researcher and a respondent ‘are talking and asking questions of one another’ usually with the help of an interview guide.1–3 During the 1–2 hours that a typical interview lasts, a researcher must juggle competing demands: maintaining the interest and openness of the respondent, listening for responses that merit further probing, capturing unspoken cues and gestures, all while simultaneously ensuring that the data collected are relevant to the health issue of interest.1 4
In the academic ideal, qualitative research is undertaken by those who possess sociocultural understanding of the study context, and are formally trained in qualitative theory, methods and analysis. In global public health research this ideal is often not feasible. Instead, research teams commonly engage locally based qualitative data collection teams who possess essential knowledge in terms of context and language, but who lack formal qualitative training. These teams are routinely trained by an external research lead, an individual who possesses a graduate-level education in public health and qualitative research, but whose contextual or linguistic knowledge is insufficient.
In the ‘local team with outside technical support’ model of qualitative research—henceforth referred to as applied qualitative research or AQR—technical support takes different forms across five overarching phases in the research. See table 1 for a breakdown of tasks for the data collection team and research lead, respectively, during the preparation and execution of data collection, transcription and translation of files, data analysis and study write-up.
Organization of qualitative research and qualitative data collection teams
This practice paper focuses on the data collection phase of the table with an emphasis on debriefings, the process where a research lead interviews data collectors soon after a data collection activity. We view debriefings as a necessary element in qualitative research, particularly as the field comes to embrace styles beyond the conventional academic ideal (one person enacting the research process from conception to publication). Furthermore, while we observe a general consensus regarding the rationale and process for undertaking many points in the five-step table (table 1), we see that debriefings receive relatively little attention in the literature and in trainings on qualitative research for public health. When debriefings are mentioned in publications, including our own,5 6 there is minimal insight into what the debriefing entailed, how it was conducted or how debriefings fundamentally informed the data collection or analysis process. Finally, given that in AQR, much of the data are collected by individuals who have limited training in qualitative research or are more familiar with quantitative survey administration, we urge researchers to more thoughtfully consider discussions on process (ensuring that the data are collected in an iterativei fashion,7 that the data set is responding to the research question and that the skills of data collectors are strengthened in real time) rather than on outputs (transcription and coding). We now outline what debriefings are, how their purpose shifts in the process of data collection and how debriefings have amplified trustworthiness in our own research.8 9
What are debriefings?
Debriefings are a discrete moment in the qualitative data collection process where a research manager sits with a data collector (or data collection team) to discuss the tenor, flow and resulting findings from a recently undertaken data collection activity. Ideally conducted after the close of a day’s data collection, debriefings are an essential supplement to qualitative methods such as focus groups, interviews or observations.ii ,10 During debriefings, the research lead takes copious notes. These notes then serve as one component of the full qualitative data set, and methods used to analyze transcripts and observational memos are also applied to debriefing notes.
Debriefings spark immediate reflection on emerging findings; they force data collectors to think through the data that have emerged and to better position findings relative to data collected by fellow data collectors either that same day or to date. Debriefings allow research teams to identify gaps in the data collected and to redirect course—whether refining a line of inquiry, reconceptualizing a research question, opting when or whether to seek out alternative perspectives (such as negative or disconfirming cases), or adding or eliminating a respondent group or research method. Debriefings are the best protection against an unfortunate scenario where, long after the close of data collection, transcripts reveal that the research team did not pursue essential lines of inquiry, or worse, that the data collected will not be able to respond to research aims. For examples of debriefing templates, see online supplementary appendices 1 and 2.
How should debriefings be done?
Debriefings serve a different purpose as the process of data collection unfolds.
At the outset of data collection, debriefings are one-on-one (the lead researcher interviews the interviewer) and largely procedural in content. In our studies, these early debriefings have been used to learn from the interviewer what could be done to improve the process of data collection. We ask the interviewer questions such as: Is it feasible to find and interview respondents in a private setting (or is the community trailing after the interviewer-respondent pair to listen in on the interview)? Did recording devices work (let’s have a quick listen and upload the recording)? Are the consent forms understandable and did you have them signed or fingerprinted (let’s put them in this waterproof folder)? Are the instruments too short or too long? Do we have concerns about respondents growing tired or bored in an interview and if so, what do you think we should do about this informant fatigue? The earliest debriefings also allow the research lead to gauge the interviewers’ strengths and weaknesses as both interviewers and qualitative researchers. Did the interviewer appear interested, observant and engaged in the data collection activity? Did they probe on valuable lines of inquiry? Did they feel capable of shifting the interview back on track if it digressed in a manner that was not informing the research question? Did they capture non-verbal cues? What would they like from the research lead in terms of troubleshooting through a difficult process? Early debriefings are a means to ensure that the messages conveyed during trainings whether procedural (getting consent) or scientific (probing) are gelling among the data collection team, and to refine or reinforce these messages if they are not.
Once roughly a quarter of the data collection process is completed, we typically conduct a refresher training on the craft of high-quality interviewing.1 4 11 This presentation often involves distributing interviewing tips and tricks (see box 1: Interview tips, a refresher), and asking data collectors to share with the group one challenge they have encountered in the preceding interviews and how they overcame that challenge.
Interview tips,* a refresher
Adhere to ethical principles
Ask for consent.
Ensure privacy throughout the interview or focus group discussion.
Convey in your actions and your questions that you respect the respondent’s autonomy.
Remove any mystery about the recorder/recording device
Put the recorder within reach of the respondent.
Tell the respondent they can turn it off at will and show them how to do this.
Assure the respondent that only researchers will listen to the recording.
Use all senses to capture details
Recognize pauses long and short.
Capture what is spoken and unspoken (gestures, glances, fidgeting, fear, smiles, sincerity, pride).
Note the smells, sights, the ‘texture’ of the interview.
Keep a conversation comfortable
Start simple.
Ask uncomplicated, unintrusive questions.
Be prepared and open to responding to questions about who you are and why you are there.
Avoid double-barrelled questions.
Know when to pause.
Give time for responses.
Refer back to comments or phrases made by the respondent in the course of the interview.
Follow the golden rules of great interviews
Avoid the temptation to interrupt the respondent.
Use open-ended questions and probes.
Don’t attach your interpretation to a response.
Ask "remarkable questions in an unremarkable tone".*
Do not judge—not with your voice, body or face.
Avoid scientific jargon
Words and phrases like ‘plural health systems’, ‘structural violence’ and ‘stigma’ "sap the power and beauty of plain language".*
Be reflexive, be conscious of your role in this endeavor
Memo how you, as the human being you are, shaped this interview.
End every interview with this question: ‘Is there anything I should have asked you that I did not ask you?’
If the respondent offers some suggestions—ask those questions!
*Informed by Harrington’s4 ‘Intimate Journalism: The Art and Craft of Reporting Everyday Life’.
As data collection progresses, the nature of the debriefing usually shifts. Procedural questions become less necessary as processes have become routinized. There is also less one-on-one engagement between the research lead and individual interviewers in favor of a debriefing session, which resembles a focus group (with the research lead serving as both a moderator and notetaker). During the debriefing session, each interviewer provides a 2–4 min summary of their interviews or focus groups, with a special interest in describing key points or new findings from their interview. Conversations regarding triangulation (comparing and contrasting findings across data collectors or data collection methods) and topic saturation (the point when similar ideas and insights are heard again and again) typically begin to emerge in this phase as interviewers are encouraged to jump in when they could contextualize, confirm or dispute a piece of information based on their own interview. For both the research lead—and the data collection teams—this is among the most enjoyable and enlightening periods of the research process. Group debriefings prompt new ways of looking at an issue, help the interviewers gauge whether a follow-up interview is necessary, force research teams to think through how to reframe old questions or create new questions for subsequent interviews, and serve as reminders to the team that an interview is a short window into a person’s life during which contextualization of experiences occurs. The debriefing often sparks vibrant conversations among data collection teams about social desirability bias, thoughts on power, autonomy and decision-making within households and communities, and the role of the interviewer and research teams generally in terms of advocacy, human rights and social responsibility. Along with conversations on triangulation, the research team typically begins to discuss reflexivity, questioning how their social standing, personal experiences and inherent biases affect the nature of the interviews.
In the final phase of the debriefing process, the research lead is almost wholly removed from the process. The team nominates one of the data collectors to serve as moderator and another data collector to serve as a notetaker. The language of the group debriefing often switches from English (or other official language in the country) in favor of the local language (or the preferred language of the data collection team and the language of the interviews). During this phase, the research lead begins to build theories, devise an outline of preliminary findings and draw up a list of key local phrasings (emic terms) that may be valuable when presenting the research to stakeholders. In our studies, at this phase of data collection, we begin to develop a slide deck that will be later presented to principal investigators and others on the conclusion of data collection.
The final debriefing occurs on the day after the conclusion of data collection. During this session, the research lead presents the slides of preliminary findings to data collectors. Slides are edited based on feedback, and data collectors are invited to practice and then present portions of the presentation to an audience of academic, ministerial or programmatic peers.
Debriefings in Tanzania
In Tanzania, debriefings informed a fundamental shift in how the research team conceptualized the research question, ‘How do women and their spouses/support networks make decisions regarding where to seek care throughout the maternal care continuum?’12 Conducted in 2011, our team initially sought to test a hypothesis that care seeking for childbirth was largely determined by factors such as cost, risk, distance and intrahousehold negotiation. Following debriefings after the earliest interviews, it became apparent that the main issues driving women and their communities away from facilities centered on issues of disrespectful maternity care by providers toward patients. Thinking that these earliest interviews represented an outlier, the data collection continued to rely on the initial data collection instrument, which did not emphasize patient–provider relationships and made no mention of disrespectful or abusive care. As data collection progressed, however, themes related to disrespect continued to emerge. The research lead presented the findings to the study’s principal investigators, who confirmed that this line of inquiry warranted pursuit. The research lead then began a literature review to identify studies that emphasized disrespectful care, and—together with the study team—modified the tools. Had the data collectors not been in regular contact with one another and the research lead, they may have disregarded or downplayed findings related to disrespect, they may have been unsure how to probe about disrespect and abuse, or they may have felt hesitant to undertake a line of inquiry that was not outlined in the tool (and may spark politically contentious debates). Along with allowing the research team to recognize and then triangulate findings related to abuse, debriefings also allowed for immediate comparisons of how male-female pairs describe their role throughout care seeking for childbirth. This immediate comparison (and the incongruences that emerged when comparing accounts across husband-wife pairs) not only generated animated discussions within the team, but also identified another new line of probing (related to births before arrival), and guided decisions regarding whether and when to conduct follow-up interviews.13
Debriefings in Sierra Leone
In Sierra Leone, debriefings strengthened our study by enhancing the research team’s reflexivity, ability to build rapport and approach to sampling. Data were collected in 2010 with the aim of examining how families understood and manage childhood illnesses.14 In the course of the earliest debriefings, it became apparent that both researchers and respondents were weary of the data collection endeavor. Data collectors said they were shaken following discussions about child illness and child death with respondents who represented the poorest of the poor. Debriefings presented an opportunity for the team to talk through their anxiety and devise coping strategies collectively. Data collectors also described challenges of building trust with respondents, given strained relations in the wake of the country’s civil war. Several respondents were frightened by the audio recorder, concerned that their voice may be shared with a much wider audience (or used to inflict harm on them or their families). Many community members were also bothered by the presence of outsiders (the data collection team) in their communities; expressing incredulity that outsiders would travel long distances to ask about child health. The data collection team used debriefings to reconsider how to best present the team and explain the purpose of the research to community leaders (in a manner that would ensure all involved that this was a peaceful endeavor, that there was no ill will or underhanded intention of the data collection team toward the community). In terms of qualitative methods specifically, debriefings served as an opportunity to reiterate messages conveyed in the data collector training. Many members of the study team had more experience with quantitative rather than qualitative data collection, so there was a tendency at the outset of data collection to use interview guides as surveys—asking questions in exactly the manner they were written with no probing. Debriefings allowed the research lead to reiterate the open nature of interviewing, and provided a forum for data collectors with more qualitative experience to demonstrate how probing is best done. Finally, debriefings allowed the team to identify respondent types whose insights could inform the research question, but who were not initially a focal group for the study (first wives, mammy queens (female leaders), spiritual healers and traditional birth attendants).15 These individuals were not initially identified as key informants, but their essential role in deciding whether and when to take a child to a health facility emerged in the earliest interviews and compelled the team to change course in favor of including these individuals.
Conclusion
Systematic debriefings are a necessary complement to more conventional qualitative approaches. Debriefings make it possible to enhance the adaptable, thoughtful and empathetic-yet-questioning nature of qualitative research among data collection teams (thereby improving both the quality of data collected and the capacity of those collecting the data), to correct course in the event of unknowable changes, insights or challenges in a given context, and to quickly share emerging data with stakeholders in programmatic, policy and academic spheres. We have outlined herein a series of steps to conduct debriefings and demonstrated how we have used debriefings in studies across two contexts. We hope this article sparks interest and debate in the literature in terms of how debriefings could be used to improve the quality of qualitative data.
Supplemental material
Supplemental material
Acknowledgments
We are thankful to the qualitative data collection teams we have worked with and debriefed with through the years. In Tanzania: Zeswida Ahmedi, Santiel Mmbaga, Zaina Sheweji, Amrad Charles, Emmanuel Massawe, Maurus Mpunga, Rozalia Mtaturo. In Sierra Leone: Robert Sam-Kpakra, Agnes Farma, Abdul Karim Coteh, Emmanuel Abdulai, Alie Timbo, Mohamed Lamin Sowe, Aminata Alhaji Kamara, Daniella Kopio, Kadie Kandeh, Jestina Lavahun, Jeremiah Sam Kpakra, Lamin Bangura, Isatu Kargbo. We thank Dr. Kerry Scott for her review of this article and for contributing her own debriefing template as an appendix. We thank Prof. Walt Harrington for sharing his wisdom on the art and craft of interviewing. Finally, we acknowledge financial support by the Deutsche Forschungsgemeinschaft within the funding programme Open Access Publishing, by the Baden-Württemberg Ministry of Science, Research and the Arts and by Ruprecht-Karls-Universität Heidelberg.
Footnotes
↵i An iterative approach refers to the process of adapting and updating data collection tools (but possibly also methods and sampling) in light of information gleaned from data collected earlier.
↵ii Beyond qualitative research, debriefings have also been described as a way to examine response error in quantitative surveys,7 and as a way to interrogate differing assumptions of quantitative and qualitative researchers in a mixed methods study.10
Handling editor Seye Abimbola
Contributors The two authors contributed equally to the manuscript.
Funding The National Institute of Mental Health of the National Institutes of Health supported coauthor SAM (Award F31MH095653) throughout her PhD work. The Olympia-Morata-Programm supports her current position as an Assistant Professor at Heidelberg University.
Disclaimer The funders had no role in the decision to publish, or preparation of the manuscript. The content is the responsibility of the authors and does not necessarily represent the views of any funder.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.