Article Text
PDF
Abstract
Changes in research practice during the COVID-19 pandemic necessitates renewed attention to ethical protocols and reporting for data collection on sensitive topics. This review summarises the state of ethical reporting among studies collecting violence data during early stages of the pandemic. We systematically searched for journal publications from the start of the pandemic to November 2021, identifying 75 studies that collected primary data on violence against women and/or violence against children. We developed and applied a 14-item checklist of best practices to assess the transparency of ethics reporting and adherence to relevant global guidelines on violence research. Studies reported adhering to best practices on 31% of scored items. Reporting was highest for ethical clearance (87%) and informed consent/assent (84/83%) and lowest for whether measures to promote interviewer safety and support (3%), for facilitating referrals for minors and soliciting participant feedback were in place (both 0%). Violence studies employing primary data collection during COVID-19 reported on few ethical standards, obscuring stakeholder ability to enforce a ‘do no harm’ approach and to assess the reliability of findings. We offer recommendations and guidelines to improve future reporting and implementation of ethics within violence studies.
- COVID-19
- Maternal health
- Child health
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Despite existing ethical guidance on how to safely collect data on violence against women and violence against children, there is no standardised or accepted guidance on ethical reporting when research on violence is published.
WHAT THIS STUDY ADDS
This study develops a 14-item checklist of best practices for the transparent and ethical reporting of violence research accounting for challenges during COVID-19 comprised of four domains: (1) Institutional Review Board approval, (2) interviewer selection, training and support, (3) sampling and engaging with respondents and (4) referrals and adverse events, and applies this checklist to 75 studies which collected data on violence published since the start of the pandemic.
Results show reporting on ethics is low, regardless of type of violence assessed or modality of data collection, with studies adhering to best practices in reporting in 31% of scored items: the highest reporting was for ethical clearance (87%) and informed consent/assent (84/83%) and lowest reporting was for measures to promote interviewer safety and support (3%), facilitating referrals for minors and soliciting participant feedback (both 0%).
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Efforts to improve the reporting of violence research are an important step to improve the quality and safety of studies and, as violence researchers, to fulfil our commitment to listen to and learn from participants while ensuring a ‘do no harm’ approach.
This study serves as a starting point to improve the reporting of violence research by proposing a checklist of items and providing strategies that can be used and adapted by researchers, journal editors, ethics committees and funders.
Introduction
Research has demonstrated increases in violence against women and violence against children (VAW/VAC) across numerous settings during the COVID-19 pandemic.1–4 This widespread evidence within a relatively short time period is due to creative use of available administrative data, as well as analysis of ongoing and new data collection efforts. In many parts of the world, data collection during the pandemic required adopting remote or other novel methods to successfully and safely reach and interview participants. Such methods were rarely used for VAW/VAC prior to the pandemic, particularly in low-income settings.5 These efforts challenged teams to ensure the appropriate adaptation of violence-specific safeguarding and ethical protocols. For example, data collected online or over the phone may leave participants vulnerable to lack of privacy, where responses could be overheard or where questionnaire forms or information might be viewed online by perpetrators or household members.6 7 In addition, shut-downs and reductions in service provision of violence and social services added complications, particularly for assuring the quality of, and continuous access to, referral services and for implementing response measures for adverse events.8 Research teams were forced to choose between collecting violence data with women and children in ethically challenging and uncertain contexts or opting to forgo primary data collection altogether.9
There remain differing opinions as to if, and how, data on VAW/VAC can be safely and ethically collected in such circumstances. Some early guidance during the pandemic suggested not to collect remote data at all, with the WHO and UN Women emphasising the mantra ‘Do not prioritise data over women’s safety’.10 Others suggested conditions which must be met in order to justify proceeding, including the ability to address safety concerns for participants, implement quality referrals and the critical use of data for immediate policy action.6 7 To date, no universal protocols exist for the design and reporting of remote research on VAW/VAC and ethical review boards are often ill-equipped to advise on violence-specific protocols even in face-to-face data collection efforts. Therefore, the decision of what VAW/VAC measures to collect and how go about setting up sufficient safeguards during COVID-19 was often made on a case-by-case basis by individual research teams.
This paper reviews reporting on ethics and safeguarding among studies where primary data on VAW/VAC were collected during the pandemic, including using remote methods to guide future research ethics and practice. In a field where methods and approaches continue to evolve and where the risk of harm is high, a commitment to transparently reporting the ethical choices research teams made is essential. We argue for greater attention to the development, implementation and reporting of ethics protocols within future studies and publications, to meet commitments to protect participant and researcher safety, to enhance data quality and to ensure researchers can learn from, and are accountable to, each other. To that end, we offer recommendations for researchers and journals across disciplines on which aspects are critical to ensure transparency, offering a 14-item checklist both to guide study design, research reporting and peer-review. Although our study presents new findings explicitly focused on data collection during COVID-19, poor reporting on ethical practices predates the COVID-19 pandemic.11 The stocktaking on ethics for VAW/VAC research comes at a critical time, when changes in data collection methodologies, advances in information technology and macrochanges across settings have raised debates around harmful practices in data collection. Results suggest the need for greater consensus, guidance and accountability in order to ensure a ‘do no harm’ approach.
Methods
Information sources and search strategy
We searched the studies compiled in the Global Tracker of Studies of VAW/VAC during COVID-19 (referred to as ‘the tracker’), compiled from Google scholar, as well as studies found via multiple listservs, newsletters and social media posts and updated weekly starting in April 2020 by the lead author (search terms: ‘COVID-19’ and ‘violence’).12 On 5 November 2021, there were 279 studies in the tracker representing a universe of 3250 hits on google scholar. Titles and abstracts were screened by the lead author and all studies including analysis of VAW/VAC measures during COVID-19 were incorporated in the tracker, including physical, sexual and emotional violence and proxy measures.
Selection process and inclusion criteria
From the tracker, we selected all peer-reviewed studies where primary data collection methods were used to collect data on VAW and/or VAC, including studies which collected data on their co-occurrence. The following types of studies were excluded: (1) those in non-English languages, (2) published in grey literature, (3) analysis of administrative or social media data, (4) modelling studies using prepandemic data, (5) studies analysing proxy measures of violence (eg, conflict, attitudes and perceptions of violence risk) and (6) data from services providers or healthcare workers. online supplemental figure A1 provides additional detail on the sample selection.
Supplemental material
Development of criteria for reporting violence research
We developed a checklist for the ethical reporting of violence research drawing on best practice guidelines for implementation of safe data collection for VAW/VAC established prior to the pandemic.13–16 In addition, as the pandemic increased use of remote data collection methods and challenges in accessing service provision, existing guidelines were augmented by key publications outlining best practices for VAW/VAC research during the pandemic.6 7 Finally, a review of literature was undertaken to explore any studies summarising or proposing guidelines for ethical reporting of interpersonal violence prepandemic, as to build on or complement existing reporting guidelines.11 17 18
We developed a 14-item checklist of best practices for reporting violence research grouped into four domains: (1) Institutional Review Board (IRB) approval, (2) interviewer selection, training and support, (3) sampling and engaging with respondents and (4) referrals and adverse events (table 1). Recognising that guidelines for the ethical reporting of violence research do not currently exist, checklist items were defined to give studies maximum flexibility for a ‘yes’ coding. For example, for item one regarding IRB approval, a ‘yes’ coding was given regardless of where the IRB was located, or the quality of the IRB assessment. For item two regarding appropriate interviewer selection, any relevant selection criteria was accepted with justification (eg, prior experience with sensitive topics, sex of interviewer, etc), rather than imposing prespecified criteria which might differ by setting, survey objectives or target population. For several items, not all studies qualified to be assessed and these were coded as ‘not applicable’. For example, interviewer selection, training and support items were not applicable for studies that exclusively collected self-administered web-based studies and items 7, 13 and 14, were only relevant to studies focused on collecting VAC data, either from minors or from other adults.
Domain and item definition for ethical reporting
Data extraction and analysis
The lead author extracted the background characteristics of each study, including the country of data collection, methodology, mode of data collection and violence measures collected, which was cross-checked by individual reviewers (online supplemental table A1). The 14-item checklist was then applied to each study, drawing on information in the main article or online supplemental material. To ensure consistency in coding, four reviewers (AP, AB, SM and RQHL) first used the checklist to score five studies independently and discussed concordance of answers. Subsequently, each study was randomly assigned to two reviewers and scored independently. Considering all studies and all items, the total percentage of discordant results after the first round of scoring was low (4%). Discrepancies were subsequently discussed and resolved, when required, by a third reviewer.
Scores for each checklist item were descriptively summarised overall and by study characteristic (eg, methodology, violence type, etc). Scores only include studies which are relevant by item or characteristic. For example, for items related to collecting data on VAC, the denominator is all applicable studies with data collection on VAC and/or among minors. In addition, a summary measure was created by averaging the proportion of items reported on (coded as ‘yes’), among the total applicable number of items (all items coded as ‘not appliable’ were not included in this score). There is no missing data for this analysis, as may be present in traditional reviews, as if studies did not report on a particular ethics item that was applicable in their study, they were coded as ‘No’. We report checklist items and summary overall, by methodology, violence and reporting type, and by mode of data. Note that in some cases, a study can fall into more than one category, thus appear for both face-to-face and web-based data collection if a combination of the two approaches were used. We do not assess risk of bias, as this review assesses ethics reporting, which is related to rigour of methodology, but is not focused on exposure outcome relationships. All descriptive analysis was conducted in Stata V.15.19 This study is exempt from ethical approval, as it uses data fully in the public domain and does not use data on human subjects. All stages of the review were documented, but a protocol was not prepared or registered. While there are no standardised reporting guidelines for rapid reviews, we report on best-practice Preferred Reporting Items for Systematic reviews and Meta-Analyses in online supplemental table A5.20
Results
Studies included
Table 2 describes the adherence to each checklist item among all 75 eligible studies. The first column under each category (n) shows the total number of eligible studies for which the checklist item is applicable (the denominator from which the score is calculated), while the second column under each category (%) reflects the percentage meeting (scoring ‘Yes’) to each checklist item, among those applicable. Most studies collected quantitative data (88%, n=66), in comparison to qualitative data (17%, n=13). The sample was similarly heavily skewed towards collection of VAW data (88%, n=67) and self-reported experience measures (75%, n=64), as compared with VAC data (17%, n=13) or proxy reports (eg, reporting by household members of violence experienced by children in the same household) (21%, n=16). Web-based methods were the most frequently used (65%, n=49), followed by telephone (21%, n=16) and face-to-face data collection (20%, n=15). The majority of publications were published in public health journals (55%, n=41), while a smaller percentage was in medical journals and other social science journals (23%, n=17 for both disciplines). Data collection occurred in the following regions: South Asia (n=15), sub-Saharan Africa (n=13), Middle East and North Africa (n=13), Europe (n=13), North America (n=12), Asia-Pacific (n=5), Latin America and the Caribbean (n=3) and global (cross regional, n=1). Recall that if studies collected more than one type of data, using multiple methodologies or in multiple settings, the study appears in multiple categories.
Descriptive statistics for ethical items by study characteristic
Ethical reporting
Results show adherence to best practices was reported on average for 31% of scored items across the 75 studies. Reporting was highest for: ethical clearance (87%) and informed consent/assent (84%/83%, assent scored for six eligible studies). Reporting was lowest for facilitating referrals for minors (0%, scored for six eligible studies), soliciting participant feedback (0%), measures to promote interviewer safety and support (3%, scored for 30 eligible studies), safe sampling designs (5%), implementation of adverse event protocols and if mandatory reporting for violence against minors was considered (both at 8%, the latter scored for 13 eligible studies). Other items were scored as follows: 33% of studies noted how interviewers were selected to support participant safety (scored for 30 eligible studies), 31% of studies report if incentives were given for participation in the study, 25% of studies report giving some type of violence referral information, 21% report any measure taken to support participant safety and privacy during the interview and 13% report specialised enumerator training on violence topics (the latter scored for 30 eligible studies). Findings suggest little overall variation on the proportion of items reported on by study methodology, type of violence and type of reporting (questions about self experience of violence vs proxy reporting)—however, there is some divergence by modality of data collection. In particular, studies using face-to-face data collection appeared to report fewer items (22% of items), while telephone-based surveys report higher adherence to ethics (35% of items). Finally, we examine ethics reporting by discipline of the journal where studies were published, finding little variation across public health, medical and other social science journals (online supplemental table A2). Tables with study-specific results by item are provided in online supplemental table A3.
Examples of best practice reporting by domain and item from the highest scoring papers in online supplemental table A4.21–28 For example, regarding interviewer selection and training, a study undertaken in Bolivia interviewing adolescents reported that ‘enumerators were training in each case by an expert on Child Safeguarding Policy, following stringent ethical guidelines on how to ask questions’, which included measures to verify privacy and use of same-sex enumerators.23 With respect to sampling and participant engagement, a study asking about violence online in Australia offered seven considerations of how participants were approached, including how ‘the survey was designed with multiple landing pages and eligibility questions (including a ‘safety trap’) to screen out ineligible participants (eg, men) from accessing the survey’ to promote participant safety.21 Finally, with respect to referrals and adverse events, a study in Ethiopia noted how women who were in need of urgent help or who had experienced severe intimate partner violence were accompanied to a local referral hospital to access counselling care units.28 Likewise, a study in India among survivors of violence noted that as per government guidelines, follow-up measures were taken by counsellors via phone to call each woman to understand their situation and offer support.27 While the variety of actions reported is diverse, these cases can serve as examples of what and how to report ethically on VAW/VAC data collection.
Discussion
Our results indicate insufficient reporting on ethics of VAW/VAC research across disciplines. Given the number of studies that fail to report checklist items, findings raise important questions about the application of existing global guidance in violence research, the limited guidance issued by IRBs and the seeming lack of criteria used and enforced by journals. Although our study includes research conducted up to November 2021 of the COVID-19 pandemic, poor reporting on ethical practices predates the COVID-19 pandemic. The limited reporting of research ethics we document is illustrative of a larger and more systemic limitation in the field of violence research. For example, a review of studies on childhood sexual abuse in India in 2018 found that only 2/3 of the 51 included studies reported approval by an ethics committee, obtaining informed consent and ensuring confidentiality for participants. Engagement with safeguarding of participants was also poor, with only 25% assessing further risk of sexual abuse and providing services, and no studies describing whether they adhered to the mandatory reporting requirement in India.11 In addition, a review of methodology and ethics in 21 studies including gender-based violence outcomes using remote data collection methods (focused on humanitarian and fragile settings) showed only four studies reported offering referral services and only five studies reported any other safety-related measures.29 Qualitative studies of study interviewers show that they often bear the psychological burden/experience secondary trauma if robust procedures to ensure both their own, and participant, safety are not in place.30–32
This lack of documentation on adherence to ethical guidance for VAW/VAC research raises serious concerns about the possibility of harm to research participants and interviewers, the quality of data and the standards of acceptability and accountability within our field. We contend that limited attention to ethics affects both participants who disclose violence and researchers who receive these disclosures, what happens when these disclosures are received, as well as the comfort participants have disclosing in the first instance. Limited ethical reporting in peer-reviewed literature also makes it challenging for violence researchers to learn from each other and for early career researchers to learn approaches to ethical data collection and reporting.
We acknowledge it is possible that both journal editors and ethics committees themselves were affected by COVID-19. For example, a study of Italian ethics committees found that the workload of committees in highly affected areas of the country increased substantially during COVID-19. This, coupled with a decrease in the ability of committee members to work, led some participants to report that ‘it was impossible to perform an accurate analysis of the submitted documentation’.33 The reprogramming of research to use remote methods required ethics committees and other research stakeholders to rapidly make decisions about new methodologies without centralised guidance. Deviations from established ethical protocols are not unprecedented, and have been deemed acceptable in some circumstances in the context of rapidly evolving humanitarian and emergency situations.34 However, a review of studies more generally with human participants during COVID-19, not specific to violence, found that even more basic ethical reporting has been insufficient—finding up to 24% of observational studies did not report approval by an ethics committee, and up to 38% did not report informed consent from participants.35 Our findings suggest that violence research during the pandemic faces similar shortcomings.
Case studies and learning from practice can help ensure ethical guidance is relevant, complete, logistically feasible and appropriate for new modalities and contexts of data collection. For example, a study reflecting on practical lessons from eight studies collecting data on VAW/VAC during COVID-19 in Brazil, Britain, Kenya, Nepal, Uganda and Zimbabwe suggests that several factors were critical in successfully redesigning studies.9 First, strong existing research partnerships were essential, with teams who were experienced in collecting sensitive data and had existing contact and rapport with participants and local referral structures. Second, it was necessary to adapt data collection strategies, with most studies pivoting to remote modalities and modifying consent and privacy protocols. For example, as part of the Maisha Fiti study in Kenya, interviewing female sex workers, the study team made an initial phone call to participants to assess privacy and safety, setting a time and day for a future interview when conditions were optimal for the interview. Third, additional safeguarding processes were necessary in the context of remote data collection. For example, in the Contexts of Violence in Adolescence Cohort (CoVAC) study in Uganda, the team hired a counselling team to coordinate referrals and revised the referral directory—recontacting all referral services to assess if they were still functional during the pandemic and their ability to act on cases, including options to engage in phone counselling and remote service provision.36 The challenges of ensuring access to quality referral services, particularly for children, are not unique to the pandemic context, however are an additional investment study that teams must consider as they plan for data collection.37 Finally, teams facilitated remote support for interviewers. These types of reflection and documentation of strategies in different contexts can help future researchers understand options and assess trade-offs in the ethical collection of violence data.
Our study has limitations. First, although we aimed to be comprehensive, it is possible that we missed studies published during the search period. Second, we only scored whether studies mentioned the presence of a particular criterion, rather than on the quality of their adherence to it, or the level of detail provided. Third, we do not exclude the possibility that studies employed good ethical practices in data collection, without reporting this explicitly in the resulting publication. Due to the diversity of possible contexts and target groups, we did not explicitly score all potential considerations for special populations which may require additional considerations, including attention to legality around diverse types of violence (eg, undocumented migrants, trafficked persons, prisoners and pregnant women). Finally, there are other generalised ethical aspects not scored here which are also relevant. These include, among others: general data protection protocols (particularly with technology-facilitated data collection via Apps or interactive voice recall), assessment of whether results are actionable and useful to communities, policy equipoise (for intervention studies), an emphasis on equity and inclusion in sampling, positionality of researchers and whether community members and survivors were included in the research design and in study steering committees, and fair, safe, adequate working conditions for data collection staff.38–40 We choose not to score these criteria, as many of these aspects fall outside the timelines of journal articles or are less likely to be documented in publications. However, these additional criteria as well as the quality or content of the criteria we propose could be further evaluated or assessed.
Conclusions
Poor reporting of ethical practices in violence research is widespread. In VAW/VAC research, there is a clear risk of harm to participants if guidance is not followed as well as an impact on the quality of the data produced. Our findings point to the importance of the development and use of reporting guidelines for research on VAW/VAC. Based on our work, the domains and checklist items outlined in table 1 provide a starting point for such guidance. For violence researchers, the checklist does not substitute for following recommended ethical guidelines, however can providing strategies that can be incorporated into the design, implementation and reporting of research studies. Both ethics committees and journal editors can assess violence research against reporting guidance, similar to the Consolidated Standards of Reporting Trials or Strengthening the Reporting of Observational Studies in Epidemiology guidance for reporting of trials and observational studies, respectively.41 42 Additionally, funders could use the checklist to assess research proposals for violence research to ensure mechanisms for safety referrals and feedback are integrated into the study from its design. Finally, the checklist could be integrated into efforts to build capacity, particularly in the context of training students, researchers and data collection teams globally. Efforts to improve the reporting of VAW/VAC research are an important step to improve the quality and safety of violence research and fulfil the commitments to listen to and learn from participants.40 As methodologies for collecting and analysing data evolve, we should continue to promote production of actionable evidence to improve understanding and practice surrounding prevention of VAW/VAC, as well as commitment to a do no harm approach.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
Not required.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Seye Abimbola
Twitter @a_peterman, @Alesscguedes, @JohtChandan, @amiyabhatia
Contributors Conceptualisation (AP, KD, AG, JSC and AB), data curation (AP, SM, RQHL and AB), formal analysis (AP), methodology (AP, KD, AG, JSC and AB), writing—original draft (AP, KD, JSC, FG and AB), writing—review and editing (all authors). AP, SM, RQHL and AB have accessed and verified the data in the manuscript. AP is responsible for the overall content as guarantor and accepts full responsibility for the finished work and/or the conduct of the study, had access to the data, and controlled the decision to publish. All authors approved the final version.
Funding The COVAC study is funded by the UK Medical Research Council (MRC), grant number MR/R002827/1, which supported the time of AB and KD. The funder did not have a role in analysis, interpretation or manuscript preparation.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Author note The views expressed in this publication are those of the authors(s) and not necessarily those of the affiliated institutions or funding agencies.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.