Article Text

Download PDFPDF

Patchy signals: capturing women’s voices in mobile phone surveys of rural India
  1. Skye Hersh,
  2. Divya Nair,
  3. Pradyot Bharadwaj Komaragiri,
  4. Raghav Kapoor Adlakha
  1. ID Insight Private Limited, New Delhi, Delhi, India
  1. Correspondence to Dr Divya Nair; divya.nair{at}idinsight.org

Abstract

Phone surveys are a rapid and cost-effective way to collect primary data for research, monitoring and evaluation purposes. But for these data to be precise, reliable and unbiased, women’s perspectives must be accurately represented. Throughout 2020, we conducted seven household surveys in rural India to understand households’ experiences of the COVID-19 pandemic and contemporaneous relief programmes. Given social distancing protocols, we conducted these surveys over the phone, using household phone numbers collected during earlier, face-to-face research. Analysing metadata from these surveys (along with women’s responses to questions about phone use), we determine how gaps in phone access inhibit women’s representation in phone surveys. We find that the prevalence of male management of household phones significantly reduces access to female respondents. This is a problem for two reasons. Firstly, men are usually the first to pick up the phone: in two surveys in which we tracked the gender of the person who picked up, men picked up 63.2% and 71.1% of the time, respectively. Moreover, only a small minority of those we reached by phone were able and willing to pass the phone to a household member of the opposite gender, when prompted (with no statistically significant difference between pass rates for women and men). This low immediate pass rate, in combination with low female pickup, led to fewer women respondents. As such, we recommend that researchers dedicate time and resources to taking appointments and making call-backs to reach more women. We also show that the use of female enumerators improves households’ willingness to participate in women-centred surveys, and call for more investment into female enumerator teams.

  • study design
  • medical demography
  • health services research

Data availability statement

Data for this article are not public.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Summary box

  • Phone surveys are a rapid, cost-effective and flexible way to collect primary data, but given the mobile divide between men and women in India, they are prone to gender bias.

  • Our metadata from seven household surveys in rural India show that the high prevalence of male mobile gatekeeping significantly reduces access to female respondents.

  • Those we reached by phone were able or willing to pass the phone to a household member of the opposite gender (when prompted) only between 7.5% and 11.1% of the time, across surveys—and because men are first to pick up the phone the majority of the time, this low pass rate means fewer women respondents.

  • When conducting phone surveys, we suggest that researchers schedule extra time to reach women through appointments and call-backs, or else report potential gender bias in their results.

  • We also find that the use of female enumerators increases households’ consent-to-survey rates for women-centred surveys, and recommend greater investment into recruiting and training female enumerator teams.

Introduction

Phone surveys have long been viewed as a low-cost, rapid alternative to face-to-face surveys for capturing data on population health status and care-seeking practices.1–3 The COVID-19 pandemic has led to the cessation of face-to-face surveys in India, and as result, phone surveys have become the preferred modality for capturing data for policy-relevant research. However, because women in rural regions, especially in South Asia, have less access to mobile phones than do men,4 5 the use of phone surveys may exacerbate the global gender data gap—the persistent absence of official statistics about women’s and girls’ lives. Without quality, gender-disaggregated data on key outcomes, policy-makers and researchers are left with little insight into the needs, experiences and progress of half the world’s population.6–8

From April to November 2020, we at IDinsight (a global advisory, data analytics, and research firm that helps global development leaders maximise their social impact) conducted seven surveys on livelihoods, nutrition and gender-related topics among rural households (HHs) in nine states across India to understand the effects of the COVID-19 crisis on rural populations (table 1). The sampling frames were created from rosters of HHs that we had visited in person between 2018 and early 2020, before the pandemic began in India. The COVID-19 phone surveys sought to examine (1) men and women’s knowledge, attitudes and practices (KAP) regarding the COVID-19 pandemic; (2) women’s digital access and digital literacy and (3) pregnant and lactating women’s access to nutrition programmes. In the interest of producing gender-responsive policy insight, we actively sought to collect data from representative samples of both men and womeni which allowed us to gender-disaggregate responses to generic questions. We also designed survey modules to gather data on female-specific experiences.

Table 1

Overview of phone surveys conducted by authors between May and November 2020

We argue that efforts are needed to quantify and address the difficulties of reaching women during phone surveys. We start by highlighting structural and normative barriers to women’s mobile access, and consider the implications of barriers on response rates. We next outline two strategies for improving response rates among women: (1) scheduling call-back appointments with male heads-of-HH to reach their female counterparts and (2) using female enumerator teams to improve the consent-to-survey rate when seeking female respondents. We close by outlining recommendations for improving survey data quality and generalisability.

Barriers to reaching women by phone in India

Indian women face barriers to phone ownership and access

India has one of the highest gender gaps in mobile ownership in the world.5 9 Various national surveys report different estimates for women’s mobile inclusion, given their use of slightly different indicators. The GSM Association, a mobile industry interest group, reports that while 79% of men in India own mobile phones, only 63% of women do (where an ‘owner’ is a sole or main monthly user of a subscriber identification module, or SIM, card).9 According to Phase I of the latest National Family Health Survey (NFHS-5), 91.2%, 91.9% and 93.4% of HHs in Andhra Pradesh, Assam and Bihar, respectively, report mobile phone ownership (‘Does your household have a mobile phone?’).10 But within these HHs, only 48.9%, 57.2% and 51.4% of women give an affirmative answer to the question, ‘Do you have a mobile phone that you yourself use?’10 (NHFS-5 estimates for the other states we studied are not available at the time of this publication.)

Rather than own their own mobile, women often request to use the common ‘HH phone’ (which is typically managed by the male head-of-HH), or borrow a phone from community contacts.11 12 Nonetheless, borrowed access is more restrictive than direct access, as women are unlikely to use a borrowed phone as freely or extensively as they would their own.5 When women do own devices, their devices are less likely than men’s to be charged or topped-up with mobile credit.13

As India’s mobile ownership gap is greater than that of other regions with similar income per capita and mobile ownership costs, structural and economic barriers alone cannot explain the divide.5 Uniquely restrictive patriarchal and gender norms play a substantial role. Many Indian HHs fear that mobile usage can raise community concerns about women and girls’ ‘purity’, as well as expose women to digital harassment or corrupting influences.5 Moreover, because women in India are often expected to fulfil homemaker or caretaker roles, they cannot claim to require phone access for socially acceptable, work-related reasons (as their male counterparts do).5 Women are more likely to use mobile phones inside the HH, and only freely if they are alone.12 14

Women’s frequency of phone use is more limited than men’s

We surveyed women about their digital and mobile use during the third round of our COVID-19 KAP survey. The testimony they provided corroborates the literature. Women report daily phone use in under half (46.1%) of HHs we surveyed, whereas nearly three-quarters (73.1%) of men in the same HHs report daily phone use (figure 1A). About one-third of women reported that they only use the phone on an emergency basis.

Figure 1

Reported phone usage rates. Source: COVID-19 KAP survey, November 2020. Observations in top graph limited to households in which we reached both male and female respondent for interview. Observations in bottom graph include households in which we reached both male and female respondent, as well as 241 dual-headed households in which we only reached only female respondent. Error bars denote 95% CI.

Furthermore, this gendered discrepancy is not limited to HHs in which there is only one phone for all members. Even in HHs with more than one phone, women report limited phone use (figure 1B). In HHs that possess two phones, over half of women still report a less-than-daily phone usage rate—and just about half of women in HHs that possess three or more phones report the same. These findings imply that even when HH phone ownership increases overall, the primary beneficiaries are male HH members.

Men pick up the phone first in most instances

During two surveys, enumerators explicitly noted the gender of the HH member who first picked up the phone; figure 2 shows this breakdown. In these surveys, a male was the first respondent to pick up the phone 63% and 71% of the time, respectively. This is likely due to men’s tendency to carry and act as the primary managers of the HH phone(s).

Figure 2

Gender of respondent who first answered the phone. Surveys featured in figure are those for which gender of respondent who first picked up was recorded. Error bars denote 95% CI.

Those who first pick up are often unable or unwilling to pass the phone to a HH member of the target gender during the same call

Our experience shows that it is generally difficult to get a successful phone pass from the person who picked up to a requested respondent during a given call—regardless of the former party’s gender. Both men and women respondents, when prompted, very seldom successfully put their opposite-gender cohabitants on the line (figure 3). This may be because, during the daytime (when surveys occur), most respondents are not co-located in the same vicinity as their opposite-gender cohabitants. But because men so often carry and act as the primary managers of the HH phone(s), these low overall phone passing rates further compound the challenge of reaching women.

Figure 3

Frequency of successful phone pass to target gender at time of initial call, by gender of household member who answered phone. Surveys featured are those for which gender of person who first picked up was recorded. Observations limited to cases in which respondent who picked up was not target gender. Error bars denote 95% CI.

Strategies for improving women’s response rates

Scheduling time for appointments and call-backs helps to reach women

Active appointment scheduling helps to mitigate the problem of low phone-passing rates. In the third round of our COVID-19 KAP Survey, we proactively sought to interview both a man and a woman from each dual-headed HH that we contacted. Typically, we reached a man at first, and after completing the male head-of-HH modules with him, we asked to speak to a woman in his home. If she was not available, we made an appointment—making an average of 2.6 total calls for all HHs (but calling each one up seven total times). As demonstrated in table 1, our female representation in this particular survey (42.6%) was higher than it was for surveys in which we targeted women but did not make appointments if we failed to reach them during our first HH contact (21.2%–37.6%). This strategy also allowed us to complete or partially complete surveys with 1169 pregnant or lactating women (PLWs) in the POSHAN Abhiyaan Survey, for which we also made up to six additional calls to HHs we initially reached in order to get a PLW on the line.

Female phone enumerators improve consent-to-survey rates for gender-related surveys

Beyond the problem of male mobile phone gatekeeping, social norms and women’s hesitancy to discuss certain topics with enumerators may deepen the issue of overall low female response rates. It can be a challenge to find participants willing to discuss their experiences around reproductive health, childbirth, breast feeding and other potentially personal subjects candidly and accurately—and it can be more challenging to find respondents willing to do so over the phone, given the difficulty of building trust and rapport without face-to-face interaction.

Our experience suggests that a female survey team may be able to help overcome these challenges, boosting HHs’ consent-to-survey rate during surveys with gender-sensitive subject matter. For our POSHAN Abhiyaan Survey in May 2020, in which we targeted exclusively PLW respondents, we had a mixed male/female enumerator team. We found that the HH refusal rate (ie, non-consent rate) was twice as high (12.2%) for male enumerators as it was for female enumerators (6.1%) (figure 4).ii

Figure 4

Households’ refusal to participate in female-focused survey, by gender of surveyor. Source: POSHAN Abhiyaan survey, May 2020. Observations include calls that resulted in completed or half-completed surveys, appointment to call back and refusals. To count as a ‘refusal’, either a man who picked up the phone would have refused to pass the phone to a pregnant and lactating women (PLW) in his household, or the PLW herself would have refused to participate. Error bars denote 95% CI.

Notable national surveys in India, such as the NFHS and the India Human Development Survey, already use female enumerator teams to conduct gender-sensitive survey modules for their in-person surveys.15 16 Our finding affirms the importance of this strategy, and justifies even greater investment into female survey teams to improve the rate of women’s inclusion in phone-based surveys.

However, our experience suggests that the organisation of female survey teams may come with operational challenges. For the third round of the COVID-19 KAP survey, we hired an all-female survey team, given the sensitivity of some of the topics, such as access to menstrual hygiene products. While we were ultimately able to hire several hundred female enumerators, pre-existing gender imbalances in the surveying industry in India meant that applicants to our all-female team were, on average, less experienced and less qualified than prior pools of enumerator applicants (which were primarily male). Once we managed to hire the all-female survey team, we had to invest more time and resources into developing training materials. With time, further investment into the recruitment of female enumerators may reduce the upfront costs of training them.

Additional considerations for improving survey quality

Design survey content with a gender lens

During the third round of the COVID-19 KAP survey, we aimed to collect evidence of the gender-specific impact of the pandemic. To do so, we examined the differential impacts of the pandemic on access to entitlements, healthcare, unpaid work, and food insecurity on men and women within the same HH. This model allowed us to hold HH factors constant, while examining differences across genders. We found that some responses (such as preferred source of pandemic-related news and perception of COVID-related risks) were correlated between men and women of the same households. Other responses varied by gender (such as reasons for increases to and type of unpaid work undertaken during the pandemic), even within the same household.

Triangulate data from both men and women to generate a more accurate picture of HH behaviour

Speaking to both a man and woman in the same HH can also help to reduce gaps and inconsistencies in the household story. There is evidence that male respondents reporting labour participation on behalf of other members of their HH tend to underestimate women’s work.17 Women may be better poised than men to report other HH members’ work statuses accurately.

On the other hand, our own surveys found that women were more likely than men to state that they did not know the answer to agriculture-related questions (particularly around fertiliser use and expenditure, agricultural borrowings in the past year and acreage). We maintain that surveys that reach respondents of both sexes in a given HH will achieve the highest overall accuracy across topics.

Define survey content with beneficiary safety and security in mind

In the context of the pandemic, we considered collecting data on violence against women, given the increase in reports of domestic violence during lockdown.18 However, the challenges associated with studying this topic by phone hindered our plans to proceed. In the context of lockdown, any abusers were likely to be in close proximity to respondents at the time of a call. By phone, our enumerators would be unable to assess the respondent’s physical context or body language for a read of the HH dynamic. Additionally, our enumerators reported that family members often instructed women to answer questions on speakerphone. Other researchers who conducted phone surveys during the first lockdown in India found that only about a third of respondents (~35%) could find a private space in which to answer questions on HH conflict.13 Without sufficient privacy, respondents probably cannot relate their true experience with such topics.5

For these reasons, we did not collect data on interpersonal violence. However, we urge other researchers to develop and pilot phone survey or training protocols to address the challenges outlined above, given the importance of valid statistics on interpersonal violence to support evidence-based policies and advocacy.19

Use weighting to reduce non-response bias

Our phone surveys featured significantly higher non-response rates than in-person surveys conducted with similar populations. An in-person survey that we conducted in November 2019 had a non-response rate of ~6%. But in November 2020, when we tried to reach respondents from the same sampling frame for our COVID-19 KAP survey, over 57% of our sample did not respond to our phone calls. Our analysis (conducted using HH data from the earlier, in-person survey) suggests that these non-respondents were, on average, poorer than our respondents.

In line with best practices,20 we constructed non-response weights with HH Poverty Probability Index scores to compensate for the relative absence of poorer respondents among our final respondents. However, HHs that do not own phones may differ from those that do along unobservable dimensions—dimensions for which we cannot account. Researchers may consider providing phones to HHs during in-person survey rounds by which to reach them later, or contacting neighbours with phones to reach the targeted HH.

Conclusion

We conducted seven HH phone surveys over 2020 in rural India. In this article, we explored challenges in reaching women and getting them to voice their opinions and experiences.

We find that male gatekeeping of HH phones inhibits the rate at which phone enumerators can reach female respondents. Because men control HH phones, they are often the first to pick up—and because they do not necessarily co-locate with their female HH members throughout the day, we observed a lower success rate getting women on the phone. In some cases, men’s policing of women’s communication with strangers may further complicate attempts to reach women; more research on this possibility is needed.

Despite these limitations, we do find that it is possible to reach sufficient samples of women respondents by phone, though doing so may require repeated calls to HHs. Researchers should prepare to make appointments with male respondents to reach their female counterparts at later times. Furthermore, our data suggest that, at least for women-centric surveys, the use of female enumerators increases HHs’ consent-to-survey rates. As such, we call for greater investment into female enumerator teams, which are currently not in common use across India’s monitoring and evaluation landscape.

Data availability statement

Data for this article are not public.

Ethics statements

Patient consent for publication

Acknowledgments

We extend our thanks to all respondents—both men and women—who provided their time and input to help us think through these topics. We warmly acknowledge Dr. Amnesty LeFevre, Dr. Suneeta Krishnan, Dr. William Thompson, Dr. Valentina Brailovskaya, Shriya Agarwal, Mitali Mathur, Rohan Raj, and Emily Coppel for providing their support during the preparation of this article. We also thank IDinsight's Data on Demand survey team, which enabled our data collection.

References

Footnotes

  • Handling editor Seye Abimbola

  • Contributors DN planned the study. SH conducted analysis and prepared the final manuscript. RKA oversaw original data collection. PBK prepared an earlier version of the manuscript and submitted the study.

  • Funding This project was financed by the Bill & Melinda Gates Foundation, the World Bank, and GiveWell.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • We always asked respondents to self-identify their gender. As over 99% of respondents’ self-identifications adhered to the gender binary, we report only those results, noting that these surveys do not represent gender-diverse identities.

  • ‘Refusing’ HHs were those in which either (1) a man answered the phone and, after hearing our enumerator explain the purpose of the call (ie, to speak to a PLW about her health needs and practices), refused to pass the phone to a PLW in the household or (2) a PLW, successfully brought onto the phone, refused to participate in the survey.