Telephone versus in-person interviews for alcohol use: results of the 2000 National Alcohol Survey☆
Introduction
Over the years, researchers who conduct large-scale surveys have debated the financial costs and data quality issues related to the way data are collected. Those who have looked to the research literature for guidance are met with few studies that have qualified findings, and results show little consistency. Thus, the decision to change the way in which data are collected is often dictated by two critical issues, cost and sample size, each having a substantial impact on how a study is conducted and the results that it yields.
In health surveys, telephone interviews have been found to provide comparable results when compared with in-person interviews (Biemer, 2001, Cannell et al., 1981, Cannell et al., 1987). However, in the alcohol field, results of studies on modes of data collection have provided very little direction for researchers. For alcohol use, findings have ranged from lower reports obtained on the telephone versus face-to-face interviews (Aquilino, 1992, Aquilino, 1994) to minimal or no differences found between modes (Aquilino, 1996; Aquilino and Wright, 1996, de Leeuw and van der Zouwen, 1988). Yet, Hochstim (1962) and Sykes and Collins (1988) found higher rates of alcohol use using the telephone as compared to in-person interviewing. Unfortunately, most of these studies used fairly crude drinking measures and did not examine the patterning of alcohol use or heavier episodic drinking. Very little is known about how mode of data collection may affect reports of alcohol-related problems or harm.
Because of the need to keep costs contained yet have a large enough sample to generalize to the national population, a decision was made to shift the mode of data collection of the 2000 National Alcohol Survey (NAS) to telephone from in-person interviews. The NAS is a large-scale survey series undertaken at approximately 5-year intervals by the Alcohol Research Group since 1965. A very important part of the survey is the ability to allow trend analyses by including comparable items that date back to surveys conducted over 40 years ago (Greenfield et al., 2000, Midanik and Greenfield, 2003). Thus, identifying and understanding potential biases in mode of administration is critical for comparative purposes, and there is a concerted attempt to adjust for any biases in comparing prevalence rates of alcohol use and alcohol-related problems in planned trend analyses.
There have been three previous studies that have examined interview mode using the NAS. The first two involved between subjects analyses (Greenfield et al., 2000, Midanik et al., 2001a). These studies assessed prevalence rates of alcohol consumption measures and alcohol-related harm from two national surveys in 1990 with different modes of administration (face-to-face versus telephone interviews). Few differences were found in alcohol consumption prevalence estimates (last 12 months) by interview mode. However, the findings indicated that lower income individuals were less represented in the telephone sample which suggests that weights should be applied to adjust for this discrepancy (Greenfield et al., 2000). However, higher rates of alcohol-related health harm and work harm in the last 12 months were found in the telephone survey as compared to the face-to-face study after controlling for demographic characteristics (gender, age, ethnicity, income, and education) and number of heavy drinking days in the past year (Midanik et al., 2001a).
While the between subjects design assesses one dimension of telephone versus in-person interview modes in which mode may be confounded by sampling design differences (random digit dialing reaches only those with residential telephones), a within subject design addresses whether mode differences influence responses with respondents serving as their own controls. Here the underlying sampling design is held constant. Based on the 1995 NAS, Midanik et al. (2001b) compared prevalence rates of alcohol consumption and alcohol-related harm measures for 1047 respondents who participated in both face-to-face and subsequent telephone followup interviews. They found 82% of their sample had the same drinking status (56.1% current drinkers; 25.9% non-drinkers) at both interviews. Approximately 18% of their sample changed: 14.8% became current drinkers and 3.2% became non-drinkers. No differences were found for mean daily volume or for reports of alcohol-related harm in the last 12 months for current drinkers at both interviews. However, the mean number of days in which respondents reported drinking five or more drinks was significantly higher in the face-to-face interviews as compared to the telephone followup. This was not significant when only those who reported at least one 5+ day were included in the analysis.
Within subject designs used to assess interview mode can be confounded by order effects. Thus, to complement the previous study, this study reversed the order of data collection so that respondents first received a telephone interview followed by an in-person interview. The purpose of this research is to address the following research questions: (1) How does mode of data collection affect drinking status (current drinker versus non-drinker last 12 months)? (2) How does mode of data collection affect reports of alcohol use and alcohol-related problems? By comparing results from both studies, an estimate of the extent of an order effect can be obtained.
Section snippets
Sample selection and recruitment
Data for this study were collected by Temple University’s Institute of Survey Research as part of the larger 2000 National Alcohol Survey data collection effort which used random digit dialing to develop the sample with list-assisted number generation, automatic detection of non-working numbers, and computer matching against yellow pages to increase the hit rate. The in-person subsample was selected using the same methodology that was employed for the NAS sample. Interviewers dialed 5042
Interview mode and drinking status
In general, respondents were consistent in their reporting of their drinking status. Approximately 67% (n=277) of the sample reported being a current drinker at both interviews while 22.9% (n=94) reported not drinking during the last year at both interviews. Only 9.7% (n=40) of the sample changed their drinking status over the 2-month time period. Eleven respondents (2.7%) of the sample reported drinking during the previous 12 months but would be considered non-drinkers based on their in-person
Discussion
As compared to the previous within subject study (Midanik et al., 2001b) which used the reverse-ordered modes of data collection (in-person, then telephone interview), a lower percentage of respondents changed their 12-month drinker status between interviews (9.8% for the current study versus 18.1%) possibly due to the shorter average time between interviews (approximately 2 months versus 4 months); however, findings from both studies indicate that most respondents who changed their drinking
Acknowledgements
This research was funded by an NIAAA Alcohol Research Center Grant #AA-05595 awarded to the Alcohol Research Group, Public Health Institute. The authors wish to thank Yu Yee and Jason Bond for their help with the statistical analysis.
References (22)
- et al.
Towards agreement on ways to measure and report drinking patterns and alcohol-related problems in adult general population surveys: the Skarpo Conference Overview
J. Substance Abuse
(2000) Ways of measuring drinking patterns and the difference they make: experience with graduate frequencies
J. Substance Abuse
(2000)Telephone versus face-to face interviewing for household drug use surveys
Int. J. Addict.
(1992)Interview mode effects in surveys of drug and alcohol use: a field experiment
Public Opin. Q.
(1994)- et al.
Substance use estimates from RDD and area probability samples
Public Opin. Q.
(1996) Nonresponse bias and measurement bias in a comparison of face to face and telephone interviewing
J. Off. Stat.
(2001)- et al.
Alcohol consumption among racial/ethnic minorities
Alcohol Health Res. World
(1998) - Cannell, C., Thornberry, O., Fuchsberg, R., 1981. Research on the Reduction of Response Error: The National Health...
- Cannell, C., Groves, R., Magilavy, L., Mathiowetz, N., Miller, P., 1987. An experimental comparison of telephone and...
- Clark, W.B., Midanik, L., 1982. Alcohol use and alcohol problems among US adults: results of the 1979 National Survey....
Cited by (0)
- ☆
Parts of this paper were presented at the 130th Annual Meeting of the American Public Health Association, Philadelphia, Pennsylvania, 13 November 2002.