A comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups

https://doi.org/10.1016/0030-5073(73)90052-4Get rights and content

Abstract

This research evaluated four methods of eliciting subjective likelihood ratio estimates. The methods differed in terms of amount and structure of interaction permitted between estimators. These processes were individual estimates, and three group processes: a Talk-Estimate process approximating an interacting group, an Estimate-Feedback-Estimate process as an approximation of a Delphi group, an Estimate-Talk-Estimate process as combination of nominal and interacting groups.

In this study the Estimate-Talk-Estimate group process was superior in approaching correct estimates in this judgmental task. This is consistent with the long research tradition which favors group as opposed to individual problem-solving in judgmental situations.

The individual Estimate process and the Estimate-Feedback-Estimate technique performed about equally well with respect to both error and variability. If anything, written feedback appeared to lead to a reduction in the quality of estimates.

Finally, the relatively poor results from the Talk-Estimate process are consistent with other studies which have pointed out dysfunctions of interacting group processes for judgmental tasks.

References (30)

  • J.P. Campbell

    Individual versus group problem solving in an industrial sample

    Journal of Applied Psychology

    (1968)
    J.P. Campbell

    Individual versus group problem solving in an industrial sample

    Journal of Applied Psychology

    (1968)
  • L.R. Hoffman

    Group problem solving

  • A.F. Osborn

    Applied imagination

    (1957)
  • M. Turoff

    The design of a policy Delphi

    Technological Forecasting and Social Change

    (1970)
  • D. Bem et al.

    Group decision making under risk of aversive consequences

    Journal of Personality and Social Psychology

    (1965)
  • P.A. Collaros et al.

    Effect of perceived expertness upon creativity of members of brainstorming groups

    Journal of Applied Psychology

    (1969)
    P.A. Collaros et al.

    Effect of perceived expertness upon creativity of members of brainstorming groups

    Journal of Applied Psychology

    (1969)
  • N.C. Dalkey et al.

    An experimental application of the Delphi method to the use of experts

    Management Science

    (1963)
  • J.F. Dashiell

    Experimental studies of the influence of social situations on the behavior of individual human adults

  • Daves, O. L. Design and analysis of industrial experiments. New York: Hafner Publishing...
  • A.L. Delbecq et al.

    A group process model for problem identification and program planning

    Journal of Applied Behavioral Science

    (1971)
  • M.D. Dunnette

    Are meetings any good for solving problems?

  • J. Dunnette et al.

    The effect of group participation on brainstorming effectiveness for two industrial samples

    Journal of Applied Psychology

    (1963)
    J. Dunnette et al.

    The effect of group participation on brainstorming effectiveness for two industrial samples

    Journal of Applied Psychology

    (1963)
  • W. Edwards et al.

    Probabilistic information processing systems: design and evaluation

    IEEE Transactions: Systems and Cybernetics

    (1968)
  • B.C. Goodman

    Risky decisions by individuals and groups

  • E.J. Hall et al.

    Group problem-solving effectiveness under conditions of pooling vs. interaction

    Journal of Social Psychology

    (1963)
  • Cited by (250)

    View all citing articles on Scopus

    The study is part of the Computer-Aided Medical Diagnosis Project at the University of Wisconsin. The purpose of the project is to develop and evaluate a computer-aided diagnostic system using subjective likelihood ratios estimated by physicians (rather than actuarial likelihoods estimated by empirical data) in a Bayesian diagnostic model. This particular study was important to the project because it became the basis for selecting a group process that would effectively aggregate the knowledge of the physician's community. Much concern has recently been evidenced concerning the role of consensus and subjective information (Winkler, 1968; Goodman, 1970).

    View full text