Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments
Published on Psychonomic Bulletin & Review (Browse Results) | shared via feedly
Abstract
With the increasing sophistication and ubiquity of the Internet, behavioral research is on the cusp of a revolution that will do for population sampling what the computer did for stimulus control and measurement. It remains a common assumption, however, that data from self-selected Web samples must involve a trade-off between participant numbers and data quality. Concerns about data quality are heightened for performance-based cognitive and perceptual measures, particularly those that are timed or that involve complex stimuli. In experiments run with uncompensated, anonymous participants whose motivation for participation is unknown, reduced conscientiousness or lack of focus could produce results that would be difficult to interpret due to decreased overall performance, increased variability of performance, or increased measurement noise. Here, we addressed the question of data quality across a range of cognitive and perceptual tests. For three key performance metrics—mean performance, performance variance, and internal reliability—the results from self-selected Web samples did not differ systematically from those obtained from traditionally recruited and/or lab-tested samples. These findings demonstrate that collecting data from uncompensated, anonymous, unsupervised, self-selected participants need not reduce data quality, even for demanding cognitive and perceptual experiments.
- Content Type Journal Article
- Category Brief Report
- Pages 1-11
- DOI 10.3758/s13423-012-0296-9
- Authors
- Laura Germine, Department of Psychology, Harvard University, 768 William James Hall, 33 Kirkland Street, Cambridge, MA 02138, USA
- Ken Nakayama, Department of Psychology, Harvard University, 770 William James Hall, 33 Kirkland Street, Cambridge, MA 02138, USA
- Bradley C. Duchaine, Department of Psychological and Brain Sciences, Dartmouth College, Moore Hall 6207, Hanover, New Hampshire 03755, USA
- Christopher F. Chabris, Department of Psychology, Union College, 807 Union Street, Schenectady, NY 12308, USA
- Garga Chatterjee, Department of Psychology, Harvard University, 33 Kirkland Street, Cambridge, MA 02138, USA
- Jeremy B. Wilmer, Psychology Department, Wellesley College, 106 Central Street, Wellesley, MA 02481, USA
- Journal Psychonomic Bulletin & Review
- Online ISSN 1531-5320
- Print ISSN 1069-9384
Geen opmerkingen:
Een reactie posten