The inaccuracy of data from online surveys: A cautionary analysis

Online recruitment methods for survey-based studies have become increasingly common in social science research. However, they are susceptible to a high rate of participation by fraudulent research subjects. The current study identified fraudulent (i.e., “fake”) participants in an online research study of parents of 13 to 18-year-old adolescents, and compared demographic, anthropometric, and subjective health data between “fake” (N = 1084) and “real” (N = 197) participants. Of 1,281 subjects who started the eligibility survey, 84.6% were coded as “fake.” “Fake” participants were less diverse in race/ethnicity and more diverse in gender. Their depression symptoms were inflated, but ratings of perceived health were comparable to “real” participants. Well-established correlations, such as that between BMI and perceived health, were not replicated with “fake” participants. Online surveys are highly vulnerable to fraudulent research subjects whose participation compromises the validity and interpretability of results. The discussion provides a guide and recommendations for improving data quality in online survey research.

This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://dx.doi.org/10.1007/s11135-023-01733-5

Files

Metadata

Work Title The inaccuracy of data from online surveys: A cautionary analysis
Access
Open Access
Creators
  1. Jennifer P. Agans
  2. Serena A. Schade
  3. Steven R. Hanna
  4. Shou Chun Chiang
  5. Kimia Shirzad
  6. Sunhye Bai
Keyword
  1. Survey research
  2. Online research
  3. Validity
  4. Data quality
License In Copyright (Rights Reserved)
Work Type Article
Publisher
  1. Quality and Quantity
Publication Date September 7, 2023
Publisher Identifier (DOI)
  1. https://doi.org/10.1007/s11135-023-01733-5
Deposited March 25, 2024

Versions

Analytics

Collections

This resource is currently not in any collection.

Work History

Version 1
published

  • Created
  • Added Scammers_paper_AcceptedManuscript_embargoed_until_8.28.24.pdf
  • Added Creator Jennifer P. Agans
  • Added Creator Serena A. Schade
  • Added Creator Steven R. Hanna
  • Added Creator Shou Chun Chiang
  • Added Creator Kimia Shirzad
  • Added Creator Sunny Bai
  • Published
  • Updated
  • Updated Keyword, Publication Date Show Changes
    Keyword
    • Survey research, Online research, Validity, Data quality
    Publication Date
    • 2023-01-01
    • 2023-09-07
  • Renamed Creator Sunhye Bai Show Changes
    • Sunny Bai
    • Sunhye Bai

Version 2
published

  • Created
  • Deleted Scammers_paper_AcceptedManuscript_embargoed_until_8.28.24.pdf
  • Added Agans et al. - inaccuracy of data - accepted to Q&Q.pdf
  • Published
  • Updated