1. bookVolume 33 (2017): Issue 2 (June 2017)
    Special Issue on Total Survey Error (TSE)
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
access type Open Access

Total Survey Error and Respondent Driven Sampling: Focus on Nonresponse and Measurement Errors in the Recruitment Process and the Network Size Reports and Implications for Inferences

Published Online: 12 Jun 2017
Volume & Issue: Volume 33 (2017) - Issue 2 (June 2017)<br/>Special Issue on Total Survey Error (TSE)
Page range: 335 - 366
Received: 01 Jan 2016
Accepted: 01 Mar 2017
Journal Details
License
Format
Journal
eISSN
2001-7367
First Published
01 Oct 2013
Publication timeframe
4 times per year
Languages
English
Abstract

This study attempted to integrate key assumptions in Respondent-Driven Sampling (RDS) into the Total Survey Error (TSE) perspectives and examine TSE as a new framework for a systematic assessment of RDS errors. Using two publicly available data sets on HIV-at-risk persons, nonresponse error in the RDS recruitment process and measurement error in network size reports were examined. On nonresponse, the ascertained partial nonresponse rate was high, and a substantial proportion of recruitment chains died early. Moreover, nonresponse occurred systematically: recruiters with lower income and higher health risks generated more recruits; and peers of closer relationships were more likely to accept recruitment coupons. This suggests a lack of randomness in the recruitment process, also shown through sizable intra-chain correlation. Self-reported network sizes suggested measurement error, given their wide dispersion and unreasonable reports. This measurement error has further implications for the current RDS estimators, which use network sizes as an adjustment factor on the assumption of a positive relationship between network sizes and selection probabilities in recruitment. The adjustment resulted in nontrivial unequal weighting effects and changed estimates in directions that were difficult to explain and, at times, illogical. Moreover, recruiters’ network size played no role in actual recruitment. TSE may serve as a tool for evaluating errors in RDS, which further informs study design decisions and inference approaches.

Keywords

Baker, R., J.M. Brick, N.A. Bates, M. Battaglia, M.P. Couper, J.A. Dever, K.J. Gile, and R. Tourangeau. 2013. “Summary Report of the AAPOR Task Force on Non-Probability Sampling.” Journal of Survey Statistics and Methodology 1(2): 90–143. Doi: https://doi.org/10.1093/jssam/smt008.10.1093/jssam/smt008Search in Google Scholar

Beaumont, J.-F. 2008. “A New Approach to Weighting and Inference in Sample Surveys.” Biometrika 95(3): 539–553. Doi: https://doi.org/10.1093/biomet/asn028.10.1093/biomet/asn028Search in Google Scholar

Burt, R.D., H. Hagan, K. Sabin, and H. Thiede. 2010. “Evaluating Respondent-Driven Sampling in a Major Metropolitan Area: Comparing Injection Drug Users in the 2005 Seattle Area National HIV Behavioral Surveillance System Survey with Participants in the RACEN and Kiwi Studies.” Annals of Epidemiology 20(2): 159–167. Doi: https://doi.org/10.1016/j.annepidem.2009.10.002.10.1016/j.annepidem.2009.10.002281843020123167Search in Google Scholar

Centers for Disease Control and Prevention (CDC). 2009. HIV-Associated Behaviors Among Injecting-Drug Users––23 Cities, United States, May 2005–February 2006. Morbidity and Mortality Weekly Report, 58, 329–332. Available at: http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5813a1.htm (accessed September 2015).Search in Google Scholar

Centers for Disease Control and Prevention (CDC). 2013. National HIV Behavioral Surveillance System Round 4: Model Surveillance Protocol. Available at: http://www.cdc.gov/hiv/pdf/NHBS_Round4ModelSurveillanceProtocol.pdf (accessed September 2015).Search in Google Scholar

Cole, S.R. and M.A. Hernan. 2008. “Constructing Inverse Probability Weights for Marginal Structural Models.” American Journal of Epidemiology 168: 656–664. Doi: https://doi.org/10.1093/aje/kwn164.10.1093/aje/kwn164273295418682488Search in Google Scholar

Compton, W., J. Normand, and E. Lambert. 2009. “Sexual Acquisition and Transmission of HIV Cooperative Agreement Program (SATHCAP).” Journal of Urban Health 86(1): 1–4. Doi: https://doi.org/10.1007/s11524-009-9373-4.10.1007/s11524-009-9373-4270549219543977Search in Google Scholar

Constantine, M. 2010. “Disentangling Methodologies: The Ethics of Traditional Sampling Methodologies, Community-Based Participatory Research, and Respondent-Drive Sampling.” American Journal of Bioethics 10(3): 22–24. Doi: https://doi.org/10.1080/15265160903585628.10.1080/1526516090358562820229410Search in Google Scholar

Dombrowski, R., B. Khan, J. Moses, E. Channell, and E. Misshula. 2013. “Assessing Respondent Driven Sampling for Network Studies in Ethnographic Contexts.” Advances in Anthropology 3(1): 1–9. Doi: https://doi.org/10.4236/aa.2013.31001.10.4236/aa.2013.31001Search in Google Scholar

Elliott, M.R. 2009. “Model Averaging Methods for Weight Trimming in Generalized Linear Regression Models.” Journal of Official Statistics 25(1): 1–21. Doi: https://doi.org/10.1.1.552.9050.Search in Google Scholar

Fay, R.E. and R.A. Herriot. 1979. “Estimates of Income for Small Places: An Application of James-Stein Procedures to Census Data.” Journal of American Statistical Association 74: 269–277. Doi: https://doi.org/10.2307/2286322.10.2307/2286322Search in Google Scholar

Frost, S.D.W., K.C. Brouwer, M.A.F. Cruz, R. Ramos, M.E. Ramos, R.M. Lozada, C. Magis-Rodriguez, and S.A. Strathdee. 2006. “Respondent-Driven Sampling of Injection Drug Users in Two U.S.-Mexico Border Cities: Recruitment Dynamics and Impact on Estimates of HIV and Syphilis Prevalence.” Journal of Urban Health 83(1): 83–97. Doi: https://doi.org/10.1007/s11524-006-9104-z.10.1007/s11524-006-9104-z170550717072761Search in Google Scholar

Gile, K.J. 2011. “Improved Inference for Respondent-Driven Sampling Data with Application to HIV Prevalence Estimation.” Journal of American Statistical Association 106(493): 135–146. Doi: https://doi.org/10.1198/jasa.2011.ap09475.10.1198/jasa.2011.ap09475Search in Google Scholar

Gile, K.J. and M.S. Handcock. 2010. “Respondent-Driven Sampling: An Assessment of Current Methodology.” Sociological Methodology 40(1): 286–327. Doi: https://doi.org/10.1111/j.1467-9531.2010.01223.x.10.1111/j.1467-9531.2010.01223.xSearch in Google Scholar

Gile, K.J., L.G. Johnston, and M.J. Salganik. 2015. “Diagnostics for Respondent-Driven Sampling.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 178(1): 241–269. Doi: https://doi.org/10.1111/rssa.12059.10.1111/rssa.12059Search in Google Scholar

Goel, S. and M.J. Salganik. 2010. “Assessing Respondent-Driven Sampling.” Proceedings of the National Academy of Sciences of the United States of America 107(15): 6743–6747. Doi: https://doi.org/10.1073/pnas.1000261107. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2872407/ (accessed September 2015).10.1073/pnas.1000261107Search in Google Scholar

Groves, R.M. 1989. Survey Errors and Survey Costs. New York: Wiley.10.1002/0471725277Search in Google Scholar

Groves, R.M. and M.P. Couper. 1998. Nonresponse in Household Surveys. New York: Wiley.10.1002/9781118490082Search in Google Scholar

Handcock, M.S. 2012. Estimating the Size of Hard-to-Reach Populations Using Respondent-Driven Sampling Data. Paper for the International Conference on Methods for Surveying and Enumerating Hard–to-Reach Populations, October 31–November 3, New Orleans, LA.Search in Google Scholar

Handcock, M.S., K.J. Gile, I.E. Fellows, and W.W. Neeley. 2014. Package ‘RDS.’ Available at: http://cran.r-project.org/web/packages/RDS/RDS.pdf (accessed September 2015).Search in Google Scholar

Heckathorn, D.D. 1997. “Respondent-Driven Sampling: A New Approach to the Study of Hidden Populations.” Society for the Study of Social Problems 44(2): 174–199. Doi: https://doi.org/10.2307/3096941.10.2307/3096941Search in Google Scholar

Heckathorn, D.D. 2002. “Respondent-Driven Sampling II: Deriving Valid Population Estimates from Chain-Referral Samples of Hidden Populations.” Social Problems 49(1): 11–34. Doi: https://doi.org/10.1525/sp.2002.49.1.11.10.1525/sp.2002.49.1.11Search in Google Scholar

Heckathorn, D.D. and J. Jeffri. 2001. “Finding the Beat: Using Respondent-Driven Sampling to Study Jazz Musicians.” Poetics 28(4): 307–329. Doi: https://doi.org/10.1016/S0304-422X(01)80006-1.10.1016/S0304-422X(01)80006-1Search in Google Scholar

Heimer, R. 2005. “Critical Issues and Further Questions About Respondent-Driven Sampling: Comment on Ramierz-Valles et al. (2005).” AIDS and Behavior 9(4): 403–408. Doi: https://doi.org/10.1007/s10461-005-9030-1.10.1007/s10461-005-9030-116344920Search in Google Scholar

Iguchi, M.Y., S.H. Berry, A.J. Ober, T. Fain, D.D. Heckathorn, P.M. Gorbach, R. Heimer, A. Kozlov, L.J. Ouellet, S. Shoptaw, and W. Zule. 2010. Sexual Acquisition and Transmission of HIV Cooperative Agreement Program (SATHCAP) 2006–2008 [United States]. ICPSR29181-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. Doi: https://doi.org/10.3886/ICPSR29181.10.3886/ICPSR29181Search in Google Scholar

Iguchi, M.Y., A.J. Ober, S.H. Berry, T. Fain, D.D. Heckathorn, P.M. Gorbach, R. Heimer, A. Kozlov, L.J. Ouellet, S. Shoptaw, and W.A. Zule. 2009. “Simultaneous Recruitment of Drug Users and Men Who Have Sex with Men in the United States and Russia Using Respondent-Driven Sampling: Sampling Methods and Implications.” Journal of Urban Health 88(1): 5–31. Doi: https://doi.org/10.1007/s11524-009-9365-4.10.1007/s11524-009-9365-4270548419472058Search in Google Scholar

Izrael, D., M. Battaglia, and M. Frankel. 2009. “Extreme Survey Weight Adjustment as a Component of Sample Balancing (a.k.a. Raking).” Proceedings from the 2009 SAS Global Forum. Cary, NC: SAS Institute. Available at: http://abtassociates.com/AbtAssociates/files/c1/c1bc376c-1931-4721-b71c-cb823a0fe809.pdf (accessed January 2017).Search in Google Scholar

Johnston, L.G. and K. Sabin. 2010. “Sampling Hard-to-Reach Populations with Respondent Driven Sampling.” Methodological Innovations Online 5(2): 38–48. Doi: https://doi.org/10.4256/mio.2010.0017.10.4256/mio.2010.0017Search in Google Scholar

Kish, L. 1992. “Weighting for unequal Pi.” Journal of Official Statistics 8(2): 183–200.Search in Google Scholar

Lansky, A., A. Abdul-Quader, M. Cribbin, T. Hall, T.J. Finlayson, R.S. Garfein, L.S. Lin, and P.S. Sullivan. 2007. “Developing an HIV Behavioral Surveillance System for Injecting Drug Users: The National HIV Behavioral Surveillance System.” Public Health Reports 122: 48–55. Doi: https://doi.org/10.1177/00333549071220S108.10.1177/00333549071220S108180410717354527Search in Google Scholar

Laumann, E.O., P.V. Marsden, and D. Prensky. 1983. “The Boundary Specification Problem in Network Analysis.” In Applied Network Analysis. A Methodological Introduction, edited by R.S. Burt and M.J. Minor. 18–34. Beverly Hills, CA: Sage.Search in Google Scholar

Lee, S. 2009. “Understanding Respondent Driven Sampling from a Total Survey Error Perspective.” Survey Practice. Available at: http://www.surveypractice.org/index.php/SurveyPractice/article/view/187/html (accessed September 2015).Search in Google Scholar

Lee, R., J. Ranaldi, M. Cummings, J.N. Crucetti, H. Stratton, and L.-A. McNutt. 2011. “Given the Increasing Bias in Random Digit Dial Sampling, Could Respondent-Driven Sampling be a Practical Alternative?” Annals of Epidemiology 21(4): 272–279. Doi: https://doi.org/10.1016/j.annepidem.2010.11.018.10.1016/j.annepidem.2010.11.01821376274Search in Google Scholar

Lee, S., Z.T. Suzer-Gurtekin, J. Wagner, and R. Valliant. 2012. Exploring Error Properties of Respondent Driven Sampling. Paper presented at the Joint Statistical Meeting, July 28–August 2, San Diego, CA.Search in Google Scholar

Lin, L., T. Finlayson, R. Iachan, M.C.B. Mendoza, and C. Wejnert. 2013. “Sampling Designs for Populations at High Risk for HIV.” Paper presented at the Joint Statistical Meeting, August 3–August 8, Montréal, Canada.Search in Google Scholar

Little, R.J.A., S. Lewitzky, S. Heeringa, J. Lepkowski, and R.C. Kessler. 1997. “Assessment of Weighting Methodology of the National Comorbidity Survey.” American Journal of Epidemiology 146(5): 439–449. Doi: https://doi.org/10.1093/oxfordjournals.aje.a009297.10.1093/oxfordjournals.aje.a0092979290504Search in Google Scholar

Lu, X., L. Bengtsson, T. Britton, M. Camitz, B.J. Kim, A. Thorson, and F. Liljeros. 2012. “The Sensitivity of Respondent-Driven Sampling.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 175(1): 1–26. Doi: https://doi.org/10.1111/j.1467-985X.2011.00711.x.10.1111/j.1467-985X.2011.00711.xSearch in Google Scholar

Marsden, P.V. 1990. “Network Data and Measurement.” Annual Review Sociology 16: 435–463. Doi: https://doi.org/10.1146/annurev.so.16.080190.002251.10.1146/annurev.so.16.080190.002251Search in Google Scholar

Martin, J.L., J. Wiley, and D. Osmond. 2003. “Social Networks and Unobserved Heterogeneity in Risk for AIDS.” Population Research and Policy Review 22(1): 65–90. Doi: https://doi.org/10.1023/A:1023509211339.10.1023/A:1023509211339Search in Google Scholar

McCarty, C., P.D. Killworth, H.R. Bernard, E.C. Johnsen, and G.A. Shelley. 2001. “Comparing Two Methods for Estimating Network Size.” Human Organization 60: 28–39. Doi: https://doi.org/10.17730/humo.60.1.efx5t9gjtgmga73y.10.17730/humo.60.1.efx5t9gjtgmga73ySearch in Google Scholar

McCreesh, N., S.D. Frost, J. Seeley, J. Katongole, M.N. Tarsh, R. Ndunguse, F. Jichi, N.L. Lunel, D. Maher, L.G. Johnston, P. Sonnenberg, A.J. Copas, R.J. Hayes, and R.G. White. 2012. “Evaluation of Respondent-Driven Sampling.” Epidemiology 23(1): 138–147. Doi: https://doi.org/10.1097/EDE.0b013e31823ac17c.10.1097/EDE.0b013e31823ac17c327790822157309Search in Google Scholar

Montealegre, J.R., J.M. Risser, B.J. Selwyn, S.A. McCurdy, and K. Sabin. 2013. “Effectiveness of Respondent Driven Sampling to Recruit Undocumented Central American Immigrant Women in Houston, Texas for an HIV Behavioral Survey.” AIDS and Behavior 17(2): 719–727. Doi: https://doi.org/10.1007/s10461-012-0306-y.10.1007/s10461-012-0306-y22961500Search in Google Scholar

Phillips, T. 2010. “Protecting the Subject: PDR and the Potential for Compromised Consent.” American Journal of Bioethics 10(3): 14–15. Doi: https://doi.org/10.1080/15265160903585602.10.1080/1526516090358560220229406Search in Google Scholar

Potter, F. 1988. “Survey of Procedures to Control Extreme Sampling Weights.” Proceedings of the Section on Survey Research Methods, American Statistical Association, 453–458. Available at: http://www.websm.org/uploadi/editor/1368363852Potter_1988_Survey_of_procedures_to_control_extreme_sampling_weights.pdf (accessed January 2017).Search in Google Scholar

Ramirez-Valles, J. 2013. Latino MSM Community Involvement: HIV Protective Effects. ICPSR34385-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research. Doi: https://doi.org/10.3886/ICPSR34385.v1.10.3886/ICPSR34385.v1Search in Google Scholar

Ramirez-Valles, J., D.D. Heckathorn, R. Vázquez, R.M. Diaz, and R.T. Campbell. 2005. “From Networks to Populations: the Development and Application of Respondent-Driven Sampling Among IDUs and Latino Gay Men.” AIDS and Behavior 9(4): 387–402. Doi: https://doi.org/10.1007/s10461-005-9012-3.10.1007/s10461-005-9012-316235135Search in Google Scholar

Salganik, M.J. 2006. “Variance Estimation, Design Effects and Sample Size Calculations for Respondent Driven Sampling.” Journal of Urban Health 83(7): 98–112. Doi: https://doi.org/10.1007/s11524-006-9106-x.10.1007/s11524-006-9106-x170551516937083Search in Google Scholar

Salganik, M. 2012. “Commentary: Respondent-Driven Sampling in the Real World.” Epidemiology 23(1): 148–150. Doi: https://doi.org/10.1097/EDE.0b013e31823b6979.10.1097/EDE.0b013e31823b697922157310Search in Google Scholar

Salganik, M.J. and D.D. Heckathron. 2004. “Sampling and Estimation in Hidden Populations Using Respondent-Driven Sampling.” Sociological Methodology 34: 193–239. Doi: https://doi.org/10.1111/j.0081-1750.2004.00152.x.10.1111/j.0081-1750.2004.00152.xSearch in Google Scholar

Schonlau, M. 2014. “Recruiting an Internet Panel Using Respondent Driven Sampling.” Journal of Official Statistics 30(2): 291–310. Doi: https://doi.org/10.2478/jos-2014-0018.10.2478/jos-2014-0018Search in Google Scholar

Simon, C. and M. Mosavel. 2010. “Community Members as Recruiters of Human Subjects: Ethical Considerations.” American Journal of Bioethics 10(3): 3–11. Doi: https://doi.org/10.1080/15265160903585578.10.1080/15265160903585578313946620229402Search in Google Scholar

Sirken, M.G. 1972. “Stratified Sample Surveys with Multiplicity.” Journal of American Statistical Association 67: 224–227. Doi: https://doi.org/10.1080/01621459.1972.10481236.10.1080/01621459.1972.10481236Search in Google Scholar

Sirken, M.G. 1975. “Network Surveys of Rare and Sensitive Conditions.” Advances in Health Survey Research Methods, NCHSR Research Proceedings 31. Hyattsville, MD: National Center Health Statistics.Search in Google Scholar

Sirken, M.G. 1997. “Network Sampling.” In Encyclopedia of Biostatistics, edited by P. Armitage and T. Colton, 2977–2986. Hoboken, NJ: Wiley & Sons.Search in Google Scholar

Valliant, R. 2013. “Comment.” Journal of Survey Statistics and Methodology 1(2): 105–111. Doi: https://doi.org/10.1093/jssam/smt010.10.1093/jssam/smt010Search in Google Scholar

Valliant, R., J.A. Dever, and F. Kreuter. 2013. Practical Tools for Designing and Weighting Survey Samples. New York: Springer.10.1007/978-1-4614-6449-5Search in Google Scholar

Verdery, A.M. and T.D. Mouw. 2012. Estimated Sampling Variance in Respondent Driven Sampling Data: Mathematical Derivations, Simulated Tests on Empirical Data, and Evidence from Other Forms of Chain-Referral Data Collection. Paper for the International Conference on Methods for Surveying and Enumerating Hard-to-Reach Populations, October 31–November 3, New Orleans, LA.Search in Google Scholar

Verdery, A.M., T.D. Mouw, S. Bauldry, and P.J. Mucha. 2015. “Network Structure and Biased Variance Estimation in Respondent Driven Sampling.” PLOS ONE 10(12): e0145296. Doi: http://dx.doi.org/10.1371/journal.pone.0145296.10.1371/journal.pone.0145296468298926679927Search in Google Scholar

Volz, E. and D.D. Heckathorn. 2008. “Probability Based Estimation Theory for Respondent Driven Sampling.” Journal of Official Statistics 24(1): 79–97.Search in Google Scholar

Wejnert, C. and D.D. Heckathorn. 2008. “Web-Based Network Sampling: Efficiency and Efficacy of Respondent-Driven Sampling for Online Research.” Sociological Methods and Research 37: 105–134. Doi: https://doi.org/10.1177/0049124108318333.10.1177/0049124108318333Search in Google Scholar

Zheng, T., M.J. Salganik, and A. Gelman. 2006. “How Many People Do You Know in Prison? Using Overdispersion in Count Data to Estimate Social Structure in Networks” Journal of American Statistical Association 101(474): 409–423. Doi: https://doi.org/10.1198/01621450500000116.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo