Factorial Survey Experiments in the Sociology of Education. Potentials, Pitfalls, Evaluation

Ajzen, Icek. 1991. The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes 50(2): 179–211.10.1016/0749-5978(91)90020-TSearch in Google Scholar

Ajzen, Icek, Thomas C. Brown, and Franklin Carvajal. 2004. Explaining the Discrepancy Between Intentions and Actions: The Case of Hypothetical Bias in Contingent Valuation. Personality and Social Psychology Bulletin 30(9): 1108–1121.10.1177/0146167204264079Search in Google Scholar

Alexander, Cheryl S., and Henry Jay Becker. 1978. The Use of Vignettes in Survey Research. Public Opinion Quarterly 42(1): 94–104.10.1086/268432Search in Google Scholar

Armacost, Robert L., Jamshid C. Hosseini, Sara A. Morris, and Kathleen A. Rehbein. 1991. An Empirical Comparison of Direct Questioning, Scenario, and Randomized Response Methods for Obtaining Sensitive Business Information. Decision Sciences 22(5): 1073–1099.10.1111/j.1540-5915.1991.tb01907.xSearch in Google Scholar

Auspurg, Karin, and Thomas Hinz. 2015. Factorial Survey Experiments. London / Thousand Oaks: Sage Publications.10.4135/9781483398075Search in Google Scholar

Auspurg, Katrin, Thomas Hinz, and Stefan Liebig. 2009. Komplexität von Vignetten, Lerneffekte und Plausibilität im Faktoriellen Survey. Methoden – Daten – Analysen 3(1): 59–96.Search in Google Scholar

Auspurg, Katrin, Thomas Hinz, Stefan Liebig, and Carsten Sauer. 2015. The Factorial Survey as a Method for Measuring Sensitive Issues. Pp. 137–150 in Improving Survey Methods: Lessons from Recent Research, edited by Uwe Engel, Ben Jann, Peter Lynn, Annette Scherpenzeel, and Patrick Sturgis. New York / Hove: Routledge.Search in Google Scholar

Auspurg, Katrin, and Annette Jäckle. 2017. First Equals Most Important? Order Effects in Vignette-Based Measurement. Sociological Methods and Research 46(3): 490–539.10.1177/0049124115591016Search in Google Scholar

Becker, Rolf. 2000. Klassenlage und Bildungsentscheidungen. Eine empirische Anwendung der Wert-Erwartungstheorie. Kölner Zeitschrift für Soziologie und Sozialpsychologie 52(3): 450–474.10.1007/s11577-000-0068-9Search in Google Scholar

Becker, Rolf, and Anna Etta Hecken. 2009. Higher Education or Vocational Training? An Empirical Test of the Rational Action Model of Educational Choices Suggested by Breen and Goldthorpe and Esser. Acta Sociologica 52(1): 25–45.10.1177/0001699308100632Search in Google Scholar

Becker, Rolf, and Heike Solga (eds.). 2012. Soziologische Bildungsforschung. Sonderheft 52 der Kölner Zeitschrift für Soziologie und Sozialpsychologie. Wiesbaden: VS Verlag für Sozialwissenschaften.10.1007/978-3-658-00120-9Search in Google Scholar

Berk, Richard A., Bruce Western, and Robert E. Weiss. 1995. Statistical Inference for Apparent Populations. Sociological Methodology 25: 421–458.10.2307/271073Search in Google Scholar

Boudon, Raymond. 1974. Education, Opportunity, and Social Inequality. Changing Prospects in Western Societies. New York: Wiley & Sons.Search in Google Scholar

Breen, Richard, and John H. Goldthorpe. 1997. Explaining Educational Differentials. Towards a Formal Rational Action Theory. Rationality and Society 9(3): 275–305.10.1177/104346397009003002Search in Google Scholar

Brüderl, Josef. 2004. Die Überprüfung von Rational-Choice-Modellen mit Umfragedaten. Pp. 163–180 in Rational-Choice Theorie in den Sozialwissenschaften. Anwendungen und Probleme, edited by Andreas Diekmann, and Thomas Voss. München: Oldenbourg Verlag.Search in Google Scholar

Brüderl, Josef, and Volker Ludwig. 2015. Fixed-Effects Panel Regression. Pp. 327–359 in The SAGE Handbook of Regression Analysis and Causal Inference, edited by Henning Best, and Christof Wolf. London et al.: Sage.10.4135/9781446288146.n15Search in Google Scholar

Buskens, Vincent, Werner Raub, and Marcel A.L.M. van Assen. 2014. Micro-Macro Links and Micro-foundations in Sociology. London / New York: Routledge.10.4324/9781315871936Search in Google Scholar

Campbell, Donald T. 1957. Factors Relevant to the Validity of Experiments in Social Settings. Psychological Bulletin 54(4): 297–312.10.1037/h0040950Search in Google Scholar

Campbell, Donald T. and Julian C. Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand-McNally.Search in Google Scholar

Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Erlbaum.Search in Google Scholar

Coleman, James S. 1990. Foundations of Social Theory. Cambridge: Harvard University Press.Search in Google Scholar

Collett, Jessica L., and Ellen Childs. 2011. Minding the Gap: Meaning, Affect, and the Potential Shortcomings of Vignettes. Social Science Research 40(2): 513–522.10.1016/j.ssresearch.2010.08.008Search in Google Scholar

Cook, Thomas D. 2001. Sciencephobia. Why Education Rejects Randomized Experiments. Education Next 1(3): 63–68.Search in Google Scholar

Dafoe, Allan, Baobao Zhan, and Devin Caughey. 2018. Information Equivalence in Survey Experiments. Political Analysis 26(4): 399–416.10.1017/pan.2018.9Search in Google Scholar

Daniel, Annabell, Martin Neugebauer, and Rainer Watermann. 2019. Studienabbruch und Einstellungschancen auf dem Ausbildungsmarkt. Ein faktorieller Survey mit Arbeitgeber/innen. Zeitschrift für Erziehungswissenschaft 22: 1147–1174.10.1007/s11618-019-00905-0Search in Google Scholar

De Leeuw, Edith D. 2005. To Mix or Not to Mix Data Collection Modes in Surveys. Journal of Official Statistics 21(2): 233–255.Search in Google Scholar

De Wolf, Inge and Rolf Van Der Velden. 2001. Selection Processes for Three Types of Academic Jobs. An Experiment among Dutch Employers of Social Sciences Graduates. European Sociological Review 17(3): 317–330.10.1093/esr/17.3.317Search in Google Scholar

Di Stasio, Valentina. 2014. Education as a Signal of Trainability: Results from a Vignette Study with Italian Employers. European Sociological Review 30(6): 796–809.10.1093/esr/jcu074Search in Google Scholar

Dülmer, Hermann. 2007. Experimental Plans in Factorial Surveys: Random or Quota Design? Sociological Methods and Research 35(3): 382–409.10.1177/0049124106292367Search in Google Scholar

Dülmer, Hermann. 2016. The Factorial Survey. Design Selection and its Impact on Reliability and Internal Validity. Sociological Methods and Research 45(2): 304–347.10.1177/0049124115582269Search in Google Scholar

Dunning, Thad. 2012. Natural Experiments in the Social Sciences. A Design-Based Approach. Cambridge: Cambridge University Press.10.1017/CBO9781139084444Search in Google Scholar

Düval, Sabine and Thomas Hinz. 2020. Different Order, Different Results? The Effects of Dimension Order in Factorial Survey Experiments. Field Methods 32(1): 23–37.10.1177/1525822X19886827Search in Google Scholar

Edgington, Eugene. 1966. Statistical Inference and Nonrandom Samples. Psychological Bulletin 66(6): 485–487.10.1037/h0023916Search in Google Scholar

Eifler, Stefanie, and Knut Petzold. 2019. Validity Aspects of Vignette Experiments: Expected What-If Differences Between Reports of Behavioral Intentions and Actual Behavior. Pp. 393–416 in Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment, edited by Paul Lavrakas, Michael Traugott, Courtney Kennedy, Allyson Holbrook, Edith de Leeuw, and Brady West. Hoboken: John Wiley & Sons.10.1002/9781119083771.ch20Search in Google Scholar

Elwert, Felix, and Christopher Winship. 2014. Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable. Annual Review of Sociology 40: 31–53.10.1146/annurev-soc-071913-043455Search in Google Scholar

Erikson, Robert, and Jan O. Jonsson. 1996. Explaining Class Inequality in Education: The Swedish Test Case. Pp. 1–63 in Can Education Be Equalized? edited by Robert Erikson, and Jan O. Jonsson. Boulder: Westview Press.Search in Google Scholar

Erlinghagen, Marcel and Karsten Hank. 2018. Neue Sozialstrukturanalyse. Ein Kompass für Studienanfänger. 2. Auflage. Stuttgart: UTB.10.36198/9783838549804Search in Google Scholar

Esser, Hartmut. 1996. What is Wrong With “Variable Sociology”? European Sociological Review 12(2): 159–166.10.1093/oxfordjournals.esr.a018183Search in Google Scholar

Esser, Hartmut. 1999. Soziologie. Spezielle Grundlagen. Band 1: Situationslogik und Handeln. Frankfurt a.M. / New York: Campus Verlag.Search in Google Scholar

Finch, Janet. 1987. The Vignette Technique in Survey Research. Sociology 21(1): 105–114.10.1177/0038038587021001008Search in Google Scholar

Finger, Claudia. 2016. Institutional Constraints and the Translation of College Aspirations into Intentions–Evidence from a Factorial Survey. Research in Social Stratification and Mobility 46: 112–128.10.1016/j.rssm.2016.08.001Search in Google Scholar

Friedman, Daniel, and Alessandra Cassar. 2004. Economics Lab. An Intensive Course in Experimental Economics. London / New York: Routledge.Search in Google Scholar

Gerber, Alan S. and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York: W. W. Norton & Company.Search in Google Scholar

Goldthorpe, John H. 2001. Causation, Statistics, and Sociology. European Sociological Review 17(1): 1–20.10.1093/esr/17.1.1Search in Google Scholar

Grusky, David B. 1994. Social Stratification: Class, Race, and Gender in Sociological Perspective. Boulder: Westview Press.Search in Google Scholar

Hainmueller, Jens, Dominik Hangartner, and Teppei Yamamoto. 2015. Validating Vignette and Conjoint Survey Experiments Against Real-World Behavior. Proceedings of the National Academy of Sciences 112(8): 2395–2400.10.1073/pnas.1416587112Search in Google Scholar

Harkness, Janet (ed.). 1998. Cross-Cultural Survey Equivalence. Zentrum für Umfragen, Methoden und Analysen. Mannheim: ZUMASearch in Google Scholar

Hedström, Peter, and Richard Swedberg. 1996. Rational Choice, Empirical Research, and the Sociological Tradition. European Sociological Review 12(2): 127–146.10.1093/oxfordjournals.esr.a018181Search in Google Scholar

Hedström, Peter, and Petri Ylikoski. 2010. Rational Choice, Empirical Research, and the Sociological Tradition. Annual Review of Sociology 36: 49–67.10.1146/annurev.soc.012809.102632Search in Google Scholar

Holland, Paul W. 1986. Causal Mechanisms in the Social Sciences. Journal of the American Statistical Association 81(4): 945–960.10.1080/01621459.1986.10478354Search in Google Scholar

Hox, Joob J., Ita G. Kreft, and Piet L. J. Hermkens. 1991. The Analysis of Factorial Surveys. Sociological Methods and Research 19(4): 439–510.10.1177/0049124191019004003Search in Google Scholar

Hughes, Rhidian, and Meg Huby. 2004. The Construction and Interpretation of Vignettes in Social Research. Social Work & Social Sciences Review 11(1): 36–51.10.1921/17466105.11.1.36Search in Google Scholar

Jackson, Michelle, and D.R. Cox. 2013. The Principles of Experimental Design and Their Application in Sociology. Annual Review of Sociology 39: 27–49.10.1146/annurev-soc-071811-145443Search in Google Scholar

Jasso, Guillermina. 2006. Factorial Survey Methods for Studying Beliefs and Judgements. Sociological Methods and Research 34(3): 334–423.10.1177/0049124105283121Search in Google Scholar

Keller, Tamás. 2018. Dare to Dream: A Vignette Survey on Self-Selection in Secondary Education Track Choice. Sociological Research Online 23(2): 1–20.10.1177/1360780418757539Search in Google Scholar

King, Gary, Christopher J. L. Murray, Joshua A. Salomon, and Ajay Tandon. 2004. Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research. American Political Science Review 98: 191–207.10.1017/S000305540400108XSearch in Google Scholar

Krumpal, Ivar. 2013. Determinants of Social Desirability Bias in Sensitive Surveys: A Literature Review. Quality and Quantity 47(3): 2025–2047.10.1007/s11135-011-9640-9Search in Google Scholar

Kuhfeld, Warren F., Randall D. Tobias, and Mark Garratt. 1994. Efficient Experimental Design With Marketing Research Applications. Journal of Marketing Research 31(4): 545–557.10.1177/002224379403100408Search in Google Scholar

Levitt, Steven D., and John A. List. 2007. What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World? The Journal of Economic Perspectives 21(2): 153–174.10.1257/jep.21.2.153Search in Google Scholar

Lindenberg, Siegwart. 1996. Die Relevanz theoriereicher Brückenannahmen. Kölner Zeitschrift für Soziologie und Sozialpsychologie 48(1): 126–140.Search in Google Scholar

Lörz, Markus, Nicolai Netz, and Heiko Quast. 2016. Why Do Students from Underprivileged Families Less Often Intend to Study Abroad? Higher Education 72(2): 153–174.10.1007/s10734-015-9943-1Search in Google Scholar

Markovsky, Barry, and Kimmo Eriksson. 2012. Comparing Direct and Indirect Measures of Just Rewards. Sociological Methods and Research 41(1): 199–216.10.1177/0049124112437712Search in Google Scholar

McDonald, Patrick. 2019. How Factorial Survey Analysis Improves Our Understanding of Employer Preferences. Swiss Journal of Sociology 45(2): 237–260.10.2478/sjs-2019-0011Search in Google Scholar

Mook, Douglas G. 1983. In Defense of External Invalidity. American Psychologist 38(4): 379–387.10.1037/0003-066X.38.4.379Search in Google Scholar

Morgan, Stephen L., and Christopher Winship. 2015. Counterfactuals and Causal Inference: Methods and Principles for Social Research. 2^nd edition. Cambridge: Cambridge University Press.10.1017/CBO9781107587991Search in Google Scholar

Möser, Sara, David Glauser, and Rolf Becker. 2019. Valuation of Labour Market Entrance Positions among (Future) Apprentices - Results from Two Discrete Choice Experiments. Journal of Choice Modelling 33: 100180.10.1016/j.jocm.2019.100180Search in Google Scholar

Mutz, Diana C. 2011. Population-Based Survey Experiments. Princeton: Princeton University Press.10.23943/princeton/9780691144511.001.0001Search in Google Scholar

Opp, Karl-Dieter. 1999. Contending Conceptions of the Theory of Rational Choice. Journal of Theoretical Politics 11(2): 171–202.10.1177/0951692899011002002Search in Google Scholar

Pager, Devah, and Lincoln Quillian. 2005. Walking the Talk? What Employers Say Versus What They Do. American Sociological Review 70(3): 355–380.10.1177/000312240507000301Search in Google Scholar

Pearl, Judea. 2010. The Foundations of Causal Inference. Sociological Methodology 40(1): 75–149.10.1111/j.1467-9531.2010.01228.xSearch in Google Scholar

Petzold, Knut. 2018. Fachspezifische Entscheidungen zum Auslandsstudium. Ein experimenteller Test der Wert-Erwartungstheorie. Zeitschrift für Erziehungswissenschaft 21(4): 817–838.10.1007/s11618-017-0788-5Search in Google Scholar

Petzold, Knut, and Petra Moog. 2018. What Shapes the Intention to Study Abroad? An Experimental Approach. Higher Education 75(1): 35–54.10.1007/s10734-017-0119-zSearch in Google Scholar

Petzold, Knut, and Tobias Wolbring. 2018. What Can We Learn From Factorial Surveys About Human Behavior? A Validation Study Comparing Field and Survey Experiments on Discrimination. Methodology. 15(1): 19-30.10.1027/1614-2241/a000161Search in Google Scholar

Petzold, Knut, and Tobias Wolbring. 2019. Zur Verhaltensvalidität von Vignettenexperimenten: Theoretische Grundlagen, Forschungsstrategien und Befunde. Pp. 307–338 in Qualitätssicherung sozialwissenschaftlicher Erhebungsinstrumente., edited by Natalja Menold, and Tobias Wolbring. Wiesbaden: Springer VS.10.1007/978-3-658-24517-7_10Search in Google Scholar

Popper, Karl Raimund. 1959. The Logic of Scientific Discovery. London: Hutchinson.10.1063/1.3060577Search in Google Scholar

Rosenbaum, Paul R. 2010. The Design of Observational Studies. New York et al.: Springer.10.1007/978-1-4419-1213-8Search in Google Scholar

Rössel, Jörg. 2009. Sozialstrukturanalyse. Eine kompakte Einführung. Wiesbaden: VS Verlag für Sozialwissenschaften.Search in Google Scholar

Rossi, Peter H., and Andy B. Anderson. 1982. The Factorial Survey Approach: An Introduction. Pp. 15–67 in Measuring Social Judgments. The Factorial Approach, edited by Peter H. Rossi, and Steven L. Nock. Beverly Hills et al.: Sage Publications.Search in Google Scholar

Rubin, Donald B. 1980. Randomization Analysis of Experimental Data: The Fisher Randomization Test Comment. Journal of the American Statistical Association 75(371): 591–593.10.2307/2287653Search in Google Scholar

Rubin, Donald B. 2008. For Objective Causal Inference, Design Trumps Analysis. The Annals of Applied Statistics 2(3): 808–840.10.1214/08-AOAS187Search in Google Scholar

Salisbury, Mark H., Paul U. Umbach, Michael B. Paulsen, and Ernest T. Pascarella. 2009. Going Global: Understanding the Choice Process of the Intent to Study Abroad. Research in Higher Education 50(2): 119–143.10.1007/s11162-008-9111-xSearch in Google Scholar

Sauer, Carsten, Katrin Auspurg, Thomas Hinz, and Stefan Liebig. 2011. The Application of Factorial Survey in General Population Samples: The Effect of Respondent Age and Education on Response Times and Response Consistency. Survey Research Methods 5(3): 89–102.Search in Google Scholar

Sauer, Carsten, Katrin Auspurg, and Thomas Hinz. 2020. Designing Multi-Factorial Survey Experiments: Effects of Presentation Style (Text or Table), Answering Scales, and Vignette Order. methods, data, analyses 14(2): 195–214.Search in Google Scholar

Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston / New York: Houghton Mifflin Company.Search in Google Scholar

Shamon, Hawal, Hermann Dülmer, and Adam Giza. 2019. The Factorial Survey: The Impact of the Presentation Format of Vignettes on Answer Behavior and Processing Time. Sociological Methods and Research DOI: 10.1177/0049124119852382.10.1177/0049124119852382Search in Google Scholar

Shi, Lulu P., Christian Imdorf, Robin Samuel, and Stefan Sacchi. 2018. How Unemployment Scarring Affects Skilled Young Workers: Evidence from a Factorial Survey of Swiss Recruiters. Journal for Labour Market Research 52: Article n°7.10.1186/s12651-018-0239-7Search in Google Scholar

Snijders, Tom A. B., and Roel J. Bosker. 2012. Multilevel Analysis. An Introduction to Basic and Advanced Multilevel Modeling. Los Angeles, CA: Sage.Search in Google Scholar

Stocké, Volker. 2010. Der Beitrag der Theorie rationaler Entscheidung zur Erklärung von Bildungsungleichheit. Pp. 73–94 in Bildungsverlierer. Neue Ungleichheiten, edited by Gudrun Quenzel. Wiesbaden: Springer VS.10.1007/978-3-531-92576-9_4Search in Google Scholar

Stroebe, Wolfgang, Volker Gardenne, and Bernard A. Nijstad. 2018. Do Our Psychological Laws Apply Only to College Students? External Validity Revisited. Basic and Applied Social Psychology 40(6): 384–395.10.1080/01973533.2018.1513362Search in Google Scholar

Thelin, Mikael and Thomas Niedomysl. 2015. The (Ir)Relevance of Geography for School Choice: Evidence from a Swedish Choice Experiment. Geoforum 67(1): 110–120.10.1016/j.geoforum.2015.11.003Search in Google Scholar

Tourangeau, Roger, and Ting Yan. 2007. Sensitive Questions in Surveys. Psychological Bulletin 133(5): 859–883.10.1037/0033-2909.133.5.859Search in Google Scholar

Trafimow, David. 2012. The Role of Auxiliary Assumptions for the Validity of Manipulations and Measures. Theory & Psychology 22(4): 486–498.10.1177/0959354311429996Search in Google Scholar

Verlegh, Peeter W. J., Hendrik N. J. Schifferstein, and Dick R. Wittink. 2002. Range and Number-of-Levels Effects in Derived and Stated Measures of Attribute Importance. Marketing Letters 13(1): 41–52.10.1023/A:1015063125062Search in Google Scholar

Wallander, Lisa. 2009. 25 Years of Factorial Surveys in Sociology: A Review. Social Science Research 38(3): 505–520.10.1016/j.ssresearch.2009.03.004Search in Google Scholar

Walzenbach, Sandra. 2019. Hiding Sensitive Topics by Design? An Experiment on the Reduction of Social Desirability Bias in Factorial Surveys. Survey Research Methods 13(1): 103–121.Search in Google Scholar

Webster, Murray Jr., and Jane Sell. 2007. Laboratory Experiments in the Social Sciences. Amsterdam: Academic Press / Elsevier.Search in Google Scholar

Wooldridge, Jeffrey M. 2013. Introductory Econometrics: A Modern Approach. 5th Edition. Mason, OH: South-Western Cengage Learning.Search in Google Scholar

Zangger, Christoph, and Rolf Becker. 2019. Experiments in the Sociology of Education: Causal Inference and Estimating Causal Effects in Sociological Research on Education. Pp. 153–171 in Research Handbook on the Sociology of Education, edited by Rolf Becker. Cheltenham, UK: Edward Elgar Publishing.10.4337/9781788110426.00017Search in Google Scholar