Outlook and Discussion

Statistical identification methods have been demonstrated to be effective and are becoming increasingly popular tools for identifying falsified interviews in surveys. Through the combined use of different analysis tools, it is possible to create comprehensive measures that can identify various types of falsification and can be particularly useful for improving and supporting traditional quality control procedures. They can be automated to flag suspicious interviewers early in the field period for more extensive quality control procedures. However, so far no empirical threshold is defined on how early results become reliable, which should be addressed in future research. Although various statistical methods for detecting falsification exist, research into new and improved methods is still ongoing. Unfortunately, appropriate data for investigating falsification and testing and evaluating new and existing detection methods are often not made available to researchers. This is due to the small number of falsified interviews, but also because falsified data are usually removed from the published data upon detection. Due to the cooperativeness of our survey institute, we were able to demonstrate the usefulness of the different statistical tools for a high-profile case of interviewer falsification. By routinely making such data available to the research community, survey institutes would make a great contribution to the discussion about interviewer falsification and facilitate the development and improvement of quality control methods.

Acknowledgments

Financial support from the Charles Cannell Fund in Survey Methodology is gratefully acknowledged.

References

AAPOR. 2003. Interviewer falsification in survey research: Current best methods for prevention, detection and repair of its effects, https://www.aapor.org/AAPOR_Main/media/MainSiteFil es/falsification.pdf (accessed January 25, 2019).

Benford, F. 1938. The law of anomalous numbers. Proceedings of the American philosophical society, 551-72.

Biemer, P. P, and S. L. Stokes. 1989. The optimal design of quality control samples to detect interviewer cheating. Journal of official statistics 5(1):23—39.

Blair, E., and S. Burton. 1987. Cognitive processes used by survey respondents to answer behavioral frequency questions. Journal of consumer research 14(2):280-88.

Blasius, J., and J. Friedrichs. 2012. Faked interviews. In Methods, theories, and empirical applications in the social sciences, ed. S. Salzbom, E. Davidov, and J. Reinecke, 49-56. VS Verlag fiir Sozialwissenschaften.

Blasius, J., and V. Thiessen. 2012. Assessing the quality of survey data. London: Sage.

Blasius, J., and V. Thiessen. 2013. Detecting poorly conducted interviews. In Interviewers' deviations in surveys, ed. P. Winker, N. Menold, and R. Porst, 67-88. Frankfurt am Main: Intemationaler Verlag der Wissenschaften.

Blasius, J., and V. Thiessen. 2015. Should we trust survey data? Assessing response simplification and data fabrication. Social science research 52:479-93.

Bredl, S., N. Storfinger, and N. Menold. 2013. A literature review of methods to detect fabricated survey data. In Interviewers' deviations in surveys, ed. P. Winker, N. Menold, and R. Porst, 3-24. Frankfurt am Main: Intemationaler Verlag der Wissenschaften.

Bredl, S., P. Winker, and K. Kotschau. 2012. A statistical approach to detect interviewer falsification of survey data. Survey methodology 38:1-10.

Brucker,H.,N.Rother,andj.Schupp.2017.IAB-BAMF-SOEP-BefragungvonGefiuchteten2016:Studiendesign, Feldergebnisse sowie Analysen zu schulischer wie beruflicher Qualification, Sprachkenntnissen sowie kog- nitiven Potenzialen. Berlin: DIW Berlin, German Institute for Economic Research, http://doku.iab .de/forschungsbericht/2017/fbl317.pdf (accessed September 20,2019).

Bushery, J. M., J. W. Reichert, K. A. Albright, and J. C. Rossiter. 1999. Using date and time stamps to detect interviewer falsification. Proceedings of the survey research methods section, 316-20.

Crespi, L. R 1945. The cheater problem in polling. Public opinion quarterly 9(4):431-45.

Duda, R. O., and P. Hart.1973. Pattern classification and scene analysis. New York: Wiley.

Finn, A., and V. Ranchhod. 2017. Genuine fakes: The prevalence and implications of data fabrication in a large South African survey. The World Bank economic review 31(l):129-57.

Goodman, W. 2016. The promises and pitfalls of Benford's law. Significance 13(3):38-41.

Groves, R. M. 2005. Survey errors and survey costs. New York: Wiley.

Gwartney, P. A. 2013. Mischief versus mistakes: Motivating interviewers to not deviate. In Interviewers' deviations in surveys, ed. P. Winker, N. Menold, and R. Porst, 195-215. Frankfurt am Main: Internationaler Verlag der Wissenschaften.

de Haas, S., and P. Winker. 2014. Identification of partial falsifications in survey data. Statistical journal of the IAOS 30(3):271—81.

de Haas, S., and P. Winker. 2016. Detecting fraudulent interviewers by improved clustering methods - The case of falsifications of answers to parts of a questionnaire, journal of official statistics 32(3):643-60.

Hanisch, J. U. 2005. Rounded responses to income questions. Allgemeines statistisches archiv 89(1):39—48.

Harrison, D. E., and S. I. Krauss. 2002. Interviewer cheating: Implications for research on entrepreneurship in Africa. Journal of developmental entrepreneurship 7(3):319-30.

Hauck, M. 1969. Is survey postcard verification effective? Public opinion quarterly 33(l):117-20.

Hood, С. C., and J. M. Bushery. 1997. Getting more bang from the reinterview buck: Identifying "at risk" interviewers. Proceedings of the survey research method section, 820-24.

Hyman, H. H., W. J. Cobb, J. J. Feldman, C. W. Hart, and С. H. Stember. 1954. Interviewing in social research. Chicago, IL: Chicago Press.

Institut fiir Arbeitsmarkt-und Berufsforschung (IAB). 2017. Revidierter Datensatz der IAB-BAMF- SOEP-Befragung von Gefliichteten. Nuremberg. http://doku.iab.de/grauepap/2017/Revidie rter_Datensatz_der_IAB-BAMF-SOEP-Befragung.pdf (accessed September 20, 2019).

Kaufman, L., and P. J. Rousseeuw. 2009. Finding groups in data: An introduction to cluster analysis. Hoboken, NJ: John Wiley & Sons, Inc.

Kemper, C. J., and N. Menold. 2014. Nuisance or remedy? The utility of stylistic responding as an indicator of data fabrication in surveys. Methodology 10:92-99.

Koch, A. 1995. Gefiilschte interviews: Ergebnisse der interviewerkontrolle beim ALLBUS 1994. ZUMA Nachrichten 19(36):89-105.

Koczela, S., C. Furlong, J. McCarthy, and A. Mushtaq. 2015. Curbstoning and beyond: Confronting data fabrication in survey research. Statistical journal of the IAOS 31(3):413-22.

Kosyakova, Y., L. Olbrich, J. Sakshaug, and S. Schwanhauser. 2019. Identification of interviewer falsification in the IAB-BAMF-SOEP Survey of Refugees in Germany, http://doku.iab.de/fdz/re porte/2019/MR_02-19_EN.pdf. (accessed April 10, 2019).

Kosyakova, Y., J. Skopek, and S. Eckman. 2015. Do interviewers manipulate responses to filter questions? Evidence from a multilevel approach. International journal of public opinion research 27(3):417-31.

Kroh, M., Kuhne, S., Jacobsen, J., Siegert, M., and Siegers, R. 2017. Sampling, nonresponse, and integrated weighting of the 2016 IAB-BAMF-SOEP survey of refugees (M3/M4) - revised version. SOEP Survey Papers 477: Series C. Berlin: DIW/SOEP.

Krosnick, J. A., and D. F. Alwin. 1987. An evaluation of a cognitive theory of response-order effects in survey measurement. Public opinion quarterly 51(2):201-19.

Kuriakose, Noble, and Michael Robbins. 2016. Don't get duped: Fraud through duplication in public opinion surveys. Statistical journal of the IAOS 32(3):283-91.

Li, J., J. M. Brick, B. Tran, and P. Singer. 2009. Using statistical models for sample design of a reinter- view program. Proceedings of the survey research method section, 4681-95.

Menold, N., and C. J. Kemper. 2014. How do real and falsified data differ? Psychology of survey response as a source of falsification indicators in face-to-face surveys. International journal of public opinion research 26(l):41-65.

Menold, N., P. Winker, N. Storfinger, and C. J. Kemper. 2013. A method for ex-post identification of falsification in survey data. In Interviewers' deviations in surveys, ed. P. Winker, N. Menold, and R. Porst, 25-47. Frankfurt am Main: Internationaler Verlag der Wissenschaften.

Messick, S. 1966. The psychology of acquiescence: An interpretation of research evidence. ETS research bulletin series (l):i—44.

Milligan, G. W., and M. C. Cooper. 1985. An examination of procedures for determining the number of clusters in a data set. Psychometrika 50(2):159-79.

Murphy, J., R. Baxter, J. Eyerman, D. Cunningham, and J. Kennet. 2004. A system for detecting interviewer falsification. Proceedings of statistics Canada symposium, 2005.

Porras, J., and N. English. 2004. Data-driven approaches to identifying interviewer data falsification: The case of health surveys. Proceedings of the survey research method section, 4223-8.

Reuband, К. H. 1990. Interviews, die keine sind: "Erfolge" und "MiBerfolge" beim Falschen von Interviews. Kolner Zeitschrift fur Soziologie und Sozialpsychologie 42:706-33.

Sarracino, E, and M. Mikucka. 2016. Estimation bias due to duplicated observations: A Monte Carlo simulation. Munich personal RePEe archive, 1-18. https://mpra.ub.uni-muenchen.de/69064/!/ MPRA_paper_69064.pdf (accessed November 30,2019).

Schafer, C, J. P. Schrapler, and K. R. Muller. 2004. Identification, characteristics and impact of faked and fraudulent interviews in surveys. Discussion Paper.

Schnell, R. 1991. Der EinfluB gefalschter Interviews auf Survey-Ergebnisse. Zeitschrift fir Soziologie 20:25-35.

Schnell, R. 2012. Survey-interviews: Methoden standardisierter Befragungen. Wiesbaden: VS Verlag fur Sozialwissenschaften.

Schrapler, J. P, and G. G. Wagner. 2005. Characteristics and impact of faked interviews in surveys: An analysis of genuine fakes in the raw data of SOEP. Allgemeines statistisches archiv 89:7-20.

Schreiner, I., K. Pennie, and J. Newbrough. 1988. Interviewer falsification in Census Bureau surveys. Proceedings of the survey research method section, 491-96.

Simmons, K., A. Mercer, S. Schwarzer, and C. Kennedy. 2016. Evaluating a new proposal for detecting data falsification in surveys. Statistical journal of the IAOS 32(3):327-38.

Slomczynski, К. M., P. Powalko, and T. Krauze. 2017. Non-unique records in international survey projects. Survey research methods 11(1):1—16.

Stokes, S. L., and P. Jones. 1989. Evaluation of the interviewer quality control procedure for the postenumeration survey. Proceedings of the survey research method section, 696-98.

Storfinger, N., and P. Winker. 2013. Assessing the performance of clustering methods in falsification using bootstrap. In Interviewers' deviations in surveys, ed. P. Winker, N. Menold, and R. Porst, 46-65. Frankfurt am Main: Internationaler Verlag der Wissenschaften.

Swanson, D., M. Cho, and J. L. Eltinge. 2003. Detecting possibly fraudulent or error-prone survey data using Benford's Law. Proceedings of the survey research method section, 4172-77.

Tourangeau, R., F. Kreuter, and S. Eckman. 2012. Motivated underreporting in screening interviews. Public opinion quarterly 76(3):453-69.

Turner, С. E, J. N. Cribble, A. A. Al-Tayyib, and J. R. Chromy. 2002. Falsification in epidemiologic surveys: Detection and remediation. Technical papers on health and behavior measurement 53:1-12.

Winker, P. 2016. Assuring the quality of survey data: Incentives, detection and documentation of deviant behavior. Statistical journal of the IAOS 32(3):295-303.

Winker, P, K. W. Kruse, N. Menold, and U. Landrock. 2015. Interviewer effects in real and falsified interviews. Statistical journal of the IAOS 31(3):423-34.

Ziegler, M., C. J. Kemper, and B. Rammstedt. 2013. The Vocabulary and Overclaiming Test (VOC-T). Journal of individual differences 34:32-40.