Question Order Experiments in the German-European Context

Abstract

In this paper, we investigate the context stability of questions on political issues in cross-national surveys. For this purpose, we conducted three replication studies (N1 = 213; N2 = 677; N3 = 1,489) based on eight split-ballot design experiments with undergraduate and graduate students to test for question order effects. The questions, which were taken from the Eurobarometer (2013), included questions on perceived performance and identification. Respondents were randomly assigned to one of two experimental groups which received the questions either in the original or the reversed order. In all three studies, respondents answered the questions about Germany and the European Union/Europe differently depending on whether the question was asked first or second in the question sequence. Specifically, when answering a subsequent question in a question sequence, the preceding question seems to have functioned as a standard of comparison. Our empirical findings also suggest that the likelihood of the occurrence of such context effects can be reduced by implementing informed questionnaire design strategies.