‘Do Events Change Minds or Samples? A Framework for Identifying Compositional Bias in Event Focused Causal Inference’ with Klara Müller (Mannheim)

23 October – 12:00 to 13:00

From political psychology, we know that major political events can shape attitudes, emotions, and behaviours. Survey methodology also finds that external shocks can affect who responds to surveys. This talk brings these insights together, proposing that political events may influence not only public opinion and behaviour but also patterns of survey participation. If an event changes who takes part in a survey, and these shifts are related to political outcomes of interest, especially through unobserved characteristics, then estimates of causal effects may be biased. Drawing on the increasingly popular design of unexpected events during survey fieldwork, I introduce a framework to formally disentangle the causal effect of an event from what I term compositional bias, which refers to bias arising from shifts in the survey sample following the event. I present strategies to adjust for bias based on observable characteristics and extend recent sensitivity analysis methods to assess how strong unobserved confounders would need to be to substantively bias these estimates. I demonstrate the framework using the well-studied case of the rally around the flag effect after the 2015 Charlie Hebdo terrorist attacks in France, showing how sample changes may distort conclusions about the public's response. By addressing these challenges, the framework strengthens causal inference in event focused research and improves the measurement of public opinion and political behaviour in dynamic political contexts.