Estimands, Assumptions, and the Challenges of Evaluating Policies: Lessons from Staggered Adoption

Abstract:
Evaluating the impact of policies and programs, especially in health, is a major goal of statistics, requiring both careful study design and data analysis methods. Causal inference through staggered adoption or roll-out has become a common approach. This can occur in a randomized study (a stepped-wedge design) or an observational study (staggered adoption difference-in-differences, among other approaches). In both cases, careful attention needs to be paid to the estimand of interest and the assumptions about treatment effect heterogeneity. This goes beyond exchangeability into two less discussed aspects of causal inference: positivity and consistency. In this talk, I will explore what stepped-wedge trials can teach us about these important questions. I describe one analysis method that approaches stepped-wedge trials from an estimand- and assumption-first framework. I then connect it to observational studies through the target trial emulation framework. Finally, I will give some general lessons, warnings, and food for thought that we can draw from these areas and should consider when evaluating policy impacts.
Lee Kennedy-Shaffer is an Assistant Professor (Educator-Scholar Track) in Biostatistics. Lee received his PhD in Biostatistics under Dr. Michael Hughes in the Harvard T.H. Chan School of Public Health and conducted epidemiologic research there with Drs. Marc Lipsitch and Michael Mina in the Center for Communicable Disease Dynamics. He was an Assistant Professor in the Vassar College Department of Mathematics and Statistics from 2020–2024.
His research focuses on randomized and observational study designs and methods for the analysis of infectious disease interventions. This includes mathematical modeling, cluster-randomized trials, and quasi-experimental designs, all with an eye toward broader population health impacts than are usually addressed by individually randomized trials. He has worked on COVID-19 data collection and analysis as well, in particular accounting for the timing and correlation of infections in interpreting test results. This work has been published in journals such as Science, Statistics in Medicine, Clinical Trials, the American Journal of Epidemiology, and the American Journal of Public Health, among others. In addition, he has written on the history of statistics, FDA policy, statistics education, and causal inference in baseball.
Time and date
Thursday 12 March
9:30-10:30am AEDT (UTC+11)
This seminar will be held via Zoom and will be recorded. Please click this URL to join:
https://monash.zoom.us/j/81840665216?pwd=H6xcf0fBWja7Gvv0sUc5YoSlQgah5t.1
Or, go to https://monash.zoom.us/join and enter
Meeting ID: 818 4066 5216
Passcode: 000690
