Again in 2012 I wrote concerning the fundamental 2 x 2 distinction in distinction evaluation (two teams, two time durations). Columbia public well being in all probability has a higher introduction.
Essentially the most well-known instance of an evaluation that motivates a 2 x 2 DID evaluation is John Snow’s 1855 evaluation of the cholera epidemic in London:
(Picture Supply)
I’ve since written about a number of the challenges of estimating DID with glm fashions (see right here, right here, and right here.), in addition to combining DID with matching, and issues to be careful for when combining strategies. However loads of what we find out about distinction in variations has modified within the final decade. I will attempt to give a quick abstract based mostly on my understanding and level in the direction of some references that do a greater job presenting the present state.
The Two-Means Mounted Results mannequin (TWFE)
The very first thing I ought to talk about is extending the 2×2 mannequin to incorporate a number of handled teams and/or a number of time durations. The generalized mannequin for DiD additionally known as the two-way fastened results (TWFE) mannequin is one of the simplest ways to characterize these type of eventualities:.
Ygt = ag + bt + δDgt + εt
ag = group fastened results
bt = time fastened results
Dgt= therapy*publish interval (interplay time period)
δ = ATT or DID estimate
Getting the proper commonplace errors for DID fashions that contain many repeated measures over time and/or the place therapy and management teams are outlined by a number of geographies presents two challenges in comparison with the essential 2×2 mannequin. Serial correlation and correlation inside teams. There are a number of approaches that may be thought of relying in your scenario.
1 – Block bootstrapping
2 – Aggregating knowledge into single pre and publish durations
3 – Clustering commonplace errors on the group degree
Clustering on the group degree ought to present the suitable commonplace errors in these conditions when the variety of clusters are massive.
For extra particulars on TWFE fashions, each Scott Cunningham and Nick Huntington-Klein have nice econometrics textbooks with chapters devoted to those subjects. See the references under for more information.
Differential Timing and Staggered Rollouts
However issues can get much more sophisticated with DID designs. Take into consideration conditions the place there are totally different teams getting handled at totally different instances over quite a lot of time durations. This isn’t only a thought experiment making an attempt to think about probably the most tough research design and pondering for the sake of pondering – these type of staggered rollouts are quite common in enterprise and coverage settings. Think about coverage guidelines adopted by totally different states over time (like adjustments in minimal wages) or think about testing a brand new services or products by rolling it out to totally different markets over time. Understanding consider their influence is essential. For some time it appeared economists could have been slightly responsible of handwaving with the TWFE mannequin assuming the estimated therapy coefficient was giving them the impact they needed.
However Andrew Goodman-Bacon refused to take this interpretation at face worth and broke this down for us figuring out that the TWFE estimator was making an attempt to provide us a weighted common of all potential 2×2 DID estimates you would make with the information. That really sounds intuitive and useful. However what he found that’s not so intuitive is that a few of these 2×2 comparisons could possibly be evaluating beforehand handled teams with present handled teams. That is not a comparability we usually are fascinated by making, however it will get averaged in with the others and might drastically bias the outcomes significantly when there’s therapy impact heterogeneity (the therapy impact is totally different throughout teams and trending over time).
So how do you get a greater DID estimate on this scenario? I will spare you the main points (as a result of I am nonetheless wrestling with them) however the reply appears to be the estimation technique developed by Callaway and Sant’Anna. The documentation in R for his or her bundle walks by loads of the main points and challenges with TWFE fashions with differential timing.
Moreover this video of Andrew Goodman-Bacon was actually useful for understanding the ‘Bacon’ decomposition of TWFE fashions and the issues above.
After watching Goodman-Bacon, I like to recommend this discuss from Sant’Anna discussing their estimator.
Under Nick Huntington-Klein gives an important abstract of the problems made obvious by the Bacon decomposition made above and the Callaway and Sant’Anna methodology for staggered/rollout DID designs. he additionally will get into the Wooldridge Mundlack strategy:
A Be aware About Occasion Research
In quite a lot of references I’ve tried to learn to know this challenge, the time period ‘occasion research’ is thrown round and it looks as if each time it’s used it’s used otherwise however the creator/speaker assumes we’re all taking about the identical factor. On this video Nick Huntington-Klein introduces occasion research in a manner that’s the most clear and constant. Watching this video may assist.
References:
Causal Inference: The Mixtape. Scott Cunningham. https://mixtape.scunning.com/
The Impact: Nick Huntington-Klein. https://theeffectbook.web/
Andrew Goodman-Bacon. Distinction-in-differences with variation in therapy timing. Journal of Econometrics.Quantity 225, Challenge 2, 2021.
Brantly Callaway, Pedro H.C. Sant’Anna. Distinction-in-Variations with a number of time durations. Journal of Econometrics. Quantity 225, Challenge 2, 2021,
Associated Posts:
Modeling Claims Prices with Distinction in Variations. https://econometricsense.blogspot.com/2019/01/modeling-claims-with-linear-vs-non.html
Was It Meant to Be? OR Generally Enjoying Match Maker Can Be a Dangerous Thought: Matching with Distinction-in-Variations. https://econometricsense.blogspot.com/2019/02/was-it-meant-to-be-or-sometimes-playing.html
