Andy Timm writes: I’m curious if you have any suggestions for dealing with item nonresponse when using MRP. I haven’t seen anything particularly compelling in a literature review, but it seems like this has to have come up. It seems like a surprisingly large number of papers just go for a complete cases analysis, or […]
Archive of posts filed under the Causal Inference category.
Fun example of an observational study: Effect of crowd noise on home-field advantage in sports
Kevin Quealy and Ben Shpigel offer “Four Reasons the N.F.L. Shattered Its Scoring Record in 2020”: No. 1: No fans meant (essentially) no home-field advantage With fans either barred or permitted at diminished numbers because of public-health concerns, the normal in-game din dropped to a murmur or — at some stadiums — to a near […]
Many years ago, when he was a baby economist . . .
Jonathan Falk writes: Many years ago, when I was a baby economist, a fight broke out in my firm between two economists. There was a question as to whether a particular change in the telecommunications laws had spurred productivity improvements or not. There a trend of x% per year in productivity improvements that had gone […]
Estimating the college wealth premium: Not so easy
Dale Lehman writes: Emmons_Kent_Ricketts_College_Still_Worth_ItHere’s the article referenced on Marginal Revolution today. I thought it might be of interest and worth blogging about. It is quite thorough and fairly complex. The results are quite striking – and important. My big concern relates to a critical variable – financial literacy. On page 14 they claim that it […]
The accidental experiment that saved 700 lives
Paul Alper sends along this news article by Sarah Kliff, who writes: Three years ago, 3.9 million Americans received a plain-looking envelope from the Internal Revenue Service. Inside was a letter stating that they had recently paid a fine for not carrying health insurance and suggesting possible ways to enroll in coverage. . . . […]
Claim of police shootings causing low birth weights in the neighborhood
Under the subject line, “A potentially dubious study making the rounds, re police shootings,” Gordon Danning links to this article, which begins: Police use of force is a controversial issue, but the broader consequences and spillover effects are not well understood. This study examines the impact of in utero exposure to police killings of unarmed […]
Statistical fallacies as they arise in political science (from Bob Jervis)
Bob Jervis sends along this fun document he gives to the students in his classes. Enjoy. Theories of International Relations Assume that all the facts and assertions in these paragraphs are correct. Why do the conclusions not follow? (This does not mean that the conclusions are actually false.) What are the alternative explanations for the […]
Smoking and Covid
Paul Kedrosky wrote: This paper is getting passed around today, with its claim that there not only isn’t a causal relationship between smoking and COVID, but possibly a protective role. This sort of thing drives me crazy about pre-prints. If your data suggests a conclusion that runs counter to decades of prior work with better […]
Fake data simulation: Why does it work so well?
Someone sent me a long question about a complicated social science problem involving intermediate outcomes, errors in predictors, latent class analysis, path analysis, and unobserved confounders. I got the gist of the question but it didn’t quite seem worth chasing down all the details involving certain conclusions to be made if certain affects disappeared in […]
This one pushes all my buttons
August Wartin writes: Just wanted to make you aware of this ongoing discussion about an article in JPE: It’s the same professor Lidbom that was involved in this discussion a few years ago (I believe you mentioned something about it on your blog). Indeed, we blogged it here. Here’s the abstract of Lidbom’s more recent […]
How to reconcile that I hate structural equation models, but I love measurement error models and multilevel regressions, even though these are special cases of structural equation models?
Andy Dorsey writes: I’m a graduate student in psychology. I’m trying to figure out what seems to me to be a paradox: One issue you’ve talked about in the past is how you don’t like structural equation modeling (e.g., your blog post here). However, you have also talked about the problems with noisy measures and […]
Will the pandemic cause a decline in births? We’ll be able to resolve this particular debate in about 9 months . . .
The fallacy of the one-sided bet I’m gonna be talking about a news article and research paper asking the question, “Will coronavirus cause a baby boom, or is that just a myth?” And my problem is the fallacy of the one-sided bet: By asking the question, is there a positive effect or is it zero, […]
What about that new paper estimating the effects of lockdowns etc?
A couple people pointed me to this article, “Assessing Mandatory Stay‐at‐Home and Business Closure Effects on the Spread of COVID‐19,” which reports: The most restrictive non‐pharmaceutical interventions (NPIs) for controlling the spread of COVID‐19 are mandatory stay‐at‐home and business closures. . . . We evaluate the effects on epidemic case growth of more restrictive NPIs […]
“They adjusted for three hundred confounders.”
Alexey Guzey points to this post by Scott Alexander and this research article by Elisabetta Patorno, Robert Glynn, Raisa Levin, Moa Lee, and Krista Huybrechts, and writes: I [Guzey] am extremely skeptical of anything that relies on adjusting for confounders and have no idea what to think about this. My intuition would be that because […]
Does regression discontinuity (or, more generally, causal identification + statistical significance) make you gullible?
Yes basically. This one’s pretty much a perfect example of overfitting, finding a discontinuity out of noise, in that if you just draw a smooth line through each graph, it actually looks better than the discontinuous version. We see this a lot: There’s no discontinuity in the data, but it’s possible to make a discontinuity […]
No, this senatorial stock-picking study does not address concerns about insider trading:
Jonathan Falk writes: As you have tirelessly promoted, a huge problem with NHST is that “insignificant” effects on average can mask, via attenuation bias, important changes in subgroups. Further, as you have somewhat less tirelessly pointed out, you need much bigger samples to reliably see anything in subgroups, particularly when (ok.. you’re back to your […]
Flaxman et al. respond to criticisms of their estimates of effects of anti-coronavirus policies
As youall know, as the coronavirus has taken its path through the world, epidemiologists and social scientists have tracked rates of exposure and mortality, studied the statistical properties of the transmission of the virus, and estimated effects of behaviors and policies that have been tried to limit the spread of the disease. All this is […]
More on the Heckman curve
David Rea writes: A slightly more refined version of our paper on the Heckman Curve [discussed on blog last year] has been published in the Journal of Economic Surveys. The journal will also publish a response by James Heckman, as well as a reply from us. As you predicted, James Heckman’s critique of our work […]
“Inferring the effectiveness of government interventions against COVID-19”
John Salvatier points us to this article by Jan Brauner et al. that states: We gathered chronological data on the implementation of NPIs [non-pharmaceutical interventions, i.e. policy or behavioral interventions] for several European, and other, countries between January and the end of May 2020. We estimate the effectiveness of NPIs, ranging from limiting gathering sizes, […]
Debate involving a bad analysis of GRE scores
This is one of these academic ping-pong stories of a general opinion, an article that challenges the general opinion, a rebuttal to that article, a rebuttal to the rebuttal, etc. I’ll label the positions as A1, B1, A2, B2, and so forth: A1: The starting point is that Ph.D. programs in the United States typically […]