Skip to content

Hilda Bastian and John Ioannidis on coronavirus decision making; Jon Zelner on virus progression models

1. Hilda Bastian writes:

Doing nothing for which there is no strong evidence is doing something: it’s withholding public health interventions that, on the balance of what we know, could save a lot of lives and trauma – including the lives of a lot of healthcare workers.
Secondly, the need for societies to be able to monitor the impact is an argument for putting more effort into monitoring. Weaknesses in that is not a reason to not act. . . .

We do not know “the” case fatality rate, but that won’t be the same everywhere, dependent as it is on regional differences like health system capacity and levels of antibiotic resistance for secondary pneumonia. And while it means best and worst case scenarios are far apart, that does not of itself give best case scenarios greater weight. . . .

Could there be fiascos from over-reaction? Yes, there could, but several countries have introduced measures that are draconian, appear to have pegged outbreaks back, and are loosening the measures. Could there be fiascos from under-reaction? Well, we already have some of those. . . . there is broad consensus that this is a public health emergency, and we have to take action, not just sit there studying the situation and waiting for better information before acting. I think the stakes are too high to ignore the public health community urging us to act in favor of a “hot take” from someone who doesn’t seem to have done his homework.

She was clarifying some issues raised by this post from John Ioannidis, who wrote:

At a time when everyone needs better information, from disease modelers and governments to people quarantined or just social distancing, we lack reliable evidence on how many people have been infected with SARS-CoV-2 or who continue to become infected. Better information is needed to guide decisions and actions of monumental significance and to monitor their impact. . . . Three months after the outbreak emerged, most countries, including the U.S., lack the ability to test a large number of people and no countries have reliable data on the prevalence of the virus in a representative random sample of the general population. . . .

When reading this article I automatically translated “representative random sample” to “representative sample” (where the representativeness could be be achieved using poststratification). There are lots of ways to get a representative sample.

2. I’ve been talking a bit with Jon Zelner about coronavirus progression models. Zelner writes:

I think in some ways that the whole transmission modeling approach, when done well, shares a lot of DNA w/MRP [multilevel regression and poststratification, also known as Mister P]. Like this paper, Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV2), by Ruiyun Li, Sen Pei, Bin Chen, Yimeng Song, Tao Zhang, Wan Yang, and Jeff Shaman that just came out—one of the times where I think we can say the material is indeed tabloid-worthy—directly models both the transmission process and the observation process that together give rise to hard-to-interpret patterns of who is observed testing positive for coronavirus.

One of the things that makes these nonlinear/diff-eq models kind of interesting in this way is also that sometimes you can’t explain the geometry of the epidemic curve with just the observed data, i.e. the peak is too high to be explained by just the small fraction of observed cases. So modeling the dynamic process actually informs the estimates of the age-specific reporting rates, etc.

You know that saying, There’s nothing so practical as a good theory? That’s what’s going on here. Latent-variable models are necessary both to understand the process and to make predictions about observables.

See here, here, and here for more on these models.


  1. Well, we are back to the debate implicating the ‘precautionary principle’. Right? The debate took some very interesting turns in the debate over GMO foods and Monsanto Pesticide use

    • Rahul says:

      Somehow next to covid, GMO and Monsanto seem so trivial and distant now.

      So does all that hullabaloo over global warming.

      • Otto says:

        The hullabaloo about global warming will come back. For example, remember that the heatwaves can be quite deadly: “More than 70,000 additional deaths occurred in Europe during the summer 2003.”

        • jim says:

          Extreme cold kills twice as many people as extreme heat in the US every year – even though the states that experience extreme cold have very low population densities, while states that experience extreme heat have intermediate to high population densities.

          Almost all of Canada’s population is within a few hundred miles of it’s southern border.

          • Phil says:

            I don’t think anyone really knows how many deaths per year are caused by either extreme heat or extreme cold. See for example.

            There are lots of other negative effects of global warming, including death from reasons other than overheating. Global warming as an issue is not going away.

            • Rahul says:


              If there’s a cause and no one really even feels the acute effects of death from it (like you mention for golbal warning) then I am reasonably content not to look at it (in the order of priorities).

              I’d rather focus on something like a Pandemic where the cause of death can be more tightly linked and work on mitigating such causes.

              • Compared to the current Pandemic, the typical level of air pollution in the LA basin is not a major deal. But of course, given enough time, it shortens a lot of lives, causing heart attacks, strokes, asthma, etc. The difference in air quality now compared to 25 years ago when I toured Harvey Mudd College to see if I wanted to go there is dramatic. The two days I was there the ozone was so high it burned my lungs within minutes, and the school was calling for all students to stay indoors. It was a considerable detractor for the school.

                But it’s hard to link air pollution to death in a direct quantifiable way. Still, I’m quite sure it’s an important factor and we should do something about it.

                The big problem with global warming is knowing whether any given “cure” is better or worse than the disease. Many people in 3rd world countries die each year due to being basically poor: malnutrition, bad roads, unsafe transportation, disease, etc. Cutting back on our use of energy resources potentially cuts back on development which harms these people. The problem with global warming isn’t that it doesn’t kill people, it’s that fixing it also potentially kills people, and without quantification of all these things, it’s hard to know which actions to take.

                We keep having the same old bullshit debate about *whether* global warming is a thing… that’s the wrong debate, but it’s the debate people like, because they don’t have to talk about their values and priorities. The real debate is: what are the costs and benefits? To understand how fraught that is, consider that there are horrible racist people who think “a bunch of brown people in india who are taking our jobs will die” is a *benefit*

              • Rahul says:


                I think it’s about proximate effects. Problems spread out over time people can deal better with. Societies can adjust.

                e.g. Assume in some weird sense we moved to a world with 2% additional mortality related to some new disease but over a period of next 10 years. I think we could cope with it much better.

                Hell, it may not be entirely bad to live in a world with say 10% less people. The quality of the new equilibrium is not the problem. It’s the transitions that are difficult and the more rapid the change the more traumatic it is.

                That’s my point. Worry more about acute causes.

              • I agree that society can adapt better to slower changes, but I still think if people are suffering and dying we should take actions even if they’re not immediate, proximate, and acute. I do agree though that when an opportunity exists to quickly hammer down a single proximate cause of mayhem we should do that first and quickly.

              • Rahul says:

                I would love to know how much the US Government (for example) spent over the last decade on (say) global warming research versus Epidimiology / Virology of Pandemics.

                My hunch is the ratio could be scandalously skewed by most rational standards.

              • I suspect you’re absolutely right.

              • Zhou Fang says:

                Both are tiny relative to bloated military budgets fighting terrorism.

              • Anonymous says:

                Daniel Said:

                “The real debate is: what are the costs and benefits?”

                Excellent. +10

                But we can’t answer this until we making falsifiable forecasts: What is the probability of x% temp increase by x year? What is the probability of x% sea level rise by x year?

            • jim says:

              I don’t think we should make policy on the basis of “forecasts” or “predictions” that are unfalsifiable and more than half a century in the future.

              • One of the major problems is that a good many experts are didactic and are compelled to find fault with their rivals. Then followers are mandated to follow their leaders. It frustrates me since childhood. So many academics would be anxious about their prestige. It was a turn off for me. I admire someone who keeps in chill.

              • Zhou Fang says:

                “Falsifiability” is a canard. You can’t (or at least shouldn’t) “falsify” the claim that if you jump in front of a bus, you’d die. That conclusion instead comes out of a logical extrapolation from known facts.

                Any policy making is going to be based on some implicit forecast or predictions in any case.

              • Phil says:

                I don’t think putting things in “quotes” somehow makes them “wrong”. I also don’t see how your “statement” is morally justifiable.

                Here, you can read about water pollution from mines that closed more than 50 years ago and that is harming people today:

                51 years ago some guy named jim was probably saying everything was fine and that anyone who “forecast” that the mines would cause problems down the road was making unfalsifiable statements about things that might happen more than 50 years in the future, and that they shouldn’t make policy on things like that.

              • Jon H says:

                “I don’t think we should make policy on the basis of “forecasts” or “predictions” that are unfalsifiable and more than half a century in the future.”

                They’re not unfalsifiable. The forecasts and predictions of 10, 20, 30, 40, 50 years ago can be compared against the real world conditions of today and the conditions over time leading up to today.

              • jim says:

                “You can’t (or at least shouldn’t) “falsify” the claim that if you jump in front of a bus, you’d die.”

                I think we can successfully generalize from the existing evidence.

                If a forecast doesn’t have exact data and isn’t within a reasonable time frame, it’s useless.

              • Zhou Fang says:

                So we generalise from existing evidence with climate predictions.

              • jim says:

                Phil Said: “51 years ago some guy named jim was probably saying…”

                I wrote several rebuttals and they’re all so good it’s hard to choose! :) But I guess I’ll keep it simple.

                For the damage caused by mining, humanity is a thousand time better off. And for the damage caused by energy development, easily a million times better off.

              • jim says:

                “So we generalise from existing evidence with climate predictions”

                If we generalize from existing evidence about disaster predictions our examples would be Paul Ehrlich’s population bomb, Peak Oil, the End of Growth….none of which materialized.

                If I had to guess right now I’d say climate models are accurate – temps are falling on the low end of the range. That’s the evidence we have right now. But the external temperature doesn’t predict heat deaths. Just like the “ecological carrying capacity” doesn’t predict the population or food supply. technology intervenes.

              • Phil says:

                jim, You had a bunch of great rebuttals and that’s the one you chose?! It doesn’t even address the point of my example! Actions now can have negative consequences in the future. Sure, OK, they can also have positive consequences in the future, but the point is that actions now have future consequences. They really do.

                Here’s a thought experiment: you have a button that will make you fabulously wealthy if you push it. But there’s a 10% chance it will kill 10 million people. Is it morally acceptable to push the button? If I understand you correctly, your answer is No if those people would die immediately, but Yes if the button would (with 10% probability) kill 10 million people in 51 years. I think that’s an immoral position.

          • Navigator says:

            You may want to double-check that. Sure, hypothermia is a problem, but physiologically, humans have no protection in extreme heat, beyond sweating.

            When extremely cold, shelter, clothing, fires, etc. are good viable options, but when over 42C, the brain suffers greatly.

            It is possible that more people die in winter when viral infections abound, but you rarely hear about mass freezing of people, even in poor countries (unless they are climbing up Mt. Everest).
            You do, however, hear about extreme heat waves and many fatalities in a short period of time.
            The absolute number may not matter as much as over what period fatalities occur. Winters are long, but heat waves are short and intense.

            • jim says:

              “physiologically, humans have no protection in extreme heat, beyond sweating. ”


              Clothing and fire that keep you warm are no different than shade and fans and air conditioning that keep you cool. It’s all technology. Naked bushmen have no more protection against the cold than they have against the heat.

              “You do, however, hear about extreme heat waves and many fatalities in a short period of time.”

              Interestingly you can have heat deaths in a place like Seattle, where the highest temps are around average day time summer temps elsewhere in the country. The average summer day in Dallas is +20 degrees from the average summer day in Seattle; an extended period of +95 weather in Seattle is a major heat wave complete with heat warnings. In Dallas, it’s ho hum.

      • Andrew says:


        Various bad things can interact in bad ways. Antibiotic resistance, viral outbreaks, floods, etc., don’t go well together. 4 horsemen and all that.

        • Rahul says:


          Maybe. But here’s a bad guy right at the door (this pandemic) and here we are caught napping and agonizing about some distant danger (e.g. Global Warming).

          My point is that we have mostly got our priorities wrong. We should have been worrying about more proximate dangers.

          It’s like a morbidly obese guy obsessed with eating organic food. You are more likely to die of cardiac arrest than residual pesticides.

          • Zhou Fang says:

            Global warming is part of the same picture as infectious pandemics. Increased severity of pandemic disease is one of many consequences of global warming. Picture this current crisis and add a refugee and food supply crisis on top. It wouldn’t be pretty.

            Generally speaking the core solution also shares similar elements. Specifically, international institutions to deal with these cross border threats. I consider the reaction to this crisis to be a fairly good model for eventual outcomes under global warming. (i.e. the cycle of it’s a hoax, to finger pointing.)

            • Rahul says:

              It’s about prioritisation and optimal allocation of resources.

              Sure all the threats you mention are credible. My point is are we allocating enough reaources to say pandemics vs say global warming.

              • Zhou Fang says:

                We aren’t allocating enough resources to both global warming and pandemics.

              • Zhou Fang says:

                About 0.2% of the federal budget is spent on climate change related activities, of which 94% go to programs like nuclear energy that aren’t primarily about climate change. The CDC’s budget is about 0.14% of the GDP, though about 42% of that is “protecting americans from infectious disease or ensuring global disease protection”. So, uh, the US actually spends more on pandemics than global warming.

                But why choose global warming as a point of comparison?

              • Zhou Fang says:

                Er, I meant 0.14% of the federal budget.

                Of course then you add private research spending on top, and I’d wager that if you start including pharma company budgets climate change research would look even tinier.

      • That is one take Rahul.

        There are many highly intelligent experts who would disagree that the Monsanto case has led to trivial effects.

        I was pointing to the application of the precautionary principle as a function of trying to compute also the costs and benefits of adopting any one course or multiple course responses.

        To be fair, John Ioannidis, in that essay, was making an excellent point in his reference to the autopsy series:

        ‘In an autopsy series that tested for respiratory viruses in specimens from 57 elderly persons who died during the 2016 to 2017 influenza season, influenza viruses were detected in 18% of the specimens, while any kind of respiratory virus was found in 47%. In some people who die from viral respiratory pathogens, more than one virus is found upon autopsy and bacteria are often superimposed. A positive test for coronavirus does not mean necessarily that this virus is always primarily responsible for a patient’s demise.’

        • Steve says:

          I understand the point Ioannidis was making, but it is a little silly in the present context. The deaths from COVID-19 that are being recorded in China, Italy and now here are clearly cases where the virus was the direct result of the death. It is not just one of many illnesses that a person dying happened to have. I understand his point that we don’t have good numbers and in situations with co-morbidities, it is not always straightforward to say what the cause was, but come on, are we suppose to ignore our eyes. At some point, qualitative analysis is just fine.

          • Hi Steve,

            Thanks for your thoughts. I speculate that we would need access to the pathology reports, which are not in the public information pipeline as of yet. A major takeaway for me is those elderly people with one or more co-morbidities are most vulnerable. The immune system compromised from the get-go.

            To an extent, many compromise their health by drinking, eating, and smoking. Sleep too is compromised.

            I base my state of immunity hypothesis in watching a briefing with Chinese Doctors that Harlan Krumholz arranged.


            The entire Twitter thread that Harlan Krumholz posted is quite interesting.

            I also access Peter Attia MD, who also is an insightful physician. His specialty immunology.


            What more can extrapolate from the information in the Autopsy Series? Anything? How would we know whether iatrogenic effects from the administration of medicine treatments played a role in death? It seems that we would need the pathology reports in any case.

    • Zhou Fang says:

      You don’t need a ton of faith in the precautionary principle to balance a linear-cost precaution against an exponential scale negative outcome.

      • Thank you, very succinctly put, saves me at least a thousand words 😉

        • Dale Lehman says:

          what makes you think the precaution costs are linear? That is not the way the economy works. 10% unemployment may be roughly twice the damage as 5%, but 20% we’ve only seen in the Great Depression (in the US) and that looks nonlinear to me.

          • The great depression as I understand it was a manufactured mess though. A major problem was the collapse of the money supply which took forever to recover and left prices out of whack. No one could usefully employ anyone because there wasn’t any money to do it with at prices that were commensurate with things like mortgages and loans and real-estate prices… Things that had a lot of illiquidity and therefore didn’t equilibriate even on the timescale of a decade. Basically, the great depression was a failure of mitigation.

            I am assuming that doesn’t happen.

            I would also very much welcome other takes on it though, it’s not like the great depression is an area I have a lot of expertise in.

            • Anoneuoid says:

              The financial/economic stuff going on is 1000x scarier than this virus to me…

              Prices have been way out of whack for a long time which is why wealth inequality has been growing. Now the bubble has been pricked and we are seeing the last ditch efforts of helicopter money, debt monetization, and negative interest rates.

              Just think about whats going to happen when people figure out they have to pay interest to keep money in the bank, and get paid interest to take out a mortgage/loan.

              Check out people like Peter Schiff and David Haggith. Many have been talking about this eventually happening for awhile.

              • Sure, I agree, but that was all *built in* there was no stopping it, this just triggered all the incredibly unhealthy financial stuff to happen now, I’ve been talking about that for several years. I think my blog was vandalized and I lost some of the posts I made back in ~ 2016 or so about the cashpocalypse.

              • Anoneuoid says:

                Sure, I agree, but that was all *built in* there was no stopping it, this just triggered all the incredibly unhealthy financial stuff to happen now,

                Yes, same with the people dying after contracting this virus. It was going to happen soon anyway.

              • jim says:

                “Prices have been way out of whack for a long time which is why wealth inequality has been growing. ”

                If you’re talking about manipulated prices, you must be referring to housing. Our prop tax has almost doubled since 2013 due to rising home values. Since the population is growing *way* faster than the housing supply, that’s not surprising. Why is the housing supply not growing very fast? Because it’s controlled by the county and cities, which stand to rake in piles of cash if they keep the housing supply tight. And even though our property tax has doubled in 7 years numerous new taxes have been passed in that time as well!

                Oh if I keep going on I’ll get myself kicked off here so I better stop!

              • Jim, not just housing but also interest rates (basically the price of money), which then reverberates across the entire price spectrum, and is also an additional contributor to bad housing prices.

                Beyond that there’s also the fact that more and more of what consumers are demanding is monopoly priced goods (copyrighted software, books, video and other entertainment, educational resources, as well as drugs and medical devices)

            • Dale Lehman says:

              All I can say is “this time is different” is a blueprint for missing a crisis.

              • True enough. But in my mind, the financial fallout was coming, and has been coming since…. about the dot-com bubble burst. That this triggered it just moves the consequences earlier by … months to a year or two… This is why in some private emails we’ve had back and forth I talked about the general unhealthiness of the economy. If you think we can actually mitigate those consequences by somehow allowing exponential growth of this virus… I’d like to see those calculations ;-)

                If you take the difference between what we’re doing now, and what would have happened if you do nothing or dramatically less than what we’re doing now at least… I can’t see any scenario where the do dramatically less comes out ahead.

                Basically things are going to be bad, and there’s nothing we can do about that… the best we can do is make things less bad, which set of actions do that? I don’t see 30 million people dead in the US as anything like acceptable or less costly, and I do think that’s the scale of the risk if we just let everyone get infected as fast as possible.

              • Dale, to talk more about my take on the financial/economic situation… I see in data like ACS and just driving around the streets of LA, or talking with families at my kid’s school, etc a large number of people who are suffering from serious issues, either living on the streets, kids who rely on schools for meals, families living in housing unstable situations, etc. So to people who may not be able to buy food by the end of the month, living paycheck to paycheck, their preference for cash now compared to cash a year from now is *very high*… indicating actual time-preference for money should put interest rates up in the say 10% range perhaps, in other words, above the historical average levels. On the other hand, the Fed has been suppressing bank rates for over a decade. So basically, price controls on money have screwed up the price structure of the whole economy.

                now, let’s look at what happens if/when the price structure resets. A few months ago, single family homes in my area were selling for basically a little over 1M. At 4% with 15% down, that’s $5400/mo payment (yikes!) now… If we fix the 5400/mo payment and raise the interest rate to 10%… with 15% down, house value drops to 600k.

                If house values are actually ~2x what the market clearing price without government intervention in the prices of money would make them… then what about the market values of things like Uber or any of those other highly speculative companies that lose money by the billions per year are at? They’re all predicated on distant future earnings that are basically “just as valuable” as earnings today… A false premise caused by price manipulation.

                The expectation is that if the general consumer preference / prices for future money were reflected properly in market interest rates, the value of assets would easily be cut in half. On this basis, I expect the stock market decline of about 25% over the last couple weeks is about half of the overall decline that is already built-in to the market manipulation of interest rates and we were riding for that fall eventually anyway.

                From a “real” perspective, the problem is that we’ve reallocated people’s productive activity to gaming the system and hoarding cash at the upper eschelons of the wealth spectrum. For decades people haven’t been doing a lot of the stuff that *consumers* really need them to be doing, whatever that is… The only way we will find out what that really is, is to get cash into the hands of consumers, who will spend it where the allocations are really needed. From this perspective UBI and the current cash handouts being discussed in congress / white house are exactly what’s needed to restore a stable accurate price structure (one that reflects preferences of real people, not of government favored corporate interests)

          • Also, Dale, as I say below, the proper analysis framework is a continuous time real-option pricing one… It’s not like “cancel everything forever” is our only option, at any time we can make decisions to relax restrictions by any amount. For *short* times, the cost should be linear in the time. I think “short” here probably includes about a couple months especially with things like govt backed cash/loans/equity positions in airlines and other high fixed cost industries that have to shut down.

            whereas for short times, the costs related to the virus were growing exponentially… even if costs related to the economy were to grow say quadratically or cubically in time, the exponential rise in infections would likely swamp those other economic issues.

            There are very few situations where something growing in badness *faster* than 2^(n/3) for n in days implies you should do anything other than shut it down as much as possible *TODAY*. (and I say faster than 2^(n/3) because that’s just the infection rate, the consequences get nonlinearly worse with infection number, as mitigation becomes impossible).

          • Zhou Fang says:

            Well, it’s a pretty offhand statement, when I wrote it I was thinking mostly in terms of the costs in terms of how long we do various measures. Closing a school for 6 days is about 20% worse than closing a school for 5 days, to some level of approximation. If we end up closing schools for months on end starting a few days “too early” isn’t going to matter so much.

            There could be potentially tipping points in the economy, sure. But I am skeptical that the sorts of decisions we are having to make actually straddles them.

            • Steve says:

              I guess the real issue is that if can’t stop the virus, will slowly moving in small steps toward herd immunity actually be better than getting there very rapidly. I think the answer is yes because overwhelming the healthcare system has so many additional impacts that are probably nearly impossible to estimate but are almost all negative. If we can stop through these steps, then it seems the cost is worth it.

              Ioannidis had a good point about how we are basically flying blind right now. If we really had more testing, we might be able to be more targeted. But, Ioannidis lost me when he talked about all of the data used to estimate the CFR being biased. I’m not sure what that means in this context. What would unbiased data look like. The CFR isn’t a feature of the virus. It is the result of how the virus interacts with the population, which groups it spreads in, what the state of medical care is, how early people get treated, how much viral load people get exposed to, and so on. There is no “true” CFR.

              • Exactly, blow through the whole population in 2 months, and CFR will fly through the roof, not to mention that no-one would get medical care for any other illness or injury either. Given that singapore who has good control over the virus and no overloaded healthcare is seeing ~10-15% of infections require critical care, if 50% of the population has it the week of say April 26, then in the US ~17M people will need critical care *that week* and we can provide probably 300k critical cares if we ramp up to double capacity by then. So assume on the order of 20M people will die *that week* in the US (or a 2 week period or even 3 week period, doesn’t matter, none of those people will get care).

                But the good news is that a week or two later, after 30M are dead, and the rest of us are hopefully immune (except there are documented cases of reinfection in Japan) the rest of us can “just go back to normal”.

                It’s not even close to an option on the table to “do nothing” or anything close. Right now the only options on the table are heavy social distancing, and total isolation of everyone… Italy tried the first and wound up at the second.

              • Steve says:

                Replying to Daniel, he said, “except there are documented cases of reinfection in Japan.” Virologists I follow say no, there is an immune response. The cases of so-called “reinfection” are just people who got false negatives, maybe the virus load fell below the level to be tested, then the went home and got sick again. Or maybe the test was just bad, but there is no reason to believe people don’t develop immunity once they recover.

              • Steve, it’s also possible they got sick from a different mutant strain. I mean, a different cold goes around every year, and even if you got last years version you often might get sick this year too…

                Also, some people will be immunocompromised, so there will always be a few strange cases, but yes, I do think there’s good reason to believe there’s immunity in most patients. we don’t know what will happen long term if there’s constant mutation of this virus like there is with other coronaviruses that commonly circulate.

      • Hi Zhou,

        We are constantly applying the ‘precautionary principle in nearly every life decision, whether well-evidenced or not.

        • Anon says:

          I’ve never been able to make proper sense of the precautionary principle. If we know the risks and can quantify them, we should just use costs, benefits and probabilities of each outcome and maybe maximise expected utility, so it doesn’t apply.

          What if we don’t understand the uncertainties well enough to assign them probabilities? A strict minimax approach seems paralysing. It’s possible that all sorts of actions lead to an enormous catastrophe. How do we distinguish silly precautions from legitimate precautions? How do know that the worst case of trying to stop the coronavirus, through its effect on economics, international relations, etc, is so much better than the worst case of less draconian attempts to combat the virus?

          It seems the precautionary principle needs a middle ground between an analysis in which we only consider whether outcomes are possible and then use criteria like minimax, and an analysis in which we quantify how probable outcomes are.

          • It’s been at least 14 years since I read Cass Sunstein’s book, Laws of Fear, Beyond the Precautionary Principle, which, for me at the time made a compelling case for re-evaluating its application in environmental and public health cases.

            Below is Sunstein’s paper:


            I have to reread it myself before commenting further.

            • Peter Dorman says:

              I had my own take on the PP, which I summarized in a piece in Ecological Economics a while back:


              The core idea is very simple: there is an approach to information-efficient regulation that parallels the efficient market hypothesis in financial markets. A regulatory threshold or other specific determination qualifies if there is no systematic bias toward revising toward stringency or laxity as new information arrives; new information should be truly “new”. A positive way to put it is that regulatory decision-making should take into account what we don’t know as well as what we do and make an effort to forecast the likelihood of future information that will constrain uncertainty in one direction or another. The PP is pro-regulation to the extent we inhabit a regime that hasn’t been accounting for the likelihood of future knowledge of environmental impact and has therefore been generally biased toward tightening regulation over time. (Net of political factors!)

              I taught Sunstein’s take for about 20 years. It was useful in challenging the somewhat unreflected eco-eagerness of many of my students, but it also struck me as rather facile.

              BTW, where are the proponents of cost-benefit analysis based on the “value of a statistical life” in the coronavirus discussion? Why aren’t we seeing the economic cost of proposed measures weighed against the monetary value of expected lives saved?

              • Steve says:


              • > BTW, where are the proponents of cost-benefit analysis based on the “value of a statistical life” in the coronavirus discussion? Why aren’t we seeing the economic cost of proposed measures weighed against the monetary value of expected lives saved?

                I’ve done this back of the envelope several times in comments here the last few days. The numbers aren’t even close, it’s obvious given the possibility of enormous numbers of people in critical care (tens of millions the same week) that QALY_lost * cost/QALY outweighs the entire GDP of the country by several multiples. We should close the whole world for 2 years no problem to avoid the kind of thing that’d happen in a “do nothing” scenario..

                Obviously some kind of “do something” scenario requires a specific epidemiology model of what happens under those scenarios, I just don’t have that available… but this is the kind of thing the CDC should have just sitting on their shelf ready to go. I suspect at some point they probably did. Politics probably eliminated that.

              • Peter Dorman says:

                Regarding cost-benefit balancing with VSL, remember that this is intended to apply to marginal impacts of specific actions. Total fatalities or QALY’s may be off the charts, but the issue is whether to, say, impose a lockdown — excuse me, shelter in place — rather than a milder quarantine and for two weeks or four weeks, etc. If you believe in this stuff you have to think the curves will cross somewhere, and the job of the Royal Economist is to tell us exactly where this happens.

              • Peter, no doubt, it’d be great to have some model based estimates of economic costs under different options driven by experts in epidemiology and virology and economics… Unfortunately it seems like no one has this sitting on their shelf, so we need time to produce it. Guess what buys us a bunch of time? Big shutdowns now.

                Furthermore, if we do too little, we lose the option to do anything but a LOT later, and also suffer big exponentially growing losses. Whereas if we do “too much” now, we preserve our option to relax it later when more information is available, the value of that option offsets the cost of doing too much today.

                All the calculations point to do as much as possible as soon as possible, and then with greater information in the near future, and improved testing, and improved care, and better more integrated models, relax the controls and monitor the situation carefully to control case counts to an economically efficient level so that we balance lives lost to disease with QALY and economic losses due to other factors.

                But play it wrong, and blow through a large population early, and you have no option, you suffer enormous losses of life and economics, and you are left with undoubtedly larger total costs than a controlled spread.

                It’s not even close… The problem seems to be that people are taking a binary approach… either do nothing, or do EVERYTHING FOREVER… but the truth is DO IT ALL NOW, and then when you have control, relax to the optimal level.

              • Thanks for the link to your article. Plus you ask two good questions there at the end.

              • Andrew says:


                I have not seen any value-of-life calculations regarding coronavirus. I’m sure that such analyses are out there, but I haven’t seen them. Someone did, however, recently point me to this article which does a formal dollars-lives tradeoff for assessing the cost of allowing leaded gasoline.

          • To me whatever the “precautionary principle” is the best most gracious interpretation I can give is it’s a heuristic designed to make bayesian integration easier… Basically pick a point out in the tail not too deep so that it’s ridiculously unlikely, calculate the costs associate with that, and balance them against the costs of “do nothing”… In that sense, properly thought out, I think it’s a perfectly fine approximation.

      • jim says:

        No one knows the relative cost of mitigation vs not mitigation or whatever.

        However, a pandemic is **WAY** different that global warming, GMO, or other civilization ending memes because the time scale is days or weeks vs decades and centuries. There is no comparison.

        On the other hand,

        “Doing nothing for which there is no strong evidence is doing something: it’s withholding public health interventions”

        Yet doing something for which there is no strong evidence is diverting resources from something else. The value of resources is amplified because of the short time frame. So if there’s “no strong evidence” for some kind of action, there better be a damned good rationale for why it will be effective, and it better be deployed in a way that is organized and effective.

        Which leads to the “shelter in place” advice, which seems to be the cheapest and easiest alternative, and can be rolled out as fast as one can say it.

        • The strong evidence of the effectiveness of social distancing *EXISTS* too. it is routinely used in outbreak situations to good effect. Ebola for example. It also has a mechanistic underlying theory that explains its effectiveness… So shelter in place is exactly what’s needed right now.. let the fallout shake out in the healthcare system… and then re-evaluate in several weeks when the scale of the situation is better understood and more specific plans can be put in place.

          So much of this fraught arguments come down to treating this as an all-or-nothing decision: either we shut down everything now and for the rest of some long period of time, or we do nothing.

          The proper framework for evaluation is as a continuous time real-option pricing framework…. We do something this week, we gain data next week, we modify, iterate, and work towards optimal balance of cost and benefit.

        • Zhou Fang says:

          There also needs to be a pretty damn good rationale for thinking the “something else” exists though. I don’t really see shelter in place as competing for resources with other measures.

        • Brent Hutto says:

          So-called “shelter in place” can hardly be construed as a cheap or easy alternative. It involves shutting down the majority of economic activity for an indefinite period of anywhere from a couple weeks to maybe 3-4 months. That is the opposite of cheap.

          Keep it up for months (like everybody’s favorite Wuhan model) at a nation-wide level and it would be the kind of expensive that takes generations to recover from.

          The problem is, there’s no stopping rule. If you’re going to hunker down and bring the world to a halt when almost nobody is sick, then how are you possibly going to back down few weeks later when there are still new cases being diagnosed? Politically speaking, it will be impossible. They’ll just double down harder and harder because they are chasing a bogeyman.

          • You know what would take decades to recover from? 30 million dead in 4 months when it goes to ~100 people who need care per available hospital bed, and all the doctors and nurses are sick anyway.

            • Brent Hutto says:

              I know, right? And what if it mutated and killed a billion people? That would be really bad, too.

              That’s the bogeyman fear I’m talking about. Just spin an arbitrarily bad scenario and ANYTHING starts to sound reasonable by comparison.

              • That’s not an arbitrarily bad scenario, that’s about the order of magnitude of what’s expected from epidemiology models, from Kevin S Van Horn’s post below:


              • Steve says:

                It is not a bogeyman. There are no magic fairies that stop viruses from growing exponentially. Nothing will stop the virus until the population of the planet reaches herd immunity, which will require around half the population of the planet to be infected. That will kill 100 million people easily. Contrary to popular perception viruses do not burn themselves out. SARS-1 was stopped by tracking all of the exposed down and isolating them. The Ebola outbreak that just took place last month (which no one noticed), was stopped the same way. Some viruses are just easier to handle than others, but none of them just magically go away. SARS-cov-2 will reach the carrying capacity of its environment (which is us) unless we intervene and stop it from spreading.

          • Zhou Fang says:

            I don’t see “there’s no stopping rule”. There will be plenty of other places not implementing shutdowns. If they are all okay, then obviously you’d stop. Over time you’d be able to implement more reasons to figure out when you can stop. For example, with improved testing capacity you’d be able to figure out the true scale of the disease. If you become confident you can capturing every case you can correctly estimate mortality. You could suspend the rule for specific locations to see if an exponential explosion in cases comes back.

            I mean, China is lifting the shutdown policy in some locations now.

            • Steve says:

              Exactly, eventually you get a manageable number of infected and can isolate every single one and completely eliminate the possibility of transmission. Ideally, that should have happened in November of last year in China, when the cost would have been tiny compared to the cost the world is now paying, but some Communist party apparatchik decided that the economic costs of scaring everybody wasn’t worth the cost of pulling the alarm. And so here we are, and the disappointing thing is that apparently the same shortsightedness is prevalent in out society as well.

          • jim says:

            “It involves shutting down the majority of economic activity for an indefinite period of anywhere from a couple weeks to maybe 3-4 months. ”

            The more complete the shut-down, the shorter the time.

            Right now though there’s nothing close to a shut-down. While the NBA and MLS have cancelled their games, people are still gathering by the hundreds in crowded lines at grocery stores, where surely the infection is quickly passed.

            I personally can’t imagine what the alternative might be. A million tests a day? A year to complete testing. Two million a day? Half a year. So that’s obviously not the answer.

            What’s your proposal?

            • You can test 100 people in one vial and determine if zero of them have the virus running just 1 well in a PCR machine. There are PCR machines that take 4 x 96 well trays easily (probably way more). The test takes ~ 1 hr to run maybe, there’s some loading time and whatnot… but you can easily run each machine 4 times a day… That means one person can screen 153600 people in a day. You can run 100 of those machines in one lab, no problem. This implies in one lab employing a few hundred people, 15 million people can be screened daily no problem. It’d be easy to set up 20 or 30 of those labs around the country. You could screen all of the population of the US each week absolutely NO PROBLEM.

              If you’re smart about the math, you can swab a person like 3 or 4 times, put their swabs in different randomly selected vials, and back out which small subset of people could be positive by determining which people intersect the various positive vials. So at the end of the day, you’d call up a few hundred or a thousand people and get their additional swabs and have them stay home for 3 days.

              You don’t need to test daily. It’d be a perfectly reasonable strategy for many people for example to have people work from home one week, be tested during that week, if they’re clear, they can do anything they like for a week, then back to work from home…

              There are literally TONS of decent strategies that involve statistical analysis, testing, risk analysis, cost-benefit analysis, and epidemiological modeling.

              All of those things are possible *IF* you don’t have exponential out of control growth.

              • jim says:

                I’m still baffled about why shutting everything but essential services is so controversial. To my mind it’s the only thing that will work, and as you pointed out it’s very effective. and again, the more effective the shut down, the shorter the shutdown.

                IMO a 2-4 week shutdown now with a gradual opening afterward would stem the transmission of the virus significantly and buy *a lot* of time for testing and preparation. Getting through it quickly would also be the best thing for the economy, and a gradual opening would generate an optimistic outlook.

                As I watch this unfold, I’m amazed by the lack of even rudimentary planning. After 9-11, you’d think we’d have everything wired.

  2. Dan F. says:

    From the numbers that have been reported, a crude argument can be made that reported numbers of cases in some countries are gross underestimates, probably off by several fold, and perhaps off by an order of magnitude. To gauge very crudely what is going on, compare reported deaths with reported cases. For example, among the four countries reporting the most cases per capita, which are Italy, Switzerland, Spain, and Norway, in that order (I’m using the numbers from worldometer) the number of deaths per capita varies greatly – the ratio of cases to deaths in Italy is 15/1, in Switzerland is 100/1, in Spain is 20/1, and in Norway is 250/1 (I rounded a lot) (compare Korea is 100/1, China is 25/1, Germany is 400/1, USA is 65/1, Iran is 15/1). These are countries with (grossly) similar age profiles (old populations – older than China, Korea, or the USA, for example) and with good healthcare systems. There is no good reason to expect such a difference in death rates (Italy *is* older than the rest, so one does expect a somewhat higher death rate there, but not by such a degree). (One should compare localization of reported cases to population density, and I haven’t done this, but it’s not as easy as simply looking at overall population density.) My interpretation is that cases in Italy and Spain currently are undercounted by approximately an order of magnitude. The reason for the undercount is simple. Italy and Spain have already neared saturation of intensive care units, at least in heavily infected regions such as Lombardia and Madrid, and neither country has domestic manufacture of testing kits (as do Germany and Korea, for example), so both have limited testing resources. In Madrid persons who do not present fairly severe respiratory symptoms are simply being ignored, and certainly not tested, there being insufficient resources to attend them or test them. For example, my brother-in-law almost certainly has the virus (a week of fever and cough of the right sort), called the hotline and told his employer, and has been officially counted as a probable case, but has not been tested and will not be unless new resources materialize (which is possible as there is apparently an effort to procure more cheap testing kits). In fact, it seems that in Madrid those being tested are those who present with a possible need for immediate hospitalization. Given that there are serious reports that indicate that perhaps half or more of carriers are asymptomatic, this means that undercounting is substantial.

    Surely precisely this sort of reasoning is what led the quite competent public health officials in Italy and Spain to push for full lockdown of the population. With it likely that the virus is far more extended than is indicated by official figures and without the resources to do accurately gauge or localize the levels of infection, the best strategy is complete shut down. It seems to have worked in China, and soon we will see if it works in Italy or Spain (the lag is about a week from Italy to Spain).

  3. Keith O’Rourke says:

    This from Ioannidis “prevalence of the infection in a random sample of a population” does look like Cochrane “group think” – if its not randomized its useless.

    Somewhere, someone pointed out outbreaks are hard to study with random sampling and the test material likely can’t be shared.

    • Andrew says:


      Yes, when reading Ioannidis’s article I automatically translated “representative random sample” to “representative sample” (where the representativeness could be be achieved using poststratification). I took “random” to be an unnecessary rhetorical flourish. I hadn’t mentioned this when writing the post, but to clarify I will add it now.

    • That might be the Cochrane position. But in the last paragraph, in the article at hand, John relies on an Autopsy Series.

    • I believe that Vinay Prasad and John Ioannidis do not always resort to Cochrane for their hypotheses for each has critiqued randomized trials too, although John on occasion has characterized the Cochrane sources as one of the best.

      Let’s not be married to our educational training. Sheesh

  4. Anoneuoid says:

    Is there any reason the number of cases would scale approximately with approximately the square of the number of tests performed in the US?

    I started with the idea that there was a constant proportion p of the population infected but as testing ramped up we would see an increase in cases. But then we would expect cases to be something like p*numTests. Instead the number of cases is growing faster than the number of tests. An exponent near 2 works pretty well, with a proportion of 0.00155 but without some theoretical explanation for the exponent it doesn’t mean anything.

  5. I just think that the narratives of the Cov-id19 have been contradictory in some aspects. It took a while to hear how the many means by which it can spread.

    The other thing is that scientists from China are mistrusted by subsets. So that’s another hurdle to overcome.

  6. It’s interesting to read Ioannidis’ article, and then look at the Imperial College study described here: . They use point estimates for important parameters (“we assume an incubation period of 5.1 days”) instead of modeling the uncertainty in these parameters, which means their conclusions don’t adequately reflect that uncertainty.

    • Hi Kevin, thanks so much for this link, although I agree with you that it’d be better to see say a small-multiples of this graph for say 9 random draws from a prior, it’s still better to see point estimates than nothing. What it shows is that the most extreme measures, school closures and univ closures and social distancing and case isolation all put together etc, are VERY effective compared to either do nothing or even just a more lightweight version.

      Of course, as soon as they turn off their control measures… :-( still the control measures buy time to learn about how to handle the virus, and that to me is the critical thing right now.

    • Dan F. says:

      Their model and conclusions are too crude for it to matter whether they use 5.1 or a distribution on some interval containing 5.1. Exponential growth swamps all that. Including an interval means the conclusion is some distribution on 4 to 6 months rather than 5 months … but this article was meant to get a message across to politicians and decision makers. They can’t handle the uncertainties, and this sort of model is robust to those sorts of uncertainties, precisely because it is so crude.

  7. Michael Nelson says:

    So far as I can tell, the whole opinion piece amounts to nothing more than Ioannidis proclaiming, “I am scared! I am a data scientist! I choose to concentrate my fear on something I fancy I understand and may be able to influence, rather than on unknowable forces of nature against which I am powerless!”

    • Anoneuoid says:

      Sounds more like you are projecting your own fear.

      • Michael Nelson says:

        I was going to withhold commenting about his piece, but we have very little reliable data at the moment as to the extent to which Ioannidis is rubber and I am glue, and curtailing normal practices of criticism in response to a still-hypothetical “bouncing-sticking” phenomenon could have devastating consequences for the entire comment thread.

      • Michael Nelson says:

        Okay, I see that making my original comment without sharing the thought process leading up to it does come off as a purely ad hominem attack.

        I have no problem admitting to my fear, and I also think it’s reasonable to infer Ioannidis’s fear from his comparing current policy to “jumping off a cliff” and his dire warnings of “financial crisis, unrest, civil strife, war, and a meltdown of the social fabric.” I’m not criticizing Ioannidis for being scared, I’m criticizing him for maximizing concern over the unintended consequences of current COVID policy and minimizing concern over COVID itself. The potential negative consequences of COVID are at least as bad as the potential negative consequences of social distancing policies. Social distancing might lead to widespread and deadly disruptions, but pestilence leads to mass deaths, which then also lead to widespread and deadly disruptions.

        Ioannidis’s portrayal of scale of the risks is not balanced, and he really doesn’t offer any constructive alternatives. My guess about his motives–that he is more concerned about the data science flaws because he knows more about data science flaws than about a brand new virus–could be wrong, and obviously my focusing on his motives was a distraction.

    • It sounds like a sensible attitude to take under the circumstances. But that is NOT what John Ioannidis is conveying. I venture that he is trying to broaden the parameters of the discussion.

      • Michael Nelson says:

        It is a sensible response to a crisis to focus on areas where we can make a constructive contribution, rather than on areas beyond our control. If Ioannidis had wanted to do this, he might have specified what we should stop or start doing and why, or highlighted the extent of uncertainty around the consequences of various policies. These efforts would have expanded the parameters of the discussion. Instead, he obscures the parameters by branding current policies as a “fiasco” and “jumping off a cliff.”

        In essence, Ioannidis vaguely warns us that, with some unknown probability, and to some unknown extent, social distancing might lead to worse consequences than would not social distancing, while saying that not social distancing is either “brilliant or catastrophic.” If a weather forecast started out by describing the worst possible consequences of evacuating an area ahead of a hurricane, then equivocated as to whether staying in the area is a good idea, would you say that forecast had broadened the parameters of the hurricane prediction model?

        • Hello Michael,

          At least John’s article has gotten you to broaden the parameters of your views. lol. Just joking with ya.

          I appreciate your elaborating on your viewpoint. Frankly, I interpreted John’s viewpoint to highlight the prospect of ‘systems neglect’& ‘tradeoff neglect’. The meaning of the ‘systems effect’ is straightforward. He illustrates this here:

          ‘One of the bottom lines is that we don’t know how long social distancing measures and lockdowns can be maintained without major consequences to the economy, society, and mental health. Unpredictable evolutions may ensue, including the financial crisis, unrest, civil strife, war, and a meltdown of the social fabric.

          I don’t see any harm in John pointing this out b/c we may not be envisioning the consequences of longer distancing measures and lockdowns.

          Tradeoff neglect is implicated here:

          In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work. School closures, for example, may reduce transmission rates. But they may also backfire if children socialize anyhow if school closure leads children to spend more time with susceptible elderly family members if children at home disrupt their parents’ ability to work, and more. School closures may also diminish the chances of developing herd immunity in an age group that is spared serious disease.’

          Now perhaps I might have framed the concerns differently and offered more stark examples than John has The reality is that each of us interprets risks/uncertainty differently.

Leave a Reply