In the coronavirus pandemic, we’re making decisions without reliable data

https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/

Newsletters

Try STAT Plus

FIRST OPINION

A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data

By JOHN P.A. IOANNIDIS
MARCH 17, 2020

A nurse holds swabs and a test tube to test people for Covid-19 at a drive-through station set up in the parking lot of the Beaumont Hospital in Royal Oak, Mich.PAUL SANCYA/AP

The current coronavirus disease, Covid-19, has been called a once-in-a-century pandemic. But it may also be a once-in-a-century evidence fiasco.
At a time when everyone needs better information, from disease modelers and governments to people quarantined or just social distancing, we lack reliable evidence on how many people have been infected with SARS-CoV-2 or who continue to become infected. Better information is needed to guide decisions and actions of monumental significance and to monitor their impact.
Draconian countermeasures have been adopted in many countries. If the pandemic dissipates — either on its own or because of these measures — short-term extreme social distancing and lockdowns may be bearable. How long, though, should measures like these be continued if the pandemic churns across the globe unabated? How can policymakers tell if they are doing more good than harm?
ADVERTISEMENT

Vaccines or affordable treatments take many months (or even years) to develop and test properly. Given such timelines, the consequences of long-term lockdowns are entirely unknown.

We know enough now to act decisively against Covid-19. Social distancing is a good place to start

The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable. Given the limited testing to date, some deaths and probably the vast majority of infections due to SARS-CoV-2 are being missed. We don’t know if we are failing to capture infections by a factor of three or 300. Three months after the outbreak emerged, most countries, including the U.S., lack the ability to test a large number of people and no countries have reliable data on the prevalence of the virus in a representative random sample of the general population.
ADVERTISEMENT

This evidence fiasco creates tremendous uncertainty about the risk of dying from Covid-19. Reported case fatality rates, like the official 3.4% rate from the World Health Organization, cause horror — and are meaningless. Patients who have been tested for SARS-CoV-2 are disproportionately those with severe symptoms and bad outcomes. As most health systems have limited testing capacity, selection bias may even worsen in the near future.
The one situation where an entire, closed population was tested was the Diamond Princess cruise ship and its quarantine passengers. The case fatality rate there was 1.0%, but this was a largely elderly population, in which the death rate from Covid-19 is much higher.
Projecting the Diamond Princess mortality rate onto the age structure of the U.S. population, the death rate among people infected with Covid-19 would be 0.125%. But since this estimate is based on extremely thin data — there were just seven deaths among the 700 infected passengers and crew — the real death rate could stretch from five times lower (0.025%) to five times higher (0.625%). It is also possible that some of the passengers who were infected might die later, and that tourists may have different frequencies of chronic diseases — a risk factor for worse outcomes with SARS-CoV-2 infection — than the general population. Adding these extra sources of uncertainty, reasonable estimates for the case fatality ratio in the general U.S. population vary from 0.05% to 1%.

STAT Reports: STAT’s guide to interpreting clinical trial results

That huge range markedly affects how severe the pandemic is and what should be done. A population-wide case fatality rate of 0.05% is lower than seasonal influenza. If that is the true rate, locking down the world with potentially tremendous social and financial consequences may be totally irrational. It’s like an elephant being attacked by a house cat. Frustrated and trying to avoid the cat, the elephant accidentally jumps off a cliff and dies.
Could the Covid-19 case fatality rate be that low? No, some say, pointing to the high rate in elderly people. However, even some so-called mild or common-cold-type coronaviruses that have been known for decades can have case fatality rates as high as 8% when they infect elderly people in nursing homes. In fact, such “mild” coronaviruses infect tens of millions of people every year, and account for 3% to 11% of those hospitalized in the U.S. with lower respiratory infections each winter.
These “mild” coronaviruses may be implicated in several thousands of deaths every year worldwide, though the vast majority of them are not documented with precise testing. Instead, they are lost as noise among 60 million deaths from various causes every year.
Although successful surveillance systems have long existed for influenza, the disease is confirmed by a laboratory in a tiny minority of cases. In the U.S., for example, so far this season 1,073,976 specimens have been tested and 222,552 (20.7%) have tested positive for influenza. In the same period, the estimated number of influenza-like illnesses is between 36,000,000 and 51,000,000, with an estimated 22,000 to 55,000 flu deaths.
Note the uncertainty about influenza-like illness deaths: a 2.5-fold range, corresponding to tens of thousands of deaths. Every year, some of these deaths are due to influenza and some to other viruses, like common-cold coronaviruses.
In an autopsy series that tested for respiratory viruses in specimens from 57 elderly persons who died during the 2016 to 2017 influenza season, influenza viruses were detected in 18% of the specimens, while any kind of respiratory virus was found in 47%. In some people who die from viral respiratory pathogens, more than one virus is found upon autopsy and bacteria are often superimposed. A positive test for coronavirus does not mean necessarily that this virus is always primarily responsible for a patient’s demise.

NEWSLETTERS

A roundup of STAT’s top stories of the day.

If we assume that case fatality rate among individuals infected by SARS-CoV-2 is 0.3% in the general population — a mid-range guess from my Diamond Princess analysis — and that 1% of the U.S. population gets infected (about 3.3 million people), this would translate to about 10,000 deaths. This sounds like a huge number, but it is buried within the noise of the estimate of deaths from “influenza-like illness.” If we had not known about a new virus out there, and had not checked individuals with PCR tests, the number of total deaths due to “influenza-like illness” would not seem unusual this year. At most, we might have casually noted that flu this season seems to be a bit worse than average. The media coverage would have been less than for an NBA game between the two most indifferent teams.
Some worry that the 68 deaths from Covid-19 in the U.S. as of March 16 will increase exponentially to 680, 6,800, 68,000, 680,000 … along with similar catastrophic patterns around the globe. Is that a realistic scenario, or bad science fiction? How can we tell at what point such a curve might stop?
The most valuable piece of information for answering those questions would be to know the current prevalence of the infection in a random sample of a population and to repeat this exercise at regular time intervals to estimate the incidence of new infections. Sadly, that’s information we don’t have.
In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work. School closures, for example, may reduce transmission rates. But they may also backfire if children socialize anyhow, if school closure leads children to spend more time with susceptible elderly family members, if children at home disrupt their parents ability to work, and more. School closures may also diminish the chances of developing herd immunity in an age group that is spared serious disease.
This has been the perspective behind the different stance of the United Kingdom keeping schools open, at least until as I write this. In the absence of data on the real course of the epidemic, we don’t know whether this perspective was brilliant or catastrophic.
Flattening the curve to avoid overwhelming the health system is conceptually sound — in theory. A visual that has become viral in media and social media shows how flattening the curve reduces the volume of the epidemic that is above the threshold of what the health system can handle at any moment.

The novel coronavirus is a serious threat. We need to prepare, not overreact

Yet if the health system does become overwhelmed, the majority of the extra deaths may not be due to coronavirus but to other common diseases and conditions such as heart attacks, strokes, trauma, bleeding, and the like that are not adequately treated. If the level of the epidemic does overwhelm the health system and extreme measures have only modest effectiveness, then flattening the curve may make things worse: Instead of being overwhelmed during a short, acute phase, the health system will remain overwhelmed for a more protracted period. That’s another reason we need data about the exact level of the epidemic activity.
One of the bottom lines is that we don’t know how long social distancing measures and lockdowns can be maintained without major consequences to the economy, society, and mental health. Unpredictable evolutions may ensue, including financial crisis, unrest, civil strife, war, and a meltdown of the social fabric. At a minimum, we need unbiased prevalence and incidence data for the evolving infectious load to guide decision-making.
In the most pessimistic scenario, which I do not espouse, if the new coronavirus infects 60% of the global population and 1% of the infected people die, that will translate into more than 40 million deaths globally, matching the 1918 influenza pandemic.
The vast majority of this hecatomb would be people with limited life expectancies. That’s in contrast to 1918, when many young people died.
One can only hope that, much like in 1918, life will continue. Conversely, with lockdowns of months, if not years, life largely stops, short-term and long-term consequences are entirely unknown, and billions, not just millions, of lives may be eventually at stake.
If we decide to jump off the cliff, we need some data to inform us about the rationale of such an action and the chances of landing somewhere safe.
John P.A. Ioannidis is professor of medicine and professor of epidemiology and population health, as well as professor by courtesy of biomedical data science at Stanford University School of Medicine, professor by courtesy of statistics at Stanford University School of Humanities and Sciences, and co-director of the Meta-Research Innovation Center at Stanford (METRICS) at Stanford University.

About the Author

John P.A. Ioannidis

jioannid@stanford.edu

@METRICStanford

Tags

CORONAVIRUS
GLOBAL HEALTH
INFECTIOUS DISEASE
PUBLIC HEALTH

Republish this article

MARCH 19, 2020 AT 12:04 PM

“I’m sitting at home after my office closed today and still wondering why my country’s economy is being destroyed by panic.”
“…and the death rate will turn out to be about what the flu is. It seems most likely.”
If you want to know why then read more of the readers comments and you will see why. The contagion factor is excluded from the authors analysis, which makes his theory just as incomplete as the missing data he complains about.
The contagion factor of this virus is far greater than influenza- so if you don’t take measures to slow the spread you get a higher death rate because you can’t treat all the sick at once. Italy versus China is an actual example of what happens- they are already surpassing China’s death rate even though their population pales in comparison (60 million versus 1.35 billion)

Will

MARCH 19, 2020 AT 11:59 AM

In the same article that you use the Diamond Princess cruise ship as a case study for fatality rates, you estimate that 1% of the U.S population might be infected. The Diamond Princess cruise ship saw nearly 25% of the ship’s passengers infected. Perhaps multiple your “lost in the noise” 10,000 influenza-like deaths by 20+.

MARCH 19, 2020 AT 11:50 AM

The author’s choice of influenza and cruise ship results as points of comparison as basis are… just as bad, if not worse, than proceeding with incomplete data.
There is a point to not having the data, but this article is irresponsible and biased against in its assumptions. Stating ‘we don’t know if these measures work’ isn’t completely accurate either, as we see first hand the difference between what happens in some instances as opposed to others (Italy’s death toll will surpass China’s).
For those touting this author’s expertise, know there are others with greater experience indicating otherwise. The author uses 1918 as a reference, so note Frank Macfarlane Burnet (more knowledgeable about influenza than this author ever will be) indicates the actual death toll from influenza was much higher, and that these viruses can mutate and come in multiple waves- the second wave in 1918 was far deadlier.
This author fails to take that into consideration, as well as the contagion factor. Th fact that COVID19’s viral shed factor is 1,000 times greater than influenza, and it’s peak shed is during incubation when many times there are no symptoms (as opposed to influenza, which peaks after it settles into the lungs).

Julian

MARCH 19, 2020 AT 11:49 AM

Why is this article’s main source of data the Diamond Princess when we have a much, much larger data set in South Korea? As of March 15th, South Korea had tested 248,000 people, and confirmed 8,162 cases, and recorded 75 deaths. That represents a case fatality ratio of 0.9%. If governments should base their policy decisions on a range of reasonable possibilities, it seems like the South Korea example, where they have conducted the most testing, should be the benchmark – not the Diamond Princess.

Mike Schwaller

MARCH 19, 2020 AT 11:49 AM

This is misinformation but the premise is right…we really do not know until better data is available.
The author selected the cruise ship as a reasonable system to make his argument, then listed confounding variables that make his points appear credible. These variables were presented like a subordinate clause. There are major epidemiologic flaws in his approach.
He could be right, but he is brave to make his assertions this early in the game based on this cruise ship.
We need draconian measures for at least 1 month until we have better data.
MS

Annie

MARCH 19, 2020 AT 11:47 AM

Thank you for this article. It is the first one I’ve read lately that seems logical and unbiased. We had over 60,000 American deaths during the 2017-18 flu season, and yet confirmed American deaths from this disease are still under 200. I’m sitting at home after my office closed today and still wondering why my country’s economy is being destroyed by panic. It’s just the unknown factor I guess, along with media bias and politics. Hopefully more testing will be done on people who have only experienced mild symptoms, and the death rate will turn out to be about what the flu is. It seems most likely.

Bob

MARCH 19, 2020 AT 11:46 AM

Dr. Ioannidis, if you’re reading this pretty robust conversation:
several comments have alluded to preparedness of the health care system (everyone at our at-capacity university hospital seem to have fingers crossed).
Any idea how your colleagues at Stanford (particulary ER docs, intensivists, ID), as well as nursing, RT, etc (especially as it pertains to staffing) feel about preparedness?

Albert

MARCH 19, 2020 AT 11:31 AM

How about “Let’s do two things at once”?
First, social distancing, knowing that it’s (a) effective at ‘flattening the curve’ and helping health systems to better cope with the inevitable influx of severely ill patients, and (b) temporary, a society-wide acute care response that’s needed until…
Second, better data are collected to determine a more complete epidemiological profile of COVID-19. Especially now that China, South Korea, and others appear to be moving past their respective outbreak peaks, countries can move quickly to randomized serological studies to determine true mortality risk. From there, we can determine how much social distancing protocols can be eased and what pace, how many restrictions (re: travel, telework, retail, public gatherings, etc.) are still needed and for whom, and how much surveillance, public health, and medical capacity are needed to maintain watch for new clustered outbreaks.
It’s not that Team Ioannidis is right and Team Lipsitch is wrong, or vice versa. It’s that they’re both right.

Dom

MARCH 19, 2020 AT 11:25 AM

Love this article… while this seems to be hospitalizing people at an alarming rate, the fear and warnings issued are based on bad data. Hospitals need protected in order to provide standard care beyond just treating this virus but issuing a death rate between 3 and 5% when almost no one has been tested is irresponsible.
I am interested to know how many hospitalizations have occurred in the last 30 days for this virus compared to others to gauge how severe this is.
I tend to want to look at things from a place of intelligence rather than emotion.

Lylia Hoehl

MARCH 19, 2020 AT 11:51 AM

What a waste of time and materials to test everyone. If someone tests negative then comes in contact with the virus, they may then test positive.

OLDER COMMENTS »

Comments are closed.