A press release from the University of Pennsylvania demands your attention: “In the U.S., COVID-19 wasn’t sole cause of excess deaths in 2020.”
The first sentence reads, “By the year 2017, the United States was already suffering more excess deaths and more life years lost each year than those associated with the COVID-19 pandemic in 2020.”
Immediately, you think, “what does this mean? Are we overestimating COVID-19 deaths?”
Spoiler alert: the press release does not accurately translate what the researchers found. This is dangerous and irresponsible, and it comes directly from the lead author’s university press office. This is not a case of a reporter misinterpreting results or aiming for click-bait.
Let’s discuss what the term “excess deaths” means, what the research results actually tell us, where the press release went wrong, and why we should care.
What Are Excess Deaths? How Do We Calculate Them?
You have probably heard the term “excess deaths” in the last year. Calculating excess deaths can give us a quick shortcut to a rough estimate of how many people have died due to a major event such as an environmental disaster (e.g., Hurricane Maria in Puerto Rico) or a pandemic (e.g., COVID-19).
Simply put, excess deaths represent the actual number of observed deaths in the population minus the expected number of deaths over a period of time. We can estimate the expected number of deaths in a population based on prior data. This is actually easier than it seems because it would take a pandemic, war, famine, considerable policy shifts, or some other major catastrophic event to significantly move the needle on the number of deaths among a population of 328 million people. Thus, when something does cause the needle to move, we can calculate a rough estimate of how much the needle moved and attribute the cause.
Here is a simple exercise. In the U.S., about 2.85 million people died in 2019. Among those deaths, about 659,000 were due to heart disease and 599,000 were due to cancer. Combined, those two causes of death represent about 44% of all deaths in 2019. Those numbers should not change significantly from year to year without a major change in society. It would be shocking if 1,000,000 people died of cancer this year, because that is well outside of the norm. We would know that something significant must have happened to lead to these excess deaths. Perhaps there was a catastrophic radiation leak in a major population area that went undetected.
Noticing that there are excess deaths leads scientists to search for the cause. Conversely, knowing there is a cause of excess deaths leads scientists to try to calculate how many excess deaths occur.
Again, without something significant to directly increase or decrease deaths, we can pretty reliably predict how many people will die in the U.S. next year. Assuming COVID-19 has subsided, that is.
The concept of excess deaths relies on an idea known as a counterfactual. We know how many people actually died in 2020, but what we really want to know is how many people would have died in 2020 if COVID-19 had not occurred. That is a counterfactual that helps us understand excess deaths.
What does that beautiful alternate reality 2020 look like? Well, for starters, there would have been about 550,000–668,000 fewer deaths.
What Does the Research Say About 2017?
The actual research presented in “Excess Mortality in the United States in the 21st Century” is quite interesting. This paper was published in the Proceedings of the National Academy of Science, a prestigious general science journal that publishes some of the best work across multiple disciplines.
The authors use data from the U.S. and five European countries (England, France, Germany, Italy, and Spain) from 2000, 2010, and 2017. They calculate excess deaths in the U.S. compared to these European countries.
Here is a critical distinction: in this research, the counterfactual is the set of European countries the authors chose. The authors examine how many actual deaths occurred in the U.S. compared to how many deaths we would expect if the U.S. had the same mortality conditions as those European countries. In other words, how many deaths would occur in the U.S. if we had similar rates of death from heart disease, cancer, accidents, etc. as Europe?
Although not discussed in this paper, other research examines why the U.S. has higher mortality rates than European countries. Instead, this paper simply estimates how many excess deaths occur because the U.S. has higher mortality rates than Europe.
The answer? MANY.
The authors calculate that the U.S. had 226,000 excess deaths in 2000 and 400,000 excess deaths in 2017 compared to Europe (see Table 1). This is a fascinating but depressing finding.
However, the authors make an unfortunate statement to grab readers’ attention and highlight the importance of their findings:
“Because it has captured a great deal of national attention, the number of deaths from the COVID-19 epidemic in 2020 forms a timely basis of comparison. On 20 February 2021, the Centers for Disease Control and Prevention reported that 376,504 deaths ascribed to COVID-19 had occurred in the United States in calendar year 2020. That figure is similar to but below the estimated total number of excess deaths of 401,000 in the United States in 2017.”
Not terrible, but it gets worse.
The Press Release Says What?
This study came to my attention because a friend posted a link to the press release but admitted he needed some help understanding the findings. When I read the press release before reading the study (terrible mistake), I was confused.
Again, the headline for the press release reads, “In the U.S., COVID-19 wasn’t sole cause of excess deaths in 2020.”
That is not what the study finds. Do you recall the researchers examining data from 2020? Yeah, me neither, because they did not study 2020. They didn’t even study COVID-19 but somehow it is the fourth word in the headline of the press release.
But wait, remember the first sentence of the press release?
“By the year 2017, the United States was already suffering more excess deaths and more life years lost each year than those associated with the COVID-19 pandemic in 2020.”
Yikes. The comparisons are not even remotely the same.
Recall that the study compares the U.S. to Europe in 2000, 2010, and 2017. The researchers calculate excess deaths with Europe as the counterfactual. A simple version of the CDC analysis compares the U.S. in 2020 to the U.S. in 2019 (with some small adjustments). The CDC calculates excess deaths with the U.S. as the counterfactual. Apples to oranges here.
If the researchers had 2020 data, they could have calculated excess deaths in the U.S. in 2020 based on both dimensions: a non-COVID world counterfactual and the Europe counterfactual. That is not what they did, but the excess death estimate under this scenario would be roughly 700,000 to 900,000. That is very different than what the press release says.
Why Should We Care?
The interpretation presented in the press release is incredibly misleading, and it appears to have its genesis in the seemingly innocuous statement the study’s authors made. It is likely that the comparison of their 2017 calculation with the 2020 CDC COVID-19 numbers was very appealing. They probably wanted to work COVID into their own research to get attention and broaden their work’s appeal and interest, just like many other researchers during the last year.
Ironically, the same journal (PNAS) that published this paper published an article on “Misinformation in and about Science” the very next day. That paper states, “Still, scientists compete for eyeballs just as journalists do. They face incentives to hype their work and to publish selectively those findings that are surprising and ‘clickable.’” Wow.
However, this is a slippery slope with consequences. The University of Pennsylvania’s Press Office turned this research into something it is not. Frankly, the headline and first sentence of the press release are criminally wrong.
My friend who posted the link is very intelligent but not an academic. He relied on the statements of the press release and asked for interpretation help from his friends.
However, my guess is that the average reader does not even get that far. To some, the click-bait headline likely makes it sound like COVID-19 deaths are overblown. That damage significantly reduces the scientific contribution this research makes.
Altmetric shows that the article is getting some attention in the media. Many of these outlets will perpetuate the inaccurate statements of the press release, although kudos to The Guardian for giving this an accurate and proper write-up.
Worse, the social media misinformation campaign has already begun.
I have one recommendation to savvy, intelligent people out there: dig into the story and do not just read the headlines or the press release. Easier said than done when academia continues to keep a giant paywall around itself, but I’ll save my rant on that problem for a later post. Don’t be afraid to ask other people for help with the interpretation.
However, the real burden to fix this is on researchers and scholars. We need to learn to better control the narrative and not overstep the boundaries of what our research can speak to. Science is under attack and misinformation is everywhere. Let’s not get in our own way on this one.