Political Overreach, Diminished Credibility
When I was asked back in June of 2021 about lessons for science from the pandemic, the gist of what I said was:
In a democracy, the community of science can be a uniquely valuable source of objective facts, but assertions by scientists will be trusted only if they are careful not to overreach by advocating on behalf of their preferred political outcomes.
(See below for my detailed response.) Three months later, in the wake of a debate about booster shots, we can see the risk associated with overreach. In the United States, no one making decisions about boosters is paying any attention to what the people who claim to be the scientific authorities are saying.
Dice, Walls and Boosters
Two opposing interpretations of the evidence about waning vaccine protection are grounded in two very different models of the course of a Covid-19 infection. In two masterful tweets, John Burn-Murdoch captured and named them: the dice model and the two-walls model.
Under the dice model, waning protection against infection implies waning protection against severe disease. Because there are fewer instances of more serious outcomes, it is more difficult to tease out statistically significant evidence of waning against a more serious outcome than against infection. But if the dice model is right, the two always go together.
Under the two-walls model, it would be possible to have waning protection for infections without waning protection for severe disease, but there is a problem with this model. Part of the beauty of the stripped-down characterization by Burn-Murdoch is that it is precise enough to surface its intrinsic logical contradiction. There is a way to patch the model to remove this contradiction, but what remains is a biologically implausible model that is starkly inconsistent with the data.
As a result, the the default presumption should be that statistically significant evidence of waning against infection also implies waning against more serious outcomes. This presumption could be overturned by tight estimates that show no waning of protection against these more serious outcomes, but not by low-powered tests that generate big uncertainty intervals.
Suppose that a drug company is trying to get approval for a new pain medication that might have some serious negative side effects. How can the company keep regulators from finding any?
Even if you have no training in statistics it is easy to understand that the sure-fire strategy is to focus attention on side effects that are rare. The smaller the number of events, the easier it is to dismiss the few that arise as chance outcomes.
This same strategy for avoiding a discovery is being used now by people who want to keep us from finding that the protection from vaccines diminishes over time.
Infrequent Events and Proxy Indicators
Data from a study of vaccine effectiveness by the Mayo Clinic shows why severe disease is hard to measure accurately. In such cases, it is better to track a proxy indicator -- a canary in the coal mine -- than the indicator of interest. Vaccine effectiveness against infection is the obvious proxy indicator for effectiveness against all outcomes, which should all vary with the number of infections. Measures of effectiveness of the existing vaccines against infection show unambiguously that the protection provided by the existing vaccines is substantially lower now than it was before the delta variant took over.
The Risk of Infection is the Canary in the Coal Mine
A reaction along the lines of “Who cares whether the probability of breakthrough infections is increasing; we only care about preventing severe disease” amounts to saying “Who cares that the canary died; we only care about saving the lives of miners.”