Two opposing interpretations of the evidence about waning vaccine protection are grounded in two very different models of the course of a Covid-19 infection. In two masterful tweets, John Burn-Murdoch captured and named them: the dice model and the two-walls model.

Under the dice model, waning protection against infection implies waning protection against severe disease. Because there are fewer instances of more serious outcomes, it is more difficult to tease out statistically significant evidence of waning against a more serious outcome than against infection. But if the dice model is right, the two always go together.

Under the two-walls model, it would be possible to have waning protection for infections without waning protection for severe disease, but there is a problem with this model. Part of the beauty of the stripped-down characterization by Burn-Murdoch is that it is precise enough to surface its intrinsic logical contradiction. There is a way to patch the model to remove this contradiction, but what remains is a biologically implausible model that is starkly inconsistent with the data.

As a result, the the default presumption should be that statistically significant evidence of waning against infection also implies waning against more serious outcomes. This presumption could be overturned by tight estimates that show no waning of protection against these more serious outcomes, but not by low-powered tests that generate big uncertainty intervals.

Suppose that a drug company is trying to get approval for a new pain medication that might have some serious negative side effects. How can the company keep regulators from finding any?

Even if you have no training in statistics it is easy to understand that the sure-fire strategy is to focus attention on side effects that are rare. The smaller the number of events, the easier it is to dismiss the few that arise as chance outcomes.

This same strategy for avoiding a discovery is being used now by people who want to keep us from finding that the protection from vaccines diminishes over time.

Data from a study of vaccine effectiveness by the Mayo Clinic shows why severe disease is hard to measure accurately. In such cases, it is better to track a proxy indicator -- a canary in the coal mine -- than the indicator of interest. Vaccine effectiveness against infection is the obvious proxy indicator for effectiveness against all outcomes, which should all vary with the number of infections. Measures of effectiveness of the existing vaccines against infection show unambiguously that the protection provided by the existing vaccines is substantially lower now than it was before the delta variant took over.

A reaction along the lines of “Who cares whether the probability of breakthrough infections is increasing; we only care about preventing severe disease” amounts to saying “Who cares that the canary died; we only care about saving the lives of miners.”

There are two mathematically equivalent ways to describe the protection that a vaccine offers against such specific outcomes as severe disease:

• relative risk

• effectiveness

Specialists understand the quirks of these two measures, but the rest of us may not until someone calls attention to them. One of these quirks is that effectiveness activates a cognitive bias that misleads people about the size of any change in the protection offered by a vaccine that is in use.

In the current context, this means that statements about effectiveness will tend to minimize the significance of the fall in protection that triggered the decision to offer a third dose of the vaccines from Pfizer and Moderna.