A recently published article by Fisman et al., “Impact of population mixing between vaccinated and unvaccinated subpopulations on infectious disease dynamics: implications for SARS-CoV-2 transmission”[1], has received widespread acceptance and coverage in the mainstream media despite major flaws in its methodology[2][3].
At BIG Media, we have examined the article of Fisman et al. and find its main conclusion – that risk of infection among vaccinated people is disproportionately attributed to unvaccinated people – is an exaggerated claim and does not follow from the model presented. Their conclusions may be true in an alternate world where fewer people have acquired immunity from natural infection, or where COVID-19 vaccines are highly protective against infection. However, they certainly do not hold up in the real world (see[4]as an example).
We acknowledge that no single model can answer all questions or include all aspects of reality – that is not its purpose. It is not a “fatal flaw” for a model to be unrealistic. All models are unrealistic to a degree. But they can be useful if they test hypotheses appropriately. In order to do so, models cannot contradict accepted observations in the real world. Their parameters must align numerically with actual biological processes – e.g., the average length of time it takes for someone to recover from COVID-19. And their results must remain robust under variable conditions – that is, if you change the value of one parameter (or several simultaneously), the model’s results are not rendered meaningless.
We will refrain from trying to address all the technical issues and debatable assumptions in their model, except to illustrate two immediately obvious errors regarding immunity:
-
- It did not wane, and
- It was equally robust whether it was acquired from infection or from vaccination.
These assumptions, to put it kindly, are dated and need to be revised to reflect both peer-reviewed literature and published technical documents showing that immunity is temporary, and that it decays unequally depending on the source from which it was acquired:
Fortunately, we can modify the waning immunity assumption using an Excel version of their model, which Fisman et al. have made available for download[8].
And, what is the result of adding the loss of immunity to Fisman’s baseline model? A complete disintegration of the paper’s main message, as illustrated in the figure below.
On the left is a reproduction of Figure 2B in the Fisman article, which demonstrates its main conclusion: as less “mixing” between the vaccinated and unvaccinated occurs (i.e., as values on the x-axis increase toward 1), the risk of infection experienced by the vaccinated comes disproportionately from the unvaccinated (i.e. values on the y-axis > 1).
On the right is a graph of the model’s output under the additional assumption of waning immunity. With a single, epidemiologically supported modification, Fisman’s model now suggests that unvaccinated people represent an equal risk to the vaccinated and to other unvaccinated people.
Thus, there is no indication of disproportionate infections caused by unvaccinated people.
Figure Caption. Impact of mixing between vaccinated and unvaccinated sub-populations as they contribute to the risk of infection for varied assumptions of vaccine effectiveness (originally part of Figure 2B in Fisman et al. CMAJ article). The left panel displays a reproduction of their results using their baseline model. The right panel displays where we changed the assumption of immunity persisting forever (Fisman’s baseline value), to waning after 8 months in both subpopulations. This simple change will under-estimate the duration of immunity from natural infection, while over-estimating it for vaccine-induced immunity. Y-axis values > 1 (dashed line) indicate a greater risk to vaccinated from the unvaccinated. Values = 1 indicate no additional risk to vaccinated people. Values of closer to 0 on the x-axis imply completely random mixing between the 2 subpopulations; values near 1 imply “strong” like-with-like. VE = Vaccine Effectiveness.
Another characteristic of the Fisman et al model is that they examined how different mixing patterns between two populations – here, the unvaccinated and vaccinated – affect the spread of infection. For the most part, humans do not mix randomly, but instead do so on a spectrum of preferential (i.e., like-with-like mixing) and random mixing. For example, age-preferred mixing is frequently observed in everyday life – e.g., children are more likely to interact with other children than would be expected at random across all age groups.
It is this preferential mixing assumption that Fisman et al. try to link to policy interventions that would segregate vaccinated and unvaccinated people through vaccine mandates and passports.
The substantial differences in the model’s outcome – and, most important, their policy implications – from a change in one parameter illustrates how much these models can rely on delicate assumptions. This should also highlight the danger of arriving at premature policy conclusions based on a single model.
The stylised model published by Fisman et al. does not appear to represent real-world dynamics and presents potentially polarizing conclusions that are unfounded when considering responsible scientific practices.
Footnotes
[1] 1Original CMAJ article: Impact of population mixing between vaccinated and unvaccinated subpopulations on infectious disease dynamics: implications for SARS-CoV-2 transmission
[2] Being with unvaccinated people increases COVID-19 risk for those who are vaccinated: modelling study, CTV News
[3] Unvaccinated disproportionately risk safety of those vaccinated against COVID-19, study shows, The Globe & Mail
[4] COVID-19 vaccinations data, Government of Ontario
[5] Quantifying the risk of SARS-CoV-2 reinfection over time, Reviews in Medical Virology
[6] Neutralizing Antibodies and Cellular Immune Responses Against SARS-CoV-2 Sustained One and a Half Years After Natural Infection, Frontiers in Microbiology
[7] SARS-CoV-2 variants of concern and variants under investigation in England, UK Health Security Agency
[8] Simple 2-Patch Vaccine Model, Figshare
