In two previous articles, we discussed the difficulty in determining the cause of an individual report of an adverse event following immunization (AEFI) Taking a responsible look at adverse events following immunization, and we analysed mild AEFIs for the Pfizer vaccine in its Phase 3 trial in adolescents and young men Analysis of COVID vaccine AEFIs, Part I. In this report, we will look at severe adverse events in four large-scale studies of surveillance data.
This analysis will require everything we have learned about base rates, risk ratios, and the processes for vetting individual AEFIs.
We will talk about several AEFIs, but focus on myocarditis, which is currently the most concerning of the severe adverse events.[1]
Our goals
We want to understand severe adverse events following the Pfizer mRNA vaccine. To do so, we will explore four studies and consider the following:
- Comparisons of AEFIs among vaccinated and unvaccinated populations, on the scale of millions of participants
- Identify the most troubling of the well-known AEFIs (myocarditis)
- Give risk ratios for myocarditis in the entire population and in the most vulnerable age groups for it. Risk ratios will be for the vaccinated over the unvaccinated, but also the COVID infected over the vaccinated.
- The excess incidence rate of myocarditis from vaccination.
- Why different studies give different results.
Myocarditis
Myocarditis is an inflammation of the heart muscle, with severity from mild to quite serious, or even fatal. It can affect the ejection fraction of the heart, and can lead to heart failure. The majority of myocarditis cases are non-severe and heart functioning often – but not always – returns to normal. Myocarditis most often afflicts young men.[2]
Myocarditis is given special attention in this article because of the numerous developing reports of it as both a consequence of COVID infection and as an AEFI of the mRNA vaccination for COVID.[3][4]These reports demonstrated that AEFI-associated myocarditis occurred most often within a few days of receiving the vaccine, and usually after the second dose.
Experimental design, limitations, and bias
Four peer-reviewed analyses of severe AEFIs to the Pfizer vaccine provide a range of risk ratios and excess incidence rates. The reasons for these variations are, in part, rooted in experimental design.[5][6][7][8]How so?
(a) The participants vary. In order to compare incidence rates of an adverse event, we must have medically comparable populations. This can be very difficult to achieve, and even studies done on the same general population may include or exclude participants based on different criteria. Differences in who is included can be important.
(b) Study time periods may vary. Medical scientists strive to choose the optimal study window. These windows vary between studies and will affect incidence rates.
- There is a known bias toward the most common AEFI disease types in the study period. All of the studies so far use relatively short study windows to identify the AEFI (20 to 30 days after a dose), but some diseases, such as chronic autoimmune disease, need much longer study periods to present, and could not be considered.[9]
(c) The criteria for vetting AEFIs differ between studies. This means that some studies will include an event that another study might exclude.
(d) Surveillance methods may differ. Some countries use active surveillance of AEFIs (Israel), while others use passive surveillance (United States).[10]The active surveillance systems are more likely to identify adverse events.
(e) Different use of control groups. Some studies have not used any control. Others use a matching or twinning system, where the outcomes of pairs of medically similar individuals are used. Historical data may also be used as control, although it is important that the relevant medical characteristics be appropriate (that is, very similar) for comparison to the study group.[11]
(f) Statistical analysis, particularly when breaking out by age group, can be inconsistent. If this is not done correctly, important results can be missed.[12]This is shown to be a material consideration for myocarditis.
(g) Different levels of clinical suspicion can affect whether an adverse event is detected. There can be under-reporting and over-reporting.[13]
(h) Mild adverse events were specifically not included in the four studies we examine here. Asymptomatic cases of any of these diseases will be missed, although COVID infection was confirmed by PCR testing.
(i) Studies may record AEFIs by dose in multi-dose vaccinations, rather than by the individual going through the course of a vaccination program. This requires careful analysis in order to be useful for real-world decisions that we would like to make with knowledge of risks to an individual going through a course of vaccinations.
It should be noted that these particular studies do not involve experiments on people, but analysis of data. For regions or healthcare organizations that keep detailed records, this can be done after the fact.
Design comparison of the four studies
The table below summarizes key differences in the four studies. Of particular note are studies A, B, and C from Israel, where the AEFI system uses active surveillance. The system is not dependent on submissions from doctors. Any patient record is automatically submitted to a digital system, which can be accessed by researchers. Study D, from the U.S., uses a passive reporting system, and it is reasonable to suspect that AEFIs could be under-reported.
Another key difference is the study period. Study C is superior from this perspective. In this study, AEFIs were scrutinized for 30 days after the second dose was administered. The other studies used a 42-day window from the date of Dose 1. Because not all second doses were administered in 21 days, some second-dose observation periods may have been short in Study A, B, and D. Furthermore, we will see that the second dose from the Pfizer mRNA vaccine results in significantly higher cases of myocarditis, making the clinical observation period following the second dose critical.
Study C reports AEFI incidence by dose. Given that the Pfizer vaccine is a two-dose regimen, we needed to be certain that our calculations accounted for this, so AEFI rates were per 100,000 persons rather than per 100,000 doses. Individuals who did not complete their second dose also needed to be accounted for in the rate calculation.
This fact that the second dose seems to be more likely to result in AEFIs creates another problem for Study D; the control group is a 21-day delay after vaccination. Such a procedure places inordinate trust on the assessment of the period in which an AEFI might occur.
Because of its more precise observation period, Study C may have design superiority for identifying the true rates of myocarditis due to vaccination, but Study A also has unique value. Study A seeks to identify the effects of COVID for a set of similar diseases against its own control. This will allow a comparison of COVID disease outcomes to vaccine AEFIs.
In the chart, “Object 1” refers to the first variable, which for all four studies is the vaccinated group. “Object 2” refers to a potential second variable, which for Study A is an analysis of secondary disease conditions following COVID infection.
Table summarizing key points of each of the four studies. Studies A, B, and C are conducted from data in Israel, while study D is of individuals from the United States. Key differences in the studies include the method of controlling the study (control group), the study period, and some of the aims. The data from the U.S. uses passive surveillance, while the Israeli data uses active surveillance and is more likely to find adverse events.[14][15][16][17]
Defining our criteria: risk ratios and risk differences
A risk ratio (RR) is the incidence rate of one medical outcome – a disease event, in this study – divided by the incidence rate (a base rate) of another outcome. Depending on which outcomes are being compared, a high RR may be desirable or undesirable. Risk ratios are sometimes called relative risk.
A risk difference (RD) is the incidence rate of the group exposed to the disease (or vaccine) minus the incidence rate of the unexposed group. This is sometimes called attributable risk, excess incidence rate, or excessive risk. For example, if we are trying to discern the true number of AEFIs that may be caused by a vaccine (as a rate), we might use this term. In such a case, the unexposed group – who comprise the experimental control – might be the placebo group in a Phase 3 trial, or unvaccinated people in a surveillance study.[18]
Risk ratios and risk differences will be reported per 100,000 persons going through the two-dose vaccination schedule in this analysis, unless otherwise stated.
The use of risk ratios and risk differences together is instructive. A high risk ratio gives confidence that the AEFI is being caused by the vaccine. In such a case, the risk difference becomes the estimate of how many cases per 100,000 are being caused by the vaccine. Thus, risk ratios are seen as qualifiers, and risk difference as the true (causal) rate estimate. The calculations of RR and RD in this article are exactly as defined above. The studies often employed slightly different calculation methods. These differences are not material.
Readers should be aware that all rates, ratios, and differences are estimations only.
Results I: a broad look at AEFI incidence rates
The table below summarizes the results for the study of the Pfizer vaccine AEFIs from each of the studies. Note that only Study A and D covered a wide range of AEFIs, so only they could be included in this table. While both Study A and D covered a large number of potential AEFIs, they did not examine the exact same list of adverse events, making an exhaustive comparison impossible. We therefore include a subset of the results.
Study A shows a higher incidence rate of AEFIs for all but one disease. This is likely due to experimental design elements such as the passive surveillance method of Study D, which was expected to miss events.[19]Study D lumped myocarditis and pericarditis results together, which compromised the assessment of myocarditis incidence rates.
The conclusion from Study D was that there is no material severe adverse events incidence rate increase due to the vaccine. But these results are questionable for the reasons of experimental design.
The main conclusion from Study A was that there was a significant incidence rate increase for myocarditis to 2.2 events per 100,000 with a risk ratio of 3.5 and a risk difference of 1.6 events per 100,000. This result is for the entire tested population.
The elevated cases of lymphadenopathy bears mention. Lymphadenopathy is a swelling of the lymph nodes and is a relatively common symptom in the general population. The risk ratio from vaccination is 2.4 and at a risk difference of over 46, occurs in a significant number of patients. It is thought that this adverse event is an immune response to the vaccine, and is considered to be mild. This will be studied further, but the current main concern is that lymphadenopathy can be a cancer symptom. Thus vaccine-caused lymphadenopathy can create a dilemma to diagnosticians of cancer patients or individuals being screened for cancer. Is the swelling a sign of cancer, or only an immune response to the vaccine? This question may result in unnecessary biopsies and patient stress.[20]
Table showing key AEFI results from Study A and Study D.[21][22]Study D shows no significant AEFI incidence rate increases due to the COVID vaccine. This conclusion is in doubt due to the passive surveillance methods of Study D, and the lumping of pericarditis with myocarditis. Study A shows that myocarditis and lymphadenopathy incidence rates are higher for the vaccinated group.
Results II: COVID infection secondary disease rates
Study A provides critical context to the AEFI analysis by including a study of the same adverse events on individuals who had a COVID infection. This analysis included a twinned control group and utilized a 42-day study window. A number of these events (secondary diseases) occur in significantly higher numbers as a result of COVID infection than as AEFIs to the mRNA vaccination. This includes kidney disease, arrhythmia, and myocadiac infarction.
Table showing the risk differences per 100,000 persons for COVID infection or mRNA vaccination from Study A.[23]For every disease except for lymphadenopathy, COVID infection results in higher numbers of adverse events than for vaccination. For context, it should be noted that 100% of people who are vaccinated face the risks associated with adverse events, whereas, to date, a fraction of the population has been infected with COVID.
Results III: a close look at myocarditis
Although, as above, COVID causes numerous significant secondary diseases at high rates, we delve deeper into the myocarditis AEFI as identified in Study A. The table below compares the results of each of Study A, B, C, and D on all individuals tested. It shows the AEFI rate, the unvaccinated control group rates, the risk ratios, and risk differences. Note that Study B lacks a control group.
Study A, B, and C show similar myocarditis AEFI rates, while Study D – which used passive reporting – showed a significantly lower rate of myocarditis. Both Study A and C suggest a population-level myocarditis risk difference rate of about 1.6 persons per 100,000.[24][25][26][27]
Comparison of the results of the four studies, for all individuals in each study. Myocarditis AEFIs appear to be vaccine-causal, with low but notable risk differences of about 1.6.[28] [29][30][31]
Myocarditis rates stratified by gender and age
Study C delved into the relationship of gender and age on myocarditis AEFIs. The key results have been adapted as a table and bar chart below. Females develop myocarditis at a much lower rate than men, and younger men develop myocarditis at much higher rates than older men. For example, about 0.25 per 100,000 excess cases of myocarditis were shown in women, while there were 13.69 excess cases per 100,000 in men aged 16-19.
This equates to an estimated risk of vaccine-caused myocarditis in men aged 16-19 at 1 in 7,357, or 0.014%. This is similar to the COVID death rate reported in the 16-17 age group of 0.013% as reported in the article Analysis of the Pfizer COVID vaccine trial for children aged 5 to 11. This near 1:1 ratio of a severe AEFI to a disease death rate does not compare well to other AEFIs and their associated vaccine-preventable diseases, as discussed in the article Analysis of COVID vaccine AEFIs, Part I. This is a case of a disease that does not kill many people in an age group in which vaccination shows the highest risk for a particular AEFI. Most of the myocarditis cases in the Israel studies were categorized as mild (for myocarditis), but one led to death.
Age and gender stratified data table adapted from Study C. All rates are per 100,000 persons. This data shows that females experience myocarditis as a result of the mRNA COVID vaccine in very low numbers. The incidence rate among men is significantly higher, especially among young men. The RD among men in the 16-19 age category is estimated at 13.59 cases per 100,000, which is 1 in 7,357.[32]
Age and gender stratified AEFI analysis from Study C. All rates are per 100,000 persons.[33]
Interpretation
In comparing four studies, we see a range of estimates for the rate of myocarditis AEFIs. This is the result of experimental design, different populations, and, likely, the reporting system used. Reported numbers are shown to be highly dependent on the demographic classification system used.
From the viewpoint of an entire population, the Pfizer mRNA vaccine, to date, shows a reasonable safety profile.[34]COVID has been shown to cause numerous secondary diseases, while the high rates of vaccine-caused myocarditis in young men is troubling. The estimated risk of vaccine-caused myocarditis in men age 16-19 appears to be on the order of 1 in 7,357. This has led some health agencies to question the safety of mRNA vaccines for young men.[35]
Also of concern is the high incidence of lymphadenopathy following vaccination, which could be an indicator of serious health issues and causes challenges in cancer screening.
Longer-term studies will undoubtedly follow as more data is acquired and adverse events come to be better understood.
References
[1] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[2] National Organization for Rare Disorders, Myocarditis
[3] Montgomery, Jay, et al., 2021, Myocarditis Following Immunization With mRNA COVID-19 Vaccines in Members of the US Military, JAMA Cardiol., V 6 N 10, 1202-1206
[4] Wallace, Megan, and Sara Oliver, June 23, 2021, COVID-10 mRNA vaccines in adolescents and young adults: Benefit-risk discussion, ACIP Meeting
[5] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[6] Witberg, Guy et al., 2021, Myocarditis after Covid-19 Vaccination in a Large Health Care Organization, N. Engl. J. Med.V385, 2132-2139
[7] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[8] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[9] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[10] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[11] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[12] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[13] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[14] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[15] Witberg, Guy et al., 2021, Myocarditis after Covid-19 Vaccination in a Large Health Care Organization, N. Engl. J. Med.V385, 2132-2139
[16] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[17] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[18] Kim, Hae-Young, 2017, Statistical notes for clinical researchers: Risk difference, risk ratio, and odds ratio, Restorative Dentistry and Endodontics, V 42, N 1, 72-75
[19] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[20] Tu, Wendy, et al, 2021, COVID-19 Vaccination-related Lymphadenopathy: What to be aware of, Radiology: Imaging Cancer, V 3, N 3
[21] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[22] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[23] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[24] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[25] Witberg, Guy et al., 2021, Myocarditis after Covid-19 Vaccination in a Large Health Care Organization, N. Engl. J. Med.V385, 2132-2139
[26] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[27] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[28] Barda, Noam, et al, 2021, Safety of the BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Setting, New England Journal of Medicine, 385
[29] Witberg, Guy et al., 2021, Myocarditis after Covid-19 Vaccination in a Large Health Care Organization, N. Engl. J. Med.V385, 2132-2139
[30] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[31] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[32] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[33] Mevorach, Dror, et al., 2021, Myocarditis after BNT162b2 mRNA Vaccine against Covid-19 in Israel, N. Engl. J. Med.V385, 2140-2149
[34] Klein, Nicola, et al, 2021, Surveillance for Adverse Events After COVID-19 mRNA Vaccination, JAMA 2021: 326(14) 1390-1399
[35] Mitchell, Laine, Nov 23, 2021, Some Albertans asked to turn down Moderna vaccine over rare side effect, DH News
I appreciate your comment, George.
Well done Lee, this add some clarity to the variance of risk factors of the vaccine I have heard and read about.