{"subscriber":false,"subscribedOffers":{}} An Overview Of Vaccine Development, Approval, And Regulation, With Implications For COVID-19 | Health Affairs

Cookies Notification

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more.



An Overview Of Vaccine Development, Approval, And Regulation, With Implications For COVID-19

  1. Aaron S. Kesselheim ([email protected]) is a professor of medicine and the director of the Program on Regulation, Therapeutics, and Law in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, in Boston, Massachusetts.
  2. Jonathan J. Darrow is an assistant professor of medicine in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School.
  3. Martin Kulldorff is a professor of medicine in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School.
  4. Beatrice L. Brown is a research assistant in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School.
  5. Mayookha Mitra-Majumdar is a research scientist in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School.
  6. ChangWon C. Lee is a research assistant in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School.
  7. Osman Moneer is a medical student at the Yale School of Medicine, in New Haven, Connecticut.
  8. Jerry Avorn is a professor of medicine in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School.
PUBLISHED:Free Accesshttps://doi.org/10.1377/hlthaff.2020.01620


The Food and Drug Administration (FDA) approves vaccines when their benefits outweigh the risks for their intended use. In this article we review the standard FDA approach to vaccine evaluation, which underpins its current approaches to assessment of vaccines to prevent coronavirus disease 2019 (COVID-19). The FDA has established pathways to accelerate vaccine availability before approval, such as Emergency Use Authorization, and to channel resources to high-priority products and allow more flexibility in the evidence required for approval, including accelerated approval based on surrogate markers of effectiveness. Among the thirty-five new vaccines approved in the US from 2006 through October 2020, about two-thirds of their pivotal trials used the surrogate outcome of immune system response, and only one-third evaluated actual disease incidence. Postapproval safety surveillance of new vaccines, and particularly vaccines receiving expedited approval, is crucial. This has generally been accomplished through such mechanisms as the Centers for Disease Control and Prevention (CDC) and FDA Vaccine Adverse Event Reporting System, the CDC Vaccine Safety Datalink, and the CDC Clinical Immunization Safety Assessment Project. Adverse events detected in this way may lead to changes in a vaccine’s recommended use or withdrawal from the market. Regulatory oversight of new vaccines must balance speed with rigor to effectively address the pandemic.


As complex biological products administered to millions of generally healthy people, vaccines have been among the most carefully evaluated medical products. Historically, it has taken years to move a vaccine from initial discovery to Food and Drug Administration (FDA) approval.1 But the unprecedented impact of the coronavirus disease 2019 (COVID-19) pandemic has brought attention to the process of vaccine development and evaluation and how it can best be expedited. Vaccine regulatory assessment demands a balance of efficacy, safety, and speed. In this article we review current and potential approaches to this critical FDA role.

Vaccine Clinical Testing And Approval

Vaccine approval comes under FDA authority through the Federal Food, Drug, and Cosmetic Act and is also governed by the Public Health Service Act, which regulates biological products. The process and requirements for vaccine approval and regulation therefore follow a pattern similar to those for other medical products, including preclinical testing, human testing, and postapproval safety monitoring. The FDA Center for Biologics Evaluation and Research is responsible for vaccine approval and regulation.

Once in vitro testing and animal studies help identify the appropriate dosage and provide pharmacokinetic and toxicology data,2 a manufacturer submits an Investigational New Drug (IND) application to the FDA. It contains preclinical data, a description of the proposed manufacturing process and quality control procedures, and a description of planned human trials.3 After the manufacturer submits a valid IND application, such trials can proceed. If preliminary data from these trials raise safety or efficacy concerns, the FDA may request additional studies or pause the trials.

Phase I clinical studies assess vaccine safety, dosage, and capacity to induce an immune response in a small number of healthy subjects. Phase II trials evaluate initial safety and efficacy in a larger population, perhaps a few hundred participants. Phase III trials provide more definitive evidence of a vaccine’s efficacy. They are usually large, randomized, blinded, and controlled, and they involve hundreds to thousands of subjects. Because vaccines are administered to healthy people, there is a low tolerance for adverse events, even rare ones.4 This requires a larger sample size than would be needed, for example, for a study of a new antibiotic to treat an acute infection. As a result, the Phase III trials that generate the pivotal data supporting FDA approval are generally much larger for vaccines than for other medical products.5 If additional safety or—less commonly—efficacy questions remain, the manufacturer may commit to one or more Phase IV studies to be conducted after approval. Phase IV studies are randomized trials and other investigations into new drugs conducted after FDA approval.

The FDA can rely on several programs to expedite development and regulatory review of new vaccines by channeling agency resources to high-priority products and accepting greater uncertainty by allowing more flexibility in the evidence required for approval.6 Three programs expedite FDA approval: fast track, breakthrough therapy, and accelerated approval.

With fast-track evaluation, for products that are designed to prevent a life-threatening disease or condition and that have the potential to address an unmet need, manufacturers receive the benefit of heightened internal prioritization by the FDA during clinical development and can submit portions of the licensing application on a rolling basis. With breakthrough therapy designation, intended for products that may offer a substantial benefit over existing options, manufacturers receive these fast-track benefits plus more formalized FDA response-time commitments. Under accelerated approval, permission to market a product may be based on surrogate measures, such as antibody levels, that might not be well established but are seen as reasonably likely to predict clinical benefit. In June 2020 the FDA indicated that no acceptable surrogates yet existed for a COVID-19 vaccine and that unless agreement is reached with the FDA on the use of an appropriate surrogate, primary endpoints should be limited to severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection confirmed serologically or virologically.7 But the agency left open the possibility that future insights into COVID-19 immunology might lead to definition of an acceptable surrogate.

Other programs authorize special access to a vaccine before FDA approval: expanded access and Emergency Use Authorization.8 Expanded access allows patients with serious or life-threatening conditions to request experimental products from the manufacturer before FDA approval. For example, in 2014 the FDA allowed expanded access to a meningococcal group B vaccine (Bexsero) during an outbreak at Princeton University more than a year before the vaccine was approved.9 In a declared public health emergency the FDA commissioner can issue an Emergency Use Authorization, allowing more widespread use of a vaccine before it meets the substantial evidence criteria for FDA approval, so long as the FDA determines that the product’s potential benefits outweigh its potential risks.10 On February 4, 2020, the secretary of health and human services (HHS) declared that COVID-19 posed such a threat, and in March the FDA issued an umbrella Emergency Use Authorization covering certain ventilators and other products. There has been substantial debate over whether and how Emergency Use Authorization should best be used for to COVID-19 vaccines and how best to define potential benefits and risks.11

In the non–emergency authorization pathway, once a vaccine successfully moves through Phase III trials, the manufacturer submits a Biologics License Application. The FDA generally solicits input from the Vaccines and Related Biological Products Advisory Committee (VRBPAC), an outside group of experts, for advice on the approval decision. The FDA usually follows the recommendations of the committee but is not legally required to do so.12 The role of VRBPAC attracted considerable attention following controversies surrounding the use of Emergency Use Authorizations for COVID-19-related treatments that lacked clear evidence for efficacy, with several authorities and professional groups demanding that approval by the advisory committee be required before any large-scale deployment of a COVID-19 vaccine.13

The FDA has generally refrained from setting minimum efficacy thresholds for vaccines. However, a 2007 FDA guidance document indicated that accelerated approval of a vaccine for a pandemic influenza virus can potentially be supported by evidence showing that the lower bound of the 95 percent confidence interval for the percentage of subjects achieving seroconversion (the production of detectable antibodies) was at least 30 percent.14 Similarly, in the early months of the pandemic, the FDA released guidance on COVID-19 vaccine evaluation indicating that the product should reduce disease incidence or severity in at least 50 percent of subjects, with a lower bound of the 95 percent confidence interval of more than 30 percent.15 While FDA guidance documents are generally not binding, after concerns emerged that political pressure might force the FDA to approve a vaccine before Election Day even if it did not meet efficacy or safety criteria, in October 2020 the FDA took the usual step of releasing updated guidance reaffirming its efficacy threshold and announcing minimum standards for vaccine safety evaluation.16

Some vaccines achieve levels of efficacy of 80 percent or higher in clinical trials,9 whereas annual influenza vaccines may achieve more modest levels, similar to the efficacy threshold the FDA initially proposed for COVID-19 vaccines.15 Early figures released by Pfizer and BioNTech in November 2020 described results of Phase III study data showing that their mRNA-based vaccine candidate appeared to be more than 90 percent effective in preventing COVID-19.17 Sometimes, vaccine efficacy as measured in clinical trials may imperfectly predict effectiveness in routine care owing to differences in real-world patient characteristics and practice patterns (such as whether a person receives both doses of a two-administration vaccine).

Even with modest efficacy, however, a vaccine can reduce disease incidence, hospitalization, mortality, and disability,18 either directly or through herd immunity if its uptake is sufficiently widespread. Between 2008 and 2018 the annual influenza vaccine varied in efficacy between 19 percent and 60 percent, with a mean efficacy of 45 percent, in part because of the difficulty of predicting which strain of influenza will become widespread in any given year.19,20

Safety Studies

For any vaccine, efficacy must be weighed in light of the risk of disease occurrence and the incidence and severity of vaccine adverse effects.

For any vaccine, efficacy must be weighed in light of the risk of disease occurrence and the incidence and severity of vaccine adverse effects. Not all adverse reactions can be detected during preapproval clinical trials. Rare but serious adverse reactions are a particularly salient concern for a pandemic vaccine intended to be administered to healthy members of nearly the entire population in a short period of time. Even moderately large trials might not be sufficiently powered to define important, rare safety risks. Study participants also might not be fully representative of the population to be vaccinated in terms of their age, race/ethnicity, frailty, comorbidities, genetics, or pregnancy status. It is therefore necessary to conduct postmarket safety surveillance to understand how the vaccine performs in a real-world setting. This is particularly important for vaccines developed under an expedited timeline or those that use molecular approaches never before deployed in any marketed product, both of which are characteristics of several COVID-19 vaccines.

The contemporary postapproval surveillance and safety system for vaccines involves Phase IV postapproval studies and other postapproval oversight and analysis: the CDC and FDA Vaccine Adverse Event Reporting System (VAERS), the CDC Vaccine Safety Datalink, and the CDC Clinical Immunization Safety Assessment (CISA) Project.

Phase IV studies to obtain additional efficacy and safety data may be conducted at the discretion of the manufacturer or sought by the FDA at the time of vaccine licensure.21 Case-control or cohort studies designed to study a particular adverse event are common Phase IV study designs.22 However, analyses of required Phase IV studies across all drugs and biologics have found that they are frequently not completed on time, if at all.23

VAERS,24 established in 1990, is a spontaneous reporting system in which clinicians, manufacturers, and the public can voluntarily report adverse events after vaccination. It allows the CDC and the FDA to monitor new, unusual, or rare adverse events and to determine whether further studies are warranted.25 At the Uppsala Monitoring Centre, the World Health Organization assesses the output of VAERS in light of findings from similar approaches around the world. One limitation of VAERS is underreporting, with the reporting of sensitivities to the system varying widely across vaccines and types of adverse events.26 This problem is also well documented for the FDA’s analogous drug adverse event reporting system. Because spontaneous reports lack denominator data and reflect voluntary, unsystematic reporting, VAERS is most relevant as a tool for generating hypotheses for other studies and usually cannot be used alone in determining causality.27

To more systematically study potential safety problems, the CDC established the Vaccine Safety Datalink in 1990. It aggregates data from health systems around the US, representing about ten million patients.28 Each site contributes routinely collected electronic health data that can be used to monitor vaccine safety and conduct studies of rare and serious adverse events.29

The CISA Project,30 using the statistical signals reported by the Vaccine Safety Datalink, enables vaccine safety experts to conduct detailed clinical reviews of patients who had an adverse event possibly caused by a vaccine and to identify possible risk factors.31 These studies are particularly important for understanding adverse events in certain populations, such as pregnant women and immunocompromised patients, who are typically excluded from prelicensure clinical trials.

One approach used by the Vaccine Safety Datalink is Rapid Cycle Analysis, in which weekly data feeds are analyzed using sequential statistical methods. When a prespecified threshold is exceeded, it may indicate a potential problem requiring evaluation. For example, the year after the measles-mumps-rubella-varicella (MMRV) vaccine was introduced in 2006, after the administration of about 43,000 doses,32 the Vaccine Safety Datalink detected the possibility of one additional febrile seizure per 2,000 children receiving the vaccine. This led to a change in national recommendations, which removed the preference for the MMRV vaccine over separate measles-mumps-rubella and varicella vaccines.33

Many vaccine-related adverse events may be unexpected. Using codes from the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, a novel tree-based statistical scanning approach makes it possible to evaluate thousands of different potential adverse reactions that would otherwise generate hundreds of false positives based on chance alone.34 W. Katherine Yih and colleagues used this approach to evaluate the quadrivalent human papillomavirus vaccine and found only mild adverse reactions such as injection-site rashes.35 To complement the CDC postmarket safety surveillance, the FDA uses data from the Centers for Medicare and Medicaid Services (CMS) and the FDA’s Sentinel Post-Licensure Rapid Immunization Safety Monitoring (PRISM) System, which was inaugurated during the 2009 H1N1 pandemic.36 The FDA is also setting up a new system based on commercial insurance claims data to replace or complement the Sentinel PRISM System.

If any of these surveillance approaches reveals a mild or very rare adverse reaction, it may lead to an additional cautionary statement on the product’s labeling. Labeling changes for safety problems after a new vaccine’s approval have been less common than for new drugs.

When more serious problems arise after a vaccine has been approved and included in the CDC guidelines, the Advisory Committee on Immunization Practices (ACIP) may revise its recommendation, recommending either the use of a different vaccine or no vaccination at all. Even the unconfirmed possibility of a serious problem can lead to voluntarily market withdrawal. This occurred with LYMErix, a vaccine developed to prevent Lyme disease. After 1.4 million doses were administered, fifty-nine cases of arthritis were reported to VAERS. Although the rate was similar to that seen in unvaccinated individuals and a postlicensure study by the manufacturer did not find a higher rate of adverse reactions among vaccine recipients, the manufacturer withdrew the vaccine from the market, citing poor sales that were likely a result of press coverage and the risks of ongoing litigation.37

The FDA can also initiate the removal of a vaccine from the market if it determines that statutory benefit-risk requirements are no longer satisfied. This is rare, but it occurred in 1999 after the administration of about 1.2 million doses of the Rotashield vaccine against rotavirus infection. During prelicensure trials, the number of cases of intussusception—in which part of the intestine telescopes into itself and causes bowel obstruction—was statistically indistinguishable from the background rate. But fifteen cases were reported to VAERS within a year of the vaccine’s introduction. A more systematic study using the Vaccine Safety Datalink found that the Rotashield vaccine was associated with an increased risk for intussusception in infants. Although the vaccine was still considered useful in countries where many infants die from diarrheal disease, ACIP determined that the risks for intussusception did not outweigh the benefits of the prevention of diarrheal disease in the US, where such disease is more manageable. Two subsequent rotavirus vaccines were introduced in 2006 and 2008. They were thoroughly evaluated using VAERS spontaneous reports,38 the Vaccine Safety Datalink near-real-time weekly monitoring system,39 and the FDA Sentinel PRISM System40 and were found to be sufficiently safe.

A safety challenge particular to vaccines is the risk for immune enhancement, in which vaccinated subjects may develop more severe disease when exposed to the target pathogen than those who were not vaccinated. Although unusual, such a finding in the Philippines was the cause of the suspension of the Dengvaxia vaccine against dengue fever. Such rare reactions further increase the importance of effective and vigorous pharmacovigilance programs.

Past safety evaluations have sometimes used a comparator vaccine, well-care visits, historical population-based incidence rates, or self-controls in which a risk window soon after vaccination is compared with a comparator window from the same patient before or further away from vaccination. These programs can also provide information on the comparative effectiveness and safety of different vaccines directed against the same condition. For COVID-19, both the Vaccine Safety Datalink and the FDA and CMS surveillance systems are likely to be used to conduct near-real-time Rapid Cycle Analyses to quickly detect any potential safety problem and unsuspected adverse reactions.

Current Landscape Of Vaccine Approvals And Postmarket Studies

To provide context for the assessment of vaccines against COVID-19, we assessed the characteristics of pivotal trials of all new vaccines approved in the past fifteen years and reviewed their required postmarket studies. The methods and full results of this analysis are in the online appendix.41 We identified thirty-five novel vaccines approved in the US between 2006 and July 2020, including six that were the first vaccine approved for that disease (“first-in-disease” products), including Gardasil for human papillomavirus and Trumenba for meningococcal group B infections. More than half of the new vaccines were for adults (n = 20, 57 percent), and the number approved each year was stable throughout the period (appendix exhibit 1).41

Sixty-one pivotal trials were conducted for these novel vaccines. All were randomized, and most were double-blinded. About half (n = 28, 46 percent) used active controls, in which an already-approved vaccine product was compared with the experimental vaccine. The remainder were either placebo controlled (n = 23, 38 percent) or self-controlled (in which comparisons of antibody levels or other outcomes were made within individuals before versus after vaccination; n = 10, 16 percent). About two-thirds of the trials used the surrogate outcome of immunogenicity, measuring a change in antibody levels or a similar biomarker; only about one-third evaluated whether the vaccine actually reduced the incidence of the targeted disease (appendix exhibit 2).41 A similar division was seen even for first-in-disease vaccines (n = 15), in which nearly half of the trials relied on a surrogate measure of efficacy. Sub-unit-based vaccines were considerably more common than whole-pathogen vaccines and were far more likely to rely on laboratory tests to determine efficacy rather than actual clinical endpoints (76 percent versus 38 percent, respectively).

The pivotal trials enrolled a median of 2,415 patients (interquartile range: 884–4,605), with a median of 1,713 patients (IQR: 466–3,084) in the intervention group, and lasted for a median of 18.0 months (IQR: 8.7–27.2) (appendix exhibit 3).41

Of the thirty-five vaccines, thirty-two had commitments or requirements for postapproval studies. Twenty vaccines had statutorily mandated postmarket study requirements, including nineteen under the Pediatric Research Equity Act of 2003 for testing in children, six under the accelerated approval pathway for confirmatory testing of products based on non-well-established surrogate measures, and two under the FDA Amendments Act Section 505(o)(3) authorities for products with potentially serious safety questions.

Vaccine Injury Compensation

The National Childhood Vaccine Injury Act of 1986 established the National Vaccine Program to direct vaccine research and development and ensure the production, procurement, and distribution of safe and effective vaccines. The act also established the National Vaccine Injury Compensation Program to compensate people with certain injuries caused by specific vaccines using a “no fault” system as an alternative to litigation.42 The program is funded by a seventy-five-cent tax levied on each dose of CDC-recommended children’s vaccine.

Not all vaccines are covered under the National Vaccine Injury Compensation Program. The Public Readiness and Emergency Preparedness Act of 2005 authorized the HHS secretary to establish the Countermeasures Injury Compensation Program, which has been administered by the Health Resources and Services Administration since 2010. This program is designed to compensate individuals injured by countermeasures, including vaccines, that are administered during public health emergencies such as pandemic influenza and COVID-19.43 The standards for compensation are similar to those of the National Vaccine Injury Compensation Program: The requester has the burden of proving that they sustained a certain injury covered by the program within an allowable period after receipt of the countermeasure. As in the National Vaccine Injury Compensation Program, manufacturers are granted immunity from liability except in cases of willful misconduct.


Our review of novel vaccine trials from the past fifteen years showed consistency in some of the characteristics of the trials, including randomization and blinding. About half of the trials used active controls as comparators. Most pivotal trials enrolled large numbers of patients and required one to two years or longer to complete. We also found that most vaccine trials used surrogate measures of efficacy, predominantly immunogenicity, rather than demonstration of differences in the rate of disease incidence.

When immunogenicity is used as a surrogate measure to support vaccine approval, it is important for that surrogate to be well validated for predicting clinical protection. It then falls to postapproval Phase IV studies or other oversight activities by the manufacturer or the FDA to confirm the expected benefit in typical “real-world” populations. Nearly all new vaccine approvals in the past fifteen years came with postapproval commitments or requirements.

For pandemic vaccines, the approach to an approval decision must be calibrated to the fact that the new product will be administered to very large numbers of healthy people in a short period of time, shaping the preapproval benefit-risk determination. Because of the clinical and ethical implications of precipitating rare severe adverse effects, initial evaluation requires large randomized trials of considerably greater size than are needed for approval of a new drug, as well as meticulous postapproval safety surveillance. The time required to accrue adequate person-time experience in a trial can also be greater for a new vaccine compared with a new drug; unlike a trial for a medication to treat an acute condition, a vaccine trial must go on for many months before a statistically significant difference can be seen in the incidence of a condition that might not occur in most patients without a vaccine. This is particularly true when clinical events (such as disease incidence) are studied, rather than a surrogate marker such as antibody levels. Similarly, in an environment that demands the fastest possible availability of a vaccine for a pandemic disease, it is vital to not “short circuit” the assessment of safety within a clinical trial even after an efficacy endpoint is reached. This explains the FDA’s October 2020 decision to require that all clinical trials of COVID-19 vaccines continue for two months after the final subject was vaccinated, to ensure better detection of potential postvaccination safety problems.

If a vaccine candidate is expected to be only partially effective (for instance, reducing disease occurrence or severity in only 50 percent of people receiving it), the requirements for sample size are even more demanding. Simple measures of immunogenicity may be acceptable if the immune response is already well understood and the vaccine has a mechanism of action similar to existing vaccines, but they may be less justifiable if the pathogen is a new one, its immunology is incompletely understood, the vaccine embodies a new technology not employed previously, or some combination of these considerations.

After vaccine approval, it is imperative that systems be in place to detect signals of adverse events once the vaccine is in widespread use.

After vaccine approval, it is imperative that systems be in place to detect signals of adverse events once the vaccine is in widespread use. As with medications, approaches that require voluntary spontaneous reports are likely to be less useful than those that use routine surveillance of clinical events in millions of patients in typical care systems. Approaches such as the CDC Vaccine Safety Datalink are in place to make this possible. After a massive, perhaps nationwide immunization program against SARS-CoV-2, provision will also have to be made for compensating people who develop complications after vaccination, perhaps based on existing programs addressing this need for prior vaccines.


During the past several decades, health care systems throughout the world have accumulated substantial evidence, experience, and insights about vaccine development, use, and surveillance. Although the COVID-19 pandemic is unprecedented in the past century, insights from past programs for vaccine development, evaluation, approval, and surveillance can provide valuable understanding about vaccines for this new clinical, public health, and policy challenge.


Support for research by Aaron Kesselheim, Jonathan Darrow, and Jerry Avorn comes from a grant from Arnold Ventures and the Harvard-MIT Center for Regulatory Science. No other authors report any grant support for their work. The authors have no conflicts of interest. An unedited version of this article was published online November 19, 2020, as a Fast Track Ahead Of Print article. That version is available in the online appendix.


Loading Comments...