On the face of it, the Food and Drug Administration’s (FDA’s) regulatory mandate is straightforward: Approve drugs that are safe and effective for their intended use. However, as demonstrated by several recent decisions regarding drugs to treat conditions such as Alzheimer’s disease, Duchenne muscular dystrophy, and amyotrophic lateral sclerosis (ALS), the reality is often far more complex, necessitating a balance between speedy patient access and reasonable certainty about drug benefit. When should the FDA demand more evidence, even if it will mean that a promising medicine might take longer to reach patients? And when should it exercise “regulatory flexibility” based on the risk tolerance often exhibited by those facing terrible diseases without good treatment options?
In answering these questions, it’s important to remember the FDA’s critical role in keeping the market clear of drugs that don’t work—a role that has been increasingly challenged in today’s political, economic, and social climate.
Brainstorm Cell Therapeutics And NurOwn
The proper limits of regulatory flexibility were at the forefront of a September 27, 2023, FDA advisory committee meeting to assess whether BrainStorm Cell Therapeutics had provided adequate evidence to support approval of NurOwn, its investigational stem cell therapy for ALS. NurOwn’s rocky path to the advisory committee was more than a decade in the making, culminating in Phase 3 trial results reported in 2020. In that randomized, placebo-controlled study, NurOwn missed all primary and secondary endpoints by wide margins and was associated with a “modest excess in deaths” in patients who received the intervention.
For the past three years, BrainStorm has mined NurOwn’s Phase 3 results for signals of benefit through analyses that were not pre-specified. This can help inform future studies, but it is not a scientifically acceptable way to prove efficacy due to the inability to control for important biases. As explained by the FDA’s NurOwn reviewers, “Post-hoc subgroup analyses in general have high risk of finding false positive results due to lack of control for multiple hypothesis testing and potential confounding due to imbalance in the measured/unmeasured baseline prognostic factors brought about by breaking the randomization.” In plain English, this means that when you go looking through the data for positive results on analyses that weren’t planned in advance, you might keep looking till you find something that looks exciting—but, unfortunately, that something might just be a fluke. Moreover, you lose the benefits of randomization by selecting new subgroups that might differ in substantial and potentially nonobvious ways from the broader group of randomized participants. For example, if you look only at patients with a certain functional score, that subgroup might also happen to have a different assortment of environmental factors, diet, age at diagnosis, and rate of progression compared to the way those factors are assorted in the full participant population.
Despite these statistical concerns, BrainStorm relied on post-hoc analyses to support its insistence that NurOwn works, at least for patients with “mild or moderate ALS.” Rather than initiating a study to test that theory in a rigorous way, BrainStorm chose to rest its hopes on the possibility that the FDA would extend its regulatory flexibility much further than it has in the past. As a small company, BrainStorm indicated it could not afford to carry out another NurOwn trial without the profits generated by market approval. Perhaps tellingly, neither industry investors nor larger pharmaceutical companies have come forward to push the drug along, which might be expected if NurOwn were as promising as BrainStorm claims. It is also not clear whether the company pursued alternative funding opportunities available through the FDA’s Orphan Products Grants Program or its Rare Neurodegenerative Disease Grant Program.
In another worrisome signal, BrainStorm still has not told participants whether they were randomized to receive NurOwn or placebo, making it difficult to assess individual claims of perceived benefit. It also refused to share trial analyses with the ALS Association as part of that organization’s process for independent expert review prior to endorsing any drug for FDA approval.
The Advisory Committee Meeting
Given NurOwn’s Phase 3 results, among other problems, the FDA initially refused to accept the company’s application for approval, ultimately doing so only after BrainStorm followed a procedure to file its application over the FDA’s protest. Following intensive patient advocacy demanding public discussion of NurOwn, nudged along by BrainStorm’s claims about the drug, the FDA scheduled the September advisory committee meeting. The agency’s briefing materials concluded that the data were “grossly deficient” as to NurOwn’s efficacy, safety, and even consistent manufacturing.
During the open public hearing, some patients, family members, and other advocates, including several NurOwn Phase 3 trial participants, offered moving testimony urging the FDA to approve a drug that they believe to be effective, or at least promising enough for approval. Nonetheless, the advisory committee voted 17–1 (with one abstention) that the available evidence did not meet the required evidentiary standard. Notably, the committee’s patient representative—a person living with ALS—voted with the majority against the drug. The FDA is not bound to follow advisory committee votes but seemed unlikely to approve NurOwn given its position to date on the drug; despite prior approvals of drugs with failed trials, this seemed a bridge too far. With the writing on the wall, on October 18, 2023, BrainStorm announced it had withdrawn its application and is working to design and launch a second pivotal trial with the FDA’s input.
Why Not Just Approve?
Many in the ALS community were understandably devastated by this result, some because it reflected another failure in fighting this awful disease and others because they believed the FDA should approve any drug that offers dying patients a “glimmer of hope.” After all, some wonder, what’s the worst that could happen? As more than one patient commented during the advisory committee meeting, risks associated with NurOwn are mere possibilities in contrast to a certain death from ALS. Why not approve NurOwn—or other drugs in similar circumstances—so those who want to try it have the opportunity do so?
The answer to this reasonable question goes beyond the desires and choices of individual patients. Even if one believes that competent adults should be allowed to take risks that might cause them to die sooner, more miserably, or financially worse off than they otherwise would have, there are good reasons for the FDA to reject unproven drugs. First, doing so will help promote the generation of strong evidence to support the treatment decisions facing all patients, and second, it will minimize interference with promising drug development.
Before the FDA was statutorily required to ensure the safety and effectiveness of drugs prior to allowing them on the market, “snake oil” ran rampant, making it difficult to differentiate between worthwhile therapies and those that might only impact a patient’s wallet. Patient testimonials can be convincing, but—as the advisory committee’s industry representative colorfully noted—diet drug testimonials never come from people who’ve gained weight. This is why conducting well-designed trials is so important, namely, to learn both the good and the bad about a drug, including what works and for whom—and what’s not effective. Allowing drugs on the market without compelling evidence of effectiveness from such trials would weaken the statutory requirement that companies undertake studies designed to rigorously demonstrate a drug’s benefit. In that case, some companies might choose instead to conduct ambiguous studies that merely preserve the plausibility of benefit, with gaps filled by positive anecdotes; this would maximize the likelihood of marketing approval and eventual profit, since many patients will advocate for access in the absence of better alternatives.
Relatedly, when the FDA approves drugs that are mediocre at best, it becomes harder to conduct trials of new, possibly better alternatives. Why? Because, as the industry representative also noted, when new medicines enter the standard of care, it becomes more likely that patients will be taking them. Those patients may be uninterested in participating in studies of new drugs—a problem in itself—or they may be unwilling to forgo the approved product even if substantial uncertainty remains about its benefit. That means new drugs will have to be tested against the approved but unproven product, or in combination with it, making results difficult to parse. For example, if the new drug fails to show comparative improvement, is it because neither product works or because both do?
There’s one more argument in the “just approve” corner, but it too cannot withstand scrutiny. A major concern for the FDA, companies, and patients is that a failed trial will lead to Type 2 error: dropping a drug that is in fact beneficial, or that could be under the right conditions. Conducting further trials to identify those conditions can be resource intensive—and sometimes cost-prohibitive, a concern plaguing BrainStorm. Yet, that does not mean the solution is for the FDA to approve unproven drugs. By that logic, it would be hard to justify requiring companies to expend resources on conducting any trials at all, since approval would not necessarily depend on the results. Instead, the better approach involves identifying more effective mechanisms to address trial costs (in addition to funding the upstream science critical to support drug development); developing new funding opportunities to advance promising but still unproven drugs; and creating additional incentives to demonstrate drug benefit.
Importantly, the FDA also allows patients with serious or life-threatening diseases who lack appropriate therapeutic alternatives and are unable to participate in trials to be treated with unapproved drugs through its expanded access pathway (commonly known as compassionate use). This approach acknowledges both that some patients truly do not have time to wait for approval and that drug approval in the face of uncertainty is not the only road to access. Although the FDA nearly always authorizes expanded access requests, this opportunity could be improved by raising clinician and patient awareness, reducing logistical hurdles, encouraging companies to participate, and addressing cost and other concerns.
Conclusion
Unfortunately, drugs don’t work just because we want them to. It is reasonable for the FDA to consider disease severity and unmet treatment need when evaluating the adequacy of evidence available to support drug approval—for example, exercising regulatory flexibility to accept a single positive trial rather than two (as it did when approving Relyvrio for ALS) or compelling biomarker data even when clinical endpoints failed (as it did for Qalsody, another recently approved ALS drug). But this sort of regulatory flexibility must have limits. Based on the evidence currently available, NurOwn clearly falls outside the lines; its approval would have bent FDA standards to their breaking point. We can debate the right balance between speed and certainty, but we cannot forget that the FDA is what stands between us and a world in which patients have their pick of unproven drugs but lack both meaningful new therapies and critical information to guide their treatment decisions.
