FDA: Lessons Learned

Insights from "Reputation and Power"

I previously summarized an organizational history of the FDA. Here are some takeaways from that book.

Lessons Learned

Approval is Final

Because approval of a drug is so symbolically powerful, and effectively stakes the FDA's reputation to a given drug, the FDA only does so very deliberately. It is a social technology that reduces the immense complexity of an IND application (consisting of clinical trials, endless manufacturing and absorption data, etc.) into a binary YES/NO that physicians and nurses can rely upon. Reversing a decision is reputationally damaging, and the FDA has occasionally faced criticism from consumer protection groups and even internal FDA employees that it is too unwilling to withdraw unsafe drugs from the market. This is problematic when drugs are approved based on surrogate endpoints instead of clinical endpoints, or when safety problems emerge after approval, as with Vioxx. 

 Carpenter on this: 

It is interesting in light of these conflicts that, in the wake of the Vioxx tragedy of October 2004, higher FDA officials (including many long-term careerists) engaged in an organizationally motivated embrace of the status quo by defending randomized controlled trials and by disparaging pharmacoepidemiology. For different reasons, Deputy Commissioner Scott Gottlieb, CDER officials Sandra Kweder, Robert Temple, and others did not want to cede more control of the pharmaceutical market to David Graham and his colleagues at ODS. Yet it was also an extension of the familiar, an area where the agency had already developed capacity. Clinical trials have advantages when they are randomized and placebo-controlled. They also have drawbacks. Often tests are done on homogenous patient populations, among patients who differ in many ways from the patients who will utilize the drugs in clinical situations. Clinical trials usually have an endpoint, and can often be too brief to allow analysts to detect whether the drug in inducing adverse events, particularly for toxicity, hepatotoxicity (or liver damage), and cardiovascular outcomes. 

On pressuring the FDA through reputation

The FDA is responsive to both reputational and political pressure. The former is best thought of as arguing with the FDA on its own ground, which can be done from multiple perspectives. Patient advocacy groups can push the FDA to approve cancer drugs on less evidence; "thought-leader" physicians can sing the praises of an innovative drug and call the FDA slow; FDA whistleblowers can testify before Congress that the FDA is too deferential to pharma; consumer protection groups can call the FDA's approval based on a surrogate endpoint "reckless". All these approaches seek to push the FDA in one direction or another, but fundamentally accept the legitimacy of the FDA and especially in the case of those pushing for more regulatory caution, harken back to some idealized version of the FDA as a rhetorical device. In the early 2000's, as the FDA was perceived to have relaxed its regulatory standards, especially on cancer, this occurs more often, with long-time FDA critics like Sydney Wolfe unfavorably compared the current FDA to the old FDA .

A recent example of reputational pressure was Eric Topol's open letter to the FDA in October 2020 that criticized the emergency approvals of convalescent plasma, hydroxychloroquine and remdemisvir, and pressured the FDA Commissioner to delay approval of a Covid-19 vaccine. Though many have criticized his actions, its method is illustrative: Topol, who has immense reputational power in academic medicine through decades of leading large clinical trials, publicly attacking Merck during the Vioxx scandal, and having critiqued the FDA in the past for its lack of action on Vioxx, was well-positioned to push the FDA to be more cautious. In addition, the emergency approvals Topol criticized (with the possible exception of Remdemisvir, and maybe Convalescent Plasma if you play with subgroup analysis....) seemed to have been regulatory bets that did not pan out, which left the FDA in a weak position. The politicization of HCQ in particular made Topol's arguments extremely appealing among the reputational audience (large medical journals, elite media, etc. ) that the FDA caters to, which all dislike Trump. The combination of a weakened FDA and a strong attacker were the likely reasons for Topol's success (and in the view of Alex Tabarrok and many others, the rest of the US's disaster, since any delay in vaccine approvals likely cost many lives.) Here an in-depth read on this. 

The pro-regulatory counter to the previous paragraph is that vaccine approval was only an obviously good idea in retrospect and that vaccine hesitancy would rise with a rushed process. I think both of those claims are wrong, and I think Ezra Klein argues this well in a recent piece. 

Political Pressure

Political pressure takes the form of more direct action: Congressional committees can ask FDA officials to justify their actions and have public hearings that embarrass an agency that prefers less public attention. They can threaten FDA funding and if they're playing hardball, threaten legislative action that directly alters FDA authority.  

The challenges to FDA practice which resulted in sustained reform are those that combine both approaches, along with policy solutions that can be proposed at the right political moment. Conservative think tanks had long wished to tie some FDA funding to drug approvals and hold the FDA to a deadline and when conservatives made sustained and substantial gains in the 80's and 90's, the FDA, first informally and then through legislative change, moved in that direction. 

Sustained media attention also seems to be important for pushing the FDA, but is not essential. The NCI-FDA disputes were not as high-profile as the 1980's AIDS activism but effected reform that was just as important.

An important but vague "audience" that the FDA defers to is the medical community as a whole. This might be thought of as the "elite consensus" in medicine. By directly incorporating respected medical scientists and doctors onto Advisory Committees, the FDA acedes to this audience but also partially neutralizes it. Pharmaceutical companies seek to use this in their favor as well by recruiting big-name academic stars to head clinical trials or research divisions and thereby "borrow" some of their reputation. A sustained critique of the FDA from these directions would also likely be a powerful pressure. For instance, if Advisory Committees repeatedly disagreed with the FDA in one direction, they could likely shame it into changing course.

Theoretically this might be strengthened if a sitting Congressman/woman then called a hearing to directly ask FDA officials about controversial decisions or if sympathetic media ran pieces highlighting this discrepancy. This would be a direct reputational attack on the FDA and a veiled political threat because it would raise the specter of legislative reform of the FDA with enough political attention.

Regulatory Caution is Often Right

As disappointing and non-contrarian as it is to agree with a large slow-moving government bureaucracy, my impressionistic summary of Reputation and Power along with background knowledge from Ending Medical Reversal and Malignant is that the FDA has usually been proven right in its caution. Approvals based on surrogate endpoints sometimes work, but in Malignant Vinay Prasad makes a strong argument that this often doesn't speed approvals and that a substantial number of drugs approved on surrogate endpoints are never properly followed up on. Low regulatory standards in cancer drugs have led to a proliferation of low-value treatments approved on the basis of surrogate endpoints that don't predict clinical (AKA useful) endpoints and that may not actually provide any benefit outside of carefully selected clinical trial participants.

A full argument that strict regulation is required for medical innovation is too long for this paper, but Ending Medical Reversal is an excellent argument along these lines. Here is a summary of it. I fully agree with Cowen and Tabarrok that the FDA was far too slow during Covid-19, but I think the libertarian critique of the FDA (usually) goes too far. 

The classic response to this is the invisible graveyard of people who died because of regulatory caution or slowness in approving new drugs. I don’t have enough background knowledge to be confident about this point.

The FDA is More Powerful Than You Think

Apart from holding pre-market approval power over a drug, the FDA also intensely regulates drug experimentation in the first place. It also regulates drug labeling. By virtue of regulating a drug's intended use (which has substantial effects on insurance coverage) it also informally regulates medical practice in general. And in a decentralized fashion, by being able to disqualify Institutional Review Boards (IRB's), it also regulates clinical research on a fine-grained level.

This IRB-mediated power means the FDA can effectively ban individual physicians or entire institutions from research: 

To the degree that overly-cautious IRB's (who are in turn fearful of FDA attention) constrained Human Challenge Trials, this seems like an under-appreciated cost of the FDA and one that has not previously faced any sustained challenge. The lack of a constituency that is visibly harmed by overly cautious IRB's makes developing political pressure on this more challenging, but this seems valuable. 

FDA and Covid vaccines

As a thought exercise, it is worth considering what a successful attack against Topol on the vaccine delay question might have entailed. Academic stars in medicine that argued for faster vaccine approval, in contradiction to Topol, would have been rhetorically effective. A less politically polarized Covid-19 response in general would have neutralized the specific anti-Trump claims that Topol made. 

Not being in an election year and so proximal to November 2 would have removed the incentive for anti-Trump individuals and institutions to view an early vaccine approval as a Trump victory and likely alleviated some concerns that the vaccines were being "rushed through." Elite media that favored faster approval and that continually raised vaccine approval salience would not have so favorably amplified Topol's open letter. 

There were some prestigious names contra Topol on vaccines like Walid Gellad (who has served on FDA committees and is a prominent scientific voice in the health policy world) but few or none with Topol's star-power and connections. The prominent economists who pushed for faster approval were all, fairly or not, associated with a deregulatory perspective, which likely made their arguments less credible to the FDA. If the FDA had not burned their credibility on HCQ early on, they would have been less vulnerable to Topol's reputational attack. The somewhat Rationalist-aligned academics and institutions that consistently pushed for vaccine approval and human challenge trials have no cachet in the medical community, and so cannot effectively engage in this reputational battle as insiders.