Book Review: Malignant

Malignant is a somewhat atypical book in the cancer space, as Prasad admits in his introduction. It reads a lot more like a combination of investigative journalism and outraged policy wonk (perhaps The Weeds podcast would like him on?) than my preconception of the 'typical' cancer book, which, according to Amazon, is cancer cookbooks, 'holistic' ways to cure cancer, and a sprinkling of cancer biology books.

Malignant’s simplified thesis is that over the last 25 years or so, increasingly lax regulatory standards, pervasive financial conflicts of interests, and other factors have led to a proliferation of marginally useful cancer drugs, in addition to a few truly useful game-changers, all at impressively high prices.

To be honest, when I first heard about the book, I wasn’t too excited. Cancer has never been my biology wheelhouse, and I wasn’t too excited about a book that I initially thought would be warmed-over criticism of cancer drug prices or something like that.

But my instincts were wrong: like in Ending Medical Reversal, the focus of the book—cancer drugs—is highly relevant to a ton of other questions.

You might not think that “progression-free survival” should matter to anybody but an oncologist, but the utility or lack thereof of a metric is increasingly important in a metric-focused world. Like Prasad suggests when he quotes Plato, “They see only their own shadows, or the shadows of one another”, there are many difficult problems in clinical medicine that should make us think deeply about medical epistemology. Good thing, according to his CV, he was an undergraduate philosophy major.

Cancer research, in addition to being philosophically interesting, is also morally compelling: it features the best and worst of modern medicine, the miracle cures like Imatinib and occasionally immunotherapy, and the arguably useless drugs that squeak by with a barely significant increase in surrogate endpoints.

So in addition to recommending “Malignant” to those in healthcare, I recommend it to anybody interested in decision-making under uncertainty, health policy, and the pharmaceutical industry. From a technological stagnation perspective there’s some interesting grist as well.

On to the book’s thesis. Prasad makes so many separate points that summarizing in any comprehensive way would be unmanageable, so here is a tentative summary: Lax approval standards + financial conflicts of interests + byzantine cost shifting in American healthcare + irresponsible hype from many sources => very expensive, very mediocre drugs, with no real competition, and no market or regulatory forces driving costs down.

In this story, the original sin in cancer research is lax approval standards at the FDA. These come in many forms, of which the most important is FDA approvals based on surrogate endpoints that are not followed up with hard survival data. Another problem: randomized controlled trials (RCT’s) conducted in ways that don’t mirror clinical practice, which give the experimental arms an advantage by comparing them to treatment inferior to the standard-of-care .

On regular vs accelerated approval:

In the US market, there are two types of regulatory approvals that the FDA can give companies. The first is regular approval. With regular approval, the FDA has largely declared itself satisfied with the efficacy of a drug and tends only to ask for further safety information. The second is accelerated approval. Accelerated approval is sometimes referred to as provisional approval. With this stamp, the FDA essentially says that, while it thinks the drug is promising, the company must have a future commitment to prove the drug improves a clinical endpoint, such as survival, once it is on the market.

On surrogate endpoints:

the FDA loves using them as the basis for drug approval. In just six years (2009–2014), the FDA approved 83 cancer drugs, of which 31 were approved on the basis of response rate and 24 on the basis of PFS, making a total of 55 (66%) approvals on the basis of surrogate outcomes. Beyond FDA drug approvals, surrogates are used widely in cancer clinical research, and their use has increased over time.

An important point is that “Surrogate endpoints often fail to predict which therapies improve survival" which wouldn’t really be a problem as long as drugs using surrogate endpoints were funneled through the accelerated (provisional) pathway, and the FDA followed up as they’re supposed to.

The surrogate endpoint used should be a reasonably good predictor of clinical benefit, but in addition, the company will still have to prove a clinical benefit in follow-up trials. As Prasad tells it, both requirements of Accelerated Approval are not being met by all cancer drugs. In more plain language: drugs are approved based on questionable proxies for important outcomes and then not actually followed up properly.

He presents data showing that 5-6 yrs after approval, only ½ of the promised post marketing commitments had been completed, ¼ were delayed, and ¼ had never been started. As I understand it, this is something the FDA is ignoring at its discretion, so the FDA could fix this pretty easily.

Though surrogates are supposed to speed approval, Kemp and Prasad note that “…before one can validate a surrogate, one needs many trials that measure both the surrogate and the hard endpoints in a specific setting. As a simple matter of fact, it may take years to build that knowledge base.”

Surrogates only speed trials up a little:

Adjusting for the line of therapy, a measure of disease severity, and the speed with which a trial can accrue, a measure of both desirability of the study and prevalence of the condition, we found that surrogates speed drugs to market by 12% over only accepting overall survival. In other words, if drug development routinely takes 7.3 years, a move to overall survival would increase time to 8.2 years

Some surrogate markers have poor evidence

Every time a surrogate endpoint was used to approve a cancer drug, we performed a systematic review of the literature to find a study documenting the strength of the correlation. We figured reasonable people can disagree about how strong is strong enough, but no rational person would defend the absence of having studied the question. We found for 14 out of 25 accelerated approvals (56%) and 11 out of 30 regular approvals (37%) there was nothing. No, not a study showing a poor correlation. I mean nothing

Another big problem in clinical cancer research is an RCT conducted in resource-poor settings with non-standard-of-care therapies justifying new drugs in standard-of-care settings. Prasad isn’t arguing against global oncology, but against comparing new drugs to drugs that are only used in low-income countries, and not routinely used in the US.

Compounding all of these problems is the strict inclusion criteria used in RCTs. The patients enrolled in RCTs are younger and have fewer comorbidities than the average patient who will later receive the drug.  In some cases the criteria are so strict that only 30% of patients with a given cancer who already received a specific drug are eligible for enrollment in a clinical trial testing a second-line therapy, so only a fraction of a fraction are eligible.

In observational studies that try to find the effect of drugs on the average patient, RCT’s with healthy populations seem to overestimate how effective a drug is, which means cancer drugs may be, on average, even worse than their already modest clinical trial outcomes!

To fix this, Prasad proposes “the choice of patients, control/comparison arm…will all be designed to reflect the US patient population.”

To disagree with Prasad here a bit, let’s look at it from pharma’s perspective. Clinical trials are enormously expensive and fraught with uncertainty. Too many adverse events (which may be more common with sicker patients, and impossible to predict) can sink a drug. Avoiding polypharmacy with patients with comorbidities is probably challenging. Perhaps looser inclusion criteria would decrease patient compliance rates in trials. Lots of reasons for strict inclusion criteria that are not easily wished away. 

On the other hand, we might propose that if a drug is not effective enough to demonstrate a clinical benefit in an imperfect scenario (which is more like the real-world than a strict-inclusion trial) we’re not interested in the drug. That’s a fair argument, and Prasad gestures in that direction by arguing for “trials…powered to detect clinically meaningful benefits. These won’t be trials looking for survival benefits measured in days….guidance from American and European professional societies can be used.”

I think he would agree that there’s some hard trade-offs here, and that a high-impact area would be figuring out ways to make clinical trials cheaper and easier to run: if nothing else, Covid-19 has demonstrated that we need to streamline RCT’s quite a bit.

Conflicts of Interest

The second main theme is financial conflicts of interest. It’s not that the average physician is all that influenced by a free steak dinner once a year or a handful of free branded pens—though those likely influence behavior at the margin—but that the top physicians in a field, the ‘thought leaders’ receive quite a bit of $ from pharma: “125 guideline writers…84% had taken personal payments from pharma...average was just over $10,000 ($10,011), with a huge range ($0 to $106,859)”. Prasad documents a few studies showing similar findings in that vein and has some neat studies looking at physicians’ Twitter activities and how positively they talk about new drugs. In addition, the practice of paying oncologists a fixed % of the drugs they administer, Prasad notes, perversely incentivizes more costly drug administration.

Prasad also criticizes the practice of patient advocacy groups taking money from pharmaceutical companies and notes that they rarely, if ever, push back against marginal drugs and criticize cost-effectiveness programs. This is a classic collective action problem: the costs of expensive drugs are diffusely spread over the American public, the benefits are concentrated among the patients who receive the new drug and pharmaceutical companies that sell them expensive new drugs, so of course the latter will rarely criticize high drug prices.

An interesting semi-counter-example outside of cancer is the Cystic Fibrosis Foundation, which has innovated in the venture philanthropy space. They partnered with Vertex pharmaceuticals, donated tons of money to Vertex directly, and over the last 10 years produced Trikafta, the triple-therapy combo, which seems to be a real “game-changer”, and came about in part because of a unique philanthropic pharma collaboration. Most patient advocacy groups aren’t capable of that, but I think it’s worth exploring.

The revolving door between FDA regulatory officials and pharma executives is also, Prasad argues, a problem. This creates an implicit incentive to be somewhat lenient on pharma from the FDA perspective, so as not to foreclose the possibility of retiring from the FDA and moving into industry. I somewhat agree, but I’m less optimistic than he is about this: “Obviously, this will disincentivize working for the FDA, which is already an underappreciated job. A compensatory measure would be to pay reviewers fairly, on par with what they would earn in the private sector.” I know very little about government salaries but my vague impression is that it’s not easy to raise salaries in that sector. I think this policy could have an unfortunate side-effect of lowering the level of talent in the FDA even further.

The payment system for off-label cancer drugs also comes in for critique. Prasad argues that while the mandate that CMS has to pay for off-label use of drugs has good intentions, the guidelines that CMS uses to make its decisions has not done a good job of sticking to the evidence, and should be reformed.

Prasad rightly criticizes the overwhelming hype in cancer research: “Every new drug seems to be a miracle, breakthrough, game changer, or cure, irrespective of how well it works or for how many people”, and wants to save the headlines for drugs that are truly transformative. A high bar, but the constant overuse of hyperbole likely erodes public trust in scientists, and trust is slow to gain and easy to lose. Much of this fault can be placed on journalists and university press release offices, but he also provides examples of regulatory officials (like former FDA head Scott Gottlieb) making much ado over only incremental improvements.

Even cost-effectiveness studies seem to buy into cancer drug hype. Prasad:

Fourth is the big one—a problem I often harp on—some of these cost-effectiveness analyses assumed the effectiveness part. In other words, some of these studies measured surrogate endpoints but modeled/projected/imagined what benefits these drugs might have on the endpoints that matter—survival and quality of life.

Before you conduct a cost-effectiveness analysis, you have to know that a therapy is effective. You have to prove someone lives longer or better as a result of a therapy. You cannot take a response rate and turn it into quality of life. You cannot assume a longer PFS is the same as living longer. When you assume efficacy and use those numbers in your value calculation, you are essentially making things up…In order not to look foolish, it is a good idea to perform a cost-effectiveness analysis if, and only if, the efficacy of the therapy (that is, the fact it works) is actually known.

The book is not all doom and gloom. He has several proposals to make things better, which he advocates, as a smart technocrat should, rolling out in incremental testable fashion. Try the reforms out, measure outcomes, and move forward from there. Eminently sensible!

Some of the changes he proposes: studies that use survival as their primary endpoints, enroll wider groups of patients, use standard-of-care control arms, and use ASCO guidelines for meaningful improvements.

His more radical proposal is to sever the [clinical trial] portion of pharma companies, fuse it to the FDA, and have a pre-specified agreement by the FDA that basically says “if you meet those endpoints, it’s approved”. These changes would raise the bar for cancer drugs, which would incentivize fewer but more groundbreaking drugs, at the expense of questionably useful but abundant cancer drugs.

Overall, I think this book lays out a refreshingly honest overview of cancer research and proposes sensible ways to fix what’s gone wrong. He doesn’t pull any punches with his critiques, but is also generally fair-minded about giving the other side their due.

The only argument that I wish he’d delved a bit into is the technological stagnation argument: what if the primary reason for mediocre cancer drugs is that the low-hanging fruit in biomedicine have been picked, and the lax approval standards, increasing RCT sizes, etc. are more of a response than a cause? To counter that argument, he could bring up the true game-changers in cancer, like Imatinib and immunotherapy, which would argue against that stagnation trend.

For those outside of the cancer and clinical trial space, the book is also an excellent introduction to medical legalese like “disease-free survival” and “partial response”, and there’s a section at the end with some practical tips on how to choose an oncologist for you or your loved ones.

A random note: Prasad is diligent about crediting collaborators and random people he quotes, which is nice to see. Every other paragraph is “and this random med student who came up with an idea did a study with me.”

And to counter a possible critique: Prasad is not some EBM “ONLY RCT!” fetishist—He recognizes where to be more or less flexible about evidence. For instance, he agrees that third-line therapies, which are by their nature less studied than first-line, can be studied in more flexible ways, like with surrogate endpoints, and that individuals with very high genetic risk for cancer (BRCA1 carriers, for instance) should be treated prophylactically without RCT’s because their risk for cancer is so high (around 30%+ lifetime risk?).

In the case of end of life care, he makes the common-sense observation that if patients have limited life expectancies, initiation or continuation of drugs with benefits that take years to accrue, like statins, is probably a bad idea.

Overall, 5/5, an excellent and highly readable book!