February 15, 2012 (Vol. 32, No. 4)
Henry I. I. Miller, M.D. Physician and fellow Stanford University
Bogus “Study” Gives Agency More Credit Than Is Due
I read with interest—and mounting skepticism—Patricia Dimond’s Insight & Intelligence™ piece about FDA, “FDA New Drug Approvals in 2011 Outpace Recent Past,” on GEN’s website. Some of its assertions and assumptions lacked essential context and disclosures.
Consider, for example, the headline, “FDA New Drug Approvals in 2011 Outpace Recent Past.” Although it’s true that the FDA’s 35 approvals are better than the 21 last year, a single data point doesn’t a trend make, so it’s useful to look at approvals during recent five-year intervals. From 2007–2011, the FDA approved 123 new drugs; from 2002–2006, 129 drugs; and 1997–2001, 178 drugs.
With that historical context, maybe the concept of outpacing isn’t such a good choice for the headline.
Partly on the basis of the single 2011 data point and a flawed report by an advocacy and lobbying group called the Friends of Cancer Research, Dr. Dimond concludes that “FDA did a pretty good job despite the carping from the pharma industry, financial community, and some patient advocacy groups,” endorsing Friends of Cancer Research’s conclusion that there is a need “for strong financial and public support of the FDA.”
To evaluate the strength of this news/advocacy piece, it’s useful to examine the Friends of Cancer Research study, which was published last year in Health Affairs. It found that between 2003 and 2010 the FDA approved 32 new cancer drugs versus 26 by the European Medicines Agency.
The FDA supposedly both approved more cancer drugs and did so more quickly: FDA approval averaged 182 days, while the EMA averaged 350 days. According to Ellen Sigal, chairman and founder of Friends of Cancer Research, FDA’s regulatory delays and intransigence toward industry are nothing more than an urban legend.
The facts argue otherwise. The only urban legend in evidence appears to be the conclusions of the study itself. Several things are disturbing about the methodology and the possible sources of bias or conflict of interest on the part of the authors that were not mentioned either as a disclaimer in the article or in the media (including Dr. Dimond’s) coverage of it.
First, the timing and location (for example, Europe vs. the U.S.) of drug approvals depends in large part on where and when drug companies decide to submit their applications for approval, how aggressive they are, and whether the drug has been previously approved elsewhere.
The quest for approvals is not a race from the same starting gate. Of the 25 drugs in the study that were approved in both the U.S. and Europe, just two were submitted to European regulators first—one by a mere 11 days, the other by a single day. By contrast, most applications were submitted to the FDA several months or in some cases two or three years earlier.
Second, not a single media report mentioned that the FDA often uses various tricks to “stop the clock” or even delay the start of the clock by “refusing to file” an application for marketing approval that it has received. Thus, the agency’s “review times” often have little correlation to the calendar. The Health Affairs article itself cites as one of its methodological limitations that it accessed only “official review times.”
The results would have been more meaningful if the authors had reported the actual number of calendar days from the date of regulators’ initial receipt of the application until the date of approval. Moreover, the FDA’s user fee authority requires the agency to demonstrate that it meets strictly defined approval timelines, creating a potent incentive to “game” the official review times.
Third, when deciding on drug approvals, European regulators may take into consideration cost and cost-effectiveness (including comparisons to other available therapies), whereas the FDA cannot. In other words, European regulators sometimes use the review that is ostensibly for a drug’s safety and efficacy as a first screen for cost-effectiveness, and sometimes they delay or deny approval on those grounds.
Fourth, neither the extensive coverage of the Health Affairs study nor the article itself disclosed that two of the authors of the study, Ellen Sigal and Samantha Roberts, have close ties to the FDA. According to her bio, Sigal is “co-chairman of the board of directors of the Reagan-Udall Foundation, a partnership designed to modernize medical product development, accelerate innovation, and enhance product safety in collaboration with the U.S. Food and Drug Administration.”
Prior to joining Friends of Cancer Research, Roberts was a post-doctoral fellow in the FDA’s Center for Biologics Evaluation and Research. The website of the Friends of Cancer Research—the professional home of all three of the authors of the study—lists one of its activities as “FDA Advocacy”, under which is the claim that FDA is “responsible for advancing the public health by helping speed innovations that make medicines and foods more effective, safer, and more affordable.”
Few scholars who study the FDA would agree that the agency has promoted innovation. Excessively risk-averse, capricious, and at times even hostile FDA regulation has made drug development extraordinarily difficult in recent years. Bringing a new drug to market now requires on average 12 to 15 years and costs more than $1.4 billion—in no small part because the average length of a clinical trial increased 70% between 1999 and 2006.
I’ve cited the downward trend in drug approvals, a trend that is destined to continue because in 2010 the number of applications for approval of new drugs was the lowest in decades. But perhaps the most ominous statistic of all is that drug manufacturers recoup their R&D costs for only one in five approved drugs. This fraction is likely to decrease further as FDA demands ever-larger, ever-more expensive clinical trials to accumulate data on the safety of drugs in development.
As to how good a job FDA is doing, a recent survey of venture capital firms revealed that they have begun to avoid investment in early-stage pharmaceutical and medical device companies, which are therefore increasingly moving overseas.
Thirty-six percent of respondents said they plan to increase investments in life science companies in Europe, while only 13% intend to increase investment in U.S. companies; and 31% said they plan to decrease investment in life science companies in the U.S., compared to 7% that intend to decrease investment in Europe.
The reasons? Sixty-one percent of the investors cited regulatory challenges as the primary reason; more specifically, they alluded to dysfunction, unpredictability, and risk-aversion at Obama’s FDA.
Finally, the abstract of the Friends of Cancer Research study itself reveals what may be the real agenda: “Our findings reinforce the need for strong financial and public support of the Food and Drug Administration…”
That certainly fulfills the Friends’ mandate for “FDA advocacy”: Undoubtedly we will see this study used to urge the Congress to increase FDA’s appropriations and to support renewal of the agency’s user fee legislation when it comes up for reauthorization later this year.
In fact, many observers of the FDA feel that regulators could do far more with far less and that the agency has become an obstacle to pharmaceutical and medical device innovation. Instead of concentrating on getting more drugs through the pipeline to patients, FDA has focused on the kinds of compliance actions that punish drug companies and their executives but please activists and congressional critics while offering little if any benefit to patients.
The agency has been particularly hard on oncology drugs. In May of 2010, the FDA rejected pixantrone for treating non-Hodgkin lymphoma, a blood cancer that kills almost 12,000 Americans a year. The rationale for the rejection was not that the drug was not effective—in clinical trials it was a qualified success—but that the trial was not “flawlessly executed,” an absurd standard.
Policy research, news reporting, and advocacy are very different undertakings. The Friends of Cancer Research and Dr. Dimond should give some thought to that.
Henry I. Miller, M.D. (firstname.lastname@example.org), is the Robert Wesson Fellow in Scientific Philosophy and Public Policy at Stanford University’s Hoover Institution and a fellow at the Competitive Enterprise Institute. He was the founding director of the FDA’s Office of Biotechnology.