FDA Drug Approval: Type I vs. Type II Error Trade-Offs

Introduction: The Cost of Every Decision

People in every walk of life must make decisions that affect, to some degree, the well-being of others. Everything comes with a cost — not simply in financial terms, but also in terms of opportunities no longer available once a particular decision is made. One chooses a course of action and thereby foregoes the other choices that were available. Each choice carries consequences, both anticipated and unforeseen. This benefit-cost dynamic and the web of consequences that result from the choices we make are an inescapable aspect of life (Miller, Benjamin, and North, 2005).

Type I and Type II Errors in Drug Approval

A hasty decision may have harmful, unintended consequences that might have been identified with more thorough pre-decision analysis. On the other hand, some people may be condemned to extended deprivation while lengthy analysis and deliberations continue. One can err on the side of immediacy — a Type I error — or on the side of caution — a Type II error. There are consequences, known and unknown, for both (Miller et al., 2005).

The Thalidomide Scandal and the FDA's Shift

Miller et al. point out that, since 1962, when the Food, Drug, and Cosmetic Act of 1938 was amended in the wake of the Thalidomide scandal, the length of time from the filing of an application for review to the agency's decision regarding the product increased dramatically — from several months to several years (2005). The FDA had previously been more likely to commit Type I errors, in which drugs were approved more quickly at the cost of more detailed, long-term study for harmful side effects. Following that amendment, the FDA's approval process shifted more toward the Type II error, whereby people may suffer from conditions and ailments for many years while potentially useful medicines are subjected to lengthy and financially costly testing, analysis, and review (Miller et al., 2005).

Review Times and the Role of Staffing

However, Carpenter, Chernew, Smith, and Fendrick found that the drug review process had been growing shorter over the preceding two decades (2003). These researchers examined whether the user fees paid by pharmaceutical companies had any influence on the length of the review process. The industry has claimed that its financial contributions through user fees sped up the review process, but the findings of Carpenter et al. do not support this claim. Instead, the research shows that the faster pace of the review process was due to where financial resources went — specifically, to FDA staffing — not where those resources originated. The more funding allocated to review scientists, the more scientists are on staff, and the more work gets completed in less time. Notably, the study found that the decrease in turnaround time from application submission to completed review began several years before the passage of the Prescription Drug User Fee Act, and therefore before the collection of those user fees began (Carpenter et al., 2003).

The question to consider is whether the increased efficiency observed by Carpenter et al. means that the FDA is moving back toward a Type I error orientation from the Type II orientation observed by Miller et al. Viewing the drug review process along the dimension of review rate alone might suggest that a Type I error orientation is reasserting itself within the agency. Drugs are being approved faster, which could imply less examination for potentially harmful side effects. However, if the number of qualified review scientists is increasing due to greater resources devoted to staffing, then the increase along the manpower and man-hour dimension must also be taken into account. Increased manpower results in more hours of work per unit of time, meaning that more evaluation and analysis is taking place simply because more workers are employed to handle these tasks.

Interpreting the Data: Error Orientation or Efficiency?

The exact answer would depend on how the workload is distributed among staff. If more scientists are each handling more individual reviews — thereby speeding up the process for each review — then the error orientation has shifted back toward Type I. On the other hand, if the additional staff are deployed to perform multiple concurrent analyses of the same drug, then a Type II error orientation would still be present, yet now achieved with a smaller investment of time. From a consumer's perspective, the latter scenario is clearly preferable.

Carpenter, D., Chernew, M., Smith, D. G., & Fendrick, A. M. (2003, December 17). Approval times for new drugs: Does the source of funding for FDA staff matter? Health Affairs, w3, 618–623.

Conclusion: Workload Distribution and Consumer Perspective

Miller, R. L., Benjamin, D. K., & North, D. C. (2005). Terrible trade-off. In R. L. Miller, D. K. Benjamin, & D. C. North, The economics of public issues (pp. 10–17). Boston: Addison Wesley.