Pipeline Physics

Pipeline Physics produces profit
Gary Summers, PhD	1700 University Blvd, #936
President, Pipeline Physics LLC	Round Rock, TX 78665-8016
gary.summers@PipelinePhysics.com	503-332-4095

---

How discovery impacts development

Suppose you are asked to select ten apples from a bushel of one hundred. If 85 of the apples are good ones, your task is easy. Even random selection will probably produce great results. However, if 85 apples are rotten, so the bushel contains only 15 good ones, your ability to distinguish good apples from bad apples must be exceptional. Otherwise, you will select many rotten apples.

In PPM, where's the front-end?

Project portfolio management (PPM) works the same way, so the front-end of a pipeline (phase-gate system), which creates the choices, is critically important. The front-end affects the following qualities of the PPM:

1. The value one can create.
2. The quality of project evaluations required for success.
3. Whether the pipeline is punishing or forgiving of project selection errors.
4. The number of projects (throughput) that can flow down the pipeline while achieving acceptable downstream success rates.
5. Cycles of booms and busts. (Having a better set of projects dampens these cycles.)

PPM's current best practices ignore the front-end, focusing on project evaluation and selection instead. Rather than improving one's set of choices, they accept the set as "givens" to an optimization problem.

Discovery's impact on development

Let's see the impact of the front-end by exploring discovery's impact on drug development. Correctly analyzing drug development pipelines and phase-gate system requires using my pipeline physics model. Pipeline physics replaces the static pipeline model that founds many pharmaceutical pipeline analyses.

Static pipeline: This model assumes each phase's attrition rate is (conditionally) independent of its preceding gate's attrition rate. For example, the model assumes that if one improves the design of phase II trials or changes the criteria for advancing compounds from phase II to phase III, phase III's attrition rate remains unchanged. This model is static because it implies a pipeline's shape is unchanging.

Despite its widespread use, the static pipeline model is implausible. Define a marketable compound as one that is safe and effective and an unmarketable compound as one that is not. Changing phase II's design or advancement criteria will affect the quality of compounds advancing to phase III. A greater or lower percentage of these compounds will be marketable, which will affect phase III's attrition rate.

Pipeline physics: Pharmaceutical executives need a model that correctly describes the relationships among the phases, showing how phase I affects phase II and how phase II affects phase III. With this model executives could manage upstream processes to produce the downstream results they desire. I developed pipeline physics to fulfill this need.

Pipeline physics uses the following terms:

• Marketable compound: A compound that is safe and effective.
• Unmarketable compound: A compound that is unsafe or ineffective.
• False-positive: Advancing an unmarketable compound to the next phase.
• False-negative: Canceling a marketable compound.
• False-positive rate: A phase's false-positive rate is the fraction of the unmarketable compounds it evaluates that it advances to the next phase.
• False-negative rate: A phase's false-negative rate is the fraction of the marketable compounds it evaluates that it cancels.
• Resolution: A phase's ability to distinguish marketable compounds from unmarketable ones.
• Throughput: The percent of compounds a phase advances, which is typically called a phase's success rate. (The term throughput is preferred to success rate because some advancing compounds are unmarketable. Advancing such compounds makes a pipeline less successful.)
• Base rate: A phase's base rate is the percent of compounds it evaluates that are marketable.

Pipeline physics proposes equations that describe how these variables interact, including how one phase affects the next. For example:

• Increasing a phase's throughput (1) increases its false-positive rate, (2) decreases its false-negative rate and (3) reduces the fraction of advancing compounds that are marketable, thus reducing the next phase's base rate.
• Decreasing a phase's throughput (1) reduces its false-positive rate, (2) increases its false-negative rate and (3) increases the fraction of advancing compounds that are marketable, thus raising the next phase's base rate.

In each case above, two of the results have are partially offsetting. Increasing (decreasing) throughput raises productivity, but reducing (raising) the next phase's base rate lowers productivity. For this reason, adjusting throughput often has a marginal impact on drug development costs. In contrast, increasing the phases' resolutions is a much more promising approach.

These relationships differ from those of the static pipeline model, and accordingly the two models make dramatically different predictions. For an example, see my discussion, "How to make drug development more productive."

With pipeline physics, let's assess discovery's impact on development, using Figure 1, which presents a drug development pipeline. To simplify the analysis, with only a small impact on its results, Figure 1 groups phase III and the new drug application (NDA) together.

Discovery affects development by determining phase I's base rate, which is the fraction of phase I compounds that are marketable. Consider two extremes, which likely encompass all therapeutic areas in drug development:

• If 5% of phase I compounds are marketable false-positive rates are high, false-negatives are disastrous and reducing development costs to reasonable levels requires exceptional resolution.
• If 30% of phase I compounds are marketable, drug development incurs fewer false-positives, suffers less damage from false-negatives and realizes low costs even when resolution is less than stellar.

Table 1 compliments Figure 1 by providing industry-wide data on oncology pipelines. Rows 1 and 2 present each phase's throughput (success rate) and attrition rate.1 Row 5 presents the capitalized cost of each phase.2

Rows 3 and 4 present new statistics from pipeline physics. Row 3 presents reasonable values for each phase's resolution, with phase II's resolution exceeding phase I's resolution, as appropriate. The table assigns infinite resolution to phase III and the NDA, meaning they perfectly distinguish marketable compounds from unmarketable ones. While unrealistic, this assumption simplifies the example without affecting its results too much. All these values are guesses and one would prefer to have estimates produced by analyzing pipeline data, as I mention below.

In Row 4 the base rate for phase I is estimated from the industry data and the model. Industry-wide, 6.7% of oncology compounds that enter phase I successively traverse the pipeline.1 Given the industry throughput rates, the assumed resolutions and the equations of pipeline physics, the 6.7% pipeline success rate implies that 13.4% of phase I compounds are marketable (phase I's base rate). The other base rates follow from the phases' throughputs and resolutions.

Table 1: Oncology pipeline statistics.

	Phase I	Phase II	Phase III & NDA
Throughput	63.9%	28.3%	36.9%
Attrition rate	36.1%	71.7%	63.1%
Resolution (signal-to-noise ratio)	0.4	1.2	∞
Base rate	13.4%	15.9%	36.9%
Capitalized cost per compound ($ millions)	32.14	69.68	234.05
Cost per drug ($ millions)			1,781

Notice the following qualities of the pipeline:

• With its poor resolution and high throughput, phase I barely raises the base rate, making phase II's base rate equal to 15.9%.
• With its greater resolution and lower throughput, phase II raises the base rate for phase III to 36.9%.
• If 13.4% of phase I compounds are marketable but only 6.7% are marketed, 50% of marketable oncology compounds are mistakenly canceled by phases I and II. Raising the phases' resolutions, to fix this problem, should be highly profitable, although one should confirm this intuition by performing an analysis with pipeline physics.
• The cost of developing an oncology drug is $1.781 billion.

Suppose one raises phase I's base rate by 25%, from 13.4% to 16.75%, producing an additional 3.35 marketable compounds for every hundred that discovery sends to phase I trials. Table 2 presents the results.

Table 2: Oncology pipeline statistics with phase I's base rate raised 25%.

	Phase I	Phase II	Phase III & NDA
Throughput	63.9%	28.3%	44.45%
Attrition rate	36.1%	71.7%	55.55%
Resolution (signal-to-noise ratio)	0.4	1.2	∞
Base rate	16.75%	19.82%	44.45%
Capitalized cost per compound ($ millions)	32.14	69.68	234.05
Cost per drug ($ millions)			1,480

Improving the front-end raises phase III's success rate from 36.9% to 44.45%, and it reduces the cost of a drug by 17% to $1.48 billion.

While impressive, the benefits could be even greater. Pipeline physics provides equations that calculate the optimal shape of a pipeline. For the data in Table 2 the optimal shape sets phase I and II throughput to 58% and 44%, respectively. Having this optimal shape reduces the cost of a drug to $1.41 billion, completing a 21% decline from the initial value of $1.78 billion. Analysis shows that all the improvement comes from increasing phase II's throughput, which was much lower than its optimal value.

Against these gains, one must apply the costs and risks of improving discovery. Importantly, the analysis shows the outcomes discovery investments can create. Producing an additional 3-4 marketable compounds for every hundred that discovery sends to phase I trials reduces development costs by $300 million.

The above analysis guesses phase I's and phase II's resolutions. Preferably, one would measure these resolutions from pipeline data. Until recently this measurement was impossible. One never learns the results of canceled compounds, and without classifying canceled compounds, as marketable or unmarketable, one cannot estimate the resolutions.

The pipeline physics model enables statistical analysis to overcome this problem, and given sufficient data, the statistics estimate phase I's and phase II's (1) base rate, (2) false-positive rate, (3) false-negative rate and (4) resolution. The statistical analysis estimates the resolution produced by NPVs, expected values and raw clinical data as well, and it compares a company's performance at compound selection to the resolution produced by the these metrics.

To learn more about pipeline physics see my discussion, "Managing drug development pipelines." To see the mathematical model and statistical analyses, see my pipeline physics research proposal. (Contact me for the password needed to view the proposal.)

1 Hay, M., D. Thomas, J. Craighead, C. Economides and J. Rosenthal (2014), "Clinical development success rates for investigational drugs," Nature Biotechnology, 32(1), pp. 40-51.

2 Paul, S., D. Mytelka, C. Dunwiddie, C. Persinger, B. Munos, S. Lindborg and A. Schacht (2010), "How to improve R&D productivity: the pharmaceutical industry's grand challenge," Nature Reviews Drug Discovery, 9(3), pp. 203-14.

© 2014 Pipeline Physics. All rights reserved.