Pipeline Physics produces profit
Gary Summers, PhD	1700 University Blvd, #936
President, Pipeline Physics LLC	Round Rock, TX 78665-8016
gary.summers@PipelinePhysics.com	503-332-4095

Scoring models + complex optimization models = water and oil don't mix

Often PPM software includes both scoring models and optimization. One builds a scoring model, scores projects and then selects projects via an optimization model. This mix of scoring models plus optimization can fail in three ways:

If the correlation of project scores with project value is sufficiently low, selecting projects via a complex optimization model will create numerous avoidable project selection errors. Why? Complex optimization models are sensitive to project evaluation errors, so they should only be used when project evaluation errors are small. When the correlation of project scores with project value is low, project evaluation errors are too large for sophisticated selection techniques. (See my discussion, "Where's the feedback?" and my forthcoming discussion, "Project selection: simple or sophisticated?")
Sometimes projects scores are better at predicting project probabilities of success than at predicting project values. In these situations, one should never use optimization. Maximizing the sum of projects' probabilities of success is nonsensical. Moreover, the relationship between project scores and project probabilities of success is nonlinear (see below).
One should never use optimization when the relationship between project scores and project value is nonlinear. In PPM, portfolio optimization has a linear objective function, so it implicitly assumes a linear relationship between project scores and project values.

Let's look more closely at problem three. Suppose the relationship between the project scores and project values is nonlinear. This relationship is common when projects produce a few grand successes and many also-rans. Pharmaceutical drug development and Hollywood movies are archetypical examples. More common examples are projects that exhibit the 80-20 rule, where a small set of projects produces most of the results. For example, twenty percent of projects produce eighty percent of the results.

Now suppose the projects are scored on a ten point scale. One project has a score of 9.0 and its cost can fund two other projects, which have scores of 5.0 and 4.5. Optimization will cancel the former project and fund the two mediocre ones because 9.5 > 9.0. This is a problem because the project scoring 9.0 is more likely to be a blockbuster than the other projects, which are probably average. Canceling a potential blockbuster to fund two average projects is a costly mistake.

After reading my discussions, many managers wish to share their experiences, thoughts and critiques of my ideas. I always welcome and reply to their comments.

Please share your thoughts with me by using form below. I will send reply to you via email. If you prefer to be contacted by phone, fax or postal mail, please send your comments via my contact page.