Action flexibility and state flexibility in PPM
An informative example
Consider a model that makes predictions: linear regression. When its assumptions are met, linear regression produces unbiased estimates of its coefficients but unbiased estimates are neither necessary nor sufficient for making good predictions.
Regression falters when it has too little data. With scant data, regression produces imprecise estimates and the estimates can be so poor that the regression predictions are abysmal. How does this happen? All data contains useful information and error, or more commonly, signal and noise. With sufficient data, regression averages away the noise but when data is scant, noise remains. Then regression fits its model to the noise, rather than to the signal. This phenomenon is called overfitting.
Now consider two approaches to regression when data is scant:
- Fit the full regression model and accept the imprecise result.
- Fit a smaller regression model that has fewer variables.
The second approach has two curiously juxtaposed properties. When compared to the full model, which more accurately describes reality, its coefficients are wrong. Technically, the smaller model is misspecified, which makes its coefficients biased. However, the simple model makes better predictions than the full model. With fewer variables to estimate, the simpler model achieves greater precision. In total, the simpler model produces a trade-off: greater precision but greater bias as well. When information is scant, the trafe-off is beneficial.
This result is outstanding. Ignoring some impactful variables improves predictions. (For examples of this phenomenon in the field of finance, see my discussion, "What modern portfolio theory reveals about PPM.")
Why does the simple regression outperform the full regression when data is scant? Consider two sources of loss from a model or a decision method:
Total Loss = Loss from incompleteness + Loss from imperfections
With regression, the losses are the cost of forecasting errors. When data is scant, the loss from imperfections (imprecise coefficients) is considerable. Simplifying the model by removing some variables reduces imprecision, thereby reducing this loss. However, removing variables increases the loss caused by having an incomplete model so there is a trade-off. When data is scant, the trade-off is beneficial.
Now suppose one selects a model by minimizing the total loss. As data becomes progressively more scant, one minimizes the total loss by removing more variables, simplifying the model. In this way, increased uncertainty favors simpler models. (To learn more about the two-loss model, see my discussion, "Why some C-level executives are skeptical of PPM.")
Action and state flexibility
Now let's introduce the ideas that name this discussion.
- Action flexibility tailors decisions to the environment and adapts to environmental change, even to small subtle changes. By including all of the impactful variables, the full regression model strives for action flexibility.
- State flexibility is when a decision works well in a great variety of situations. Generally, these decisions are not optimal in any particular situation but they produce a decent result in most situations. One can think of state flexibility as insurance. Forgoing optimality is the cost and achieving reasonable results with high probability is the benefit. By excluding some impactful variables, the simpler regression models achieve state flexibility.
If done well, so one makes few errors, action flexibility is superior to state flexibility. However, implementing action flexibility is difficult. By considering many details and making many adjustments, action flexibility offers numerous opportunities for decision errors. If errors occur frequently or are severe, striving for action flexibility is harmful.
When action flexibility fails, simplifying decision-making and adjusting less frequently to the environment improves performance. This result is outstanding. Given sufficient uncertainty so that action flexibility commits too many decision errors, state flexibility has both less risk and a higher expected value.
Having introduced action flexibility and state flexibility, let's consider project portfolio management (PPM).
State flexibility in PPM
While some experts advocate action flexibility, by recommending the most sophisticated models and methods, techniques that produce state flexibility are more common. Principally, the field increases state flexibility via three methods:
- Simple models: The regression example suggests that well-designed, simpler PPM methods can be state flexible. Some of the simpler PPM practices, like cutoff values and rankings are similar to methods in studied in the field called Fast & Frugal Heuristics. One can find scholarly and practical introductions to the field the books Simple Heuristics that Make Us Smart and Gut Feelings: the Intelligence of the Unconscious.
- "What if" analysis: One can use a complex or simple optimization model and create a state flexible portfolio by performing a "what if" analysis. One varies some parameters of the project evaluation or project selection models, and for each variation, calculates the "optimal" portfolio. Then one invests in projects that are most frequently in the "optimal" portfolios. These projects are good choices in a variety of circumstances, so investing in them produces a state flexible portfolio.
- Simulation optimization: Like "what if" analysis, smulation optimization performs repeated optimizations while varying the parameters in project evaluation and project selection models. For each optimization it estiamates the risk of the "optimal" portfolio by estimating the variation in the portfolio's potential results. By testing many portfolios it searches for portfolios that maximize value while limiting risk.
"What if" analysis and simulation optimization have some qualities of robust decision-making, which is a decision method being pioneered by the RAND Corporation. Robust decision-making couples the simulation approach of simulation optimization with the explicit rejection of maximizing utility (or expected value) in favor of seeking state flexibility, which one finds in "what if" analysis. Wikipedia provides a nice introduction to robust decision-making.
Each approach has benefits and limitations:
- Simpler selection techniques can be robust to modeling errors but only if the models are well-designed.
- "What if" analysis, as typically practiced, uses models that are simpler than the models of simulation optimization but more sophisticated than the simplest selection methods such as cutoff values. "What if" analysis provides a middle ground but if uncertainty is substantial, it may still suffer too much from modeling errors.
- Simulation optimization, as typically practiced, uses complex models, so it more sensitive to modeling errors and errors in information.
Which approach is best? Understanding the appropriate situations and best use of each approach are part of my research on project selection. Additionally, I am developing a new approach that eschews static portfolio models for a dynamic model of pipelines, called pipeline physics.
Pipeline physics
Pipeline physics reorients PPM from a predict-and-plan approach to one that strives to minimize the five losses listed above. Achieving this result allows a process to increase action flexibility, without commiting errors, which creates the most value. You can read learn about the two approachs from my discussion, "Does PPM need a new paradigm?"
Manufacturing, supply chain management, project management and product development have already made this transition. Here are some examples:
- Just-in-time achieves action flexibility via many practices, including reducing the lead-time to fulfill orders. Reducing lead-time reduces the lengths of forecasts, so forecasts better match demand. If lead-times are shortened enough, forecasting is entirely omitted and production responds to sales, reported by checkout scanners. This practice transforms manufacturing from a push system (based on predicted sales) to a pull system (based on actual sales).
- Agile project management greatly decreases the cost of developing and changing project features, including the cost of creating the wrong features. With low costs and rapid feedback, one can quickly test many features, keep what works and slough off what doesn't. Action flexibility increases.
- Modular designs exploit a two-stage decision process, starting with architecture and ending with components. The initial decisions develop a modular architecture. This design has extreme state flexibility, so designers can delay decisions about a product's components until after uncertainty has resolved. Once uncertainty resolves, designers can select a product's components to match technology, systems, competition and customer needs, achieving action flexibility.
How does PPM make this transition? It exploits a key quality of pipeilnes (phase-gate systems): uncertainty and risk decrease as projects progress down a pipeline. (See my discussion "How PPM differs from modern portfolio theory.") I call this new approach pipeline physics, and you can learn about it from my discussion, "Managing drug development pipelines" and my pipeline physics research proposal. (Contact me for the password needed to view the proposal.)
After reading my discussions, many managers wish to share their experiences, thoughts and critiques of my ideas. I always welcome and reply to their comments.
Please share your thoughts with me by using form below. I will send reply to you via email. If you prefer to be contacted by phone, fax or postal mail, please send your comments via my contact page.
© 2014 Pipeline Physics. All rights reserved.