Difference between theory and practice: it's disappearing
Decision theory recommends the same decision method regardless of risk: maximize expected utility. (Utility is a concept that relates risk and value.) Meanwhile, many managers intuitively perceive that sophisticated decision methods, like utility maximization, cause decision errors.
Project portfolio management (PPM) illustrates these opposing views. Decision analysis recommends evaluating projects via decision trees and selecting projects via portfolio optimization while evaluating each portfolio by using a utility function or Monte Carlo analysis. Meanwhile, many managers believe these practices are so sophisticated they are harmful.
What causes the difference between theory and managers' intuition? Traditional decision theory contains the following critical assumption: managers' decision models correctly account for all unpredictable events and outcomes via probabilities. Of course, practice never satisfies this assumption, and this distinction causes theory to diverge from practice and justifies managers' suspicion of sophistication.
Let's specify the difference more precisely. Economists define risk and uncertainty as follows:
- Risk: Unpredictable events that one can describe and model via probabilities.
- Uncertainty: Unpredictable events that cannot be modeled with probabilities. These events arise from several sources:
- Ambiguity: Situations are ambiguous when information needed to define probabilities is missing, as examplified by the Ellsberg paradox.
- Unknown-unknowns: Some events and outcomes are unknown, so decision-makers cannot explicitly address them. Black swan events.1 are unknown-unknowns.
- Complexity: Some situations are so complex one cannot fully understand them and reason about them correctly.
- Bounded rationality: Bounded rationality is the technical term for the cognitive limitations and proclivities of human beings.
John Maynard Kaynes described the difference cogently:
By "uncertain" knowledge … I do not mean merely to distinguish what is known for certain from what is only probable. The game of roulette is not subject, in this sense, to uncertainty … The sense in which I am using the term is that in which the prospect of a European war is uncertain, or the price of copper and the rate of interest twenty years hence, or the obsolescence of a new invention … About these matters there is no scientific basis on which to form any calculable probability whatever. We simply do not know!
—John Maynard Keynes, 1937*
With the above definitions, one can state the differences between traditional decision theory and practice:
- Traditional decision theory assumes that uncertainty does not exist, that all unpredictable events, and the relationships among them, are known and modeled correctly via probabilities.
- In practice, managers always face uncertainty. Their understanding, decision methods, and if made, decision models are always incomplete and imperfect.
How did this divergence arise?
Before decision theory, economic theory could not adequately address uncertainty and risk. Many economists proposed ad hoc remedies, but none proposed a satisfactory mathematical approach. As one might imagine, this caused a crisis.
Decision theory solved the crises. Developed by mathematicians John von Neumann and Leonard Savage, decision theory provided a formal axiomatic method of theorizing about unpredictable events. For resolving the crisis, mainstream economics embraced it quickly and fully.
However, the solution came with a cost. Decision theory only addressed risk, excluding uncertainty from analysis. Having adopted decision theory, many economists believed that theorizing about uncertainty was impossible, as exemplified by these quotes from economic Nobel laureates:
"[Uncertainty] seems to lead only to the conclusion that no theory can be formulated for this case."
- Kenneth Arrow, 1951*
"In situations of risk, the hypothesis of rational behavior on the part of agents will have usable content, so that behavior may be explainable in terms of economic theory….In cases of uncertainty, economic reasoning will be of no value."
- Robert Lucas, 1981*
By excluding uncertainty from analysis, decision theory created the difference between theory and practice, causing consternation for managers ever since.
During and after the crisis, a minority of economists addressed uncertainty. Three efforts produced insights that still affect economics. In 1921, Frank Knight introduced the idea of uncertainty to describe situations in which probabilities exist but cannot be known. In modern terms, the situations Knight described are called ambiguous. The Ellsberg paradox exemplifies them.
G. L. S. Shackle offered a stronger distinction: managers confront situations containing unknown-unknowns and truly unique situations and such situations violate the assumptions of probability theory. To Shackle, the nature of probabilities do not match the nature of managerial decision-making.
Finally, the most impactful development was Nobel laureate Herbert Simon's concept of bounded rationality. The behavioral theory of the firm, behavioral finance and the heuristics and biases literature, which compares human decision-making to decision theory, arose from Simon's ideas.
For decades, traditional decision theory dominated economics, and while this domination continues, the heuristics and biases literature and managers' tepid reciption to traditional decision theory caused economists to question its practical utility. Economist Robin Hograth expresses this growing sentiment:
"Like many graduate students of my generation, I was totally seduced by statistical decision theory. But statistical decision theory never really became the universal tool that many imagined it would … My belief is that statistical decision theory fails in many important problems - particularly future-choice problems - because humans are incapable of characterizing the uncertainties of the world in which they operate."
- Robin Hograth, 2010*
I've called decision theory traditional decsion theory, which is unorthodox, so I could distinguish the well-known theory from some recent research. This research is expanding decision theory to include uncertainty, perhaps closing the gap between theory and practice. The new research includes four mathematical frameworks for studying the impact of uncertainty on decision-making, as introduced by the following papers.
- Heiner, R.A. (1983), "The origin of predictable behavior," American Economic Review. vol. 73, no. 4, pp. 560-595.
- Bookstaber, R. and J. Langsam (1985), "On the optimality of coarse behavior rules," Journal of Theoretical Biology, vol. 116, no. 2, pp. 161-193.
- Forster, M. and E. Sober (1994). "How to tell when simpler, more unified or less ad hoc theories will provide more accurate predictions," British Journal of the Philosophy of Science, vol. 45, pp. 1-35.
- Al-Najjar, N. I. and M. M. Pai (2014), "Coarse decision making and overfitting," Journal of Economic Theory, vol. 150, pp. 467-486.
Mty research applies these frameworks to PPM, pipeline management and drug development. I strive to (1) transform managers' intuitions into knowledge that can be learned and applied more effectively and (2) modify mathematical decision methods and models to better manage uncertainty, which raises productivity.
You can learn more about my research from my:
- Discussions of drug development.
- Drug development pipeline models and diagnostic metrics. (Contact me for the password needed to view this research.)
1In his book, The Black Swan, Tesla describes black swans as pivotal events that one could be aware of but that are ignored for a myrid of psychological reasons. Organizational issues and even their rarity and unusualness can cause managers to ignore burgeoning catastrophes and bonanzas as well.
*The above quotes are from:
- Kaynes, J. M. (1937), "The general theory of employment" The Quarterly Journal of Economics, vol. 51, no. 2, pp. 213-214.
- Arrow, K. J. (1951), "Alternative approaches to the theory of choice in risk-taking situations," Econometrica, vol. 19, no. 4, p. 417.
- Lucas, R. E., Jr. (1981), Studies in Business Cycle Theory, Cambridge: MIT Press, p. 224.
- Hograth, R. (2010), "Subways, cocunuts, and foggy minefields: an approach to studying future-choice decisions," in The Irrational Economist (E. Michel-Kerjan and P. Slovic, eds.), p. 22.
After reading my discussions, many managers wish to share their experiences, thoughts and critiques of my ideas. I always welcome and reply to their comments.
Please share your thoughts with me by using form below. I will send reply to you via email. If you prefer to be contacted by phone, fax or postal mail, please send your comments via my contact page.
© 2014 Pipeline Physics. All rights reserved.