How to count cards in blackjack
Traditional decision theory recommends distinguishing the quality of decisions from the quality of results. One makes decisions to maximize expected value but because of risk, the results could still be poor. If that happens, one made good decisions but received a bad outcome because of bad luck. For example, one selects projects to maximize portfolio value, but potentially from bad luck, the selected projects fail.
The recommended distinction arises from a restrictive assumption of decision theory: models used for making decisions completely and correctly represent the uncertainty that affects a decision. Of course, this assumption is incorrect, and the mistaken assumption causes theory and practice to diverge. The problem is acute for project portfolio management (PPM), and it causes consternation among managers, who recognize that something is wrong, but do not what is amiss. They see PPM's best practices, and lacking a better excuse for declining them, say, "My organization isn't ready." In truth, they believe something is awry, and they are right.
Fortunately, recent advances in decision theory are relaxing the restrictive assumption, producing important implications for PPM and drug development. (To learn about the new research in decision theory, see my discussion, "The difference between theory and practice: it's disappearing.") However, PPM still adhere's to the older assumptions. To better understand the consternation they cause, and see how theory and practice diverge, let's consider a task that is simpler than drug development, a game with simple rules and well-defined probabilities: blackjack.In blackjack, a player can beat the house by counting cards. A player can choose from many counting schemes, some simple, others complex. Sophisticated schemes use more information and use it in superior ways, keeping more counts and having more incisive rules that relate the counts to betting. In contrast, the simplest schemes keep only one count and relate the count to betting via coarse rules.
Which scheme is best? Let's see the answers produced by theory and practice:
- Theory: Via Monte Carlo simulation, a technique used in PPM, scholars studied the schemes. The more sophisticated schemes have higher expected values. Without surprise, if one uses more information and uses it better, one achieves superior results.
- Practice: The results from actual casino play differ from the Monte Carlo simulations. When used in a casino by a skilled player, the simple schemes make money but sophisticated schemes fail dreadfully. The sophisticated schemes perform worse than not counting.
How can this be?
Decision errors! Casino play requires applying basic strategy (when to hit, stick, double and take insurance) and card counting while facing distractions from noise and conversations. One must count at the speed of the deal as well, which can be quick. These distractions and difficulties cause counting errors, and these errors cause one to raise bets at the wrong times. Players mistakenly bet more when the house has the advantage, producing large losses.
The technical term for a person's limited abilities is bounded rationality, and it is a source of uncertainty that the Monte Carlo analysis did not consider. Failure to consider all the uncertainty faced by decision-makers is why theory differed from practice.
PPM suffers the same shortcoming. Often, its experts too easily dismiss bounded rationality and ignore additional sources of uncertainty that aflict drug development, such as modeling errors, unknown-unknowns and a myriad of organizational issues that affect decision-making and implementation. Drug development executives must manage all the uncertainty they face (practice), even though PPM's mathematical methods ingore's much of it (theory).
Let's continue the blackjack example. Suppose a blackjack player hires a consultant who espouses decision theory. The consultant advises the player to maximize expected value by using the most sophisticated counting scheme and to distinguish between good decisions and bad results. The player accepts this advice, enters a casino, makes many counting errors and loses money. The player exits the casino poor, but he congratulates himself on making optimal decisions, even though he feels unlucky.
Here's a different result. The player selects a sophisticated counting scheme, enters the casino and begins loosing. However, rather than blaming the losses on bad luck, as prescribed by decision theory, the player reflects on his results and realizes that he is making mistakes. He switches to the simplest counting scheme and wins back his money.
Like the blackjack example, organizing drug development to better manage uncertainty requires deviating from from traditional decision theory (by applying its recent advances). However, the solution is not as simple as adopting the rudimentary decision methods. Rather, it requires identifying where and how uncertainty causes errors, which are canceling marketable compounds (false-negatives) and advancing unmarketable compounds downstream (false-positives). One must organize drug development to reduce the frequency of both errors while maintaining, or even increasing, pipeline throughput. For a prescription for achieving these goals, see my discussion, "How to make drug development more productive.
After reading my discussions, many managers wish to share their experiences, thoughts and critiques of my ideas. I always welcome and reply to their comments.
Please share your thoughts with me by using form below. I will send reply to you via email. If you prefer to be contacted by phone, fax or postal mail, please send your comments via my contact page.
© 2014 Pipeline Physics. All rights reserved.