Summing Up
What is the Proper Role of Judgment in Decision-Making?
There is a seemingly universal (and currently popular) quest for rational processes—what Hamilton Carvalho terms "cognitive repairs"—to counter the foibles of human judgment. Nevertheless, the predominant opinion among respondents to this month's column is that judgment has a dominant role to play in some decisions while it is an element in all decisions. That assumes, of course, that we can arrive at some agreement about what the term means.
Seena Sharp equated judgment with intuition. As she said, "Intuition is far more valuable in decisions regarding people …" However, intuition lags reality. "Use it to consider if the new idea makes sense, but don't give it more weight than it deserves." B. Graham equated judgment with a "gut check." He put it this way: "Seems to me that a 'gut check' on big decisions is always prudent. I realize that is the sort of bias these authors warn about, but the application of their methods shouldn't reduce decision-making to a formula…"
Phil Clark had a more encompassing view of the term: "The discussion you opened is not contradictory to sound judgment. It only emphasizes it…. Judgment is an encapsulation of all those elements … data, facts, processes, etc… that go into decisions from which we have learned in the past." Joe Schmid said: "Our education and work environments are analysis rich and have become synthesis poor… Synthesis is what happens when, after absorbing a subject, there is a leap in thinking that one can't quite explain."
Zuff Deo explained the semantic difficulties this way: "We do use various faculties and when we can't figure out which one we are using we call it gut feel."
There were a number of views regarding the role of judgment. Wayne Brewer commented that "judgment is needed for creating a vision, decision tools are for optimizing a decision." Mark Andrew said, "Judgment can work well in 'high validity' situations—those that are encountered frequently and provide accurate feedback…. Most situations, especially in the business world, have poor validity, in which case intuition is a poor guide. Vance Kirklin appeared to disagree with this, saying that, "Transformative decisions … (strategic decisions that transform organizations…) require a level of intuition. Organizations that preclude intuition from their decision-making will … never be, or cease to be, transformative."
Others had a more expansive view of the role of judgment. As Donald Shaw put it, "Just because we have the technology to record terabytes and terabytes of data does (not) mean we have the tools to understand what it is saying. Judgment is still crucial…." Debra Bordignon put an interesting twist on the issue, saying, "Where systems, tools, and artificial intelligence reach their limits, human judgment IS the decision support tool."
Several were quite eloquent. Aim asked: "So, I need to make a proposal to my girlfriend. What should I use to make the correct decision? Intuition or regression analysis?" Paulina Czentor added: "And going one step further, let intelligent machines make judgment-free decisions. No flawed reasoning, no more human 'interference'… replacing hordes of managers. What will one do with those?" Yan Song reminded us that, "As long as there is still mystery and new discovery to be made in life, human judgment will remain indispensable."
Are we naturally biased on the question? What is the proper role of judgment in decision-making? What do you think?
Original Article
In the last several years, a veritable tsunami of advice on how to make decisions has hit the Internet and what few shelves remain in our local bookstores. The advice is a distant relative of early ideas about decision theory in which we were advised to construct decision trees, mapping outcomes, attaching values to each one, and estimating probabilities that various combinations of outcomes might occur. Judgment entered into the construction of the resulting "decision trees," but the process itself was a way of injecting a certain amount of objectivity and analysis into the decision to be made.
In recent years, we have been advised to make certain decisions in a "blink" by Malcolm Gladwell, to "think twice" by Michael Mauboussin, and to think "fast and slow" by Daniel Kahneman. The replacement of customs and biases with data, "big" or "small," has been intended, at least in part, to drive out such things as tradition, habit, and even superstition in endeavors ranging from child rearing to professional sports. After all, wasn't the book and film, Moneyball, at least in part a glorification of the triumph of statistics and probabilities over intuition and managerial judgment in professional baseball?
Two recent books add to the genre of advice on decision-making. One advises us how to make better decisions. The other helps us ensure that we don't allow our decisions to get sidetracked (or sidetrack them ourselves).
In their book Decisive, the Heath brothers cite four major reasons—all linked to common human traits—why we make poor choices and how to avoid doing it. They are: (1) the "narrow framing" of problems that makes us miss options; (2) the "confirmation bias" that leads us to give undue credence to information confirming a decision while ignoring other information; (3) the injection of "short-term emotion" into the decision process; and (4) overconfidence that we naturally display about the future (something that may be peculiar to only certain of the world's cultures, by the way).
They advise us to do such things as: (1) widen our options by emphasizing the "and" over the "or" in formulating them; (2) reality-test assumptions by reviewing them with more objective associates or making small tests; (3) seek ways of attaining distance by looking at a decision through someone else's eyes or focusing on the long-term impact of the decision; and (4) prepare to be wrong by setting limits on outcomes (similar to a "stop loss" order in stock trading).
Harvard Business School's Francesca Gino cites findings from her own research studies and those of her colleagues in her book Sidetracked, to warn us of three types of forces that derail our decisions: forces coming from within, from our relationships, and from the outside. Among those that emanate from within are an inaccurate and often inflated view of ourselves that leads us to treat advice inappropriately at the wrong times, "infectious emotion," and a tendency to adopt an overly narrow focus. To cope with these she suggests ways of achieving greater self-awareness (mitigating our biases by soliciting expensive advice, which we are more likely to take seriously, for example), taking our emotional temperature (determining when our feelings regarding a decision "were triggered by an event unrelated to the decision at hand"), and "zooming out" for broader perspective (for example, by asking "What information am I missing?").
I didn't Google the texts of these books, but there is no mention of the word "judgment" in their tables of contents or indexes, and I don't recall the use of the word in the texts. In fact, if there is a sense that one gets from all of this work, it is that we are our own worst enemies when it comes to making and implementing good decisions. We need tools to correct the errors and biases of our own judgment. This is puzzling, because we are frequently reminded that the ability to exercise judgment is what sets humans apart from other forms of life. (Perhaps judgment is what leads us to adopt recommendations such as those of these authors.) Is there still a role for judgment in decision-making? What do you think?
To Read More:
Francesca Gino, Sidetracked: Why Our Decisions Get Derailed, and How We Can Stick to the Plan (Boston: Harvard Business Review Press, 2013)
Malcolm Gladwell, Blink: The Power of Thinking Without Thinking (New York: Little, Brown and Company, 2005)
Chip Heath and Dan Heath, Decisive: How to Make Better Choices In Life and Work (New York: Crown Business, 2013)
Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus, and Giroux, 2011)
Michael Lewis, Moneyball (New York: W. W. Norton & Company, 2003)
Michael J. Mauboussin, Think Twice: Harnessing the Power of Counterintuition (Boston: Harvard Business Press, 2009)
However, intuition for other business decisions (regarding customers, features, distribution channels, new products or services, marketing approaches and much more) is less and less reliable in a changing marketplace - and in fact, may be misleading.
Far too often, decision-makers are shocked when the results are disappointing. And they rationalize their great decision with "The customer doesn't get it." Or "We were ahead of our time."
Intuition is an aggregation of the past: experience, data, facts, beliefs. All of these reflect yesterday and may no longer be true. More importantly, they don't capture what's changing and emerging - the signs of today and tomorrow - and the only time a company sells.
Intuition lags reality. Use it to consider if the new idea makes sense, but don't give it more weight than it deserves.
People who are not lost, in other words, psychologically balanced, know that they're here and progress by simply getting on with it.
That is very different than most "old-school" decision-making techniques that were based on personal judgment, limiting the number of people involved in decisions, and relying on limited information. Even if each of the participants is competent, and the information is of high quality, how could it possibly match the collective wisdom of a much larger group of experts across the spectrum of associated domains with higher quality information at their disposal? But in today's business environment we have an important complication. We're living in a world of specialists where the norm is for someone to spend their entire careers in one or two functional areas. So the error and bias factors are even greater today than a generation ago when managers making key decisions were trained as generalists, and circulated from one functional area to another in the years following business school, in preparation for the day one would rise to the challenge of running that company.
So, to answer the question, "Is there a role for judgment in decision making?" - Yes, of course. But we must harness the collective intuition, judgment and wisdom of many people today as a baseline to get where we once were with the general management decision quality of yesterday. The good news is we have better information at our disposal and the better tools to tie more relevant perspectives together to make higher quality decisions than ever before. But I would argue that far too many executives in positions of power today think they can apply old-school judgment to complex decisions in areas where they have no education or experience. They are truly dangerous - as Howard Gardner points out in his "5-year-old mind" analogy - and we see the consequences far too often.
You may have been burned in the past using sound "investment advice" from an "expert". A person building sound judgment likely will not fall into that trap again. The next time you may ask better questions, do better research, or even check your emotional committments to a decision based on the past experience. I view judgement not as an independent element of decision-making but a dependent element needed to force you to widen your decision base.
A person making the same poor decisions over and over or a person who narrows their thinking would not fit my definition of judgement.
Another way to think of it is that judgment is needed for creating a vision, decision tools are for optimizing a decision..a much more analytical exercise that would be a lower level in the hierarchy since decisions are in support of visions.
with the linear techniques of statistics." Human beings are not machines. They seek meaning. Interacting with each other is one of the ways they do that. And those interactions may be planned or unplanned, objective or subjective. In any event they can change the way things get done without leaving a record in any data base. Data may be helpful if analyzed properly, but how to do that is a big question concerning big data. Just because we have the technology to record terabytes and terabytes of data, does mean we have the tools to understand what it is saying. Judgment is still crucial.
On the former, analysis can often lead to rationalization. On the latter, a contract is only as good as the volition of the parties involved. A sensitive gut is an important warning device.
1) Directional Causality (Does A cause B or does B cause A);
2) Reason behind the Relationship;
3) Whether the relationship is bound by a third factor or if it is just a spurious relationship;
4) Is the relationship temporary or will it continue into the future; or
5) How to exploit the relationship.
This is where judgment needs to come into play--to bring the numbers to life in a story that provides meaning that can be acted upon.
Also, keep in mind that:
1) If your sample size is large enough, all relationships become "statistically significant" even if they are not "meaningfully significant" in a business sense. This can cause a focus on "ghosts" rather than on the true, critical issues.
2) You cannot measure the future since it has not yet occurred, yet the future is where decisions make their impact.
Hence, numbers alone have serious shortcomings. And if that's all you look at, then you are like a driver of a car who only looks at the dashboard numbers and never looks out the window. You miss the panorama of insight that keeps you safely on the road to the future.
We cannot rule out entirely the role of judgment in decision making. People are bound to add their " common sense" to other factors while reaching the conclusive stage.
For much more on this, read Conditions for Intuitive Expertise:
A Failure to Disagree by two experts who thought they would be on different sides of the question you ask (Daniel Kahneman and Gary Klein). FYI, the word "judgment" appears 105 times :-)
http://www.chrissnijders.com/eth2012/CaseFiles2012/Kahneman,%20Klein%20-%202009%20-%20Conditions%20for%20intuitive%20expertise%20a%20failure%20to%20disagree.pdf
Has anyone ever made a perfect decision? Good and bad decisions can only be evaluated in hindsight.
Statistical or mathematical analysis is useful but the assumptions in the modelling are something to be aware of.
At the "decision moment" of making a decision all the analysis comes to naught. You could subject yourself to all the analysis you want. It could inform you but you have to use your discernment or judgement to choose otherwise.
The development experience from going through a bad decision (or something which we think in hindsight is one) is also useful if we choose to learn from it.
It's the freedom to choose that makes us unique to all other species.
But if its your key competitive differentiator, beware of making data your primary basis for decisions. Data is too often looking back - and not forward. Data didn't see the internet coming, didn't see the impact on social technologies coming, didn't see the risks in CDOs - and so many more big changes that got so many data driven companies in trouble. Its judgement and intuition that spots "something has changed and the old rules don't apply" that keeps you leading and in business over a long period of time.
It strikes me that the books cited (above) attempt to provide that sort of "judgment": to be aware of human predispositions and to seek a more thorough consideration. Just because they may not use the word, that doesn't mean that judgment is not included in the ultimate decision.
On the other hand, some leaders seem more enthusiastic about implementing decisions they've made that relied upon their own judgment to a significant degree.
Another interesting dimension of this: some leaders who agree that the data available argues for a decision different from the one their "gut" advises may then almost be hoping for that decision to fail -- thus allowing them to revert to decisions heavily based (or even entirely based) on their own judgment!
be prepared to manage paradox. Similarly, in decision-making devoid of meticulous judgment, we might arrive at the wrong decision precisely, and thus must face the consequences that follow.
I am not surprised that the word 'judgment' is hard to come across in today's publications. Why? Because the word 'judgment' sounds very judgmental in our post-modern, all-inclusive, preference-driven, value-free world.
How did we get to this point? You can find a very good answer to this question in Alisdair MacIntyre, After Virtue.
What can we do about this allergy to the world 'judgment'?
Well, instead of judgment, I prefer to use 'prudential discernment', 'prudential decision' making or 'prudential evaluation'. It sounds less scary, and yet it allows to teach the 'obscure and almost clerical sounding' art of judgment, which is nothing else than the exercise of practical reasoning.
Judgment is unavoidable whenever uncertainty is involved. Of course there are ways to reduce the judgment content, but it cannot be completely eliminated. This is why intuition - the judgment forming part of our unconscious - plays a part in difficult decisions whether we like it or not.
For experts, like tennis players or negotiators or marketing specialists, judgment is everything. They do not have time to think things through from first principles or the uncertainties are too complex to reduce to a formula. Without judgment they would be unable to act.
The instinct, insight, sixth sense, feeling, you name it, is derived from a human being's encounters to the extent that human judgment and decision making techniques become one.
I would therefore conclude that it is practically impossible to exclude human judgment from decision making as they are inexplicably intertwined.
I think that having various ways to solve problems and make decisions is crucial as it expands your thinking abilities. Uncosncious Competence has a inherent danger in that unconscious habits might be formed.
I wrote about this in blog post asking - What model are you using? http://goo.gl/24goRO
case) that it was no longer the whole truth. After 1968 (Carterphone) and 1974 onwards as the anti-trust case proceeded to the break up of AT&T, no one believed any longer in that myth. The cell phone revolution, not to mention the Internet were additional nails in that coffin. Still, to the end the self-assurance of top management brooked no doubt. It is fascinating to me how big disruptive innovations ( Christensen) are based on a hunch, an insight, a dream, anything but following a rational decision-making model. We make major life ( and death) decisions by "judgment"; as Noel Tichy and Warren Bennis write in their book JUDGMENT, it is a learned skill, often propelled by errors made. As the divorce rate, and failed M&A's show, many people lack the judgment ex ante.
Slogging through a few hundred case studies of Harvard and ECCH is not foolproof, but a good start on developing
such a capability.
Transformative decisions require a level of intuition, in my opinion, since there is no way to identify and measure every internal and external variable, much less build them into a decision-matrix that covers every possible vector and intersection point. Attempting to collect and analyze "everything" often leads to analysis paralysis, and/or mundane, non-transformative decisions.
Operational decision-making often has the advantage of foundational standardized policies, procedures and processes, which remove many variables and generate baseline and benchmark data to be analyzed. Incremental changes are easier to identify, apply and measure.
As Dr. Heskett mentioned, it is the human ability to make judgement calls that sets us apart. More often than not, transformative decisions require leaps of intuition, otherwise, they are probably not transformative, but simply extensions of existing ideas.
It seems it might be valuable to look at the decision-making processses utilized by successful individuals and organizations. How did Gates and Allen arrive at the conclusion that operating systems were the key to building Microsoft? How did Page and Brin conclude revenue could be generated from "free" access to information from across the world-wide web? Or Zuckerberg and company with Facebook, etc, etc, etc?
Organizations that preclude intuition from their decision-making will, in my opinion, never be, or cease to be, transformative in nature. Is data important? Absolutely - but it is also limiting when we feel a need to know everything before we make a decision.
The power of the human brain to make leaps of logic should not be minimized. In fact, it should be recognized for its miraculous nature and leveraged in every instance.
If we had perfect and infinite information, decision makers would have an easier job and they would be perfectly correct all the time. Alas, that is not the world I (we) live in. Luck, judgment, gut-feeling, (women's) intuition, co-incidence, timing, faith, courage of conviction, confidence, contra-intuition ("observe the masses and do the opposite"), best practices, follow the trend-buck the trend, experience ALL have a role to play in decision making.
If we take judgment out of decision making we may as well take THINKING out of decision making. Taking judgment out of decision making is similar to tying both hands behind our backs and trying to get things done.
A copernician revolution is expected.
The provocative title of this article points out a real problem of modern societies, this highly inter-connected and reflexive world of ours.
Most organizations have very poor decision making processes (if any) because they assume that good decision making is the natural duty of a decision maker based on (supposed) robust analysis, advices from peers and/or collaborators and, ultimately, personal knowledge or expertise.
Most companies do not use lucid scientific approaches in risk analysis and have forgotten the proper role of statistics for multi-criteria decisions in uncertain or human-biased contexts.
This weakness is obvious in many areas of strategic decisions : R&D, capex investments, recruitments/assigments, finance...
One symptom of this situation is the loss of credibility of quantified risk analysis in Finance because its usage was flawed by over confidence (let alone greed), lack of competency, moral hazard and political interests. It allowed the systemic crisis of 2008-2009 (among others)...all in all, a result of poor human judgments.