toby groves [email protected] @tobylgroves linkedin tobygroves judgment removing the mystery

Toby [email protected]@tobylgrovesLinkedin TobyGroves

JudgmentRemoving the

Mystery

mailto:[email protected]

• The role of contrast and perspective

• A basic understanding of probabilities

• Recognizing the limitations inherent in our thinking and our technology

• An understanding of the invisible forces that influence judgment (biases)

• The neuroscience behind judgment processes

• The role of the the emotional brain in logical judgment

• Judgment under pressure

Understanding Judgment

• Logic- Normative theory of how one should reason. Formal logic.• Mathematics/computer science• Hard answers-precisely defined operations• Not descriptive theory of how humans actually reason• Does not generally consider psychology

• Critical Thinking- Everyday, real life reasoning.• Using creativity• Deals with psychology. Beliefs and recognizing biases• Gray areas, complex, unclear and changing situations• Reflective and independent thinking

• Judgment- decisions made, and conclusions reached that drive actions and behaviors in how you deal with a situation or set of facts

What is Judgment?

• Rational drug design• The technical aspect of models for drug design are easier than the human

endeavor of interpretation

• Placebo effect without believing in the placebo?• Unconscious effects• Cyclosporin A

• Placebo studies using the meaning response

• We are constantly primed- consciously and subconsciously and unconsciously

Judgment: Logical and Rational thought

Judgment

(Beaman et al. 1979; Diener et al., 1976)

Good Judgment –Including Ethical Judgment

What it is not…• Whatever society accepts- If this were true, you could take a poll and you would believe

whatever the result is• Not science- Science deals with how people “ought to act”, not considering biases,

psychology, or emotions• Not the law- Laws can deviate from what is ethical, and rules have a counterintuitive

effect on ethical thought processes.• Intuition- We may have formed habits in thinking that bias us. Familiar habits trigger

oxytocin.

Judgment


Good Judgment-Including Ethical Judgment

What it is…

Endless number of models, theories, and definitions-for our purposes today we will view ethics as:• Well founded standards of critical thinking of fairness and of right and wrong• Understanding our own reasoning processes, thinking critically about these processes,

and seeking to understand our unique biases• A continuous effort to understand and develop one’s own reasoning and standards• Understanding the important role the emotional brain plays in judgment and critical

thinking

Judgment: Removing the Mystery


What is judgment?

Perception- Becoming aware of something through the senses

Interpretation- The action of explaining the meaning of something

Judgment- Forming an opinion, estimate, notion or conclusion from circumstances presented to the mind

• Critical thinking skills allow us to:• Understand connections between ideas and concepts• Construct and evaluate arguments• See inconsistencies and gaps in reasoning• Assess the relevance of ideas• Examine the justification of your own beliefs

• Critical thinking should not be confused with:• Being critical or argumentative

(Indeed, critical thinking plays an important role in cooperative reasoning)

What is Critical Thinking?

• Halpern Critical Thinking Assessment dimensions of critical thinking

• Argument analysisRecognizing the difference between conclusions and

assumptions• Seeking reasons why certain considerations would support or

fail to support certain conclusions• The difference between uninformed opinions and reasoned

judgment

Critical Thinking


• Thinking as hypothesis testing• Looking at a sample and drawing conclusions and testing to

see if that conclusion is reasonable• Hasty generalizations from samples of behavior• Not considering control conditions (The change you

witnessed may have happened with or without the item you identified)

Critical Thinking


• Likelihood and uncertainty• A basic understanding of probabilities• Likelihood of outcome in uncertain situations• An extreme event is likely to be followed by a less

extreme event (regression to the mean)• If a coin flip results in 3 heads in a row, what is the likely

outcome of the 4th flip?

Critical Thinking


• Decision making and problem solving (judgment)• Decision making -Selecting from alternatives based on

relevant information• Problem solving -Finding solutions to a situation, requires

seeing connections in data that may not appear to go together

Research in judgment skills

Critical Thinking

• Scientific method• Ask a question• Do background research• Construct a hypothesis• Test your hypothesis with an experiment• Analyze your data and draw a conclusion• Communicate your results

• Limitations-Goal is not prove a rule but to disprove (null hypothesis)• Randomly assign subjects• Exactly the same treatment of subjects with no outside influences that bias

the results (remove contextual differences)

Judgment: Critical Thinking

• Important in the scientific method:

• Validity-Are you actually measuring what you intended to measure? Internal and external validity. The extent to which a concept, conclusion or measurement is well founded and actually corresponds to the real world.

• Reliability-The ability to achieve similar results under consistent conditions.


• Critical thinking• Is thinking about your thinking process• Analyzing the thinking process and the outcome • Supporting your beliefs and actions with solid reasons• A meta-thinking skill that requires

• Theory• Understanding the rules of reasoning• Laws of logic and methods of scientific reasoning• Understanding what not to do (not just what to do)

• Knowledge of common mistakes in reasoning (biases)


• Attitude• Openness• Willingness to be wrong• Exploring biases and assumptions• Accepting, even seeking criticism• Preferring to figure out problems on your own so you really

understand them-not handed the answer• Analyzing your errors in judgment

• Instead of excuses, curiosity about what biases or other errors may have resulted in the incorrect judgment

Critical Thinking- A meta-thinking skill that requires:

• You may use:• Inductive reasoning-From the bottom up approach-Seek strong

evidence for the truth of a conclusion-seeks only to give probable indication of truth-not absolute-is inherently uncertain

• The old dictionary definition of deriving general principles from specific observations is considered out of date.

• Deductive reasoning-Top down approach-Start broad and work towards specific conclusion-Relies on original premise being correct.


• The story of Cook County Hospital

Judgment: When Less is More

Situational AwarenessAttractors and Distractors

The “more information is always better” bias

Research shows that doctors have more confidence in their diagnoses when more data is available, even though the accuracy of their diagnoses does not improve with that additional data-Joseph Kidd -Stock market performance/weather forecasts

Judgment: When Less is More

Creeping determinism

After an event has occurred, we feel very sure that the outcome is what we would have predicted

Research had people predict an outcome and the level of certainty of their outcome, then after the outcome occurred, they asked them again- and people gave much different answers, either changing their answers (saying they had predicted what ended up occurring, or changing their level of certainty very dramatically.

Complicated-Many different moving parts that all fit together like a puzzle (An engine)

Complex-Many similar parts that interact in ways that are not fully predictable (A group of people). What happens is a mystery.

Does this distinction matter? Absolutely-it changes the way we search for data

Judgment: Complex or Complicated-Understanding the true nature of your problem

Choice Complexity and Professional JudgmentStudy on bias in professional decision-making using Doctors

• Decision to send patient-a 67 year old farmer for hip replacement surgery

• First group of doctors told they forgot to try one drug-Ibuprofen. Would

they call patient back from surgery to try the drug? 50% said pull then back

• Second group of doctors told they forgot two drugs-Ibuprofen & Piroxicam.

Would they call patient back from surgery? 72% let patient go on to surgery

Why? Choice complexity increases chance of going with default option

Redelmeier & Shafir (1995)

• Puzzles • Source dependent (data)• Clear conclusion• Every piece of new information makes answer clearer• Search will fall in the normal curve

• Mysteries• Receiver dependent (interpretation skills)• Answer does not become clearer with additional piece of information• Making sense of the information we already have makes answer clearer• Making judgments about situations that are uncertain• Search is open-may be outlier data, or uncorrelated

Judgment: Puzzle or Mystery

Judgment: Choke or Panic

Choke- Thinking too much-When you are thinking about the steps and should just be on autopilot

Panic- Thinking too little, when you should be following the steps that you learned

Choking vs. Panic

The part of the brain that handles new learning is different than the part of the brain that handles memory and memorized activities.

-Memorized tasks-Cerebrum-automatic and extremely fast

-New learning- cerebellum handles motor learning, parts of temporal lobe handle other types of learning-slower

Time Management

Judgment: Research Techniques

Quantitative- Relies on strict assumptions (normal distribution, no outliers, variables are mutually exclusive), data driven-complicated-what the data is

Qualitative- Recognize assumptions may be biased, except for underlying assumption that people can perceive the world differently-complex-what the data means

Big Data and Judgment

• What makes “BIG” data different from the data we’ve always used?

• Has a different direction• Usually we have a problem and form a hypothesis and then

gather data• In big data-we have this pile of data and want to see what

there is to learn

Judgment

What is the nature of your data?

Understanding the nature of the data requires different perspectives

One or a few things-middle-

Many things

Is the problem you face one of logic, or one of insight?

• Insight- Understanding the hidden nature of something.• Requires a new perspective to achieve a solution

Judgment: Insight

There are ten bags, each containing ten weights, all of which look identical. In nine of the bags each weight is 16 ounces, but in one of the bags the weights are actually 17 ounces each.

How is it possible, in a single weighing on an accurate weighing scale, to determine which bag contains the 17-ounce weights?

Judgment: Insight

How is it possible, in a single weighing on an accurate weighing scale, to determine which bag contains the 17-ounce weights?

Take 1 from the 1st bag, 2 from the 2nd, 3 from the 3rd etc. Then weigh all those coins. If all the bags weigh 16 ounces you will have 55 ounces (10+9+8+7+6+5+4+3+2+1). Any amount in excess of the 55 ounces will determine which bag contains the 17 ounces (two ounces over = bag 2 ,if it is 7 ounces over = bag 7 etc)

Judgment: Insight

Insight-Spatial

Three cards lie face down on a table, arranged in a row from left to right. We have the following information about them. a. The Jack is to the left of the Queen b. The Diamond is to the left of the Spade c. The King is to the right of the Heart d. The Spade is to the right of the King. Which card - by face and suit - occupies each position?

Insight

Three cards lie face down on a table, arranged in a row from left to right. We have the following information about them. a. The Jack is to the left of the Queen b. The Diamond is to the left of the Spade c. The King is to the right of the Heart d. The Spade is to the right of the King. Which card - by face and suit - occupies each position?

Jack and hearts, King of Diamonds, Queen of Spades

2, 4, 6, 30, 32, 34, 36, 40, 42, 44, 46, 50, 52, 54, 56, 60, 62, 64, 66, ___?

Judgment: Insight

Think of a stranger you saw today, perhaps the person who served you coffee-anyone.

-Picture that person-Now write a description of that persons face-You would now have more difficulty picking

that person out of a lineup

Judgment: Insight

Dissecting something-breaking it down into pieces, damages insight. It loses meaning as a

whole.

Judgment: Insight

Judgment: Insight-Removing barriers to Rapid Cognition

Rules of Improv

• Improvisers get a suggestion of a location, object, or relationship• Make up everything on the spot• But…guided by basic rules


Rules of Improv

Rule #1 Yes and… This is the holy grail of improv-this allows scenes to be created out of nothing. This rule says we must agree with the reality created by the other actor and add to it which is agreed to in turn. Disagreeing “blocks” the scene and this is the end of the scene-nothing but frustration comes from this disagreement.


Rules of Improv

Rule #2 Act/React- Everyone on stage should be making an effort to contribute to the scene-take responsibility and confront fear. Put an idea out there and trust that your fellow actors will make the best of it. No holding back-be bold! If other actors are on a roll-let them keep going, but be ready as soon as the scene allows. When everyone doesn’t pull their weight – the scene dies.


Rules of Improv

Rule #3 Make someone else look good and you look good- Everyone is a supporting actor and instead of looking out for yourself, look out for the other actors. This builds trust, acceptance, responsibility and commitment. Have the other actors back. Never, ever put someone else on the spot to get yourself out of one. Don’t block, don’t say something “standard” that stalls the scene.


Rules of Improv

Rule #4 Be Vulnerable and Honest- Don’t try too hard to be funny, it will feel forced. Be as natural as possible, this is always most interesting and moves the scene forward. Don’t hide behind standard answers and responses-be authentic!


Rules of Improv

Rule #5 There Are No Mistakes- Everyone is doing the best they can-no matter what happens, everything is used-a bad joke, someone’s frustration, etc. This shows that the reward that comes with the risk outweighs staying safe.


Rules of Improv

This basic structure frees you up to think openly, like the doctors at Cook County hospital when they used the Goldman Algorithm

(Dan Ariely, Duke University)

• Search for level of certainty that doesn’t exist• We subconsciously assign tremendous weight to our

assumed outcomes and automatic character assessments• This assessment causes profound biases our

perceptions • Colors our judgment• Results in automatic assumptions and predictions• Causes subliminal susceptibility to seek subliminal

information

Emotional Need for Certainty

Negotiation Research in Auditing

• The more ambiguous the accounting issue, auditor likely to accept clients preferred treatment.

• The more important/increased pressure surrounding the accounting issue, auditor more likely to concede

• The greater number of possible alternative treatments, the less likely the auditor was to insist on their judgment

• Clients perceive they are more likely to persuade auditors if firm has short tenure

• Auditors are much more likely to waive smaller adjustments that aggregate to a material amount than those that are individually material (Small Steps Phenomenon)

Brown & Wright (2008)

Take a “Mental” Walk

The importance of taking breaks

• Heavy concentration causes fatigue• Fatigue results in diminished attentional control

• Give your mind an easier task every 20-30 minutes for a few minutes• Take a longer break every 2-3 hours• Improves control and judgment• Can allow elusive answers to appear

Our mind will force the breaks if you don’t take them voluntarily-Research shows that we spend 30%-47% of our waking hours daydreaming

We benefit from taking a walk-even if it is a “Mental Walk” –spurs those spatial parts of our brain.

Judgment and Pressure

Task Type: Creative vs. Logical

Creative TasksSuffer more from pressures- limitations, distractions, and structure/rules

Logical tasksCan improve under low pressure but suffer increasingly as pressure increases into

moderate/extreme

Physical tasksCan improve under moderate pressure but suffer under extreme pressure

Judgment and Pressure

The Effect of Rules and Sanctions• Narrows our thinking processes• Control the tendency to act only in our self interest (Self protection-Good

Samaritan laws)• The tendency to focus on the letter of the rule rather than the spirit.

• Is tendency to interpret rules narrowly due to the characteristics of the rules themselves?

• Principles versus Rules based accounting standards• CFO’s less likely to make aggressive accounting choices under “Principles”

based system (Agoglia, 2009)• CFO’s came to more similar conclusions under principles than rules based

system• U.S. accounting standards have become so precise as to invite opportunistic

interpretation by corporate executives (Agoglia, 2009)• The mention of sanctions gives a different frame

Mood and Professional Judgment


• Can something as simple as mood affect professional judgment?

• Study:

•Auditors in a “good” mood state found inventory values to be an average of about 5% less than the client valuation

•Auditors in a “poor” mood state found inventory values to be an average of about 8% less than the client valuation

Recognizing BiasesConfirmation Bias- The tendency of people to favor information that confirms their beliefs or hypotheses. Studies show we tend to seek information that confirms rather than properly tests the validity of what we are told

Selective Perception-A cognitive bias wherein individuals are subconsciously attracted to stimuli that falls in their range of reasonable expectation and are oblivious to other stimuli.

Belief Perseverance- Our tendency to seek information that is consistent with our pre-existing beliefs.

Biased Processing of Disconfirming Information- The subconscious bias towards being more critical of evidence that disconfirms our initial beliefs than evidence that confirms it.

Toby [email protected]@tobylgrovesLinkedin TobyGroves

JudgmentRemoving the

Mystery

mailto:[email protected]

toby groves [email protected] @tobylgroves linkedin tobygroves judgment removing the mystery

Documents