real life experience as a pedagogical principle

Real-Life Experience is not Real-Life Experience is not (Sufficient) Pedagogical (Sufficient) Pedagogical

GuidanceGuidance

Human Computer Interaction ConsortiumHuman Computer Interaction ConsortiumWinter Park, ColoradoWinter Park, Colorado

January 31, 2008January 31, 2008John C. ThomasJohn C. Thomas

Outline (Landauer Paradigm)Outline (Landauer Paradigm)

• Comments on the Carroll, Rosson, Borge, Xiao, Jiang paper– Praise

– Nits

• A Random Walk through related work.

• This is all from the perspective of what is best for the learner to learn ---

Praise…Praise…

• Use of cases to be applauded…• BUT, they go way beyond this to:

– Think about why cases seem to work.– Consider how to select cases.– Design a way to incorporate cases into an overall

pedagogical framework.– Think about how to improve narrative aspects while

leaving important minimalistic features.– Design ways to enhance collaborative learning with

tools and teaching (e.g., about roles and processes).

Potential NitPotential Nit

• (Is that just the gleam in the eye of a louse?)

• Do we really need “educational institutions”? Why not just use real world experience itself?

• Isn’t experience the best teacher and not some scaled-down version?

• Slight detour to consider this question…

Comparisons of FieldsComparisons of Fields

• Tiger Woods: 2007 Earnings = $10,867,052.

• Nobel Prize in Computer Science =

Comparisons of FieldsComparisons of Fields

• Tiger Woods: 2007 Earnings = $10,867,052.

• Nobel Prize in Computer Science=– OOPS…there isn’t one! But, the Nobel Prize

in Mathematics is = $1, 414,427.16– Turing Award now $ 250,000

Therefore, we can concludeTherefore, we can conclude(Main Metric of Merit in American Society;(Main Metric of Merit in American Society;

only Money Matters)only Money Matters)

• Golf is ~ 10 x as important as mathematics

• Golf is ~ 40 x as important as computer science

Of course, money isn’t everythingOf course, money isn’t everything

• Computer scientists get to work in a variety of controlled environments…

While golfers are often required to work in While golfers are often required to work in weather-beaten venues such as these:weather-beaten venues such as these:

Carroll’s MetricCarroll’s Metric

• Results 1 - 10 of about 2,130,000 for "Human Computer Interaction"

• Results 1 - 10 of about 18,300,000 for "Tiger Woods".

Carroll’s MetricCarroll’s Metric

• Results 1 - 10 of about 2,130,000 for "Human Computer Interaction"

• Results 1 - 10 of about 18,300,000 for "Tiger Woods".

• Results 1 - 10 of about 665,000,000 for golf

Can golfers learn to be good golfers via Can golfers learn to be good golfers via “real life” experiences alone?“real life” experiences alone?


• No.

• Most good golfers have good teachers.

• Most great golfers have had great teachers.

• Most golfers without instruction do not improve after the first year despite lots of practice & despite fairly immediate feedback of results.


• No.

• Here’s why in a nutshell: – The relationship between OUTCOME and

ACTION is too complex.

For example….For example….

• A putt may go left of target because: – The golfer misread the slope.– The golfer misread the grain.– A puff of wind knocked the ball off course.– The putter blade hit the ball off center.– The putter blade did not hit the ball normal to the path

of the ball.– The golfer’s swing path was not aimed where they

thought it was aimed.– The putter blade path was curved.– The ball was not clean. – The ball was malformed.

That is bad enough…That is bad enough…

• What does a golfer typically “learn” when they miss a putt to the left? Do they say:

“I wonder….whether I misread the slope, the grain, a puff of wind hit the ball, whether the ball was dirty or malformed or my putter path was non-linear, off-center or the putter path was non-normal to the ball path…what could it be?”

No, of course not. They say: No, of course not. They say:

• “Oh, I pulled it.” (Or some other unproven hypothesis that is subsequently treated as fact).

• Equally bad…or worse…the putt goes in the hole and the golfer concludes: “I hit it perfectly” when, in reality, some combination of errors may have resulted in a sunk putt.

Solution in golf?Solution in golf?

• Expert attribution of error (not the novice’s)

• Use of “toy” situations that allow separation of concerns; e.g.,– Dave Pelz (MIT physicist) developed:

– Laser + mirror on putter for aiming– Machines to allow multiple identical trials– Track to give feedback on putter path– Clip-on to give feedback on point of putter contact

What is the situation in HCI?What is the situation in HCI?

• Arguably…much like putting • Users can be confused for numerous reasons and only

have some ability to localize these to the actual sources.• A project may fail for numerous HCI reasons as well as

for political reasons, economic reasons, etc. • HCI is “worse” in some ways because an amateur golfer

may get to play many hundreds or thousands of rounds and feedback is usually seconds after action.

• An HCI designer in the “real world” may only work on a much smaller number of projects and much of the feedback may be months after decisions are made.

Another Example: DrawingAnother Example: Drawing

• People can flounder around for a lifetime and not learn to draw very well….

• In a course that teaches a few important principles, nearly anyone can learn to draw.

• During the course, instructor points out principles; gives much faster and more accurate feedback than student can achieve working alone; uses tools and techniques to separate concerns.

• (Developed by Betty Edwards, Drawing on the Right Side of the Brain.).

• Individual humans are typically not capable of learning these complex skills alone. (Same for collaboration?)

On very rare occasions: On very rare occasions:

• A few individuals seem to learn complex skills like drawing or golf (or HCI?) with very little structured education. Why?

• Innate talent? In some cases. Perhaps, more likely…

• In a huge pool of equally talented people, the vast majority will get “trapped” in local maxima; a small minority will make a few crucial choices to get to the “globally good” part of the terrain and incrementally improve.

Nits? Nits?

• I would mention that while cases are about activity in a context…they way they are typically used, the case presumes a separation into “context which is there” and “problem to be solved.”

• At some point, people need to learn to question the particular partition presumed by the case.

• The 10,000 person conference call.

Learning from Failure Learning from Failure

• In organizations and communities, it is useful to learn from errors.

• Yet, for social and political reasons, hard to uncover errors.

• It is possible to collect and use failure stories by applying various “transformations”

Techniques for morphing "Negative Stories”Techniques for morphing "Negative Stories”

• Anonymity• Projective spaces; e.g., British Navy Admiral cartoon

• "Trusted Source & registered anonymity"

• Re-framings: "I almost did X, but -- Deus Ex Machina -- hence, goodness"

• "I had intra-psychic conflict; almost did X, but thankfully did ~X; hence, goodness"

• "As an experiment, on a small scale, we did X and discovered badness; hence, ~X"

Half-baked ideas…Half-baked ideas…

• In instruction, it is critical to deal with motivational issues.

• Superstructures to help people “see” what is important in cases may extend beyond logical argument to the perceptual, play, dramatic, aspects.

• At some point in a curriculum, there may be a place for playing with the cases, not just using the cases.

Why do I self-down? Because I’m an idiot? Why do I self-down? Because I’m an idiot? Rational LivingRational Living

Riff on Tversky articleRiff on Tversky article

Over time…Over time…• Coaches begin by reinforcing unusually good

performance and

• Criticizing unusually bad performances.

• Because of “regression to the mean”, praise behavior is generally followed by poorer performance while criticizing behavior is generally followed by better performance.

• Eventually, coaches engage only in critical behavior…

• An individual may well do the same.

Among functions of an expert tutor:Among functions of an expert tutor:

• Focusing attention on what is relatively more important.

• Preventing student from prematurely settling on a "theory of the domain" that may be wrong or limited.

• Selection and organization of cases.

• Separation of concerns.

• Motivating that further progress is possible.

Informal questionnaireInformal questionnaire

• Preliminary work in the development of an e-learning system.

• Given in 2003 to 55 people; 18 responded with 36 incidents of “exceptionally good learning that they had experienced.”

• American adults, aged 30-70.• Motivational aspects were given as

important attributions in about half the cases.

Possible Discussion items…Possible Discussion items…• Some functions of an expert tutor:

• Focusing attention on what is relatively more important.

• Preventing student from prematurely settling on a "theory of the domain" that may be wrong or limited.

• Motivating that further progress is possible.

• Selection and organization of cases.

• Separation of concerns.

• Some questions:

• Can technology help with these tasks?

• Can an individual learn to perform these functions for themselves?

• Can social systems enable individuals to perform these functions for each other?

Excellent Learning and Excellent Learning and Teaching ExperiencesTeaching Experiences

A Highly Informal Survey and ResultsA Highly Informal Survey and Results

John ThomasJohn Thomas

February 21, 2003February 21, 2003

Respondents: Respondents:

• Adults age 30-70

• Americans; college grads

• Eighteen people responded with 36 incidents or experiences

Example:Example:

• "Stop trying to hit the ball. Forget the ball. You're not here to hit the ball. You are here only to execute a beautiful swing. If you happen to be standing near the ball and it gets in the way and rockets down the fairway, so be it."

Example:Example:

• I was trying to explain to someone about a "mole" in chemistry. They kept thinking it was like a measure of weight or volume. With some exasperation, I finally said, "NO! It's like eggs. They come in a dozen. It's just like a dozen...only there are a lot more than a dozen in a mole. Avagadro's number to be exact." And, they got it.

““Venues” of ExperiencesVenues” of Experiences

• Ten were “informal learning”

• Ten were “adult education”

• Twelve were “public education”

Subject Areas: DiverseSubject Areas: Diverse

• Motor Learning• Artistic and Musical• Science• Mathematics• Cooking• Computing• Political Science• Brazilian History

Agent of Learning: Agent of Learning:

• Formal Teacher: 21 cases

• Spouse or Friend: 5 cases

• Peer learning mentioned: 6 cases

Attribution of ExcellenceAttribution of Excellence

• Teacher as a person: 5• Passion of the teacher: 3• Use of specific media: 8• Small input large change: 7• Change or refocus perception: 7• Principles and Practices Interplay: 3• Teacher showing confidence in students: 3• Length and Breadth of the Utility of the subject: 2• Motivation (overcoming fear or increasing importance): 4• Relating the exotic to something familiar: 3

Attribution of Excellence (cont).Attribution of Excellence (cont).

• Use of tools: 1• Using “Real” work: 1• Demanding synthesis: 2• Intensity: 1• Use of Imagery: 1• Imitation: 2• Changing feedback: 2• Coolness of subject matter: 1• Building carefully and slowly: 1

Elicit from DiversityElicit from Diversity• Pp. 7-8: “AND HOW MANY MIGHT DO WHAT FEW-

ALONE COULD NOT EVEN THOUGH EACH OF THE MANY HAS LESS STRENGTH.” (Bears)

• Pp. 11-12: “WHAT IS IMPOSSIBLE FOR ONE MAY BE POSSIBLE FOR MANY” (Multi-colored paintings)

• P 44. “IF THERE IS NOT ONE AMONG US WHO CONTAINS SUFFICIENT WISDOM MANY PEOPLE TOGETHER MAY FIND A CLEAR PATH.” (Council)

• P. 65: “WHAT IS IMPOSSIBLE FOR ONE MAY BE POSSIBLE FOR MANY” (Ropes)

““WHAT IS IMPOSSIBLE FOR ONE WHAT IS IMPOSSIBLE FOR ONE MAY BE POSSIBLE FOR MANY” MAY BE POSSIBLE FOR MANY”

• THINKING OF THIS,• They wove ropes• which were long as well as thick• and with which those who were struck by Ocean• and washed from their footing• might be restrained by others• who were more secure.

Iroquois “Rule of Six”Iroquois “Rule of Six”

• You are in a meeting room. Your calendar says the meeting is supposed to start at 10 am. The clock on the wall says 10:15. John is not here yet. You think: “John doesn’t really care about this project.”

• According to the “Rule of Six” you need to generate five additional explanations for the current situation.

Iroquois “Rule of Six”Iroquois “Rule of Six”

• Your calendar entry is wrong

• The clock on the wall is wrong

• John comes from a culture where 15 minutes is not “late”

• John was unavoidably delayed in traffic

• John was waylaid by the Vice-President and even now is talking up the project

Seek to Understand the Heart of OthersSeek to Understand the Heart of Others

• The Iroquois reflect on how giant tree sloths became extinct and how even now bear and deer are more difficult to find; they decide to understand more about how their four-footed brothers live and ensure the world is arranged for their prosperity.

• Later, when confronted with a war-like tribe with superior weaponry, they see that this other tribe, unlike the Iroquois, has a strict division of labor between men and women. The Iroquois use this, first to learn the arts of war and then, when battle comes, to “freak out” their opponents by sending five women to fight their braves.

Small Successes EarlySmall Successes EarlyBased on the story of Old Grandfather who invented Clothing

Small Successes EarlySmall Successes Early

• “Take off your skins, Carry them if you must…

Proceed in this unvaried manner

for many days –

until I make a sign to you.

“IN THIS MANNER OVER SOME DAYS

I hope to change the thinking of the People

leading them as I might lead them

over some great mountain

with slow and careful steps.”

Her voice was neither loud nor soft

as she asked her questions.

It was such that those nearby could easily hear,

yet those at some great distance

could easily turn away and hear no more.

Small Successes EarlySmall Successes Early

• Her father never more than answering any specific question, those among the People coming and going, so that curiosity or discomfort with the new were equally accommodated.

Gradually the People learned – each at their own pace – the nature of the path of one who walks North…

and yet returns.

“Old Father,” she began –“I was wondering whether you have any use for that softest skin…”

Reality CheckReality Check

Who Speaks for Wolf?Who Speaks for Wolf?Visual by www.PDIimages.comVisual by www.PDIimages.com

Application to “Dynamic Learning Environment”Application to “Dynamic Learning Environment”

• Circle and Epicircle Search– Focus on developing workable system– Peripheral search: joined MERLOT; read about e-learning; participated

in panels; constant inquiry of social network• Elicit from Diversity

– Team make-up and summer students• Small Successes Early

– Three iterations of “PowerPoint Prototypes”– Two successive field deployments

• Reality Check– Ersatz measures: questionnaires gave promising results. People liked

system and found it easy to use – Experiment to determine whether it really worked for learning

• Who Speaks for Wolf?– Early interviews with curriculum developers; course developers; content

providers; potential users; potential “catchers” in IBM; used earlier comparative study for initial requirements

““Systems Thinking” proposal to NYNEX CEOSystems Thinking” proposal to NYNEX CEO

• Understanding the heart of others– Need to understand what would appeal to the

CEO– Also needed to understand nature of my own

management: infinite regress of permission– But also a “rebel at heart”

Further ReadingFurther Reading• Alexander, C. A., Ishikawa, S., Silverstein, M., Jacobson, M, Fiksdahl-King, I. and Angel, S. A

pattern language. New York: Oxford Press, 1977.• Gamma, E., Helm, R., Johnson, R. and Vlissides, J. Design patterns: Elements of reusable

object oriented software. Reading, MA: Addison-Wesley, 1995.• Johansen, B. E. Forgotten founders: How the American Indian helped shape democracy.

Cambridge, MA: Harvard Common Press, 1987.• Johansen, B. E. Debating democracy: Native American legacy of freedom. Sante Fe: Clear

Light, 1998.• McKee, R. Story: Substance, structure, style and the principles of screenwriting. New York:

Harper & Row, 1997.• Thomas, J. Narrative technology and the new millennium, Knowledge Management Journal

(2), 14-17, 1999.• Thomas, J. Kellogg, W. A., & Erickson, T. A. The knowledge management puzzle: human and

social factors in knowledge management, IBM Systems Journal, 40(4), 863-884, 2001.• Underwood, P. Who speaks for Wolf: A Native American learning story. San Anselmo, CA:

Tribe of Two Press, 1983. • Underwood, P. Three strands in the braid: A guide for learning enablers. San Anselmo, CA:

Tribe of Two Press, 1994.• Underwood, P. The walking people. San Anselmo, Ca: Tribe of Two Press, 1993.

real life experience as a pedagogical principle

Technology