Chapter 5Chapter 5

LearningLearning

What is Learning?What is Learning?

Learning: Learning: experience leads to a experience leads to a relatively permanent change in relatively permanent change in behaviorbehavior

Conditioning: Conditioning: a behavior becomes a behavior becomes associated with a stimulusassociated with a stimulus

Stimulus: Stimulus: anything that influences anything that influences behaviorbehavior

Classical ConditioningClassical Conditioning

Classical (Pavlovian) Conditioning:Classical (Pavlovian) Conditioning: a response a response (salivation) (salivation) naturally naturally elicited by one stimulus elicited by one stimulus (food) (food) comes to be elicited by a different comes to be elicited by a different stimulus stimulus (bell)(bell)

Ivan Pavlov: Ivan Pavlov: Russian physiologist, Russian physiologist, discovered classical conditioningdiscovered classical conditioning

4 Classical Conditioning Terms4 Classical Conditioning Terms

1. unconditioned stimulus (US):1. unconditioned stimulus (US): stimulus that naturally (stimulus that naturally (reflexivelyreflexively) ) “elicits” a response“elicits” a response– foodfood

2. unconditioned response (UR):2. unconditioned response (UR): response (response (reflexivelyreflexively) elicited by ) elicited by the unconditioned stimulusthe unconditioned stimulus– salivationsalivation

4 Classical Conditioning Terms4 Classical Conditioning Terms

3. conditioned stimulus (CS):3. conditioned stimulus (CS): through through pairing with an US (food) the CS pairing with an US (food) the CS (bell) comes to elicit the same (bell) comes to elicit the same response (salivation) response (salivation)

4. conditioned response (CR): 4. conditioned response (CR): same same as as the unconditioned response the unconditioned response (salivation) (salivation) BUTBUT is elicited by the CS is elicited by the CS (bell) NOT the US (food)(bell) NOT the US (food)

Classical Conditioning ProcedureClassical Conditioning ProcedureBefore ConditioningBefore Conditioning

Food (US)

Salivation (UR)

Bell(CS)

No Salivation (CR)

Classical Conditioning ProcedureClassical Conditioning ProcedureDuring ConditioningDuring Conditioning

Bell (CS)

Food (US)

Salivation (UR)

Classical Conditioning ProcedureClassical Conditioning Procedure

After ConditioningAfter Conditioning

Bell (CS)

Salivation (CR)

J.B. Watson’s Little Albert StudyJ.B. Watson’s Little Albert Study

By “pairing” a loud noise with a white By “pairing” a loud noise with a white rat (cute), the white rat became a CS rat (cute), the white rat became a CS for fear in little Albertfor fear in little Albert

What is US? > Loud NoiseWhat is US? > Loud NoiseWhat is UR? > CryingWhat is UR? > CryingWhat is CS? > White RatWhat is CS? > White RatWhat is CR? > CryingWhat is CR? > Crying

Factors Affecting ConditioningFactors Affecting Conditioning(not in book)(not in book)

Order of presentation: Order of presentation: Conditioning is Conditioning is much more effectivemuch more effective if CS precedes if CS precedes US US (e.g., first the white rat, then the noise)(e.g., first the white rat, then the noise)

Inter-Stimulus Interval (ISI): Inter-Stimulus Interval (ISI): about 1/2 about 1/2 second to a few seconds is bestsecond to a few seconds is best

Numer of CS - US pairings: Numer of CS - US pairings: usually usually many are neededmany are needed (except in conditioned taste (except in conditioned taste aversion!)aversion!)

Biological Preparedness HypothesisBiological Preparedness Hypothesis

Martin Seligman: Martin Seligman: evolution has evolution has made us more likely to become made us more likely to become conditioned to stimuli that are conditioned to stimuli that are “potentially dangerous”“potentially dangerous”– heights, thunder, animals, water, fire, heights, thunder, animals, water, fire,

people, insects, etc. people, insects, etc.

Conditioned Taste AversionConditioned Taste Aversion

An exception to the need for An exception to the need for multiple CS - US pairingsmultiple CS - US pairings

One trial learning: One trial learning: animals (and animals (and sometimes humans) will learn to sometimes humans) will learn to avoid a taste/smell that has been avoid a taste/smell that has been associated with sickness ONE time. associated with sickness ONE time. This has “survival value” This has “survival value”

Operant ConditioningOperant ConditioningOperant conditioning:Operant conditioning: organism organism

““operatesoperates” on the environment to ” on the environment to cause the occurrence (or non-cause the occurrence (or non-ocurrence) of some event (ocurrence) of some event (an if - an if - then statementthen statement))

Instrumental Conditioning: Instrumental Conditioning: another another name for operant conditioning name for operant conditioning (organism is “(organism is “instrumentalinstrumental” in its ” in its own learningown learning

Edward L. ThorndikeEdward L. Thorndike

used his “used his “puzzle boxpuzzle box” with cats to ” with cats to study operant type behaviorstudy operant type behavior

Thorndike’s Law of EffectThorndike’s Law of Effect

Behavior that leads to a pleasant Behavior that leads to a pleasant outcome is “outcome is “stamped instamped in””– e.g., cat hits lever and escapes boxe.g., cat hits lever and escapes box

Behavior that leads to an Behavior that leads to an unpleasant outcome is “unpleasant outcome is “stamped stamped outout””– e.g., cat hits lever and gets shockede.g., cat hits lever and gets shocked

B. F. SkinnerB. F. SkinnerGreatly expanded on Thorndike’s Greatly expanded on Thorndike’s

ideasideas Invented the “Invented the “Skinner BoxSkinner Box””

– A box with a lever (for a rat to press) or a disk A box with a lever (for a rat to press) or a disk (for a pigeon to peck(for a pigeon to peck

– a system for delivering fooda system for delivering food– a metal floor that could deliver a mild shocka metal floor that could deliver a mild shock

Believed all behaviors, thoughts, Believed all behaviors, thoughts, words, were “learned” and could be words, were “learned” and could be studies and, perhaps, changedstudies and, perhaps, changed

Language of Operant ConditioningLanguage of Operant Conditioning reinforcement:reinforcement: following a behavior following a behavior

with a consequence (event) that with a consequence (event) that INCREASES the probability that the INCREASES the probability that the behavior will be repeatedbehavior will be repeated

Positive: Positive: something pleasant is added something pleasant is added to the situation (e.g., candy)to the situation (e.g., candy)

Negative: Negative: something unpleasant is something unpleasant is removed from the situation (e.g., no removed from the situation (e.g., no homework)homework)

Language of Operant ConditioningLanguage of Operant Conditioning

punishment:punishment: following a behavior following a behavior with a consequence (event) that with a consequence (event) that DECREASES the probability that DECREASES the probability that the behavior will be repeatedthe behavior will be repeated

Positive: Positive: something is added to the something is added to the situation (e.g., a shock)situation (e.g., a shock)

Negative: Negative: something is removed from something is removed from the situation (e.g., fine for speeding)the situation (e.g., fine for speeding)

Four Operant ProceduresFour Operant Procedures1. Positive Reinforcement: 1. Positive Reinforcement: behavior behavior

results in something pleasant being results in something pleasant being added (behavior increases)added (behavior increases)– rat presses lever > rat gets foodrat presses lever > rat gets food

2. Negative Reinforcement: 2. Negative Reinforcement: behavior behavior results in something unpleasant being results in something unpleasant being removed (behavior increases)removed (behavior increases)– rat presses lever > bigger aggressive rat is removedrat presses lever > bigger aggressive rat is removed

Four Operant ProceduresFour Operant Procedures

3. Positive Punishment: 3. Positive Punishment: behavior behavior results in something unpleasant results in something unpleasant being added (behavior decreases)being added (behavior decreases)– rat presses lever > rat gets shockedrat presses lever > rat gets shocked

4. Negative Punishment: 4. Negative Punishment: behavior behavior results in something pleasant being results in something pleasant being removed (behavior decreases)removed (behavior decreases)– rat presses lever > rat’s food gets taken awayrat presses lever > rat’s food gets taken away

Superstitious BehaviorSuperstitious Behavior

Discovered by SkinnerDiscovered by SkinnerOrganism learns “incorrectly” that a Organism learns “incorrectly” that a

behavior produces an outcomebehavior produces an outcomeExample:Example:

– pigeon is turning as food pellet dropspigeon is turning as food pellet drops– pigeon “assumes” turning will bring pigeon “assumes” turning will bring

foodfood

Learned HelplessnessLearned Helplessness

Learned helplessness: Learned helplessness: may be one may be one cause of human depressioncause of human depression

Organism learns it has no control Organism learns it has no control over situation (environment)over situation (environment)

Martin Seligman: Martin Seligman: demonstrated this demonstrated this using dogs and shockusing dogs and shock

Learned Helplessness StudyLearned Helplessness Study

Seligman’s Learned Helplessness StudySeligman’s Learned Helplessness Study

Two groups of dogs are exposed to Two groups of dogs are exposed to shockshock– control group could escape shockcontrol group could escape shock– ““no escape” group could NOT escape no escape” group could NOT escape

shockshock

Later, when escape was possible, Later, when escape was possible, “no escape” dogs didn’t even try“no escape” dogs didn’t even try

Learned that they had Learned that they had NO CONTROLNO CONTROL

Shaping Shaping

shaping:shaping: complex behaviors are complex behaviors are learned in small steps (“reinforcing learned in small steps (“reinforcing successive approximations to a successive approximations to a goal”)goal”)

any complex behavior (e.g., dog any complex behavior (e.g., dog fetching a stick) is learned in small fetching a stick) is learned in small stepssteps

Response AcquisitionResponse Acquisition

Learning is slow at first, becomes Learning is slow at first, becomes more rapid, then levels offmore rapid, then levels off

0 5 10 15 20 25 Trials

CR

ExtinctionExtinctionRepeated pRepeated presentation of CS without resentation of CS without

US weakens, then eliminates the CRUS weakens, then eliminates the CR

0 5 10 15 20 25 Trials

CR

NO US (food)

Spontaneous RecoverySpontaneous Recovery After extinction, and then a period of rest, the After extinction, and then a period of rest, the

CR returns without any additional conditioningCR returns without any additional conditioning

0 5 10 15 20 25 Trials

CR

CR reappears

Generalization - DiscriminationGeneralization - Discrimination

stimulus generalization:stimulus generalization: response is response is made to original AND “similar” CSmade to original AND “similar” CS– Little AlbertLittle Albert

stimulus discrimination:stimulus discrimination: organism organism learns to respond to original CS but learns to respond to original CS but NOT to similar onesNOT to similar ones– pigeon learns to peck at red light but pigeon learns to peck at red light but

not at green lightnot at green light

Higher Order ConditioningHigher Order Conditioning

Can a CS be used as if it were a Can a CS be used as if it were a US to condition a new second CS?US to condition a new second CS?

YESYESA CS (bell) is used as a US to A CS (bell) is used as a US to

condition a new CS (light) to elicit condition a new CS (light) to elicit the same CR (salivation)the same CR (salivation)

HIgher Order ConditioningHIgher Order ConditioningBefore Higher Order ConditioningBefore Higher Order Conditioning

Bell (CS1)

Salivation (UR)

Light(CS2)

No Response

Higher Order ConditioningHigher Order ConditioningDuring HIgher Order ConditioningDuring HIgher Order Conditioning

Light (CS2)

Bell (CS1)

Salivation (UR)

Higher Order ConditioningHigher Order Conditioning

After Higher Order ConditioningAfter Higher Order Conditioning

Light (CS2)

Salivation (CR)

Primary and SecondaryReinforcers

primary reinforcers: naturally reinforcing

food, water, sexsecondary reinforcers: become

reinforcing through association with primary reinforcers

for humans, MONEY is the most potent secondary reinforcer

The Blocking Effect & ContingenciesThe Blocking Effect & ContingenciesA problem for Pavlov:A problem for Pavlov:

– Pavlov - Thought all that is needed for Pavlov - Thought all that is needed for conditioning is for a CS and US to be conditioning is for a CS and US to be presented togetherpresented together

Leon Kamin: Leon Kamin: showed that it was not showed that it was not this simplethis simple– prior conditioning of a tone with shock prior conditioning of a tone with shock

prevented rats from later being able prevented rats from later being able to associate a light with the shockto associate a light with the shock

Original Fear ConditioningOriginal Fear Conditioning

Tone (CS)

Shock (US)

Fear (UR)

Tone is paired with a shock

After Original Fear ConditioningAfter Original Fear Conditioning

Tone signals shock and elicits fearTone signals shock and elicits fear

Tone (CS)

Fear (CR)

Try to Condition LightTry to Condition Light

Light is presented WITH tone AND Light is presented WITH tone AND shockshock

Tone (CS)

Fear (UR/CR)

Light (CS)

Shock (US)

IMPORTANT! In contrast to “higher order conditioning,” the US “shock” is still present

Light has NOT ConditionedLight has NOT Conditioned

Light - shock association has Light - shock association has NOTNOT been been learnedlearned

Light does not tell the rat anything it Light does not tell the rat anything it doesn’t already know, rat ignores lightdoesn’t already know, rat ignores light

Prior learning of the tone-shock Prior learning of the tone-shock relationship “BLOCKED” learning of the relationship “BLOCKED” learning of the light-shock relationshiplight-shock relationship

Light (CS)

NO Fear (CR)

Contingency TheoryContingency Theory

Explains the blocking effectExplains the blocking effectFor conditioning to occur, the CS For conditioning to occur, the CS

must provide “must provide “useful information” useful information” about the US to the organismabout the US to the organism

Schedules of ReinforcementSchedules of ReinforcementContinuous Reinforcement:Continuous Reinforcement: each each

response is reinforcedresponse is reinforcedpartial (intermittent) Reinforcement:partial (intermittent) Reinforcement:

reinforcement does not follow reinforcement does not follow every responseevery response– ratio schedules: several responses are ratio schedules: several responses are

required before reinforcementrequired before reinforcement– interval schedules: a certain amount of interval schedules: a certain amount of

time must pass before next reinforcementtime must pass before next reinforcement

Fixed RatioFixed Ratio

FR-10: FR-10: a set number of bar presses (10) must be performed to get food

rat presses 10 times, gets food, rests, starts again– produces a “stepped” response recordproduces a “stepped” response record– ex. being paid for “piece work”ex. being paid for “piece work”

Variable RatioVariable Ratio

VR-10: VR-10: number of bar presses needed for food will vary but will AVERAGE a certain number (10)

Rat never knows when food is coming so he keeps pressing– produces a steady steep response produces a steady steep response

recordrecord– ex. Playing a slot machineex. Playing a slot machine– **** “Highest” rate of responding ******** “Highest” rate of responding ****

Fixed IntervalFixed Interval

FI-10: FI-10: food is available every 10 seconds IF bar is being pressed– rat senses time to as 10 sec. Mark rat senses time to as 10 sec. Mark

approaches, he presses more quicklyapproaches, he presses more quickly– produces a “scallop shaped” recordproduces a “scallop shaped” record– ex. more frequent trips to the employment ex. more frequent trips to the employment

office when more jobs are availableoffice when more jobs are available

Variable IntervalVariable Interval

VI-10: VI-10: food is available ABOUT every 10 seconds but rat must be pressing bar as the time approaches– steady low rate response recordsteady low rate response record– ex. studying for a teacher who gives ex. studying for a teacher who gives

“pop” quizzes“pop” quizzes

Cumulative records of a rat bar-pressing in a Skinner Cumulative records of a rat bar-pressing in a Skinner Box. The lines are made up of dots with each dot Box. The lines are made up of dots with each dot

(moving upward) representing a bar press.(moving upward) representing a bar press.If you want a lot of work from a rat or a person, use a If you want a lot of work from a rat or a person, use a

variable ratiovariable ratio schedule!, schedule!,

Latent LearningLatent Learning

Latent Learning: Latent Learning: learning that is not learning that is not apparent in behaviorapparent in behavior

Demonstrated in an experiment by Demonstrated in an experiment by “Tolman and Honzik”“Tolman and Honzik”

Problem for Skinner’s idea that Problem for Skinner’s idea that learning requires “reinforcement” or learning requires “reinforcement” or “punishment”“punishment”

Tolman and Honzik’s StudyTolman and Honzik’s Study rats (3 groups) learned a maze over about 2 rats (3 groups) learned a maze over about 2

weeksweeks Group A:Group A:

– no food at end, their speed through the no food at end, their speed through the maze did not increase muchat allmaze did not increase muchat all

Group B: Group B: – food at end daily, they ran the maze a little food at end daily, they ran the maze a little

faster each dayfaster each day Group C: (actually a sub-group of A)Group C: (actually a sub-group of A)

– no food till day 11, then they immediately no food till day 11, then they immediately ran the maze as fast as group Bran the maze as fast as group B

Influence of Tolman’s Study Influence of Tolman’s Study

It showed that learning It showed that learning CANCAN occur occur without “reinforcement”without “reinforcement”

Group C rats were learning about the Group C rats were learning about the maze but didn’t show it until given a maze but didn’t show it until given a good reason (food at end of maze)good reason (food at end of maze)

Learning may be naturally reinforcingLearning may be naturally reinforcing So, there is more to operant learning So, there is more to operant learning

than Skinner thoughtthan Skinner thought

© Prentice Hall, 1999

Albert Bandura’s Social Learning TheoryAlbert Bandura’s Social Learning Theory

We can learn by We can learn by watchingwatching Observational (vicarious) learningObservational (vicarious) learning - -

We observe the behaviors of others We observe the behaviors of others and the consequences of those and the consequences of those behaviors.behaviors.

Vicarious reinforcementVicarious reinforcement - If their - If their behaviors are reinforced we tend to behaviors are reinforced we tend to imitate the behaviorsimitate the behaviors

Vicarious punishmentVicarious punishment - If their - If their behaviors are punished we tend NOT behaviors are punished we tend NOT to imitate the behaviorsto imitate the behaviors

• the “Bobo Doll Study” - Bandura’s classic experiment demonstrating observational learning

• two groups of children watched an adult get either rewarded or punished for behaving aggressively with a doll

• children who saw the adult rewarded were later more likely to be aggressive when placed in the same situation

Social Learning Theory (cont.)Social Learning Theory (cont.)

Learning and Human BehaviorLearning and Human Behavior

Classical ConditioningClassical Conditioning

Explains how some phobias Explains how some phobias developdevelop– dog is not feareddog is not feared– dog bites persondog bites person– dog becomes CS for feardog becomes CS for fear

Two Factor Theory of AvoidanceTwo Factor Theory of Avoidance O. H. Mowrer (not in book)O. H. Mowrer (not in book)

– the two “factors” are the two “factors” are classicalclassical and and operantoperant conditioning conditioning

1. person learns to fear a dog via classical 1. person learns to fear a dog via classical conditioningconditioning

2. each time dog is avoided, avoidance 2. each time dog is avoided, avoidance behavior is “reinforced” by?behavior is “reinforced” by?

negative reinforcementnegative reinforcement – avoidance (removal) of dog lowers anxiety so avoidance (removal) of dog lowers anxiety so

behavior is repeatedbehavior is repeated