Paradoxical Effects of Reward

• Overtraining extinction effect: more training leads to faster extinction

• Reinforcement magnitude effect: Big rewards lead to faster extinction

• And, of course, the partial reinforcement extinction effect (PREE)

Paradoxical effects of reward: Why?

• Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF.

Discrimination Hypothesis: Test

CRF CRF EXT

vs.

PRF CRF EXT

Paradoxical effects of reward: Why?

• Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF.

• Frustration hypothesis (Amsel): animals learn to make response as a reaction to nonreward.

• Sequential theory (Capaldi): The memory of nonreinforcement becomes a cue that elicits responding.

Stimulus Control

Stimulus Control of Behavior

• Having stimulus control means that the probability of the behavior varies depending upon the stimuli present

• Most of our behavior is under stimulus control– A person that contributes to charity generously while in

church may watch every penny spent while at work

Discrimination

0

0.2

0.4

0.6

0.8

1

1.2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Trials

Re

sp

on

se

Str

en

gth

CS+

CS-

Discrimination and Stimulus Control• Discrimination is demonstrated when differential

responding occurs to two or more stimuli.

Train

Test

Reynolds (1961)

Generalization

• Generalization is when responses to one stimulus occur to other, usually similar, stimuli

• Generally, as the training and test stimuli become more different responding will decline, producing what is called a generalization gradient

Generalization GradientGuttman & Kalish (1956)

0

50

100

150

200

250

300

350

510

530

550

570

590

610

630

Wavelength (nanometers)

Res

pons

es

– pigeons reinforced for pecking a 580 nm lit key (orange-yellow) (S+) on a VI schedule

– A test session was then given where many different colored key lights were presented in extinction

S+

Interpreting Generalization Gradients

Pigeons trained to peck a moderately bright light (S+) to get food.

(S- = dim light)

After asymptote is reached, present occasional non-reinforced probe trials at various wavelengths or levels of brightness.

Excitatory andinhibitory gradients

Pigeons trained to peck at a 800 hz tone (S+), with a 500 nm light S-.

1000 Hz Tone always on

1000 Hz Tone S+ / No Tone S-

1000 Hz Tone S+ / 950 Hz Tone S-

Control group: 550 nm Light S+Experimental group: 550 nm Light S+ / 590 nm Light S-

0

100

200

300

400

500

480 500 520 540 560 580 600 620

Wavelength (nanometers)

Res

pons

es

ControlExperimental

Peak Shift Effect – Hanson (1959)

Spence’s Theory to Account for Peak Shift

-60

-40

-20

0

20

40

60

80

490 510 530 550 570 590 610 630 650 670Wavelength (nanometers)

Inh

ibit

ory

or E

xcit

ator

y S

tren

gth Inhibitory

ExcitatoryDifference

S+

S-

Interdimensional discrimination

0

100

200

300

400

500

600

700

501 530 555 576 606

Wavelength

# R

espo

nses

PseudodiscrimDiscrimination

Discrimination: S+ = 555nm Light; S- = Tone

How do we learn discriminations with complex stimuli?

How do we learn discriminations with complex stimuli?

A BA+B

Complex Discrimination: Example

Pre-exposure Devalue Test

-- Saline-LemonLiCL Sucrose-Lemon?

Lemon Saline-LemonLiCL Sucrose-Lemon?

Another example…

Pre-exposure Devalue Test

-- Saline-LemonLiCL Sucrose-Lemon?

Sal-L/Suc-L Saline-LemonLiCL Sucrose-Lemon?

Complex Discriminations: Mechanism # 2

the method of pre-exposure matters…

Mondragon & Hall (2002)

A = lemon B = salt C = sucrose X = quinine

Pre-Exposure: AXBXAXBX | CXCXCXCX

Devalue: AXLiCL

Test: BX? CX?

Question: How much does aversion generalize to BX and CX?

What’s going on?

• Juxtaposition of stimuli clearly matters

• But why? AXAXAX… produces habituation to AX

AXBXAXBX….

A

Remember: expected things are less salient or associable

B

Treating Different Stimuli Alike: Categorization

“Categorization can be viewed as the ability to treat similar, but not identical, things as somehow equivalent, by sorting them into their proper categories and by reacting to them in the same manner” (Huber, 2001)

• Classical view: categories united by a defining feature or features

• But Consider: Oak leaves v. Non-oak leavesChairs v. non chairs

What is “Chairness”

“family resemblance”

Categorization Experiments

Train Test

Scenes with Trees + New Set tree scenese

Scenes w/o trees - New Set of no-tree scenes

"A pigeon pecks rapidly at a small photograph of Harvard Yard containing trees, buildings, people, sky. After a few seconds, a hopper of grain appears and the pigeon eats. Now the scene changes to a treeless Manhattan street. The bird emits a few desultory pecks, then turns away and paces about. After a minute or so, a picture of a leafy suburban garden appears and the bird begins pecking again." (Shettleworth 1998)

Other categories pigeons can form•Aerial v. non-aerial photos

•Chairs

•Humans

•Cars

•Defective pharmaceutical capsules!

•Oak leaves versus other leaves

Human v. Non-Human

How do they do it?

• Exemplar theory: remember category members and then generalize.– Vaughn & Greene 1984: pigeons can remember

no less than 320 individual slides! Outdoor scenes randomly assigned to + or –

Exemplar theory: more evidence

• Cook (1990)– Birds versus Mammals used in slides

– Real Category Group: Birds v. Mammals

– Pseudocategory Group: Random Bird & Mammals versus Random Birds & Mammals

Feature Theory

• Individual features acquire associative value.

• Response rate to stimulus depends on total expectancy (V) evoked.

Feature Theory: Evidence

Cerella (1980): Train: Charlie Brown +, other characters –

Test: Keep all features intact, but alter whole

Prototype theory

• Abstract the “ideal” (or average) category exemplar.

• To test: train with only extreme exemplars, test with average of extremes.

Prototype Theory in Humans

Posner & Keele 1968

Conclusions:

• Not clear whether birds can extract abstract concepts in categorization experiments

• Birds may use features and exemplars

• Another animals may be capable of more complex feats.