conditional learning

Conditional learning

Charlotte Bonardi

C82NAB Neuroscience and Behaviour

A red light means....

newspaper

newspaper

journal

periódico

shinbun

The same stimulus can be associated with two different outcomes.Which association is retrieved is conditional on the context in which the stimulus is presented.

This allows the associations to represent knowledge in a versatile way.

The context appears to control access to the CS-->US association.

Conditional control of associations

english

spanish “periódico”

“newspaper”

Standard associative theories can't explain this -- the association forms and that’s it.

... maybe associative theory cannot explain all animal learning?

Questions:

Is conditional control really independent of associative learning?

If so, how does it work?

Conditional control of associations

Feature-positive discrimination:

light+tone ---> food tone --> nothing

Conditional learning in animals



Associative theory predicts that the light is strongly associated with food ... and that the tone is not

On reinforced trials the light and the tone both acquire some strength; on nonreinforced trials the tone loses it again.

As there is only limited associative strength available, eventually the light gets it all, and the tone is left with nothing.




Associative theory predicts that the light is strongly associated with food ... and that the tone is not

On reinforced trials the light and the tone both acquire some strength; on nonreinforced trials the tone loses it again.

As there is only limited associative strength available, eventually the light gets it all, and the tone is left with nothing.

But how do you know this is what is happening?


Ross and Holland 1981

Used the fact that auditory and visual stimuli elicit different behaviours in the rat:

auditory: "headjerk" visual: "rearing"

They examined responding in two groups of animals: 1 light + tone --> food tone --> nothing

2 light + tone --> food light --> nothing

Ross and Holland 1981

Used the fact that auditory and visual stimuli elicit different behaviours in the rat:

auditory: "headjerk" visual: "rearing"

They examined responding in two groups of animals: 1 light* + tone --> food tone --> nothing

2 light + tone*--> food light --> nothing

The rats learned exactly according to the Rescorla Wagner model:

light* + tone --> food tone --> nothing rearing to light

light + tone* --> food light --> nothing

headjerking to tone

But if you don't have the light and the tone simultaneous something else happens...

Ross and Holland, 1981

simultaneous


simultaneous

serial


simultaneous

serial

rear


simultaneous

serial

rear

headjerk headjerk

In serial case hard for light to associate with food – so tone can

Headjerk response suggests responding is based on the tonefoood association.... and it seems to be stronger after the light!

tone food

light

so is the light acting as a switch???

headjerk headjerk

tone food

light

Although the light is not closely followed by food, it is always followed by food delivery

Maybe the rats headjerk more to the tonewhen the light has been on because theyare already expecting food, and this boosts the headjerk CR elicited by the tone.

...or is it just associated with food too??

headjerk headjerk

grub up!

tone food

lightWe can test this by extinguishingthe light-->food association.

Will the rats headjerk to the tone whenit is followed by the light?

If they do, we cannot explain the results in terms of associative learning.

Holland, 1989

Two groups of animals:

Group FP: Lighttonefood, tone no food

Group PP: Lighttonefood, tone no food...

Holland, 1989



Group PP: Lighttonefood, tone no food, light no food

Holland, 1989



Group PP: Lighttonefood, tone no food, light no food

If the light-->food association is responsible for discrimination in Group FP, then Group PP should not show any discrimination.

This type of discrimination is called a positive patterning (PP) discrimination.

The light is still controlling responding to the tone, despite the fact that it’s not associated with anything.

In these cases standard associative theory cannot explain how thelight is controlling responding. The light may be called a conditional cue, a modulator, or an occasion setter.

Can get negative occasion setting too – turn OFF an association

SIM light & tone nothing tone shock noise shock

SER lighttone nothing tone shock noise shock

A normal Pavlovian inhibitor will suppress responding to the noise – a summation test

will this also happen in the serial case?

Holland & Lamarre, 1984

tone shock

light

so is the light acting as a negative switch???

tone food

lightor is the light just a Pavlovian inhibitor?

if you can explain the discrimination in terms of regular classically conditioned associations, associative theory can explain it – don’t need any other theory

but if associative theory can’t explain result, need something else --then light is an occasion setter

Group SIM Group SER0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

noisenoise+light

supp

ress

ion

ratio

tone food

light

Rescorla’s modulation theory (Rescorla, 1985)

Whenever a CS is presented, it must activate the US representation to get a conditioned response.

If the light is a positive occasion setter, it lowers its activation threshold -- making it easier for the CS to activate.

tone food

light

Holland’s and-gate theory (1983)

The light acts as an and-gate, allowing activation to flow from the CS to the US, and so elicit a conditioned response.

tone

clickThese theories make differentpredictions about transfer

Suppose you also pair a clicker with food; would you expect thelight to elevate responding to theclicker as well?

food

food

light

click

Rescorla’s theory says YES.

The light is altering theactivation threshold of the food, and will boost responding to any stimulus that is associated with it.

tone

light

click

Holland’s theory says NO

The light is acting as an and-gate forthe tone-->food association

-- not the click-->food association

foodtone

Hundreds of experiments later...

(i) Occasion setters do sometimes transfer their effects to other CSs paired with the same US

(ii) But this transfer effect is seldom as big as with the original CS

Many people have done this experiment (e.g., Holland, 1986; Rescorla, 1985). Here is one example, using rats:

TRAIN

light.....tone-->food tone--> no food

click-->food

light.....tone-->food tone--> no food

click-->food

TESTlight.....tone? tone?

light.....click? click?

TRAIN

Holland might say...

the animals could confuse (“generalise between”) the two CSs (the tone and the click). This might give a false and weak) transfer effect

tone food

light

food

click

light

light

Rescorla might say...

animals are “upset” by experiencing novel CS combinations (such as the click and the light) -- this could disrupt responding, and transfer to a different CS.

food

light

click

foodtone

So get some transfer to the click (supporting Rescorla) but less effect than with the tone (supporting Holland)

results are not conclusive...

a complication!

(iii) Transfer is better if the transfer CS has its own occasion setter

tone food

light

click

dark

a complication!

(iii) Transfer is better if the transfer CS has its own occasion setter

tone food

light

click

light.....tone-->food, tone--> no food

dark....click-->food click-->no food



light.....tone? tone?

dark...click?click?

SAME



light.....tone? tone?

dark...click?click?

light.....click? click?

dark...tone?tone?

SAME

DIFFERENT

Again, many people have done this experiment (e.g., Holland, 1986, Rescorla, 1985). This is one example (it actually used pigeons).

Conclusions:

Everyone (including Rescorla!) agrees Rescorla’s theory is too simple; occasion setters specific to training CS

Holland’s theory a possibility -- but why better transfer to a CS from another conditional discrimination?

So Holland (1989) suggested an occasion setter will transfer perfectly to a CS that has its own occasion setter – not to a CS that hasn’t.

Circular...!!

Bouton & Nelson (1998):

occasion setters form best when tone can acquire strength more easily than feature...

light tone food tone nothing

but tone always losing strength too.. both excitatory and inhibitory links with US:

tone food

light

maybe light inhibits the inhibitory link!


problem: can get occasion setting even without tonenothing trials

light tone food tone nothing

tone food

light

maybe light inhibits the inhibitory link!


problem: can get occasion setting even without tonenothing trials

light tone food

tone food

light

no inhibitory link to inhibit!

?

An alternative account ... configural learning

although they are given this....

they actually experience something like this...

tone after light

..when the tone and the light are presented together, the animal experiences a “configural” blend of the stimuli that is different from either

This tone/light configure is associated with food in the normal way -- and that is what produces discrimination performance.

(and extinction of the light doesn’t affect discrimination – it’s a different stimulus!)

It is difficult to discriminate between the and-gate and configural accounts of occasion setting -- but it seems the configural account is not enough to explain all occasion-setting effects.

Occasion setters can be contexts (places)smells drug states (state dependence)

maybe even apply to tasks like Stroop:

red redgreen greenblue blue

task is occasion setter telling you which association to use – the one linked with the name of the word, or that linked with the colour

General references

Bouton, M.E. (2007). Learning and Behavior. Sinauer Associates.

Domjan, M. The principles of learning and behaviour. (1998). Brooks/Cole.

Advanced: Swartzentruber, D. (1995). Modulatory mechanisms in Pavlovian conditioning.Animal Learning and Behavior, 23, 123-143.

Specific references

Bouton, M.E., & Nelson, J.B. (1998). Mechanisms of feature-positive and feature-negative discrimination learning in an appetitive conditioning paradigm. In P.C. Holland and N.A. Schmajuk (Eds.) Associative learning and cognition in animals: Occasion setting. APA, Washington, D.C.

Holland, P.C. (1983). Occasion-setting in Pavlovian feature positive discriminations. In M.L. Commons, R.J. Herrnstein, & A.R. Wagner (Eds.), Quantitative analyses of behavior (Vol. 4, pp. 183-206). New York: Ballinger.

Holland, P.C. (1986) Transfer after serial feature-positive occasion setting. Learning and Motivation, 17, 243-268.

Holland, P.C. (1989b). Feature extinction enhances transfer of occasion setting. Animal Learning and Behavior, 17, 269-279.

Holland, P.C., & Lamarre, J. (1984). Transfer after serial and simultaneous feature negative discrimination training. Learning and Motivation, 15, 219-243.

Rescorla, R.A. (1985). Conditioned inhibition and facilitation. In R.R. Miller & N.E. Spear (Eds.), Information processing in animals: Conditioned Inhibition.(pp. 299-326). Ross, R.T., & Holland, P.C. (1981). Conditioning of simultaneous and serial feature-positive discriminations. Animal Learning and Behavior, 9, 293-303.

conditional learning

Documents

light tone

light light tone

food tone

food light

tone simultaneous

red light

tone can25headjerk response

associative theory