the learned dog class 14: learning in nature. the great debate: general mechanisms or adaptive...

The learned dog

Class 14: Learning in nature

The great debate: general mechanisms or adaptive specializations...

3 views...

• Psychology

• Ethology

• Cognitive Science

Different perspectives on learning: psychology

• Psychology (especially the behaviorists...)

• Discover the elemental ‘laws’ of learning

• Expressed with respect to observable behavior (eschew internal representations, etc.)

• Because these are general laws, experiments are explicitly designed to control for species differences...

• Artificial operant responses: lever press, key press

• Stimuli: light, tones...

Different perspectives on learning: psychology

• Psychology continued, controls...

• Sterile environment

• Standardized test subjects

• Learning is abstracted from its behavioral/ecological context. That is, it can be understood outside of its biological function

• Implicit assumption: learning is good, and more learning is better

• Not withstanding the above, the approach made invaluable contributions to helping us think about learning...

Different perspectives on learning: ethology

• Learning as only one of a number of strategies that organisms use to adapt to their ecological niche.

• Learning can only be understood within its biological and ecological niche

• Indeed, tend to view ‘learning’ as a suitcase word that encompasses a range of special purpose mechanisms for solving particular problems, and the nature of the problems shape how the animal learns...

• Who is my mother, my conspecifics, my potential sweetie(s)?

• Song bird learning

• Predatory sequence learning


• Learning has a cost...

• Ability to learn is only good to the extent that...

• enhances animal’s reproductive success

• is the least cost way to do so, vis-a-vis an innate mechanism that solves the same problem.

• Selective pressure on reducing the cost of learning important things...

• Timing

• Scaffolding

Different perspectives on learning: biological constraints vs. predispositions...

• When faced with species-specific differences...

• Psychologists: biological constraints on learning

• The Breland’s instinctive drift

• Ethologists: innate predispositions to learn...

• Lorenz spoke of the “innate schoolmarm”

• Note the fundamental difference in perspective: constraint vs. predisposition

Biological constraints/predispositions

• Reflect animal’s (specie’s) response to making a living in its ecological niche...

• Bias to attend to certain modalities in certain contexts

• Rats: olfactory cues -> feeding, visual cues -> self-defense

• Pigeons: visual cues-> feeding, auditory cues -> self-defense

• Dogs: motion, smell, sound?

• What is reinforcing?

• Food vs. social vs. performing behavior vs. ...


• Up through the late 60’s, ethologists and psychologists seemed to go out of their way not to learn about each other...

• Cultural differences

• Philosophical differences

• Learning as an adaptive mechanism vs. general mechanism

• Innate vs. learned

• Purposive vs. behavior as a response to the environment

• Both have hopefully recognized that each has much to offer the other

Different perspectives on learning: cognitive science

• Cognitive science is focused on the internal representations and information processing that may underlie observed behavior...

• How might learning implemented in the brain?

• Once again, learning is removed from its biological context...

• Tends to rely on experiments/observations done by others

• General vs. specific

• Heavily influenced by computer science

The problem facing dogs...

Set of all possible actions

Set of all motivational

goals

Set of all possible stimuli

What do I do, when, in order to best satisfy my motivational goals?

The space of possible stimuli is wicked big

Dogs make choices as to what they attend to...

Set of all possible stimuli

SmellsMotion

Sounds

Dog sounds

SpeechWhistles

Modality of Stimuli

Time of Occurrence

State Space

The space of possible actions is wicked big too..

Set of all possible actions

Action

Time of Performance

Figure -8

Shake

Low shake

High -5

Beg

Down

Left ear twitch

Action Space

Who gets the credit for good things happening?

Its a very hard problem that animals like dogs solve by using commonsensical rules of paw

Yumm.

Action

Shake

Low shake

High Five

Beg

Down

Left ear twitch

Motion

Sounds

Dog sounds

SpeechWhistles

Modality of Stimuli

Time

The secret of good learners...

• Assume things close in time are related

• Assume that your actions matter. When in doubt assume that it was your action that made the difference (action more salient than external stimuli)

• Attend to novel stimuli and see what happens vis-a-vis reliability of your actions.

• Use expectation violations as driver for learning...

• Vary what you do, when you do it, and how you do it, and see what happens.

The secret of good trainers...

• Very skilled at guiding dog’s exploration of its search space

• reward

• timing

• shaping

• luring

• Just the right amount of variation

• They know their particular animal!!!!!

What happens when there isn’t a trainer? Nature cheats like crazy...

Taste aversion...

• Rats tend to avoid novel foods

• Ok, if they smell food on whiskers of another rat

• If they get sick, even hours later, they ‘blame’ the food.

• Note: funny kind of proximity.

Song Learning

Prepared to learn in easiest way at easiest time

• Song Bird Learning

• They ‘learn’ song of relatives while still in nest and surrounded by relatives

• ‘Built-in’ template of song makes it easier to recognize song and to get started practicing song

• Learning about predators...

Scaffolding: sensitive time plus template

Shettleworth, S. J. (1998). Cognition, Evolution and Behavior. New York, NY, Oxford University Press

10-50 days:

150 days:

200 days:

Age

Scaffolding: sensitive time plus template


Easiest time, easiest way Who is my predator???


Easiest time, easiest way

Associative learning scaffolded by predisposition


Easiest time, easiest way...

• Indigo Buntings

• They ‘learn’ location of Polaris (North Star) as babies in their nest

• Later use this to navigate

Imprinting

Learning to recognize mom, con-specifics, and potential mates...

• Lorenz was the first to describe the process of ‘imprinting’ in which chicks would rapidly form a preference to follow & attend to a ‘mom’ like object in its environment. He claimed it was a different type of learning...

• Occurred during a critical period and was irreversable

• Influenced behavior later in life of animal, e.g, mate selection

• From mom, animals learn conspecifics.

• There was no reinforcement signal

• Most of his claims regarding imprinting have been modified as a result of subsequent research. Better viewed as biases vs. absolutes.

Two different processes may be at work

• At least two underlying processes (from at least 2 different parts of the brain...)

• Predisposition to approach stimuli with the characteristics of an “innate mom”...

• Head & neck mostly, but presence of eyes and acoustic cues enhance tendency. Stuffed ferrets & ducks as good as mom in this regard.

• Note this is similar to neonatal infants attending preferentially to ‘face-like’ objects.

• In a sense, this is scaffolding to bias chick to attend to the likely mom in its environment.

Two different processes may be at work (cont)

• At least two underlying processes (from at least 2 different parts of the brain...)

• Predisposition to approach objects to which they have been exposed (such as mom like objects) and build more detailed perceptual models of the object.

• Fancy way to say that associative learning takes over, and the chick learns other stimuli associated with the ‘preferred’ object that provide it with incremental predictive value (presumably) in identifying mom from others.

• Aspects of the imprinted object act like US, chick learns other CS

• Scaffolding points chick in right direction and helps ensure the success of an otherwise simple learning mechanism.

Sexual imprinting occurs later and process is a bit

different occur later

Learn model of siblings and choose mate that is ‘slightly different but not too different.’


Shettleworth suggests...

• Initial imprinting is all about learning a reliable model of a specific mom.

• Subsequently, they may learn a reliable model of their sibs...

• Complicated by fact that sibs are changing form as they develop

• At some point, sexually imprinting occurs

• Here the adaptive challenge is to have a model that says, in effect, “this animal is almost like my sibs, but it isn’t one of my sibs.

But I have a dog, not a goose...

• Scott & Fuller suggested that dogs go through a critical period of imprinting between 3 and 5 weeks.

• mom, siblings and ‘ok’ species.

• appears to only take short passive presence of humans to cause dogs to imprint on humans, i.e., add humans to list of “familiar/good” species

• Ray Coppinger’s student Kathryn Lord suggests...

• Olfactory cues scaffold process (US). Maybe tactile too?

• Visual and auditory cues are associated with olfactory cues (CS)

Sequences

Who gets credit for good things happening?

stalkgrab-bite

eye

orient

kill-bitechase

Yumm..

Time

Conventional idea: back propagation from

goal

stalkgrab-bite

eye

orient

kill-bitechase

Yumm..

Time Credit flows backward

It just can’t work that way...

• If each element in sequence has 3 variants, there are 729 possible combinations of which 1 may work (ignoring stimuli)

• If there are 12 possible stimuli, there are 1,586,874,322,944 possible combinations of stimuli-action pairs to explore.

• Don’t know if it is the right sequence until goal is reached

• What happens if “variant” needs to be learned?

• Early elements usually fail even in best case.

Leyhausen’s suggestion

stalkgrab-bite

eye

orient

kill-bitechase

Time Each element is innately self-motivating and has innate reward

metric

motivation & reward

motivation & reward

motivation & reward

motivation & reward

motivation & reward

motivation & reward

Leyhausen’s prediction: Internal motivation/reward especially strong for appetitive behaviors that are likely to fail even when done right, e.g., chase...

Coppinger’s suggestion

Tendency to move on to next motor pattern in sequence varies based on breed

stalkgrab-bite

eye

orient

kill-bitechase

TimeVarying innate tendency to follow behavior with “next” in sequence

Coppinger’s suggestion

Achieving the putative goal is a side-effect, albeit a wicked useful one.

stalkgrab-bite

eye

orient

kill-bitechase

TimePropagated value from functional goal

plays incidental role

Yumm..

A couple of other big ideas here...

• Important skills are ‘perfected’ in non-functional context, i.e. what we call play

• Whether this is a side-effect of the phenomena we label play, or the reason the phenomena exists is an open question.

• Animals do lots of things for which the external reinforcement is anything but obvious and which fail more often than they succeed, so what is the reinforcement signal, perhaps it is the performance of the behavior itself...

• Leyhausen viewed process as creating discrete variations of the underlying motor pattern vs. a continuous refinement of the motor pattern...

• When excited or nervous reverts back to early variations.

Lorenz on Animal Training

Lorenz and animal training...

• Believed that the set of motor patterns that a given animal could perform was fixed in number and form. So you could not teach an animal a motor pattern that it did not already ‘know’ how to perform.

• Rather he believed that during animal training the animal was learning to repurpose a motor pattern from one motivational context to be an appetitive motor pattern in a new motivational context.

• The example of the Lippizzaner: from self-defense to foraging (note use of luring & shaping)

• Not all motor patterns are equally easy to repurpose. In particular, those that are under hormonal control tend to be difficult to repurpose

Lorenz, K. (1981). The Foundations of Ethology. New York, NY, Springer-Verlag.

The bottomline

To be a successful animal trainer...

• You must understand that learning must be viewed within the biological context of the animal you are trying to train...

• Predisposition to attend to certain classes of stimuli and in certain contexts

• Predisposition to prefer certain types of reinforcement over others (food, social interaction, play...)

• Implications of developmental processes

• Emotional reactivity

To be a successful animal trainer, you should understand that the ‘laws of learning’ are, in the words of Capt. Jack Sparrow, ‘more like guidelines.’

the learned dog class 14: learning in nature. the great debate: general mechanisms or adaptive...

Documents

otherdifferent perspectives

innate mechanism

innate predispositions

olfactory cues

adaptive mechanism

general mechanisms

observable behavior

general laws