Some applications in human behavior modeling(and open questions)

Jerry Zhu

University of Wisconsin-Madison

Simons Institute Workshop on Spectral Algorithms:From Theory to Practice

Oct. 2014

1 / 30

Outline

Human Memory Search

Machine Teaching

2 / 30

Verbal fluency

Say as many animals as you can without repeating in one minute.

3 / 30

Semantic “runs”1. cow, horse, chicken, pig, elephant, lion, tiger, porcupine,

gopher, rat, mouse, duck, goose, horse, bird, pelican, alligator,crocodile, iguana, goose

2. elephant, tiger, dog, cow, horse, sheep, cat, lynx, elk, moose,antelope, deer, tiger, wolverine, bobcat, mink, rabbit, wolf,coyote, fox, cow, zebra

3. cat, dog, horse, chicken, duck, cow, pig, gorilla, giraffe, tiger,lion, ostrich, elephant, squirrel, gopher, rat, mouse, gerbil,hamster, duck, goose

4. cat, dog, sheep, goat, elephant, tiger, dog, deer, lynx, wolf,mountain goat, bear, giraffe, moose, elk, hyena, aardvark,platypus, lion, skunk, wolverine, raccoon

5. dog, cat, leopard, elephant, monkey, sea lion, tiger, leopard,bird, squirrel, deer, antelope, snake, beaver, robin, panda,vulture

6. deer, muskrat, bear, fish, raccoon, zebra, elephant, giraffe,cat, dog, mouse, rat, bird, snake, lizard, lamb, hippopotamus,elephant, skunk, lion, tiger

7. dog, cat, ferret, fish, cow, horse, pig, sheep, elephant, tiger,lion, bear, giraffe, bird, groundhog, ox

8. antelope, baboon, cat, dog, elephant, frog, giraffe,hippopotamus, jaguar, dog, cat, horse, pig, chicken, bird,hippopotamus, cow

9. dog, cat, cow, sheep, lamb, rooster, chicken, tiger, elephant,monkey, orangutan, lemur, crocodile, giraffe, owl, bird, lion,elephant, hippopotamus, rhinoceros, penguin, seal, walrus,whale, dolphin

10. dog, cat, bear, deer, puma, opossum, moose, bear, wolf,coyote, lion, tiger, elephant, giraffe, hyena, cougar, mountainlion, antelope, elk

4 / 30

Memory search

−1.5 −1 −0.5 0 0.5 1 1.5

−1

−0.5

0

0.5

1

Guinea pig

aardvark

alligator

alpaca

amphibiananemone

ant

anteater

antelope

ape

armadillo

baboon

badger

bat

bear

beaver

bee

beetle

bird

bison

bluejay

boaconstrictor

bobcat

buffalo

bull

bumblebee

bunny

butterfly

buzzard

camel

canary

cardinal

caribou

cat caterpillar

centipede

chameleon

cheetah

chickadeechicken

chimp

chinchilla

chipmunk

clam

cobra

cockatiel

cockroach

colt

cougarcow

coyote

crab

crane

cricket

crocodile

crow

deer

dingo

dog

dolphin

donkey

dove

duck

eagle

eel

elephant

elk

emu

falcon

fish

flamingo

flea fly

fowl

fox frog

gazelle

gecko

gerbilgibbon

gilamonster

giraffe

gnat

goat

goose

gopher

gorilla

grasshopper

groundhog

grouse

gull

hamsterhare

hawk

hedgehog

heron

hippopotamus

hornet

horse

human

hummingbird

hyena

ibis

iguana

impala insect

jackal

jackrabbit

jaguar

kangaroo

koalaladybug

lamb

lemur

leopard

lion

lizard

llama

lobster

loon

lynx

manatee

maremarten

mole

mongoose

monkey

moose

mosquito

moth

mouse

mule

muskox

muskrat

musky

newt

ocelot

octopus

opossum

orangutan

orioleostrich

otter

owl

oxpanda

panther

parrot

peacock

pelican

penguin

pig

pigeon

platypus

polar bear

pony

porcupine

porpoise

prairie dog

puma

python

quail

rabbitraccoon

racoon

ram

rat

rattlesnake

raven

reindeer

rhinoceros

roadrunner

robin

rodent

rooster

salamander

sea lion

seagull

seal

shark

sheepshrew

shrimp

skunk

sloth

snail

snake

sparrow

spider

squid

squirrel

swan

tadpole

tapir

tasmanian devil

tiger

tortoise

turkey

turtle

vole

vulture

wallaby

walrus

waspweasel

whale

wildebeest

wolf

wolverine

wombat

woodchuck

woodpecker

worm

wren

yak

zebra

K = 12, obj = 14.1208

5 / 30

Memory search

−1.5 −1 −0.5 0 0.5 1 1.5

−1

−0.5

0

0.5

1

Guinea pig

aardvark

alligator

alpaca

amphibiananemone

ant

anteater

antelope

ape

armadillo

baboon

badger

bat

bear

beaver

bee

beetle

bird

bison

bluejay

boaconstrictor

bobcat

buffalo

bull

bumblebee

bunny

butterfly

buzzard

camel

canary

cardinal

caribou

cat caterpillar

centipede

chameleon

cheetah

chickadeechicken

chimp

chinchilla

chipmunk

clam

cobra

cockatiel

cockroach

colt

cougarcow

coyote

crab

crane

cricket

crocodile

crow

deer

dingo

dog

dolphin

donkey

dove

duck

eagle

eel

elephant

elk

emu

falcon

fish

flamingo

flea fly

fowl

fox frog

gazelle

gecko

gerbilgibbon

gilamonster

giraffe

gnat

goat

goose

gopher

gorilla

grasshopper

groundhog

grouse

gull

hamsterhare

hawk

hedgehog

heron

hippopotamus

hornet

horse

human

hummingbird

hyena

ibis

iguana

impala insect

jackal

jackrabbit

jaguar

kangaroo

koalaladybug

lamb

lemur

leopard

lion

lizard

llama

lobster

loon

lynx

manatee

maremarten

mole

mongoose

monkey

moose

mosquito

moth

mouse

mule

muskox

muskrat

musky

newt

ocelot

octopus

opossum

orangutan

orioleostrich

otter

owl

oxpanda

panther

parrot

peacock

pelican

penguin

pig

pigeon

platypus

polar bear

pony

porcupine

porpoise

prairie dog

puma

python

quail

rabbitraccoon

racoon

ram

rat

rattlesnake

raven

reindeer

rhinoceros

roadrunner

robin

rodent

rooster

salamander

sea lion

seagull

seal

shark

sheepshrew

shrimp

skunk

sloth

snail

snake

sparrow

spider

squid

squirrel

swan

tadpole

tapir

tasmanian devil

tiger

tortoise

turkey

turtle

vole

vulture

wallaby

walrus

waspweasel

whale

wildebeest

wolf

wolverine

wombat

woodchuck

woodpecker

worm

wren

yak

zebra

K = 12, obj = 14.1208

6 / 30

Censored random walk

[Abbott, Austerweil, Griffiths 2012]

I V: n animal word types in English

I P : (dense) n× n transition matrix

I Censored random walk: observing only the first token of eachtype x1, x2, . . . , xt, . . .⇒ a1, . . . , an

I (star example)

7 / 30

The estimation problem

Given m censored random walksD =

{(a(1)1 , ..., a

(1)n

), ...,

(a(m)1 , ..., a

(m)n

)}, estimate P .

8 / 30

Each observed step is an absorbing random walk

(with Kwang-Sung Jun)

I P (ak+1 | a1, . . . , ak) may contain infinite latent steps

I Instead, model this observed step as an absorbing randomwalk with absorbing states V \ {a1, . . . , ak}

I P ⇒(Q R0 I

)

9 / 30

Maximum Likelihood

I Fundamental matrix N = (I−Q)−1, Nij is the expectednumber of visits to j before absorption when starting at i

I p(ak+1 | a1, . . . , ak) =∑k

i=1NkiRi1

I log likelihood∑m

i=1

∑nk=1 log p(a

(i)k+1 | a

(i)1 , ..., a

(i)k )

I Nonconvex, gradient method

10 / 30

Other estimators

I PRW: pretend a1, . . . , an not censored

I PFirst2: Use only a1, a2 in each walk (consistent)

11 / 30

Star graphx-axis: m, y-axis: ‖P − P‖2F

12 / 30

2D Grid

13 / 30

Erdos-Renyi with p = log(n)/n

14 / 30

Ring graph

15 / 30

Questions

I Consistency?

I Rate?

16 / 30

Outline

Human Memory Search

Machine Teaching

17 / 30

Learning a noiseless 1D threshold classifier

θ∗

x ∼ uniform[0, 1]

y =

{−1, x < θ∗

1, x ≥ θ∗

Θ = {θ : 0 ≤ θ ≤ 1}

Passive learning:

1. given training data D = (x1, y1) . . . (xn, yn)iid∼ p(x, y)

2. finds θ consistent with D

Risk |θ − θ∗| = O(n−1)

18 / 30

Learning a noiseless 1D threshold classifier

θ∗

x ∼ uniform[0, 1]

y =

{−1, x < θ∗

1, x ≥ θ∗

Θ = {θ : 0 ≤ θ ≤ 1}

Passive learning:

1. given training data D = (x1, y1) . . . (xn, yn)iid∼ p(x, y)

2. finds θ consistent with D

Risk |θ − θ∗| = O(n−1)

18 / 30

Learning a noiseless 1D threshold classifier

θ∗

x ∼ uniform[0, 1]

y =

{−1, x < θ∗

1, x ≥ θ∗

Θ = {θ : 0 ≤ θ ≤ 1}

Passive learning:

1. given training data D = (x1, y1) . . . (xn, yn)iid∼ p(x, y)

2. finds θ consistent with D

Risk |θ − θ∗| = O(n−1)

18 / 30

Active learning (sequential experimental design)

Binary searchθ∗

Risk |θ − θ∗| = O(2−n)

19 / 30

Active learning (sequential experimental design)

Binary searchθ∗

Risk |θ − θ∗| = O(2−n)

19 / 30

Machine teaching

What is the minimum training set a helpful teacher can construct?

θ∗

Risk |θ − θ∗| = ε,∀ε > 0

20 / 30

Machine teaching

What is the minimum training set a helpful teacher can construct?θ∗

Risk |θ − θ∗| = ε,∀ε > 0

20 / 30

Comparing the three

θ∗

O(1/2 )n

θ∗θ∗

{{

O(1/n)

passive learning "waits" active learning "explores" teaching "guides"

The teacher knows θ∗ and the learning algorithm.

21 / 30

Example 2: Teaching a Gaussian distribution

Given a training set x1 . . . xn ∈ Rd, let the learning algorithm be

µ =1

n

n∑i=1

xi

Σ =1

n− 1

n∑i=1

(xi − µ)(xi − µ)>

How to teach N(µ∗,Σ∗) to the learner quickly?

non-iid, n = d+ 1

22 / 30

Example 2: Teaching a Gaussian distribution

Given a training set x1 . . . xn ∈ Rd, let the learning algorithm be

µ =1

n

n∑i=1

xi

Σ =1

n− 1

n∑i=1

(xi − µ)(xi − µ)>

How to teach N(µ∗,Σ∗) to the learner quickly?

non-iid, n = d+ 1

22 / 30

An Optimization View of Machine Teaching

A(D)

θ∗

Α (θ )−1 ∗

A

Α−1

D

Θ

DD

minD∈D

|D|

s.t. A(D) = θ∗

The objective is the teaching dimension [Goldman, Kearns 1995] ofθ∗ with respect to A,Θ.

23 / 30

Example 3: Works on Humans

Human categorization on 1D stimuli [Patil, Z, Kopec, Love 2014]

human training set human test accuracy

machine teaching 72.5%iid 69.8%

(statistically significant)

24 / 30

New task: teaching humans how to label a graph

Given:

I a graph G = (V,E)

I target labels y∗ : V 7→ {−1, 1}I a label-completion cognitive model A (graph diffusion

algorithm) such as:I mincutI harmonic function [Z, Gharahmani, Lafferty 2003]I local global consistency [Zhou et al. 2004]I . . .

Find the smallest seed set:

minS⊆V

|S|

s.t. A(y∗(S)) = y∗

(inverse problem of semi-supervised learning)

25 / 30

Example: A = local-global consistency [Zhou et al. 2004]

F = (1− α)(I − αD−1/2WD−1/2)−1y∗(S)

y = sgn

(F

(1−1

))

26 / 30

|A−1(y∗)| is largeTurns out 4649 out of 216 = 65536 training sets S lead to thetarget label completion

27 / 30

Optimal seed set 1: |S| = 3

28 / 30

Optimal seed set 2: |S| = 3

29 / 30

Questions

I How does |S| relate to (spectral) properties of G?

I How to solve for S?

30 / 30