model-based clustering for categorical and mixed data sets

HAL Id: tel-01076418https://tel.archives-ouvertes.fr/tel-01076418

Submitted on 22 Oct 2014

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Model-based clustering for categorical and mixed datasets

Matthieu Marbac-Lourdelle

To cite this version:Matthieu Marbac-Lourdelle. Model-based clustering for categorical and mixed data sets. Statistics[math.ST]. université lille 1, 2014. English. tel-01076418

https://tel.archives-ouvertes.fr/tel-01076418

https://hal.archives-ouvertes.fr

♥♠ér♦ ♦rr

❯♥rsté ♦rt♦r P P♥é

ès

♣rés♥té ♣♦r ♦t♥t♦♥

♣ô♠ ♦t♦rt

♥rsté

♣été té♠tqs

♦ès ♠é♥ ♣♦r sst♦♥♥♦♥ s♣rsé ♦♥♥és qtts t

♠①ts

♣r

tt r♦r

♦t♥ s♣t♠r ♥t r② ♦♠♣♦sé

♠trs rs Pr♦ssr ❯♥rst② ♦ t♥s ♣♣♦rtr♥ r♥ Pr♦ssr ❯♥rsté ♦♥t♣r ♣♣♦rtrs ① rtr r ♥r ② ①♠♥tr♦s ❲r Pr♦ssr ❯♥rsté ①♠♥trrst♦♣ r♥ Pr♦ssr ❯♥rsté rtr❱♥♥t ❱♥ Pr♦ssr ❯♥rsté rtr

ès ♣ré♣ré é♣rt♠♥t té♠tqs♦rt♦r P P♥é ❯ ❯♥rsté ❱♥ sq ❳

♠r♠♥ts

rst ♦ s♥r② ♦ t♦ t♥ ♠trs rs ♥ ♥ r♥♦r ♣t♥ t♦ r♣♦rt ♦♥ ts tss s s s ① ♥ ♦s ❲r♦r tr ♥ ♣rt♣t♦♥ s ①♠♥rs ♥ t P ♥s

rss é♠♥t ♣r♦♦♥s r♠r♠♥ts à ♠s ① rtrs tèsrst♦♣ r♥ t ❱♥♥t ❱♥ ♣♦r r ♥r♠♥t ♦rs s tr♦s ♥♥és ❱♦s ③ s str ♥♦♠r① ♦♥ss t♦t ♥ ♠ ss♥t♥ rté q♥t ① ①s rrs q ♣ ①♣♦rr q ♦rt♠♥t♣♣réé ♦s r♠r ♠♦r ♦♥♥é ♦♣♣♦rt♥té ♠♥tr à rr♦rs ♠♦♥ st ♠str t ♦s êtr é♠♥és ♣♦r ♠ tr♦r ♥ ♥♥♠♥t tès

r♠r r♥ q ♠ ♦♥♥é ♦ût à rr ♦rs ♥♦s ré♥♦♥s q s s♦♥t éés ss ♣rtèr♠♥t r♦♥♥ss♥t ♥rs r♥ r♠♥♥ q ♠ ♥♦ré ♣s ♠♦♥ ♠str t q t♦♦rs s♣r♥r t♠♣s ♠ ♣r♦r ♣ré① ♦♥ss

rss é♠♥t ♠s r♠r♠♥ts à ♥s♠ éq♣ ♦ù ♠♦♥♥tért♦♥ été té râ à ♥r♥ ♥

♦rs s tr♦s ♥♥és tès ♣sr é♦rr ♠♦♥ ♥s♥♠♥t r♥ ♠r rst♥ Pr t ♥ qs

r♠r ♣rs♦♥♥ ♦rt♦r P♥é ♥ ♣rtr ♥ r♥t② éè♥ réérq ♥qs r♥ç♦s t♦♠t t ♠r r♦ ♠s

P♥♥t tt tès ♣sr é♥r ♥♦♠rss ♣rs♦♥♥s♣r♠ sqs r Pr♠t ①♥r ♥♠♥ Prr♦s t P♦rt t ♥ s ♠s ♦♥t été ♣rtèr♠♥t ♠é♠♦rs râ à ♥t♦♥ ♠ t Prr r à ♦ t s ♥ ♣s ♠s ♥t ♣♦r ss♦♥ss ♥ ♣rtr ♣♥♥t ♠ rét♦♥

r♠r s ♠♠rs ♠ ♠ qs ♠♥t ♦ ♥♦♥ s ♥♥s ♣♦rr s♦t♥ t ♣r q ♥ ss sr s ♦♥t t♦s rt♥ ttr ♠ tèsrss ♥ ♠♥t♦♥ s♣é ♣♦r é♦ q ♠ s♣♣♦rt q♦t♥

♥♥ ♣rés♥t ♠s ①ss à ① q ♦♥ts♠♥t ♦é ♠♥t♦♥♥rt q s♣èr ♥ ♠♥ t♥r♦♥s ♣s rr

♦♥t♥ts

♠r♠♥ts

♦r♦r

♥ rt♦♥s ♥ ♥♦tt♦♥s

str ♥②ss stt ♦ t rt r ♦ t str♥ ♣♣r♦s ♥rts ♦♥ ♥t ♠①tr ♠♦s Pr♠tr st♠t♦♥ ♦ st♦♥ ♦♥s♦♥

♦s str♥ ♦r t♦r t

str ♥②ss ♦ t♦r t sts stt ♦ t rt ♥ ♦ str ♥②ss ♦r t♦r t ♦♠tr ♣♣r♦s ♦♥r ♠①tr ♠♦s ①trs ♦ trs t t♥t ss ♠♦ ♦♥s♦♥

♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s ♥tr♦t♦♥ ①tr ♦ ♥trss ♥♣♥♥t ♦s Prs♠♦♥♦s ♦ strt♦♥ ①♠♠ ♦♦ st♠t♦♥ ♦rt♠ ♦ st♦♥ ♦rt♠ ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥

♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s ♥tr♦t♦♥ ①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

♦♥t♥ts

①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠ ♦ st♦♥ tr♦♣♦st♥s s♠♣r ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥

♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s strt ♦♦s

♦♥s♦♥ ♦ Prt

♦s str♥ ♦r ♠① t

str ♥②ss ♦ ♠① t sts stt ♦ t rt ♥ ♦ str ♥②ss ♦r ♠① t r ♦ s♠♣ ♠t♦s t♦ str ♠① t ①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s ❯♥r♥ ss♥ ♠①tr ♠♦ ♦♥s♦♥

♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s ♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠ ♦ st♦♥ ♦rt♠ ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥

♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t ♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♦♣s ②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ tr r t sts ♦♥s♦♥

♦♥s♦♥ ♦ Prt

♥r ♦♥s♦♥ ♥ ♣rs♣ts

♣♣♥① ♦ Prt ♥r ♥tt② ♦ t ♠①tr ♦ t t♦ ①tr♠ ♣♥♥②

strt♦♥s

♦♥t♥ts

♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt ♦♦ ♦ t ♠①tr♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

♣♣♥① ♦ Prt ♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st str

t♦♥s ♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s

♦r♣②

♦r♦r

♥ t ♥ t♦ str

t qst♦♥ s ♦♠ ♥rs♥② s② t♥s t♦ t ♥rs ♣r♦r♠♥ ♦ ♦♠♣t♥ r♦r ♣rtt♦♥rs r ♥ t sts r ♥rs♥② ♥♦r♠t♦♥r t s♦ ♥rs♥② strs ♥ t ♥♦r♠t♦♥♦♥t♥ ♥ t t sts ♥ rt② ♥tt♥ ♦r t♦ ♠♥ rs♦♥s tq♥tt② ♦ t ♥ tr ♦♠♣①t② s sttst ♠t♦s r ♠♥t♦r② t♦♥②③ s t sts

str♥ s ♥ ♣♣r♦ rs t ♣r♦♠ s ② t r q♥tt②♦ t ♥ ts ♠ s t♦ r♦♣ t ♥s ♥t♦ s♣ sss st ♣r♦s ♠♥♥ s♠♠r② ♦ t t st tr♦♦t rtrst♥s ♦ t ♥ ♦r ♦t str♥ s ♥tr ♦r ♥st♥t ♥ ♦r s ♥t♦ t♦ sss ♣♥ts ♥ ♥♠s t ♥♠s rs♣t ♥t♦ ♥rtrt ♥ rtrt t rtrt r ss ♥t♦ sss♠♠♠s ss rs ♠♣♥s r♣ts

♣r♦st ♠t♦s ♣r♠t t♦ ♣r♦r♠ t str ♥②ss ♥ r♦r♦s♦♥t①t ♠♦♥ ts ♠t♦s t ♥t ♠①trs ♦ ♣r♠tr strt♦♥s s♠♠r③ t t ② t ♣r♠trs ♦ ss ♦r♦r ♥ ts ♦♥t①t tss ♣r♦st t♦♦s r t♦ ♥sr t t qst♦♥s ♦ str♥②ss t ♦ ♦ t ♥♠r ♦ sss t ♦r♣② s ♣r♦ ♦t♦♥t♥♦s t sts ♥♦t s♦rt ♥ t t r ♠♦r ♦♠♣① ♥ ts♦♥t①t t ♠ ♦ ts ♠♥sr♣t s t♦ st② ①st♥ ♣r♦st ♠t♦s ♥t♦ ♣r♦♣♦s ♥ ♦♥s t♦ str ♦♠♣① t sts

t♦ ♦ts ♦ ts ♦r

❲ ♦s ♦♥ t♦ stt♦♥s r t t sts r ♦♠♣① t s r ♥s r sr ② t♦r rs ♥ t s r t② r sr② ♠① rs r♥t ♥s ♦ rs s t♦ t♠ts r st

♠♦s str♥ ♦r t♦r t sts ♠♦s str♥ ♦r ♠① t sts

t♦r rs r t t♦ str s♥ t② t sttst♥♥ t ♠♥② ♦♠♥t♦r ♥s s t② ♥rss ♥ t rs r ♣♥♥t ♥ t s♠ ss ♥ t ♠♦s rqr r ♥♠r ♦♣r♠trs ♥ ♦rr t♦ t ♥t♦ ♦♥t t ♥trss ♣♥♥s s t ss ♣♣r♦ ss♠s t ♦♥t♦♥ ♥♣♥♥ t♥ rs ♦r

♦r♦r

ts ♣♣r♦ s s ♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s ♦t♥② tr♥t ♣♣r♦s ♥ ♣r♦♣♦s t tr ♥srs st② ♥♦♠♣t♥ t ♦♠♥t♦r ♣r♦♠ ♦ t ♠♦ st♦♥ s ♥♦t ②s s♦ ♦r♦r ts ♠t♦s ♥ sr r♦♠ ♥stt② ♦r r♦♠ ♦ ♥tr♣rtt② ♥ts ♦♥t①t ♦r ♦♥trt♦♥ ♦♥ssts ♥ t♦ ♣rs♠♦♥♦s ♠①tr ♠♦s ♦ t♦ str t♦r t ♣rs♥t♥ ♥trss ♣♥♥s ♠♥ ♦ ts ♠♦s s t♦ r♦♣ t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ②stt♥ s♣ strt♦♥s ♦r ts ♦s ♦t ♠♦s ♦♥sr t ♥trss♣♥♥s t♥ t rs ♦t ♠♦s ♣r♦ ♣r♠trs t♦ s♠♠r③ t t ♣r♦♣♦s r♦r♦s ♣♣r♦ t♦ ♣r♦r♠ t ♠♦ st♦♥ ♥ srr♥② s③t♦♥ ♦ t ♣r♠trs ♥ ♦ t t♦rt str♥ s ♠♦tt ② t t tt t♦r t r s② sss t② r ♥♠r♦s t rs t♦ ♦sr ♥trss ♦rrt t ♥rss

st② ♦ t str ♥②ss ♦ ♠① t sts s t s♦♥ ♦t ♦ts ♦r s ♣r♦♠ s ♠♦tt ② t t tt t rr♥t t sts r ♦t♥♦♠♣♦s t r♥t ♥s ♦ t ss ♣♣r♦ ss t♦r ♥②③rs♠t♦s ♥tr♣rtt♦♥ ♦ s ♠t♦ s ♦t♥ ♦♠♣① s♥ t ♣r♠trsr ♥♦t rt t♦ t rs ♥ tr ♥t s♣ tr ss ♣♣r♦s♦♥sst ♥ ♣♣②♥ s♣ ♠①tr ♠♦ ♦♥ ts t ♥ s t♦t ♦ ss strt♦♥s ♦r ♠① rs ❲ ♣r♦♣♦s t♦ ♠①tr ♠♦s t♦ ts ♣ rst ♦♥ s ss s♥ t s ♥ ①t♥s♦♥ ♦ ♥♦♥♠t♦ t♦ str t♦r t sts ♥ t ♠♦ ♦♠♥s ss♥ strt♦♥s ♥ ♥r ♦st rrss♦♥s s ts ♠♦ ♥②③s t sts t♦♥t♥♦s ♥ t♦r rs s♦♥ ♠♦ s t ♠♥ ♦♥trt♦♥ ♦ts tss s♥ t ♦s t♦ ♥②③ t sts t ♥② ♥ ♦ rs ♠tt♥ ♠t strt♦♥ ♥t♦♥ s ♠♦ s ♥ s ♠①tr ♦ ss♥ ♦♣s ts ♣rsr♥ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥ ♦r rs ♦ ♦♠♣♦♥♥t rtr♠♦r ts ♣♣r♦ ♠♦③s t ♥trss♣♥♥s ♥② ♥♦t tt s③t♦♥ t♦♦ s s ②♣r♦t ♦ts ♠♦

r♥③t♦♥ ♦ t ♠♥sr♣t

♠♥sr♣t s ♥t♦ t♦ ♠♥ ♣rts ♦rrs♣♦♥♥ t♦ t t♦ ♥s♦ t ♦ ♥trst ♦r ♣rs② t s ♦r♥③ s ♦♦s

♣tr s r ♦r ♦ t ♠♥ str♥ ♠t♦s ♥ ♥rr♠♦r t ♦ss ♦♥ t r♥t s♣ts rt t♦ ♥t ♠①tr ♠♦s t ♥tr♦s t ♥r ♥♦t♦♥s ♥ ♦rt♠s s ♥ t ♦♦♥♣trs

Prt ♦s str♥ ♦r t♦r t

♣tr ♦♥ssts ♥ t stt ♦ t rt ♦ t ♠t♦s ♣r♦r♠♥ tstr ♥②ss ♦ t♦r t sts

♦r♦r

♣tr ♣rs♥ts ♦r rst ♦♥trt♦♥ t♦ t t♦r t ♥②ssr♠♦r ♣r♦♣♦s ♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥②♥♣♥♥t ♦s s♣ strt♦♥ ♦ t ♦s ♠♦③s t♥trss ♣♥♥s ♥ ♣r♦s s♣ ♦♥t s♠♠r③♥t str♥t ♦ ts ♣♥♥s ts rsts r ♣rt ♦ t rt ♦s str♥ ♦r ♦♥t♦♥② ♦rrt t♦r t❬❱❪

♣tr ♣rs♥ts ♦r s♦♥ ♦♥trt♦♥ t♦ t t♦r t ♥②ss r♠♦r s ♠♦ ♦♥ssts ♥ ♠①tr ♠♦ r♦♣st rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ ♦♦s ♣rs♠♦♥♦s ♠t♥♦♠ strt♦♥ r t r ♣r♠trs♦rrs♣♦♥ t♦ ts ♠♦s ts rsts r ♣rt ♦ t rt ♥t♠①tr ♠♦ ♦ ♦♥t♦♥ ♣♥♥s ♠♦s t♦ str t♦rt ❬❱❪

♣tr strts ♦t ♣s ♣r♦r♠ t ♥r♥ ♦♦t ♣r♦♣♦s ♠♦s s ♣tr ♥ s♦ s s tt♦r ♦♦t ♣s s♥ t ♣r♦s ♣rs♥tt♦♥ ♦ tr ♠♥ ♥t♦♥s ♥♠♥② sr♣ts ♦♥ t♦ ♣r♦r♠ t str ♥②ss

Prt ♦s str♥ ♦r ♠① t

♣tr ♦♥ssts ♥ t stt ♦ t rt ♦ t ♠t♦s ♣r♦r♠♥ tstr ♥②ss ♦ ♠① t sts

♣tr ♣rs♥ts ♦r rst ♦♥trt♦♥ t♦ t str ♥②ss r♠♦r ♦ ♠① t sts t ♦♥t♥♦s ♥ t♦r rs ♠♦ s r r♦♠ t ♠t t♥t ss ♠♦ ♦♣ t♦ str t♦r t sts ♦r ts ♠♦ t ♦♠♣♦♥♥t strt♦♥s ♦t ♦♥t♥♦s rs r ss♥ ♥ t♦s ♦ t t♦r rs ♦♥t♦♥② ♦♥ t ♦♥t♥♦s ♦♥s r ♥r ♦st rrss♦♥s ♠♦ st♦♥ ♥ t ♣r♠tr st♠t♦♥ r s♠t♥♦s② ♣r♦r♠ ② ♠ ♦rt♠

♣tr ♣rs♥ts t ♠♥ ♦♥trt♦♥ ♦ ts tss t ♦♥ssts ♥ ♠①tr ♠♦ ♦ ss♥ ♦♣s s t ♠♦ ♣r♦r♠s t str ♥②ss ♦ t sts ♦♠♣♦s ♦ ♥② ♥ ♦ rs ♠tt♥ ♠t strt♦♥ ♥t♦♥ ts rsts r ♣rt ♦ t rt♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t ❬❱❪

♥ rt♦♥s ♥ ♥♦tt♦♥s

♥ rt♦♥s

♥r

♠♣ ♠①♠♠ ♣♦str♦r♠♣ ♠①♠♠ ♣♦str♦r st♠t♠ ♠①♠♠ ♦♦ st♠t ♠t strt♦♥ ♥t♦♥♣ ♣r♦t② strt♦♥ ♥t♦♥

♦rt♠s

♠ ①♣tt♦♥①♠③t♦♥ ♦rt♠♠ ♥r③ ①♣tt♦♥①♠③t♦♥ ♦rt♠

♠♠ r♦ ♥ ♦♥t r♦s♠ t♦st ①♣tt♦♥①♠③t♦♥ ♦rt♠

♥♦r♠t♦♥ rtr

♥♦r♠t♦♥ rtr♦♥ ②s♥ ♥♦r♠t♦♥ rtr♦♥ ♥trt ♦♠♣t♦♦

♥ ♥♦tt♦♥s

♥♦tt♦♥s ♥ ♥ t t ♦♦♥ rs

rs r ♥♦t t r ttrs t ♣r♠trs r ♥♦t t r ttrs

♠t♠♥s♦♥ ♦ts r ♥♦t ② ♦ s②♠♦s t ♥♠♥s♦♥ ♦ts r ♥♦t ② t♥ s②♠♦s

♥ rt♦♥s ♥ ♥♦tt♦♥s

❱rs ♥ ♦srt♦♥s

X i st ♦ t e r♥♦♠ rs rt t♦ ♥ ixi ♦sr s ♦ X i

x′i tr♥s♣♦s ♦ xi

x

i sst ♦ xi ♦♠♣♦s ♦ t c ♦♥t♥♦s rsx

i sst ♦ xi ♦♠♣♦s ♦ t d srt rsmj ♥♠r ♦ ♠♦ts ♦ r jZi r♥♦♠ r ♦ t ss ♠♠rs♣ ♦ t ♥ izi ♦sr s ♦ Zi

yi s♦♥ t♥t r rt t♦ ♥ i rqr① n s♠♣ ① = (x1, . . . ,xn)③ n s♠♣ ③ = (z1, . . . , zn)② n s♠♣ ② = (y1, . . . ,yn)

Pr♠trs

θ ♦ ♣r♠trs ♦ t ♠①trπ t♦r ♦ ♣r♦♣♦rt♦♥sαk ♣r♠trs rt t♦ ♦♠♣♦♥♥t kΓk ♠tr① ♦ t ♦rrt♦♥ rt t♦ ♦♠♣♦♥♥t kν ♥♠r ♦ ♣r♠trs

♠♣♦rt♥t ♥trs

c ♥♠r ♦ ♦♥t♥♦s rsd ♥♠r ♦ t♦r rse ♥♠r ♦ rs c+ d = eg ♥♠r ♦ sssn s③ ♦ t s♠♣nk s③ ♦ ss k ♦♠♣t ♦♥ t ③③② ♣rtt♦♥♥k s③ ♦ ss k ♦♠♣t ♦♥ t r ♣rtt♦♥

ss strt♦♥s

Dg(.) rt strt♦♥ ♦ s③ gG(.) ♠♠ strt♦♥

Nc(µ,Σ) crt ss♥ strt♦♥ t ♠♥ µ ♥ ♦r♥ ♠tr① Σ

Mg(.) ♠t♥♦♠ strt♦♥ ♦ s③ gP(.) P♦ss♦♥ strt♦♥

♥ rt♦♥s ♥ ♥♦tt♦♥s

ss t♦♦s

p(.; .) ♣r♦t② strt♦♥ ♥t♦♥P (.; .) ♠t strt♦♥ ♥t♦♥

φc(.;µ,Σ) ♣ ♦ Nc(µ,Σ)Φc(.;µ,Σ) ♦ Nc(µ,Σ)

Φ1(.) ♣ ♦ N1(0, 1)KL(f1, f2) r r♥ r♦♠ f1 t♦ f2 f2 rr♥

p(①;θ) ♦srt ♦♦L(θ;①) ♦srt ♦♦♦

p(①, ③;θ) ♦♠♣tt ♦♦L(θ;①, ③) ♦♠♣tt ♦♦♦

tik(θ) ♣r♦t② tt xi s r♥ ② ♦♠♣♦♥♥t k

♣tr

str ♥②ss stt ♦ t rt

♠♥ ♣r♣♦s ♦ ts ♣tr s t♦ r t trtr ♦t str ♥②ss ♦t tt ♦r ♠ s ♥♦tt♦ ①st ts ♣r♥♣② ♦s ♦♥ t ♠♦s ♣♣r♦s ♥ ♦rr t♦ ♥ t r♥t ♥♦t♦♥s♦♣ ♥ ts ♠♥sr♣trst② ♣rs♥t r♥t ♣♣r♦s ♦♠tr ♥♣r♦st t♦ str t t ♦♥② rt rq♥tst ♥ t ②s♥ ♣♣r♦s s t♦ ♥r t ♥t ♠①tr ♠♦s ♥② ♣rs♥t s♦♠rtr ♣r♦r♠♥ t ♠♦ st♦♥ ♥ ♣r♦st♦♥t①t♦ t♦② ①♠♣s strt t r♥t ♥♦t♦♥s ♥ t♦rt♠s tr♦ ts ♣tr ♥ ♦♥t♥♦s ss♥ t s t sst ♦♥

♦♦ st♦r② ② ♥ t♦ t♥s ♠st ♥ ♦t ♦ t

rrs ♦♥ ①♣r♥♦♥ t♥ ♦rt t

r ♦ t str♥ ♣♣r♦s

str♥ ♥

♦②s ♣rtt♦♥rs r ♦t♥ ♥ ♦♠♣① t sts tt ♥♦t ♥ts ♠♥sr♣t ② x = (x1, . . . ,xn) sr♥ n ♥s xi = (x1i , . . . , x

ei ) ② e

rs s ♦♠♣①t② s ♥r② ♥♦ ② t r ♥♠r ♦ ♥s♦r♠♥ t ♣rtt♦♥rs ♥r ♠ ♥♦r♠t♦♥s rtr♠♦r ts♦♠♣①t② ♥ ♥rs ② t sr♣t♦rs t♦ t ♥♠r ♦ rs ♦r t♦tr ♥tr ♦r ♥st♥ t♦r ♦r ♠① rs

str♥ s ♥r ♥sr t♦ ts ♣r♦♠ ♥rs♥② ♠rs tt ♦♠♣tr ♦♣♠♥t ♥ ts t♥q s♠♠r③s t t ② r♦♣♥

♣tr str ♥②ss stt ♦ t rt

t ♥s ♥t♦ g sss ♦r♥ t♦ ♦t ♦♦♥ ♣r♥♣s t ss ♦♠♦♥t② r♦♣♥ s♠r ♥s ♥t♦ t s♠ ss ♥ t ss s♣rt②t♦ ♥s rs♥ r♦♠ t♦ r♥t sss r str♦♥② r♥t ♥ t ①t ♥t♦♥ ♦ ss s s♣ t♦ t str♥ ♠t♦ st ② t♣rtt♦♥r t s ②s s ♦♥ ts t♦ ♣r♥♣s

♦r♥ t♦ ts ♣r♥♣s t str♥ ♠t♦s tr② t♦ tr♠♥ t t♥tt♦r z = (z1, . . . , zn) r t t♦r zi = (zi1, . . . , zig) ♥ts t ss ♠♠rs♣ ♦ t ♥ xi ② s♥ ♦♠♣t s♥t ♦♥ zik = 1 xi s t ♥t♦ ss k ♥ zik = 0 ♦trs ♦t tt sss t♦ ♥tr♣rt ♦r t s♣st ♦ t ♦♠♥ r t t ♦♠ r♦♠ ♥ str♥ ♠t♦ ♣r♦s ♥ ♥t s♠♠r② ♦ t t ♦♥② ts rst♥sss r ♠♥♥

ss ♠♠rs♣s ♦ t ♥s t♦ st♠t t t ♥♠r♦ sss s ♥r② ♥♥♦♥ s ♥ ♥t str♥ ♠t♦ ♣r♦s t♦♦st♦ ♣ t ♣rtt♦♥r ♦r t st♦♥ ♦ t ♥♠r ♦ sss str♥s tr ♠♥ ♦s t♦ st♠t t ♣rtt♦♥ t♦ ♣r♦ ♠♥♥ sss ♥t♦ st t♦♠t② t ♥♠r ♦ sss

♦r t r t sts n ♥ e r ♣rtt♦♥rs ♥ s♠t♥♦s② strt ♥s ♥ t rs ♦ ♣rtt♦♥s ♥ ts sr ♦♥ ♠♦♥t ♥s ♥ ♦♥ ♠♦♥ t rs s ♣♣r♦ s ♥♠ ♦str♥❬ ❪ t t s ♥♦t ♦♣ ♥ ts tss r ♦♥② st② t str♥♣r♦♠

♠t♦s ♣r♦r♠♥ t str ♥②ss r ♥t♦ t♦ r ♠st ♦♠tr ♠t♦s s ♦♥ s♦♠ st♥s t♥ ♥s ♥ t ♣r♦st ♠t♦s ♠♦③♥ t t ♥rt♦♥ ♥ ts st♦♥ ♦t ♣♣r♦sr t ♥ ♥r r♠♦r rtss t② r strt ♦♥ rt♦♥t♥♦s t st ♣rs♥t ♦ s♥ t ♦s t♦ s② s③ t rsts♦ s♣ stts ♦ t rt rt t♦ ♠♦r ♦♠♣① stt♦♥s t♦r ♥♠① t sts r ♥ tr ♥ t ♥tr♦t♦♥s ♦ Prt ♥ Prt

t t st ❬❪ s ♦♥ t ♣ ♠ss st st ♦♥t♥s t t♥ t♠ t♥ r♣t♦♥s ♥ t rt♦♥♦ t r♣t♦♥s ♦r t t ②sr ♥ ❨♦st♦♥ t♦♥ Pr❲②♦♠♥ ❯ s♣② ② r ♠ s t♦ ♣r♦ ♠♥♥ s♠♠r② ♦ t t st x = (x1, . . . ,xn) r ♥ xi ∈ R

2s ♥ ts ①♠♣ n = 272 ♥ e = 2

t t st t ♣rs♥tt♦♥


1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

5060

7080

90

eruptions

wai

ting

r t t st

♦♠tr ♣♣r♦s

♥rts

♥ ♦♠tr ♣♣r♦s r♦♣ t st ♦ t str♥ ♠t♦ss ♦♥ t st♥ ♠sr♠♥t t♥ ♥s ♥ ♦rr t♦ ss♥ t ♠♦sts♠r ♥s ♥t♦ t s♠ ss ② ①rt♥ t ss ♦♠♦♥t② ♦♥♣t ♥ ♥trt ♦t ♦ t ♦♦♥ t♦ ♥s rs♥ r♦♠t s♠ ss t♦ ♠♦r s♠r t♥ t♦ ♥s ♥ r♥t ss♠♠rs♣ s ♠ ♥ ①♣rss ② t ♦♦♥ ♠t♠t rt♦♥♦r st♥ D(., .) ♥ ♦r (i0, i1, i2, i3) s tt zi0 = zi1 ♥ zi2 6= zi3 :

D(xi0 ,xi1) ≤ D(xi2 ,xi3).

Pr ♣r♦♠ ♥ s rt♦♥ s t♦ sts ② t qr♣ts ts ♦t ♥r② s t♦ ♠♣t② st s♦t♦♥ ♥♠r ♦ ♦♥str♥ts rtr♠♦r t s ♥♦t r③ t♦ ♣r♦r♠ ♥ ①st ♣♣r♦ ♦♠♣ts t ♦t rtr♦♥ ♦r t ♣♦ss ♣rtt♦♥s ♥ ♦r s♠♣♦ s③ n = 40 tt str ♥ g = 3 sss t ♥♠r ♦ t ♣♦ss ♣rtt♦♥ss r♦② q t♦ 2.1018 s ♦♠♣tr ♣r♦r♠♥ 109 ♣rtt♦♥s ♣r s♦♥♥s 64000 ②rs t♦ t t ♣♦ssts

♦ rtr♦♥ ♥ ♣rt t s s t♦ ♦♣t♠③ ♦ rtr♦♥ rt♥t ss ♦♠♦♥t② r♥t rtr ♦t♥ s ♦♥ rst s ♥s♦ ♣r♦♣♦s s t r♥♥♥ ①♠♣ s rtr r s② ♦♣t♠③ ② ♥♦rt♠ ♦♥ ♥ ①st ♣♣r♦ s ♥trt

trtr ♦ ts st♦♥ rst② ♣rs♥t t tr ♠♦st ♦♠♠♦♥ rtr♦ ♥trst ♥ t rs r ♦♥t♥♦s ♦♥② t t ♠♥s


♦rt♠ s t ♠♦st ss ♦♠tr ♣♣r♦ t♦ str ♥s sr ② ♦♥t♥♦s rs ❬r❪ ♥② ts ♦rt♠ s strt ♦♥ tt t st

t s ♥tr♦ t ♠tr① ♦ t ♦ s♠♣ ♦r♥ ♥♦t ② T

♥ ♥ s

T =1

n

n∑

i=1

(xi − x)(xi − x)′,

r x = 1n

∑ni=1 xi s t ♠♥ t♦r ♦ t ♦ s♠♣ s ♠tr①

♥ rtt♥ s s♠ ♦ t♦ ♠trs

T = W +B,

r t ♥trss ♦r♥ ♠tr① W ♥ r t ♥trss ♦r♥ ♠tr① B r ♥ ②

W =1

n

g∑

k=1

n∑

i=1

(xi − xk)(xi − xk)′ ♥ B =

1

n

g∑

k=1

♥k(xk − x)(xk − x)′,

xk = 1

♥k

∑ni=1 zikxi s t ♠♥ t♦r ♥ ♥k =

∑ni=1 zik t s③ ♦ ss

k s ♠trs rtr③ ♦t ♦♥strt♦♥s ♣r♥♣s ♦ t sss♥ sss r ♦♠♦♥♦s t♥ t st♥s t♥ t ♥s ss♥ t♦ ss ♥ ts ♥tr r s♠ s♦ W s s♠ sss r s♣rt t♥ t ♥trs ♦ t sss r ♠t② t♥② s♦ B s rs t ♣rtt♦♥r ♥ str t t t st ② ♦♣t♠③♥ r♥t rtr rt t♦ ts ♠trs ♠♦♥ t♠ t ♠♦st ssr t ♦♦♥ min tr(W ) min det(W ) ♦r max tr(BW−1) ❲rr t♦ t ♦♦ str ♥②ss ② rtt ♥ s ♥ t ❬❪ ♦r ♠♦r ts

t t st ♣t♠③ rtr ♦r ♦♥t♥♦s rs

♠♥s ♦rt♠

♥ ♦② ♣r♦♣♦s t ♠♥s ♦rt♠ r♦♥ ♥ t t♦ ♣s t ❬♦❪ ss♦t t♦ st♥ D(., .) ts ♦rt♠ ♠s t♠♥♠③♥ t ♦♦♥ ♥rt

I(z,θ;x) =n∑

i=1

g∑

k=1

zikD2(xi,µk),

r θ = (µ1, . . . ,µk) ♥ r µk s t ♥tr ♦ t ss k trt♥ r♦♠♥ ♥t ♦ t ss ♥trs t ♠♥s ♦rt♠ tr♥ts t♥ t♦st♣s t ss♥♠♥t ♦ ♥ t♦ t ss ♠♥♠③♥ t st♥ t♥♠ ♥ t ss ♥tr ♥ t ♦♠♣tt♦♥ ♦ t ss ♥trs


trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s ♦♦s ss ♠♠rs♣ z

[r] = argminz

I(z,θ[r];x)

z[r]ik =

1 k = argmink′

D2(xi,µ[r]k′ )

0 ♦trs

♥tr♦ st♠t♦♥ θ[r+1] = argminθ

I(z[r],θ;x)

µ[r+1]k =

1

♥[r]k

n∑

i=1

z[r]ik xi,

r ♥[r]k =

∑ni=1 z

[r]ik

♦rt♠ ♠♥s ♦rt♠

s ts ♦rt♠ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ I(z,θ;x) t s ♠♥t♦r②t♦ ♣r♦r♠ r♥t ♥t③t♦♥s ♥ t♦ ♣ t ♦♣ (z,θ) ♠♥♠③♥ t♦t ♥rt

♠r ♠♥s ♦rt♠ ♥ ♦♣t♠③ rtr♦♥ t ♠♥s ♦rt♠ strs ♦♥t♥♦s t ② s♥ t ♥ st♥ t♥ t ♦♣t♠③st rtr♦♥ min tr(W )

①t♥s♦♥s ♦ t ♠♥s ♦rt♠ ♦♠ ♣♣r♦s tt♠♣t t♦ rt rs ♦ t ♠♥s ♦rt♠ ♦r ♥st♥ t ♠♥s ♦rt♠❬❱❪ ①t♥s t ss ♦♥ ② r♥♦♠③ s♥ t♥q ♠♣r♦♥ ts♣ ♥ t r② ♦ t ♠♥s

♦ ♠♥② sss st♦♥ ♦ t ♥♠r ♦ sss ♥ ♥♦t rt②♣r♦r♠ ② t ♥rt rtr♦♥ ♥ ♥ s♥ ts ttr s rs♥ tt ♥♠r ♦ sss g ♦r t ♦t rtr♦♥ rs ♣t ♥g ♥rss ♥ ♥ ts ♣t s r t sss r ♥♦ ♠♦r♦♠♦♥♦s t ss ♦r♣♣♥ ♥rss rst rtr♦♥ ♦♥ssts ♥st♥ t rst ♥♠r ♦ sss ♦ ts ♣t t t s r tt ts ♥♦ rtr♦♥ s ♥♦t r② r♦r♦s ♥ ♣rt t rtr♦♥ ♥ ♥♣ ♥s♦♠ ♣ts r ♦sr tr rtr r s ♦r ♥st♥ ❬r❪t t② r s ♦♥ rst ♣♣r♦


❲ s t ♠♥s ♦rt♠ t♦ str t t t st ♦r♥t♦ r r♥ t ♦t♦♥ ♦ t ♥rt ♦r r♥t ♥♠rs ♦sss r♦♠ ♦♥ t♦ t ♥ st t♦ sss ♣rtt♦♥ ♥t ss ♥trs r s♣② ② r t t sts s♦ s♠♠r③ ② t♦ ♣r♦s ♦ r♣t♦♥s t r♣t♦♥s t s♦rtt♥ t♠ ♥ rt♦♥ ♥tr t (2.09, 54.75) r s♣② t rs ♥ t r♣t♦♥s t rr t♥ t♠ ♥ rt♦♥ ♥tr t(4.30, 80.28) r s♣② t r tr♥s

t t st ♠♥s ♦rt♠ ♣♣r♦

1 2 3 4 5 6 7 8number of classes

iner

tia0

1000

020

000

3000

040

000

5000

0

♥rt

1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

5060

7080

90

eruptions

wai

ting

ttr ♣♦t ♥ ss ♥trs

r t♣ts ♦ t t str ♥②ss ♣r♦r♠ ② ♠♥s ♦rt♠ sttr ♣♦t ♥ts t ♣rtt♦♥ ② t ♦♦rs ♥ t t♥ s②♠♦s t ss ♥trs r r♣rs♥t ② t ♦ s②♠♦s ♥ t ♦♦r ♦ tr ss

♠ts ♦ t ♦♠tr ♣♣r♦s

r♥t t♦rs ♣♦♥t ♦t t rs ♥r♥t t♦ t ♦♠tr ♣♣r♦ss ♦r ♥st♥ t ♦♦ t ♥②ss ♣tr ② ♦rt ❬♦❪ ♦st tr ♠♥ rs r ♣rs♥t r

♦ st♦♥ ♣r♦r♠ ② rst ♣♣r♦s ♥ t ♦♠tr ♣♣r♦s r ♣♦ss ♥sr t♦ t str♥ ♥ ♠♥② t♦rt♣r♦♠s ♠② rs ❬♦❪ ② ♥r② st t ♥♠r ♦ sss ② s♥ rst ♣♣r♦s t s♦♣ ♦ t rtr♦♥ s rtr♠♦r ♦t♦s ♦ t ♠tr ♥ t rtr♦♥ s r ♠♣♦rt♥t s♣ts ♦ ts ♠t♦s s♥ t② ♥♦ ♠♥② ♥ ss♠♣t♦♥s r ♥r② ♥♦r ② t♣rtt♦♥rs ♦r tr ♠♣t s r s t② ♥♦ r♥t ♣rtt♦♥s


①t♥s♦♥ ♦ t ♦♥s♦♥s t♦ t ♦ ♣♦♣t♦♥ t ♦♥s♦♥s ♦ str ♥②ss s ♦♥ s♠♣ t♦ ①t♥ t♦ t ♦ ♣♦♣t♦♥t s ♠♥t♦r② t♦ ♥rst♥ s♦ t♦ ♠♦③ t t ♥rt♦♥ ①t♥s♦♥♦ t ♦♥s♦♥s ♦t♥ ② ♦♠tr ♠t♦ r ♥♦t ♦ ♥ s st ♣r♦st r♠♦r s s♦ ♠♥t♦r②

♥ t ♠ss♥ s ♦♠tr ♣♣r♦s ♥♥♦t rt② ♠♥t sts t ♠ss♥ s ♥ t② t♦ ♣r♦r♠ ♥ rtrr② ♠♣tt♦♥ ♦r t② t♦ ♥♦r ♥s t ♠ss♥ s t ♣r♦st♣♣r♦s r t♦ r♦r♦s② ♠♥ s t

♥s t♥ ♦♠tr ♥ ♥rt ♣♣r♦s

♥② ♦♠tr ♣♣r♦s ♥ ♥tr♣rt s ♣r♦st ♦♥s r♥tr ♣r♦st ♥ ss♠♣t♦♥s s♦♠ ①♠♣s r ♥ ♥ ts tss ♥♦rr t♦ ♣r♦st t♦♦s ♥ t♦ r t ss♠♣t♦♥s ♠ ② t str♥♠t♦s ♦♣ ts tss ♥ ♣r♦st r♠♦r

♥rt ♣♣r♦s

♥rts

♥ ♠①tr ♠♦s r ♥tr t♦♦s t♦ str t t ② ♣♣r♦♥ tr strt♦♥s s tr ♣r♦st r♠♦r ①♣♥s t t ♥rt♦♥ ♥ ts ♦♥t①t t ♥♦t♦♥ ♦ ss ♦♠♦♥t② s ♥ ② t ♦♦♥ t ♥s ♦ t s♠ ss rs r♦♠ t s♠ ♣r♦t② strt♦♥ ts strt♦♥ s ♥r② ss♠ t♦ ♥♠♦ ts ss♠♣t♦♥ ♥ r① ♦r ♥st♥ t ♦♠♣♦♥♥t strt♦♥ ♥ ts ♠①tr ♦ ♣r♠trstrt♦♥s t♦ ♥rs t ♠♦ ①t② ❬+❪

t♥t r ♥ ss ♠♠rs♣ ss ♠♠rs♣ ♦ t ♥ i s qtt r♥♦♠ r ♥ g ♠♦ts ♥ ♥♦t ② Zi =(Zi1, . . . , Zig) ② s♥ s♥t ♦♥ s t ss ♠♠rs♣ ♦♦s ♠t♥♦♠ strt♦♥

Zi ∼ Mg(π1, . . . , πg),

r πk ♥♦ts t ♣r♦♣♦rt♦♥ ♦ ss k s♦ ♥tr♣rt s t ♣r♦t② ♣r♦r tt ♥ ♥ rss r♦♠ ss k ss ♣r♦♣♦rt♦♥ πk rs♣ts♦t ♦♦♥ ♦♥str♥ts 0 < πk ≤ 1 ♥

∑gk=1 πk = 1 ♦t tt t str♥

♥ s t♦ st♠t t ♦ t r③t♦♥ zi ♦ t t♥t r Zi

♦♥t♦♥② ♦♥ t ♦sr t xi

sr rs ss k s rtr③ ② t strt♦♥ ♦ t ertr♥♦♠ r X i = (X1

i , . . . , Xei ) ♥ ♦♥ t s♣ X ♦♥t♦♥② ♦♥ t

r③t♦♥ zi ♦ t r♥♦♠ r Zi s strt♦♥ s ♥♦t ② pk(xi)r k s s tt zik = 1 ♥

X i|Zi = zi ∼ pk:zik=1(xi).


strt♦♥ ♦ ♦t ♦sr ♥ t♥t rs ② s♥ ts ♣r♦t②♦♠♣♦st♦♥ P (X i,Zi) = P (Zi)P (X i|Zi) t ♣r♦t② strt♦♥ ♥t♦♥♣ ♦ (xi, zi) ♥♦t ② t ♥r ♥♦tt♦♥ p(.) s ♥ s ♦♦s

p(xi, zi) =

g∏

k=1

(πkpk(xi))zik .

♥ ts ♠♦ s s t♦ str t s zi r ♦♥sr s ♠ss♥ ss ♦t♥ ♦t ♥t♦♥s ♦ t ♥t ♠①tr ♠♦ ♥ ts ♥rt♠♦ ② s♠♠♥ t ♣r♦s qt♦♥ ♦r t ♣♦ss s ♦ Zi

♥t♦♥ ♥t ♠①tr ♠♦ ♥t ♠①tr ♠♦ t g ♦♠♣♦♥♥ts ♥s t ♠r♥ strt♦♥ ♦ t r♥♦♠ r X i ts ♣ s rtt♥s

p(xi) =

g∑

k=1

πkpk(xi).

♥rt ♠♦ s♠♣♥ r♦♠ t ♠①tr ♠♦ ♥ ② s♣r♦r♠ ② t ♦♦♥ ♥rt ♠♦ ♥t♦ t♦ st♣s

t♣ t ss ♠♠rs♣ s♠♣♥ Zi ∼ Mg(π1, . . . , πg t♣ t ♦♥t♦♥ t s♠♣♥ X i|Zi = zi ∼ pk:zik=1(xi)

sst♦♥ r

③③② ♥ r ♣rtt♦♥ ❲♥ t t strt♦♥ p(xi) s ♥♦♥ t ♥t♦♥ ♦ Zi|X i = xi s strt♦rr

Zi|X i = xi ∼ Mg(ti1, . . . , tig),

r tik s t ♦♥t♦♥ ♣r♦t② tt xi s r♥ r♦♠ ♦♠♣♦♥♥t k s♥ ②

tik =P (Zik = 1,X i = xi)

P (X i = xi)=πkpk(xi)

p(xi).

❱t♦r ti = (ti1, . . . , tig) s♦ ♥s ③③② ♣rtt♦♥ ♥ s t♦ ♦♠♣tt rs ss♦t t♦ t r ♣rtt♦♥ zi

rr♦r rs ♥ sst♦♥ r r♦♠ ts ③③② ♣rtt♦♥ ♥ ♥ tsst♦♥ rr♦r e(.) ss♦t t♦ (zi, ti) ②

e(zi, ti) = 1−g∑

k=1

(tik)zik .

♠①♠♠ ♣♦str♦r r ♠♣ ♠♥♠③s t sst♦♥ rr♦r ② ss♥♥♥ ♥ ♥t♦ t ss ♥ t rst ♣r♦t② s t ♥s tsst♦♥ r r(.) s ♦♦s

∀xi ∈ X , r(xi) = k ∀k′ tik ≥ tik′ .

t♦♥ ♦ t rs ♦ t sst♦♥ rr♦r s rt ♥t ♦ t ♣r♦st ♠t♦s s♥ t ♦♠tr ♦♥s ♥♥♦t q♥t② t rr♦r rs ss♦tt♦ tr sst♦♥ r

♥rts ♦♥ ♥t ♠①tr ♠♦s

♥rts ♦♥ ♥t ♠①tr ♠♦s

♥ s ♠♦s ss♠ tt t ♦sr ♥s r ♥♣♥♥t②r♥ r♦♠ t s♠ strt♦♥ ❲ ♥♦ q② sr t s♠♣r♠tr♠①tr ♠♦s ♠ ss♠♣t♦♥s ♦♥ t ♦♠♣♦♥♥t strt♦♥s ♥ sr t ♣r♠tr ♠①tr ♠♦s ss♠ tt t ♦♠♣♦♥♥tstrt♦♥s r ♣r♠tr ♦♥s ♦t tt t ♠♦ sr♣t♦♥ ♣rs♥t rs s ♦♥ ♥t ♠①tr ♠♦s ② ♥ ♥ P ❬P❪ ♥ tstss ♦s ♦♥ t ♣r♠tr ♠①tr ♠♦s s♥ t② ♣r♠t ♥ sr♥tr♣rtt♦♥ ♦ t sss

♠♣r♠tr ♠①tr ♠♦s

♦♥str♥ts ♦♥ t ♦♠♣♦♥♥t strt♦♥s s♠♣r♠tr ♣♣r♦s ♦ ♥♦t ss♠ tt t ♦♠♣♦♥♥ts ♦♦ ♣r♠tr strt♦♥s ♦r ♦r rs♦♥s ♦ ♥tt② s♦♠ ♦♥str♥ts t♦ ♠♣♦s ♦r t ♦♠♣♦♥♥ts ♦r ♥st♥ strt♦♥s t♦ ♦♥ t♦ t ♠② ♦ ♥♠♦strt♦♥s ♦r s②♠♠tr strt♦♥s ❬❲❪

♥r♥ st♠t♦♥ ♦ t ♦♠♣♦♥♥t strt♦♥s ♥ ♣r♦r♠ ②♦rt♠s ♥s♣r r♦♠ t ♠ ♦rt♠ ❬❪ ss r♥ ♣♣r♦s❬❪ ♣ ♠①t♦♦s ❬❪ ♦s s t♦ str t t ② s♥ s♠♣r♠tr ♠①tr ♠♦

♦♥♥tt② rs s♠♣r♠tr ♠①tr ♠♦s r r② ①s♦ t② ♥ s② t t t strt♦♥ ♦r ts ①t② ♥♦s ♥♠♣♦rt♥t rs ♦ ♥♦♥♥tt② ♥ r r♥ ♦ t st♠t ♠♦rtr♠♦r t ss ♥tr♣rtt♦♥ ♥ t s♥ t ♦♠♣♦♥♥ts ♥ ♥♦t s♠♠r③ ② ♣r♠trs s ♥ t ♣r♠tr ♠①tr ♠♦s ♦♥ ts tss ♦♥② st② t ♣r♠tr ♠①tr ♠♦s ♦r t ♥r♣r♦♣rts r ♥♦ ♦♣

♣r♠tr ♠①tr ♠♦s

♥rts

♥ s ♠♦s ♠ t s♣♣♠♥tr② ss♠♣t♦♥ tt ♦♠♣♦♥♥t ♦♦s ♣r♠tr strt♦♥ s♦ pk(xi) = p(xi;αk) r αk r♦♣s t♣r♠trs ♦ ♦♠♣♦♥♥t k ♥s r s♦ r♥ ② ♣r♠tr strt♦♥ p(xi) = p(xi;θ) r θ = (π,α) ♥♦ts t ♦ ♣r♠tr rπ = (π1, . . . , πg) s t t♦r ♦ t ss ♣r♦♣♦rt♦♥s ♥ r α = (α1, . . . ,αg)r♦♣s t ♣r♠trs ♦ t ♦♠♣♦♥♥ts

♥t♦♥ ♥t ♣r♠tr ♠①tr ♠♦ ♣ ♦ t ♥t ♣r♠tr♠①tr ♠♦s t g ♦♠♣♦♥♥ts s ♥ ②

p(xi;θ) =

g∑

k=1

πkp(xi;αk).


♥tr♣rtt♦♥ t ♣r♠trs s ♠♦s r ♠♦r ♠♥♥ t♥ ts♠♣r♠tr ♣♣r♦s s♥ ss ♥ s♠♠r③ ② ts ♣r♦♣♦rt♦♥ πk♥ t ♣r♠trs ♦ ts strt♦♥αk ♣r♦ts ♦ t ss ♠♠rs♣stik r s♦ ♣r♠tr③ ② θ s♦ t② r ♥♦ ♥♦t ② tik(θ) t

tik(θ) =πkp(xi;αk)

∑gk′=1 πk′p(xi;αk′)

.

♦♠♣♦♥♥ts tr ♦ t♥ r♥ ♥ ♥♠r s ♣♣r♦ s♥♦t r② rstrt s♥ ♠①tr ♦ ♣r♠tr strt♦♥s ♥ ♣♣r♦ ♥②strt♦♥ t ♥② ♣rs♦♥ t strt♦♥s ♦♠♣♦♥♥t strt♦♥s ♥♣♣r♦ strt♦♥ t s♠ s♣♣♦rt st ② ♥rs♥ t ♥♠r ♦♦♠♣♦♥♥ts ♥ t s♠♣ s③ s t ♦♦♥ ①♠♣ s ♠①tr ♦st♥r strt♦♥s ♥ ♠♦③ ② ♦♠♣① strt♦♥s ♦r t♠①tr s ♠♦r ♠♥♥ ♥ t ♥♠r ♦ sss st②s s♠ rtr♠♦rs t ♠①tr ♠♦ s st♠t ♦♥ ♥t s♠♣ t ♣rtt♦♥r srs t♠♦ ♣r♦r♠♥ t st tr ♦ t♥ ts s t t tr ♠♦ ♥ts r♥ s ② t tt♦♥s ♦ t s♠♣♥ r♦r t s ♠♣♦rt♥ttt t ♦♠♣♦♥♥ts ♦♦ st♥r strt♦♥s r ♣t t♦ t t

①♠♣ ♥st② st♠t♦♥ ♥ Pr③♥♦s♥tt st♠t♦r t x t♦ t s♠♣ ♦ s③ n r ♥ xi ∈ R s ♥♣♥♥t② r♥ ② tstrt♦♥ rtr③ ② ts ♣ f(x) Pr③♥♦s♥tt st♠t♦r ❬Pr❪s ♥ s

p(y;θ) =n∑

i=1

1

nhK

(

y − xih

)

.

♦t tt ♥s ♥t ♠①tr ♠♦ t n ♦♠♣♦♥♥ts r ♣r♦♣♦rt♦♥ s q t♦ 1/(nh) ♥ r t strt♦♥ ♦ ♦♠♣♦♥♥t i s ♣r♠tr③② ♦♥ xi ♦r i = 1, . . . , n ❯♥r s♦♠ rrt② ♦♥t♦♥s ♦♥ t ♥t♦♥ K(.)♥ ♥r s♦♠ rt♦♥s t♥ t s♠♣ s③ n ♥ t ♥t h t ♣p(y;θ) ♦♥rs t♦ t tr ♣ f(x) s ♦r ♥st♥ ❬❩❪ ♦r ♠♦r tsr strts t Pr③♥♦s♥tt st♠t♦r ♣♣r♦♥ ♣ ② s♥t ♥♦r♠ r♥ ♦r t ss♥ ♦♥

K♥♦r♠(y) =

1/2 |y| < 10 ♦trs

♥ Kss♥(y) =1√2πe−

12y2 .

①tr ♥ ♥ ♦ rs ♣r♠tr ♠①tr ♠♦s ♥ str♥s ② ♣♣r♦♥ t strt♦♥ ♦ t rs ♥ tr ♥t s♣♦s② t strt♦♥s ♦ t ♦♠♣♦♥♥ts t♦ rs♣t t ♥tr ♦ trs s t ♠①tr ♠♦s r s t♦ ♥②③ r♥t ♥s ♦ t sts♦r ♥st♥ t ♠①tr ♦ P♦ss♦♥ strt♦♥s ❬❪ ♥ str ♥tr t t ♠①trs ♦ t♥t strt♦♥s ❬P❪ ♥ str t ♦♥t♥♦s ♦♥st t ♠①tr ♠♦s r s♦ s t♦ str ♥t♦rs ❬ ❲❪ r♥t ❬❪ ♦r ♥t♦♥ t ❬P❪ s ♦ t♦r t s ♦♣ ♥Prt Prt s ♦s ♦♥ t str♥ ♦ ♠① t ♦r t ♠♦st♦♠♠♦♥ ♠①tr ♠♦ s t ss♥ ♦♥ tt t ♥♦

♥rts ♦♥ ♥t ♠①tr ♠♦s

0 5 10 15

0.00

0.05

0.10

0.15

0.20

0.25

0.30

n = 3

0 5 10 15

0.00

0.05

0.10

0.15

0.20

0.25

0.30

n = 25

0 5 10 15

0.00

0.05

0.10

0.15

0.20

0.25

0.30

n = 100

0 5 10 15

0.00

0.05

0.10

0.15

0.20

0.25

0.30

n = 1000

r ♣ ♦ t tr strt♦♥ ♦tt r ♥ ts st♠ts♦t♥ ② t ♥♦r♠ r♥ t♥ r ♥ ② t ss♥ r♥ ♦r r r t ♥t s h = lnn

ss♥ ♠①tr ♠♦

♥ ss♥ ♠①tr ♠♦ s ♥tr♦ s♠t♥♦s② t♦ t♣r♠tr ♠①tr ♠♦ ♥ ♦rr t♦ str t Prs♦♥s r t st ❬P❪t s ♣♦r t♦♦ t♦ str ♦♥t♥♦s t ♦s sss s t♦ t♦ ♠♥rs♦♥s ♥ t ♦♥ ♥ ts ♣t ♥t♦♥ ♦ ss s ♥ ♦r♥ tt ♥tr ♥t♦♥ ♦ ss ♥ t ♦tr ♥ ts ♦♠♣tt♦♥ trtt②♣r♠ts ♥ s② ♥r♥ ss♥ ♠①tr ♠♦ ss♠s tt r♥♦♠r X i|Zik = 1 s ♥ ert ss♥ r ♦s t ♠♥ t♦r s♥♦t ② µk ♥ ♦s t ♦r♥ ♠tr① s ♥♦t ② Σk s♦

X i|Zik = 1 ∼ Ne(µk,Σk).

s ♦t♥ t ♦♦♥ ♥t♦♥ ♦ t ss♥ ♠①tr ♠♦

♥t♦♥ ss♥ ♠①tr ♠♦ t xi ∈ Re t ♦♥t♥♦s r

rs♥ r♦♠ ss♥ ♠①tr ♠♦ t g ♦♠♣♦♥♥ts ts ♣ s rtt♥ s


♦♦s

p(xi;θ) =

g∑

k=1

πkp(xi;αk) t p(xi;αk) = φe(xi;µk,Σk),

r φe(xi;µk,Σk) =1

(2π)e/2|Σk|1/2exp

(

−12(xi − µk)

′Σ

−1k (xi − µk)

)

♥ rαk =

(µk,Σk)

ss ♥tr♣rtt♦♥ ss♥ ♠①tr ♠♦ ♣r♦s s♠♠r② ♦ ss tr♦♦t ts ♥tr ♣♦st♦♥ µk ♥ ts s♣rs♦♥ ♠tr① Σk rt♥ ts♣♥♥s t♥ t ♣rs ♦ rs

Prs♠♦♥♦s ♠♦s ❲♥ t s♠♣s r s♠ t ♥♦r♠t♦♥ ♦t t♥trss ♣♥♥s t♥ rs s ♥♦t ♣rs♥t ♥ t t ♥ ♥rt sr♥ tr ♦ ♠② ttr ♦♥str♥ts ♦♥ t ♣r♠tr s♣ r rst♥ ♠♦s r ♣rs♠♦♥♦s ♠♦s ♦r ♥st♥ s ♦♥t s♣tr ♦♠♣♦st♦♥ ♦ Σk ♣r♦♣♦s ♥ ❬❪ ♦rt♥ ♣rs♠♦♥♦s ♠♦sr t ❬❪ s ts ♠♦s r s♥st t♦ t ♥t ♦ ♠sr♠♥t ♦ trs ♥ ♠② ♦ ss♥ ♠①tr ♠♦ ♥♠ rt s ♣r♦♣♦s ② r♥ ♥ ♦r♠ ❬❪ ♥② ♥♦t tt t s♣tr ♦♠♣♦st♦♥♦ Σk ♥ s t♦ str ♠♥s♦♥ t ❬❪

♦trs ♥② s♦trs ♣r♦r♠♥ t st♠t♦♥ ♦ t ss♥ ♠①tr♠♦s r r ♠♣t ♥ t s♦♥ ♦ ts ♠♦s s s♠♦♥ t♠ ♦♥ ♥ t t tr ♦♦rs st ❬❪ ①♠♦ ❬+❪♥ ①t♦♦ ❬❨❪

s ♦t rs ♦ t t st r ♦♥t♥♦s st♠t ♦♠♣♦♥♥t ss♥ ♠①tr ♠♦ st♦r♠s ♦ ♦t rs r s♣② ♥ r ♥ t st♠t ♠r♥ ♣ ♦ ♦t ♦♠♣♦♥♥ts r s♣r♠♣♦s ♠♦ s♠♠r③s t t st s ♦♦s

♠♦rt② ss π1 = 0.64 s t ss ♦ t str♦♥ r♣t♦♥ss♥ µk = (4.29, 80.00)

♠♥♦rt② ss π2 = 0.36 s t ss ♦ t r♣t♦♥ss♥ µk = (2.04, 54.51)

ss ♦ t str♦♥ r♣t♦♥s s ♠♦r s♣rs t♥ t ss ♦ t r♣t♦♥s s♥ t r♥ ♦ ♦t rs r rr rs♣t②(0.15, 40.90) ♥ (0.10, 57.68) ♥ t rs r ♣♦st② ♦rrt ♥ ♦t sss tr ♣♥♥② str♥t s rr ♥ t ♠♥♦rt②ss t♥ ♥ t ♠♦rt② ♦♥ ♥ t ♦♥t ♦ ♦rrt♦♥ s qt♦ 0.50 ♥ t ss ♦ t r♣t♦♥s t s q t♦ 0.23 ♥ tss ♦ t str♦♥ r♣t♦♥s

t t st ss♥ ♠①tr ♠♦ str♥

♥rts ♦♥ ♥t ♠①tr ♠♦s

Den

sity

1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

0.0

0.2

0.4

0.6

0.8

r♣t♦♥s

Den

sity

40 50 60 70 80 90

0.00

0.01

0.02

0.03

0.04

0.05

❲t♥

r st♦r♠s ♥ ♠r♥ ♥sts ♦ t ♦♠♣♦♥♥t ♠①tr ♠♦♦r t t st

❲ r♠r tt t s♠♠r② ♣r♦ ② t ss♥ ♠①tr ♠♦ s♠♦r ♣rs t♥ t s♠r② ♦t♥ ② t ♠♥s ♦rt♠ ♥ t ♠♥s ♦rt♠ ♦s ♥♦t ♦♥sr t ss ♣r♦♣♦rt♦♥s s♦ t♠♣t② ss♠s tt ♦t ♥s ♦ r♣t♦♥s r q♣r♦ rtr♠♦r t ss♥ ♠①tr ♠♦ ♣r♦s ♥ ♥②ss ♦ t ♥trss♣♥♥s s s♣② ② r rs t sttr ♣♦t ♦ t♣rtt♦♥ ♥ t ♣ss ♦ q♣r♦t②

t t st ♦♠♣rs♦♥ t♥ ♦t str♥ rsts

①tr ♠♦ t ♦♥t♦♥ ♥♣♥♥ ss♠♣

t♦♥

♥ ♦♥t♦♥ ♥♣♥♥ ♠♦ ♠ s♦ ♥♦♥ s ♥ ②s♦r t♥t ss ♠♦ s ♠①tr ♠♦ ss♠♥ t ♦♥t♦♥ ♥♣♥♥t♥ rs ♦ t ♦♥t♦♥ ♣r♦t② ♦ t ert r♥♦♠ rX i = (X1

i , . . . , Xei ) s rtt♥ s

P (X i|Zik = 1) =d∏

j=1

P (Xji |Zik = 1).

♦s② ts ss♠♣t♦♥ s r t♥ t ♦ ♥♣♥♥ ss♠♣t♦♥♥ t ♣♥♥② t♥ t rs s ♠♦③ ② t strtr ♥ sss♦ t strt♦♥

♥t♦♥ ♠ ♠♠♦ s ♠①tr ♠♦ ss♠♥ t ♦♥t♦♥♥♣♥♥ t♥ t rs s t ♣ ♦ t ♥ xi rs♥ r♦♠


1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

5060

7080

90

eruptions

wai

ting

r t♣ts ♦ t t str ♥②ss ♣r♦r♠ ② ♦♠♣♦♥♥tss♥ ♠①tr ♠♦ sttr ♣♦t ♥ts t ♣rtt♦♥ ② t ♦♦rs ♥t t♥ s②♠♦s t ♥trss ♣♥♥s r ♣t ② t ♣ss ♦q♣r♦t② ♦ t ss♥ ♦♠♣♦♥♥ts t ♥s ♦♥♥ t♦ t ss♦ t str♦♥ r♣t♦♥s r s♣② t r tr♥s ♥ t♦s ♦♥♥ t♦ tss ♦ t r♣t♦♥s r s♣② t rs

t ♠ ♠♦ s rtt♥ s

p(xi;θ) =

g∑

k=1

πk

d∏

j=1

p(xji ;αkj),

r αkj ♥♦ts t ♠r♥ ♣r♠trs rt t♦ r j ♦r ♦♠♣♦♥♥t k

①♠♣ ♦rrt t r♥ ② ♠ r s♣②s s♠♣ r♥② t ♠ ♠♦ t ♦r rt ss♥ ♦♠♣♦♥♥ts ♥ ts ①♠♣ t sstrt♦rr tt ♦t rs r ♥♦t ♥♣♥♥t

♥♥ rsts ♥ ♣rt ♠ ♠♦ ♦t♥s ♦♦ rsts ♥ ♣rts♥ t rqrs ♣r♠trs s sss ② ♥ ♥ ❨ ❬❨❪s t ♥ r③ ♦♦ tr ♦ t♥ t s ♥ t r♥ ts s♣rst②s rt ♥t ♦r t s♠ t sts s♥ t ♥♦r♠t♦♥ ♦ t ♥trss♣♥♥② s ♥r② ♥♦t ♣rs♥t sss ♦ t ♠ ♠♦ s s♦ ①♣♥② ts ♠♥♥ s♣t ♥ ♥ t ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥tsr ss ♦r ♥st♥ ♥ t② ♦♥ t♦ t ①♣♦♥♥t ♠② ss♥ s♠♠r③ ② t ♣r♠trs ♦ ts ♠r♥s

s ♦ t ♥trss ♦rrt t ❲♥ t ♦♥t♦♥ ♥♣♥♥ss♠♣t♦♥ s ♦t t ♠ ♠♦ srs r♦♠ sr ss ♥ s s t ss ♥♠r s ♥♦♥ t♥ t ♣rtt♦♥ ♦♠s s t ss♥♠r s ♥♥♦♥ t♥ t ♠ ♠♦ ♦rst♠ts t t♦ ttr t t t s♦r ♥st♥ t ♣♣t♦♥ ♣rs♥t ♥ ❬❱❪

♥rts ♦♥ ♥t ♠①tr ♠♦s

0 2 4 6 8 10

02

46

8

X1

X2

Zi1 = 1Zi2 = 1Zi3 = 1Zi4 = 1

r rt s♠♣ rs♥ r♦♠ t ♠ ♠♦ t ♦ ♣♥♥②t♥ rs p(xi;θ) =

∑4k=1

14N2(2k, I)

①♠♣ s ♣rtt♦♥ t t ♦♠♣♦♥♥ts ♦♠♦sst ss♥♠①tr ♠♦ ♦s t ♣ ♦ xi ∈ R

2 s ♥♦t ② f(xi) = 13φ2(xi;µ1,Σ) +

23φ2(xi;µ2,Σ) r Σ =

(

1 ρρ 1

)

♥ ρ 6= 0 ♦♣t♠ sst♦♥ rs

♠♥♠③♥ t ②s rr♦r s

r②s(xi) : zik =

1 (xi − µk)′Σ−1(xi − µk) < (xi − µℓ)

′Σ−1(xi − µℓ) t ℓ = 2− k0 ♦trs

t t ♠ ♠♦ ♥ t s♠ ♦♥♠♥s♦♥ ♠r♥ strt♦♥s t♥ t♠♦ ♥ ② f(xi) ♣ ♦ ts ♠ ♠♦ s rtt♥ s

p(xi;θ) =1

3φ2(xi;µ1,Γ) +

2

3φ2(xi;µ2,Γ) t Γ =

(

1 00 1

)

.

sst♦♥ rs ss♦t t♦ ts ♠♦ s

r♠(xi) : zik =

1 (xi − µk)′(xi − µk) < (xi − µℓ)

′(xi − µℓ) t ℓ = 2− k0 ♦trs

s t ♣rtt♦♥ st♠t ② t ♠ ♠♦ s s s♥ t st Ω r♦♣st ♥s r t sst♦♥ rs sr s ♥♦t ♠sr qs t♦ ③r♦

Ω = xi ∈ R2 : r②s(xi) 6= r♠(xi).

♥tt② ♦ t ♠①tr ♠♦s

ss♥t ♦♥t♦♥ sss ♣r♦ ② str ♥②ss r ♥tr♣rttr♦♦t t ♣r♠trs ♦ t ♦♠♣♦♥♥ts t s s♦ r tt t ♣r♠trsr ♥q ♦r ① strt♦♥ s t♦ ♠♦s ♥ t s♠ strt♦♥♠st t s♠ ♣r♠trs s♦ rr t♦ t ♥tt② ♦ t ♠♦


♥t♦♥ ♥tt② t t♦ ♠①tr ♠♦s ♥ t s♠ ♥tr♦r t ♦♠♣♦♥♥ts ♥ rs♣t② ♣r♠tr③ ② θ ♥ θ′ t♥ t ♠♦s ♥t

∀xi ∈ X p(xi;θ) = p(xi;θ′) ⇔ θ = θ′.

Pr♦♠ t♦ t r♥ ♦ t ♦♠♣♦♥♥ts ♦s② t ♠①tr♠♦s r ♥♦t strt② ♥t t ♦♥② ♣ t♦ s♣♣♥ s♥ t sss♥ r s strt ② t ♦♦♥ ①♠♣ ♦r ts s ♦ ♥♦♥♥tt② s ♥♦t sr r s♥ t ♥tr♣rtt♦♥ ♦ t ♣rtt♦♥ st②s♥t

①♠♣ ♥ ♦ t ♦♠♣♦♥♥ts t t ♦♠♣♦♥♥t ♠①tr ♠♦s ♣r♠tr③ ② θ = (π1, π2,α1,α2) ♥ θ′ = (π2, π1,α2,α1) t♥ ♦t ♣r♠tr sts ♥ t s♠ strt♦♥

∀xi ∈ X , p(xi;θ) = π1p(xi;α1) + π2p(xi;α2)

= π2p(xi;α2) + π1p(xi;α1)

= p(xi;θ′).

♥ ♦ t ♦♠♣♦♥♥ts ♥ ♥r♥ ♦t tt ts ♦♥t♦♥ ♦ ♥♦♥♥tt② ♦s ♥♦t str t ♠ ♦rt♠ s ♥①t st♦♥ t ♥ ②s♥r♠♦r t ♥ ♥♦ t st♥ ♣♥♦♠♥♦♥ ❬t❪

❲② ♥t ♠①tr ♠♦s ♥ ♦rr t♦ ♦ t ♣r♦♠ t♦t ♦♠♣♦♥♥t r♥ t ♥♦t♦♥ ♦ ♥tt② s ♥tr♦ ♦r t♠①tr ♠♦s ❬❪

♥t♦♥ ❲ ♥tt② ♠①tr ♠♦ ♥ p(xi;θ) s ♣s ② ♥t ♥

∀xi ∈ X p(xi;θ) = p(xi;θ′) ⇔ θ ♥ θ′ r q♥t

♥② ♠①tr ♠♦s r ② ♥t ♠♦♥ t♠ ♦♥ ♥ t t ♥t♠①trs ♦ ss♥ strt♦♥s t ♥t ♠①trs ♦ ♠♠ strt♦♥s ♥t ♥t ♠①trs ♦ P♦ss♦♥ strt♦♥s r ♠♥ rts st② t ♥tt② ♦ t ♠①tr ♠♦s ❬ ❨❪ ♥ ♦rr t♦ ♥ ♦ trrs♦♥♥ ♣rs♥t t t♦r♠ ♦ r ❬❪ ♠♦♥strts t ♥tt② ♦ s♦♠ ♥rt ♠①tr ♠♦s ♦r ♥st♥ t ♥rtss♥ ♠①tr ♠♦

♦r♠ ♦♥t♦♥s ♦ ♥tt② ❬❪ t P = P ♠②♦ ♦♥♠♥s♦♥ ♠t strt♦♥ ♥t♦♥s t tr♥s♦r♠s ψ(t) ♥♦r t ∈ Sψ t ♦♠♥ ♦ ♥t♦♥ ♦ ψ s tt t ♠♣♣♥ M : P 7→ ψs ♥r ♥ ♦♥t♦♦♥ ♣♣♦s tt tr ①sts t♦t ♦rr♥ () ♦ P stt F1 ≺ F2 ♠♣s Sψ1 ⊆ Sψ2 t ①st♥ ♦ s♦♠ t1 ∈ Sψ1 t1 ♥♥♣♥♥t ♦ ψ2 s tt lim

t→t1ψ2(t)/ψ1(t) = 0 ♥ t ss ♦ ♥t

♠①trs ♦ P s ② ♥t

Pr♠tr st♠t♦♥

Pr♦♣♦st♦♥ ♥tt② ♦ t ♥rt ss♥ ♠①tr ♠♦ ❬❪ ss ♦ ♥t ♠①trs ♦ ♥rt ss♥ strt♦♥s s ② ♥t

Pr♦♦ t Φ1(.;µ, σ2) ♥♦t t ss♥ ♠t strt♦♥ ♥t♦♥ t

♠♥ µ ♥ r♥ σ2 > 0 ts tr ♣ tr♥s♦r♠ s ♥ ② ψ(t) =exp(σ2t2/2−µt) rr t ♠② ①♦r♣② ② Φ1(xi;µ1, σ

21) ≺ Φ1(xi;µ2, σ

22)

σ1 > σ2 ♦r σ1 = σ2 t µ1 < µ2 ♥ ♦r♠ ♣♣s t Sψ =(−∞,∞) ♥ t1 = +∞

♦r s♦♠ ♠①tr ♠♦s r ♥♦t ♥t t r ♦ ♥trst s♥ t②♣r♦ ♠♥♥ rsts ♥ ♣rt ♥ s♥ tr ♣r♠trs s♠ ♥t

♥r ♥tt② s t ♥tt② ♦♥t♦♥ ♦ t♦♦ str♥♥t ss rstrt ♦♥t♦♥ ♥♠ ♥r ♥tt② s ♥tr♦

♥t♦♥ ♥r ♥tt② ♠♦ s ♥r② ♥t ♥t ♣r♠tr s♣ r t ♠♦ s ♥♦t ♥t ♣ t♦ t ♦♠♣♦♥♥t r♥ s ♠sr q t♦ ③r♦

s ♦♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♠♥ ts ♥ ♦s ❬❪ ♥ s♥t ♦♥t♦♥ ♦ t ♥r ♥tt② ♦rt ♠①tr ♦ ♠t♥♦♠ strt♦♥s s ♠♦ s st ♥ Prt rts ♦♥ t ♣r♦♦ ♦ ts ♥r ♥tt② r ♥ ♦r s♦♠ ♠①tr♠♦s r ♥♦t ♥r② ♥t s♦ tr ♣r♠trs ♥♥♦t ♥tr♣rt sstrt ② t ♦♦♥ ①♠♣

①♠♣ ♦♥♥r ♥tt② ♦ t ♠①tr ♦ ♥♦r♠ strt♦♥st xi ∈ [a1; b2] r♥ ② t ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♥♦r♠ strt♦♥s♦s t ♣ s

p(xi;θ) = πU [a1, b1] + (1− π)U [a2, b2],

r θ = (π, a1, b1, a2, b2) U [., .] ♥♦ts t ♣ ♦ ♥♦r♠ strt♦♥ ♥r a1 < b1 a2 < b2 a1 < a2 b1 < b2 ♥ a2 < b1 s ♠♦ s q♥t t♦t ♦♦♥ tr♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♥♦r♠ strt♦♥s ♦s t ♣ s

p(xi;θ′) = ε1U [a1, a2] + ε2U [a2, b1] + (1− ε1 − ε2)U [b1, b2],

r θ′ = (ε1, ε2, a1, a2, b1, b2) ε1 = π a2−a1b1−a1

♥ ε2 = π b1−a2b1−a1

+ (1− π) b1−a2b2−a2

s

t strt♦♥ ♦ xi ♥ ♠♦③ t t♦ r♥t ♣r♠tr③t♦♥s θ ♥ θ′


trtr ♦ ts st♦♥ str♥ ♦ t st ② ♥t ♠①tr ♠♦rqrs t st♠t♦♥ ♦ t ♠♦ ♣r♠trs ♥ ts st♦♥ ♣rs♥t tt♦ ♠♦st ♣♦♣r st♠ts ♥ t ♠①tr ♠♦ ♦♥t①t t ♠①♠♠ ♦♦st♠t rtr ♥♦t ② ♠ ♥ t ♠①♠♠ ♣♦str♦r st♠t rtr♥♦t ② ♠♣ ♦r ♦t st♠ts ♣rs♥t t st♠t♦♥ ♦rt♠s ♥tr trs s♣ t♦ t ♠①tr ♠♦s


♥♥♥ ①♠♣ ♥ ts st♦♥ ♥t♦♥s ♥ ♦rt♠s r ♥ ♥ ♥r③ s ♥ t② r strt ② t ♦♦♥ r♥♥♥ ①♠♣ ①trt r♦♠t rt ②s♥ ♦♥ ♥ ♥r♥ ♦♥ ①trs ♦ strt♦♥s ♦ r♥ ♥rs♥ ♥ P ♦rt ❬❪

t xi ∈ R ♥ t t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦♦s t ♣ s rtt♥ s

p(xi;θ) =2∑

k=1

πkφ1(xi;µk, σ2k),

r θ = (π1, µ1, σ21, π2, µ2, σ

22) ♥ r φ1(.;µk, σ

2k) ♥♦ts t ♣ ♦

t ♥rt ss♥ r N1(µk, σ2k)

♥♥♥ ①♠♣ ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr

①♠♠ ♦♦ st♠t♦♥

♥ ts st♦♥ ♥ t ♦♦ ♥t♦♥ ♥ t st♠t ♦ t ♠①♠♠♦♦ ♦s sr t ♠♥ ♣r♦♣rts ♥ ♥tr♦ t ♥♦t♦♥♦ ♦♠♣tt ♥ t ♦♦ ♥t♦♥ ss♦t t♦ t ♥♠ ♦♠♣tt♦♦ ♥t♦♥

①♠♠ ♦♦ st♠t

♥ ♦♦ ♥t♦♥ ♦s t ♦ ♥♦r♠t♦♥ ♦♥t♥ ♥ tt st ♦r s t s ♠♦r ♦♠♦rt t♦ ♦r t t ♦rt♠ ♦ ts♥t♦♥ ♣rs♥t t ♥t♦♥s ♦ ♦t ♥t♦♥s

♥t♦♥ ♦♦ ♥t♦♥ ♦r ♥ s♠♣ x ts ♥t♦♥ ♦♠♣t t t ♣♦♥t θ s ♥ ② p(x;θ) =

∏ni=1 p(xi;θ)

♥t♦♥ ♦♦♦ ♥t♦♥ ♦♦♦ ♥t♦♥ ♦♠♣tr♦♠ t s♠♣ x ♥ t ♦♥ t ♣♦♥t θ s ♥ ②

L(θ;x) =n∑

i=1

ln p(xi;θ).

♦r ♥ s♠♣ x ♦♠♣♦s t n ♥s xi ∈ R t♥ t ♦♦ ♥ t ♦♦♦ ♥t♦♥s r ♥ s

p(x;θ) =n∏

i=1

2∑

k=1

πkφ1(xi;µk, σ2k) ♥ L(θ;x) =

n∑

i=1

ln2∑

k=1

πkφ1(xi;µk, σ2k).

♥♥♥ ①♠♣ ♦♦ ♥ ♦♦♦ ♥t♦♥s


♥ rq♥tst r♠♦r ♥t t♦ ♥r ♦r♥ t♦ t ♥♦r♠t♦♥ ♥② t t ♦ ♥tr ♣♣r♦ s t♦ sr t ♠①♠♠ ♦♦ st♠t♠ ♥♦t ② θ

♥t♦♥ ♠ ♠①♠♠ ♦♦ st♠t s ♥ ②

θ = argmaxθ

L(θ;x).

s t ♦♦♦ ♥t♦♥ ♦r s♠r② t ♦♦ ♥t♦♥ s tr♥t ts ♦♥t♦♥ s ♥r② r ♦r t ♠①tr ♠♦s t♠ s ♦t♥ ② s♦♥ t qt♦♥s ♥♥ ♦ t r♥t ♥ ♥♦♥ ♣♦st ♥t ss♥ ♠tr①

∇L(θ;x) = 0.

Pr♦♣rts ♦ t ♠ ♠ s ♣♦♣r st♠t s♥ t s ♥r rstrt ♦♥t♦♥s ♦♦ ♣r♦♣rts

t s ♥q t ♣r♦t② t♥♥ t♦ s t s♠♣ s③ r♦s t♦ ♥♥t② t s ♦♥sst♥t t s s②♠♣t♦t② ♥s t s s②♠♣t♦t② ss♥ t s②♠♣t♦t② ♠♥♠③s t r r♥

ts ♦♥ t ♦♥t♦♥s ♥♦♥ ts ♣r♦♣rts r ♥ ♥ ♦r② ♦ P♦♥tst♠t♦♥ ♣tr ② ♠♥♥ ♥ s ❬❪ s t ♠ s♦♦ ♣r♦♣rts t s ♥tr t♦ st② ts ①st♥ ♥ t ①sts t ♠t♦s♣r♦r♠♥ ts st♠t♦♥

♥r② ①st♥ ♥ t ♥q♥ss ♦ t ♠ s ♥♦t r♥t ♦rt ♠①tr ♠♦s ♥ t ♦♦♦ ♥t♦♥ ♥ ♥♦t ♣♣r ♦♥s t r♥♥♥ ①♠♣ ♥ s s t ♦♦ ♥t♦♥ ♥ t♥ t♦ t♥♥t② s stt♦♥ ♥♠ ♥r② ❬❪ ♥♦s ♥♦♥sst♥t st♠t♦rs♥ s s t st♠t♦r r②♥ ♥ ♥♦♥ ♥t ♦♦♦ ss♦ sr

❲ ♥t t♦ t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ ♦♥ ts♠♣ x ss♠ tt µ1 = x1 ♦sr ♠♦ ♥r② s♥

limσ21 → 0σ22 > 0

L(θ;x) = ∞.

♥♥♥ ①♠♣ ♥r②

♥t♦♥ ♠ ♥ ♥♦♥ ♦♦♦ ❲♥ t ♦♦♦♥t♦♥ s ♥♦t ♣♣r ♦♥ t ♠ s ♥ s

θ = argmaxθ∈θ:L(θ;x)<+∞ ♥ ∇L(θ;x)=0

L(θ;x).


♦t tt t ♦♦♦ ♥t♦♥ s ♥r② sr ♦ ♦♣t♠ ♥rs t t② ♥ ♥♥ t ♠ s ♣♥♦♠♥♦♥ s ♥♦ strt ♦♥t r♥♥♥ ①♠♣

❲ ♥rt s♠♣ ♦ s③ r♦♠ t ♦♠♣♦♥♥t ♥rt ♠①tr♠♦ ♦s t ♣r♠trs r ♥ s ♦♦s

π1 = 1/3, π2 = 2/3, µ1 = −1, µ2 = 3.5 ♥ σ21 = σ2

2 = 1.

❲ ss♠ tt t ♣r♦♣♦rt♦♥s ♥ t r♥s r ♥♦♥ ♥ ♥tt♦ st♠t t ♠♥s ② ♠①♠♠ ♦♦ r s♣②s t ♦♦♦ ♥t♦♥ ♦r♥ t♦ t s ♦♥ ♦t ♣r♠trs ♥ ♥♦sr tt ts ♥t♦♥ s t♦ ♦♣t♠ ♦ ♦♥ s ♦t r♦♥(−1, 3.5) t ♦ ♦♥ s ♦t r♦♥ (3.5,−1)

♥♥♥ ①♠♣ ♦♦♦ ♦♣t♠ t ♥♥♦♥ ♠♥s

−4 −2 0 2 4 6

−4

−2

02

4

µ1

µ 2

r ♦♦♦ s ♦r t s♠♣ ♦ s③ ♦r♥ t♦ t s♦ (µ1, µ2)

♦ ①♣t s♦t♦♥ ♦r t ♠①tr ♠♦s t sr ♦ t ♠ ♥♦st♦ s♦ qt♦♥s ♥ ♥♦ ♥②t s♦t♦♥ rt ♦♠♣tt♦♥ ♦ t♠ s ♥♦t s② s ♦ t s♣ ♦r♠ ♦ t ♦♦♦ ♥t♦♥ s♠ ♦♦rt♠s ♦ s♠s ♦ ♣ ♦ s♦♠ trt ♣r♦rs s♦ s t t♦♥♣s♦♥ ♦rt♠ t ♦r ♥st♥ ♥ ♠r ♦♣t♠③t♦♥t♦rt ♥ ♣rt s♣ts ② ♦♥♥♥s rt ♠r ♥ st③ ❬❪ ♦r ts ♠♣♠♥tt♦♥ s ♦t♥ ♦♠♣① s♥ t♥♦s t ♦♠♣tt♦♥ ♦ t rts ♦ t ♦♦


♠①tr ♠♦s ♥ ss♠♥t s ♦ t ♥♥t♦♥ ♦ t ♠♦rt♠ ♦s t ♠♣♠♥tt♦♥ s s♠♣ s♥ ♥♦ rt ♦ t ♦♦s ♥♦ s ts ♦rt♠ s s♣③ ♦r t ♠ss♥ t rst② ♥t ♥♦t♦♥ ♦ ♦♠♣tt ♦r t ♠①tr ♠♦ ♥ s♦♥② t t

sr t ♥ ♦♠♣tt ♦r s♠♣ x t ♥rt ♠♦s ss♠ tt t r♥ ♦ ♥ xi ♥♦s t ♣r♠♥r② s♠♣♥ ♦zi s Prr♣ ♥rt ♠♦ ♥ t♦♥ s t♦r z s ♥♦srt s ♦♥sr s ♠ss♥ s x s ♥♠ t ♦sr t t♦♣ (x, z) s ♥♠ t ♦♠♣tt ♥ t s♠ ② t ♦♦♦ ♥t♦♥ ♦♠♣t ♦♥ (x, z) s ♥♠ t ♦♠♣tt ♦♦♦ ♥t♦♥ ♥ t srtt♥ s ♦♦s

L(θ;x, z) =n∑

i=1

ln p(xi, zi;θ)

=n∑

i=1

ln

(

g∏

k=1

(πkp(xi;αk))zik

)

=n∑

i=1

g∑

k=1

zik ln (πkp(xi;αk)) ,

② strt♥ r♦♠ t rt♦♥ t♥ t ♣ p(xi;θ)p(zi|xi;θ) = p(xi, zi;θ) ♦♥♥ t ♦♦♥ rt♦♥

L(θ;x) = L(θ;x, z) + e(z,x;θ).

r e(z,x;θ) = −∑ni=1

∑gk=1 zik ln tik(θ) s L(θ;x) ≥ L(θ;x, z) s♥ tik(θ)

s ♣r♦t②

♦r t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t ♦♠♣tt ♦♦♦ ♥t♦♥ ♦♠♣t ♦♥ θ ♦r t s♠♣ x ♥ t ♣rtt♦♥ z s ♥ s

L(θ;x, z) =2∑

k=1

nk lnπkσk

− 1

2

2∑

k=1

n∑

i=1

zik(xi − µk)

2

σ2k

− n ln√2π.

♥♥♥ ①♠♣ ♦♠♣tt ♦♦♦

♦rt♠s ♦r ♠①♠♠ ♦♦ st♠t♦♥

♠ ♦ t ♠①tr ♠♦s s ♥r② ♦t♥ ♥ ♠ ♦rt♠ sst♦♥ s ♦t t♦ t ♣rs♥tt♦♥ ♦ ts ♦rt♠ ♥ ♦ ts ①t♥s♦♥s


♠ ♦rt♠

Prs♥tt♦♥ ♦ t ♠ ♦rt♠ ①♣tt♦♥①♠③t♦♥ ♦rt♠rtr ♥♦t ② ♠ s ♣r♦♣♦s ② ♠♣str r ♥ ♥ ♥ ❬❪ ts ♦♠♥s ♦ ♣♣t♦♥ r str t♥ t ♠①tr ♠♦s s♥ ts s♣③ ♥ t s ♦ ♠ss♥ s ♥ t ♦♥t①t ♦ t ♠①tr ♠♦st ss ♠♠rs♣s ♦ t ♥s r ♥tr♣rt s ♠ss♥ s ♦ttt ts ♦rt♠ ♦s t♦ str t st t ♠ss♥ s ② ♠♥② rstrt ss♠♣t♦♥s ♠♥ ♥t ♦ ts ♦rt♠ s ts s♠♣t② s♥ t ♦♣t♠③s t ♦♦ ♥t♦♥ t♦t ♦♠♣t♥ ts rtsrtr♠♦r s♥ ts ♠♣♠♥tt♦♥ ♥ ♣r③ t st②s ♥t ♥ tss ♦♥r♦♥t t r t sts s st♦♥ s st ♥ ♦r ♦ ts ♦rt♠t rr ♥♥ ♠♦r ts ♦ rr t♦ ♦rt♠ ♥ ①t♥s♦♥s ② ♥ ♥ rs♥♥ ❬❪

♥t♦♥ ♦ t ♠ ♦rt♠ ♠ ♦rt♠ s ♥ trt ♦♥ strt♥r♦♠ ♥ ♥t ♦ t ♣r♠tr tr♥ts t♥ t t♦ ♦♦♥ st♣s t ♦♠♣tt♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦ st♣ ♥ ts ♠①♠③t♦♥ ♠ st♣

trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t Q(θ;θ[r]) r

Q(θ;θ[r]) = Eθ[r] [L(θ;x, z)] ,

st♣ st θ[r+1] s s

θ[r+1] = argmaxθ

Q(θ;θ[r]).

♦rt♠ ♠ ♦rt♠

t♦♣♣♥ rtr ♦ rtr r ♥r② s ♠♦st ♦♠♠♦♥ ♦♥ ♦♥ssts♥ st♦♣♣♥ t ♦rt♠ ♥ t ♥rs ♦ t ♦♦♦ s ♦r t♥ ttrs♦ ε ♦s♥ ② t sr s♦ ♥

L(θ[r+1];x)− L(θ[r];x) < ε.

s♦♥ ♦♥ ①s ♥ ♥ t ♥♠r ♦ trt♦♥s ♣r♦r♠ ② t ♦rt♠

♦s t ts ♥r♥t t♦ t ♠①tr strtr st♠t♦♥ ♦ t ♣r♠trs s ♦ ♦r ♠①tr ♠♦ t♦t ♦♥str♥t t♥sss t ♥r♥ ♦ s ♠♦ ♥ ♠ ♥ t ss ♠♠rs♣s ♦t ♥s r ♥♦♥ s ♠①tr ♠♦ ♦s t st♠t s trt ♥t sr♠♥♥t ♥②ss s♦ ♥ t s r ♥♦♥ ♥ ②s ♥rr♥ str♥ ♣r♦♠ ♦r ♥st♥ t ♠①tr ♠♦s ♦s t ♦♠♣♦♥♥t


strt♦♥s ♦♥ t♦ t ①♣♦♥♥t ♠② ♥ ♥ ♥♦ ♦♥str♥t t♦tr ♥ ①♣t② ♦♠♣t

❲ ♦♥sr t ss ss♥ ♠①tr ♠♦ ♦s t ♦♠♣♦♥♥tsr ♥ ② trt♦♥ [r] ♦ t ♠ ♦rt♠ s rtt♥ s

st♣ t ♦♥t♦♥ ♣r♦ts

tik(θ[r]) =

π[r]k p(xi;α

[r]k )

p(xi;θ[r])

.

st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

π[r+1]k =

n[r]k

n, µ

[r+1]k =

1

n[r]k

n∑

i=1

tik(θ[r])xi,

σ2[r+1]k =

1

n[r]k

n∑

i=1

tik(θ[r])(xi − µ

[r+1]k )2,

r n[r]k =

∑ni=1 tik(θ

[r])

♥♥♥ ①♠♣ ♠ ♦rt♠

Pr♦♣rts ♦ t ♠ ♦rt♠ ❯♥r rstrt ss♠♣t♦♥s ❬❲❪ t♠ ♦rt♠ ♣r♦s sq♥ ♦ st♠ts θ[r] ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t ♦♦♦ ♥t♦♥ s ♦♣t♠♠ ♦♥② ♣♥s ♦♥ t ♥t③t♦♥θ[0] ♥ t ♦♦ ♥t♦♥ ♥rss t trt♦♥ ♦ t ♠ ♦rt♠

∀[r], L(θ[r+1];x) ≥ L(θ[r];x).

s ♦rt♠ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t s ♠♥t♦r② t♦ ♣r♦r♠ srr♥t ♥t③t♦♥s ♥ ♦rr t♦ ♦♣ t♦ t t ♠ ♥ ♦tr r ♦ t♠ ♦rt♠ s ts s♣ ♦ ♦♥r♥ ♥ ts ♦rt♠ ♥ ♦♥r s♦②s♣② ♥ t sss r ♦r♣♣ s ♠♥② t♦rs ♥ ♥trst♥ t rt♦♥ ♦ t ♠ ♦rt♠ s ♦r ♥st♥ ❬❱ ❪ tr tsr♣t♦♥ ♦ t ♠ ♦rt♠ ♣♣ ♦♥ t ss♥ ♠①tr ♠♦s ♣rs♥ttr ♦ ts ①t♥s♦♥s r♥ ts rs


❲ ♦♥sr t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t♥♦♥ ♣r♦♣♦rt♦♥s ♥ r♥s ♥ s s t ♠ st♣ ♦♥② ♦♥ssts ♥ ♦♠♣t♥ µ[r+1]

k s♥ t ♦tr ♣r♠trs r ♥♦♥ r s♣②s t s ♦ t ♦♦ ♦♠♣t t trt♦♥s ♦ t♦r♥s ♦ t ♠ ♦rt♠ r♥ ♣r♥t t tr♥s s ♥t③ t(−1,−1/2) ♥ ♦♥rs t♦ t ♠ t r♥ ♣r♥t t sqrss ♥t③ t (3.5, 3) ♥ ♦♥rs t♦ ♦ ♠①♠♠ ♦ t ♦♦♥t♦♥

♥♥♥ ①♠♣ ①♠♣ ♥ ♠ ♦rt♠

−4 −2 0 2 4 6

−4

−2

02

4

µ1

µ 2

r ♦♦♦ s ss♦t t t♦ sq♥s ♦ ♣r♠trs ♣r♦♥ ② ♥ ♠ ♦rt♠

①t♥s♦♥s ♦ t ♦rt♠

♦rt♠ ♦♠t♠s t s♦t♦♥ ♦ t ♠ st♣ s ♥♦t ①♣t ♥ s s t ♥r③♠ ♠ ♦rt♠ ♥ s ♠ st♣ s s♦ r♣② ♠ ♦♥ ♦♥② rqrs t ♥rs ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦ s t trt♦♥ [r] t st♣ s ♥♥ t ♠st♣ tr♠♥s θ[r+1] s s

Q(θ[r+1];θ[r]) ≥ Q(θ[r];θ[r]).

♠ ♦rt♠ ♣s t ♠♦♥♦t♦♥ ♣r♦♣rt② ♦ t ♥rs ♦ t ♦♦♥t♦♥ ♦r trt♦♥ s ♥rt r♦♠ t ♠ ♦rt♠ ♦r ts♦rt♠ rqrs ♠♦r trt♦♥s t♥ t ♠ s♥ ts ♦♥r♥ s s♦r


♥ ♦rt♠s ♥ ♦rr t♦ ♦r♦♠ t tr ♠♥ rs♦ t ♠ ♦rt♠ str♦♥ ♣♥♥② t t ♥t③t♦♥ ♣♦♥t ♦ ♦♣t♠♠ ♦♥r♥ ♥ s♦ ♦♥r♥ t t♦st♠ s♠ ♦rt♠ s♣r♦♣♦s ❬+❪ ♦rt♠ ♥♦r♣♦rts st♦st st♣ s st♣ t♥t st♣ ♥ t ♠ st♣ rt ② t r♥♦♠ ♠♣tt♦♥ ♣r♥♣ sq♥ ♥rt ② t s♠ ♦rt♠ ♦♥rs t♦ ♥q stt♦♥r② strt♦♥♦s t♦ p(θ|①)

trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t t ♦♥t♦♥ ♣r♦ts

tik(θ[r]) =

π[r]k p(xi;α

[r]k )

p(xi;θ[r])

.

st♣ s♠♣ t ss ♠♠rs♣ s s

z[r]i ∼ M(ti1(θ

[r]), . . . , tig(θ[r])).


θ[r+1] = argmaxθ

L(θ;x, z[r]).

♦rt♠ s♠ ♦rt♠ ♦r t ♠①tr ♠♦s

♦rt♠ s st♦♣♣ tr ♥♠r ♦ trt♦♥s ♦s♥ ② t sr ♦ttt ♥♦tr rs♦♥ ♦ ts ♦rt♠ ♥♠ s♠ ♣r♦s ♥ ♠♦st sr②♦♥r♥ t♦ t ♥q stt♦♥r② strt♦♥ ❬❪ t s tr ♦ t♥ rs♦♥ s♠t ♥♥♥ ♠ ♦rt♠ ♥ t s♠ ♦rt♠ ♥ t st♣ ♥♥♥ s ♥tr♦ tr t s st♣ ♥ ♦rr t♦ r t ♠♣t ♦ tr♥♦♠ ♣rtrt♦♥s ♣r♦r♠ ② t s st♣ s rt♦♥ ♥rss t t♥♠r ♦ trt♦♥s s ♥ t s♠ strts t ♦rs t s♠ ♦rt♠t♥ t t♥s t♦ t ♠ ♦rt♠ ♥ t ♥♠r ♦ trt♦♥s ♥rss

♦rt♠ sst♦♥♠ ♠ ♦rt♠ ❬❪ s ♥r ♦rt♠ t♦ ♦♠♣t t st♠t ♥ t♦ ♥ t ♣rtt♦♥ ♥r t sst♦♥♣♣r♦ s t ♣r♦s t ♦♣ (θ, z) ♠①♠③♥ t ♦♠♣tt ♦♦♦

argmax(θ,z)

L(θ;x, z).

♥ t st♠t ♦ t ♠①♠♠ ♦♠♣tt s s ♥ ♥♦♥sst♥t❬♦❪ ts rsts ♥ ttr t♥ t♦s ♦ t ♠ ♥ t s♠♣ s③ ss♠ ♥ ♥ t sss r s♣rt rtr♠♦r t ♠ ♦rt♠♦♥rs str t♥ t ♠ ♦rt♠ ♠ ♦rt♠ ♥♦r♣♦rts sst♦♥ st♣ t♥ t st♣ ♥ t ♠ st♣ ♦r♥ t♦ t ♠♣ ♣r♥♣s ts ♦♥r♥ s♣ s ①♣t t♦ str t♥ t ♦♥r♥ s♣ ♦t ♠ ♦rt♠


trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ s st♣ t Q(θ;θ[r]) r

tik(θ[r]) =

π[r]k p(xi;α

[r]k )

p(xi;θ[r])

.

st♣ ♠♥♠③ e(z,x;θ[r]) s♦

z[r]ik =

1 tik(θ[r]) ≥ tiℓ(θ

[r]) ∀ℓ = 1, . . . , g0 ♦trs


θ[r+1] = argmaxθ

L(θ;x, z[r]).

♦rt♠ ♠ ♦rt♠ ♦r t ♠①tr ♠♦s

♠r ♠ ♦r t s♣r ss♥ ♠①tr ♠♦ ♥ ♠♥s ♦rt♠ ♠♥s ♦rt♠ s q♥t t♦ t ♠ ♦♥ ♥ t ♠♦ t♥ s t s♣r ss♥ ♠①tr ♠♦ t q ♣r♦♣♦rt♦♥s ❬❪

rs ♦ t ♠①♠♠ ♦♦ ♣♣r♦s ♥ t ①t♥s♦♥s♦ t ♠ ♦rt♠ r ts ♠♥ rs tr ♣r♦♠s st② ♥r♥t t♦ t♠①♠♠ ♦♦ ♣♣r♦ ♣♣ ♦♥ ♠①tr ♠♦s

rst ♦♥ s t t② t♦ ♥ t ♦ ♠①♠♠ ♦ t ♦♦♥t♦♥ ♦t tt ts ♣r♦♠ s ♠♦r ♣rs♥t ♥ t s♠♣s r s♠s♥ t ♦♦ ♥t♦♥ ♥ r② ♠♣②

s♦♥ ♦♥ s t♦ t ♣♣r♦♥ ♦ t ♦♦ ♥t♦♥ ♥ ts ♥t♦♥ s ♣♣r♦♥ ♦r t♦r t t ♥ ♣♣r♥♦♥ ♥ ♦tr stt♦♥s s ♦r ♥st♥ t tr♦sst ss♥♠①tr ♠♦ s t st♠t rtr♥ ② t ♦rt♠ ♥ ♦♥ t♥r② ② ♥ s s ts st♠t s s♦ s ♥ ♥♦♥sst♥t

tr ♦♥ s ♦t t rrt② ♦♥t♦♥s r ♦t♥ ♦t ♦rt s♠ t sts s t st♠t♦♥ ♥ ♥♦ ♥ ♦rtt♥

①♠♠ ♣♦str♦r st♠t♦♥

②s♥ r♠♦r ♥ t ②s♥ r♠♦r t ♣r♠tr θ s ss♠ t♦ ts r♥♦♠ r ♦s t ♣r♦r strt♦♥ s ♥♦t ② p(θ) sstrt♦♥ ♦♥t♥s t ♥♦r♠t♦♥ ♦♥ θ ♥ ② ♥ ①♣rt s t tr♠♣r♦r ♥ ♥tr♣rt s ♦r t♦ ♦sr t t r r r♥t ♣r♦rstrt♦♥s ♥ tr ♠♣t ♦♥ t ♥r♥ ♥ ♥♦t ♥ s♣② ♦rt s♠ t sts s strt♦♥s r ♣rs♥t ♥ ②s♥ ♦ ②P ♦rt ❬♦❪ ♥ r ts ♦♥ t ②s♥ r♠♦r t ♣r♦rstrt♦♥ ♦♥t♥s t ♥♦r♠t♦♥ ♥ ② ♥ ①♣rt t strt♦♥ r♦♠


♥r♥s r ♠ s t ♣♦str♦r ♦♥ ♣♦str♦r strt♦♥ ♦♥t♥s t♣r♦r ♥♦r♠t♦♥ ♥ ② t ①♣rt p(θ) ♥ ② t t (x) s t tr♠♣♦str♦r ♥ ♥tr♣rt s tr t♦ ♦sr t t

P♦str♦r strt♦♥ ♥ ♦♦ ♥t♦♥ ②s r ♥♦s ttt ♣♦str♦r strt♦♥ p(θ|x) s ♥ s

p(θ|x) = p(x|θ)p(θ)p(x)

.

♦t tt t ♥♦r♠t♦♥ ♥ ② t t s rt t♦ t ♦♦ ♥t♦♥p(x|θ) s t ♥t♦♥ ♦ t ♦♦ ♥t♦♥ s r s♥ ts ♥t♦♥♦♥t♥s t ♥♦r♠t♦♥s ♥ ② t t ♥ ♦t rq♥tst ♥ ②s♥r♠♦rs s ♥t♦♥ s s♦ ♦♠♠♦♥ s ♦r ♦t ♦♠♠♥ts ♥p(x) s ♦♥st♥t ♦r♥ t♦ θ t ♦♦♥ rt♦♥ s s ♥ p(x) s ♥♦t♦♠♣t

p(θ|x) ∝ p(x|θ)p(θ).

♥ ♥ts ♦ t ②s♥ ♣♣r♦ ②s♥ ♣♣r♦s ♦rt ♠①tr ♠♦ r t ♥ ♥t ①tr ♥ r♦ t♥ ♦s ② rürt♥ttr ❬❪ r♦♠ r ♦♥ ♥ ①trt t ♦r ♦♦♥ ♠♥qts

♣r♦r s s♠♦♦t t ♦♥ t ♣r♦♠s ♦ t ♥rt s♦t♦♥

s ♠t♦s t ♥t♦ ♦♥t t ♣r♠tr ♥rt♥t② ② st② ♥ s r rrt② ♦♥t♦♥s r ♦t s♠ t

st ♠①tr t s♠ ♦♠♣♦♥♥t ♣r♦♣♦rt♦♥s s♥ t② ♦ ♥♦t r② ♥s②♠♣t♦t ♥♦r♠t②

r ♠♣♠♥tt♦♥ s ♥♦t ♦♠♣① ♥ t ♦♠♣♦♥♥t strt♦♥s ♦♥ t♦ t ①♣♦♥♥t ♠② ♥ ♥ s s t ♦♥t ♣r♦rstrt♦♥s ♣r♦ ①♣t ♣♦str♦r strt♦♥s ❬♦❪

❲ ♦♥sr t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t♥♦♥ ♣r♦♣♦rt♦♥s ♥ r♥s ♥ ts s θ = (µ1, µ2) ss♠♥♣♥♥ t♥ t ♣r♦r strt♦♥s ♥ s t r②s ♥♦♥♥♦r♠t ♦♥s s♦

p(θ) = p(µ1)p(µ2) t µ1 ∼ N (ξ, κ) ♥ µ2 ∼ N (ξ, κ),

r ξ ♥ κ r ②♣r♣r♠trs r s♣②s t s ♦ t♣r♦r ♥ ♦ t ♣♦str♦r strt♦♥s ♦♠♣t ♦♥ s♠♣ ♦ s③

♥♥♥ ①♠♣ ②s♥ r♠♦r


−4 −2 0 2 4 6

−4

−2

02

4

µ1

µ 2

♣r♦r strt♦♥

−4 −2 0 2 4 6

−4

−2

02

4

µ1

µ 2

♣♦str♦r strt♦♥

r Pr♦r ♥ ♣♦str♦r strt♦♥s ♦r ♦♠♣♦♥♥t ♥rt ♠①tr♠♦ t ξ = 1 ♥ κ = 9

♠♦♦t t ♥ ♥rt s♦t♦♥ ②s♥ r♠♦r ♥ ♦s♦♠ ♥r② ♣r♦♠s ② s♥ ♣r♦r strt♦♥s ♣r♦s s♠♦♦tt s strt ♦

r♦♠ t s♠♣ x rs♥ r♦♠ t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t ♥♦♥ ♣r♦♣♦rt♦♥s t µ1 = 0 ♥ µ2 = x1 t ♠ st♦ ♥r t ♣r♠tr θ = (σ2

1, σ22) rq♥tst ② ♥ sr r♦♠

♥r② s♦t♦♥ s strt ♥ ♥♥♥ ①♠♣ ♥ ②s♥② ss ♥♣♥♥ ss♠♣t♦♥ t♥ ♣r♦r strt♦♥s ss♦t t ♦♥t ♣r♦r strt♦♥s ♥♦s tt

p(θ) = p(σ21)p(σ

22) r 1/σ2

1 ∼ G (c0, C0) ♥ 1/σ22 ∼ G (c0, C0) .

♥ t ♣♦str♦r strt♦♥ p(θ|x) =∑z∈Z p(θ|x, z)p(z|x) r Z =

1, 2n t♥ ts strt♦♥ s ♣♣r♦♥ ②

p(θ|x) ≤∑

z∈Z

p(θ|x, z) ♥ p(θ|x, z) = p(σ21|x, z)p(σ2

2|x, z).

♥ 1/σ2k|x, z ∼ G

(

c0 +♥k2, C0 +

∑

i:zik=1(xi−µk)

2

2

)

♥ s♥ t ♠♦

♦ G(α, β) s α−1β

♥ α ≤ 1 t♥

p(θ|x) < +∞ ∀θ.

♥♥♥ ①♠♣ ♠♦♦t t ♥ ♥rt s♦t♦♥

♦t tt t ♦♥t ♣r♦r strt♦♥s r ♦t♥ s s♥ t② s♥♥t②s♠♣② t ♥r♥ ♦r t ♦ ♦ t ②♣r♣r♠trs ♣r♠trs ♦t ♣r♦r strt♦♥ ♥ t ♥ t r②s ♥♦♥ ♥♦r♠t ♣r♦r s


♥♦t s♦t♦♥ s s♦ t♦ ① t ②♣r♣r♠trs ② s♥ ♥ ♠♣r②s♥ ♣♣r♦ tr♠♥s t ②♣r♣r♠trs ♦r♥ t♦ t ts ♦r ♥st♥ ❬❪ ♦r t ss♥ ♠①tr ♠♦

②s♥ ♥r♥ ②s♥ ♣♣r♦s ♦t♥ ♥ s♠t♦♥ ♠t♦s r♦ ♥ ♦♥t r♦ ♠♠ t♦ ♥rr ♦ tr ♦♣♠♥t s r♥ts t s rt t♦ t ♦♠♣tt♦♥ ♣♦r ♥ t ♣♦str♦r strt♦♥♦♥t♥s t ♦ ♥♦r♠t♦♥ ♦t θ ♥♦r♠t♦♥ ♥ ② t ①♣rt ♥ ② tt ♥② ♥r♥ ♦♥ θ r s ♦♥ ts strt♦♥ t s s♦ ♥tr t♦ ♦♣tt ♣♣r♦ s♠r t♦ t ♦♥ s ♥ t rq♥tst r♠♦r ♦ ♥t t♦♦t♥ t st♠t ♦ t ♠①♠♠ ♣♦str♦r st♠t ♠♣ ♥♦t ② θ

♥t♦♥ ①♠♠ ♣♦str♦r st♠t ♠①♠♠ ♣♦str♦rst♠t s ♥ ②

θ = argmaxθ

p(θ|x) = argmaxθ

p(x|θ)p(θ).

♠r ♥ t♥ t ♠ ♥ ♠♣ ♦t tt t ♣r♦r ♦♦s ♥♦r♠ strt♦♥ t♥ p(x|θ) ∝ p(θ|x) ♥ s s t ♠ s s♦ q t♦t ♠♣

s t ♠♣ s rs♦♥ st♠t t t ♥ t t♦ ♦t♥ t♥ s♦ r♣ ② t ♠♥ ♦r t ♠♥ ♦ t ♣♦str♦r strt♦♥ ♦tttr st♠ts r ♠♦r s② ♦t♥ ♠♠ ♦rt♠s ❲ ♥♦ t t♠♥ ♦rt♠s ♣r♦r♠♥ t ②s♥ ♥r♥ ♦♥ θ

♦rt♠s ♦r ♠①♠♠ ♣♦str♦r st♠t♦♥

♦rt♠ ♦r ②s♥ st♠t♦♥

♥ ♠ ♦rt♠ ♥ ♠♦ t♦ ♣r♦ t♠♣ ♦r t st♠t♦ t ♠①♠♠ ♣♥③ ♦♦ ❬r❪ s ♦t s ② s♥ t♦♦♥ rt♦♥ ♦t♥ ② ♣♣②♥ t ②s r ♥ ② s♥ t ♦rt♠♥t♦♥

argmaxθ

p(θ|x) = argmaxθ

L(θ;x) + ln p(θ).

♦ ♦t♥ t ♠♣ t ♠ st♣ ♦ t ♠ ♦rt♠ ♦♥ssts ♥ t ♠①♠③t♦♥♦ t ①♣tt♦♥ ♦ t ♦♠♣t t ♣♦str♦r strt♦♥ p(θ|①, ③) t trt♦♥[r] t ♠ st♣ tr♠♥s t ♣r♠tr θ[r+1] s s tt

θ[r+1] = argmaxθ

Q(θ;θ[r]) + ln p(θ).


t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ ♦s ♦♥② t ♠♥sr ♥♥♦♥ ♣r♦r strt♦♥ ♦ t ♣r♠trs s ♥ ♥ s♦ t ♠ st♣ s rtt♥ s

µ[r+1]k =

σ2kξ + κ

∑ni=1 tik(θ

[r])xi

σ2k + κn

[r]k

.

♥♥♥ ①♠♣ ②s♥ st♠t♦♥ ♦ ss♥ ♠①tr

♦rt♠s ♥ ②s♥ st♠t♦♥

trtr ♦ ts st♦♥ ❲ ♥♦ ♣rs♥t s♦rt ♦r ♦ tr ♠♥ ♦rt♠s s t♦ ♥r t ♣r♠trs ♦ ♠①tr ♠♦ t s s♠♣rt tr♦♣♦sst♥s ♦rt♠ ♥ t tr♦♣♦st♥s s♠♣r rr ♥t♥ ♠♦r ts ♥ r♣♦rt ♦♥ ♦♥t r♦ ttst t♦s ②P ♦rt ❬❪ s ♦rt♠s r ♠♠ ♦♥s ♦s t r♦ ♥s t ♣♦str♦r strt♦♥ p(θ, z|x) s t stt♦♥r② strt♦♥ s t②s♠♣ sq♥ ♦ ♣r♠trs ♦r♥ t♦ tr ♣♦str♦r strt♦♥ s♥ ts♣♣r♦ ♦s s t♦ ♣r♦r♠ t ②s♥ ♥r♥

♠r ♠♦sts♦r♥ stts ♥♦♥ r♥t ♥t③t♦♥s ♦ t♦rt♠s ♥ t ♠♠ ♦rt♠s ♥ ♥ rr ♥ r♦ r♦♥ r ♥♦t t♦rt② s♥st t♦ t ♦ ♦♣t♠ tr ♦r s ♥♦t s♦♣rt ♥ ♣rt s ♦r ♥st♥ ❬❪ ♥ tr r tr♣♣♥ stts r ♠♦sts♦r♥ stts rqr♥ s♦ r ♥♠r ♦ trt♦♥s t♦ s♣r♦♠ t♠ tt t ♦rt♠ s ♥r② st♦♣♣ ♦r

s s♠♣r

♥ s s♠♣r s t ♠♦st ♣♦♣r ♣♣r♦ t♦ ♣r♦r♠ t②s♥ ♥r♥ ♦ ♠①tr ♠♦ s♥ t ss t t♥t strtr ♦ t ts ♦rt♠ s t ♦♥ ♦♥t♦♥ strt♦♥s r♦♠ t s s② t♦s♠♣

s s♠♣r ♥ ♠①tr ♠♦s s s♠♣r s ♥ trt ♦rt♠ ♦s ♦♥ trt♦♥ s s♣t ♥ t♦ ♠♥ st♣s ♦r t ♠①tr ♠♦ r♠♦r♥ ts ♦rt♠ tr♥t② s♠♣s t ss ♠♠rs♣s ♦♥t♦♥② ♦♥t ♣r♠trs ♥ ♦♥ t t ♥ t ♣r♠trs ♦♥t♦♥② ♦♥ t ss ♠♠rs♣s ♥ ♦♥ t t s ts stt♦♥r② strt♦♥ s p(θ, z|x) tr♦r tsq♥s ♦ t ♥rt ♣r♠trs r s♠♣ r♦♠ tr ♣♦str♦r strt♦♥p(θ|x)


s ♦rt♠ ♥ p(θ|①) s ♠r♥ stt♦♥r② strt♦♥ strtsr♦♠ ♥ ♥t θ[0] t♥ tr♥ts t♥ t♦ st♣s t trt♦♥[r] t ♣r♦r♠s t t♦ ♦♦♥ st♣s

z[r] ∼ z|θ[r],x

θ[r+1] ∼ θ|z[r],x.

♦rt♠ s s♠♣r ♦r t ♠①tr ♠♦s

♠♣♥ ♦ t ss ♠♠rs♣ ♥♣♥♥ t♥ ♥s ♦st♦ s② s♠♣ t t♦r z s♥ p(z|θ[r],x) =

∏ni=1 p(zi|θ[r],xi) ♥ z

[r]i

s s♠♣ r♦♠ t ♦♦♥ ♠t♥♦♠ strt♦♥

z[r]i |θ[r],xi ∼ M(ti1(θ

[r]), . . . , tig(θ[r])).

♠♣♥ ♦ t ♣r♠trs ❲♥ tr s ♥♦ ♦♥str♥t t♥ t ♣r♠trs ♦ r♥t sss t ♦♦♥ ♦♠♣♦st♦♥ s s t♦ s♠♣ θ[r+1]

p(θ[r+1]|z[r],x) = p(π[r+1]|z[r])g∏

k=1

p(α[r+1]k |z[r],x).

♦t tt π s ♥♣♥♥t ♦ t t ♦♥t♦♥② ♦♥ t ss ♠♠rs♣s s ♣r♦r ♦ π s t ♦♥t r②s ♥♦♥ ♥♦r♠t ♣r♦r ♥ s s t♣r♦r ♥ t ♣♦str♦r strt♦♥s ♦ t ss ♣r♦♣♦rt♦♥s r rs♣t② ♥②

π ∼ Dg

(

1

2, . . . ,

1

2

)

♥ π|z[r] ∼ Dg

(

1

2+ ♥[r]

1 , . . . ,1

2+ ♥[r]

g

)

,

r r♠♥ tt ♥[r]k =

∑ni=1 z

[r]ik ❲ ♥♦ strt ts ♦rt♠ t t

r♥♥♥ ①♠♣


❲ ss♠ tt t ♣r♦r ♦ σ2k s G−1(c0, C0) ♥ tt t ♣r♦r ♦ µk

♦♥t♦♥② ♦♥ σ2k s N1(b0, B

−10 σ2

k) trt♦♥ [r] ♦ t s s♠♣r♥ p(θ|x) s stt♦♥r② strt♦♥ s rtt♥ s ♦♦s

∀i = 1, . . . , n z[r]i |xi,θ[r] ∼ M2(ti1(θ

[r]), ti2(θ[r]))

π[r+1]|z[r] ∼ D2

(

1

2+ n

[r]1 ,

1

2+ n

[r]2

)

∀k = 1, 2 µ[r+1]k |x, z[r], σ2[r]

k ∼ N1(b[r]k , B

[r]k )

∀k = 1, 2 σ2[r+1]k |x, z[r], µ[r+1]

k ∼ G−1(c[r]k , C

[r]k ),

r b[r]k =B0b0+

∑ni=1 z

[r]ik xi

B0+n[r]k

B[r]k =

σ2[r]k

B0+n[r]k

c[r]k = c0 +n[r]k +1

2♥ C

[r]k =

C0 +12(∑n

i=1 z[r]ik (xi − µ

[r+1]k ) + B0(µ

[r+1]k − b0)

2)

♥♥♥ ①♠♣ s s♠♣r

♠♣ s♠♣♥ ♦♥t♦♥ s t s s♠♣r s t♦ ♣r♦r♠ ♥♠r♦ trt♦♥s t s s♦t② ♥ssr② tt st♣ ♥♦s s♠ s♠♣♥ t♠ t ♦♥t♦♥ strt♦♥s ♦ z[r]

i ♥ π[r+1] r ①♣t t s♠♣♥ ♦α[r+1] ♥ ♠♦r ♦♠♣t ♦♥t ♣r♦r strt♦♥s r s♦ ♥r②s s♥ t② ♣r♦ ss ♣♦str♦r strt♦♥ s t s♠♣♥ ♦ α[r+1]

s s② ♥ tr s ♥♦ ♦♥str♥t t♥ t ♣r♠trs ♥ t s r ts♠t♦♥ ♦ p(α[r+1]

k |z[r],x) s t♦♦ ♠ t♠ ♦♥s♠♥ ♥♦tr ♣♣r♦ t♥t s s♠♣r s t♦ s

tr♦♣♦sst♥s ♦rt♠

♥ ♠ ♦ t tr♦♣♦sst♥s ♦rt♠ s t♦ s♠♣ sq♥♦ θ ♦r♥ t♦ ts ♣♦str♦r strt♦♥ p(θ|x) s ♦rt♠ rqrs ♥ ♥str♠♥t strt♦♥ ♥♦t ② q(.;θ) ♥ t rs♣t t♦ t ♦♠♥t♥♠sr ♦ t ♠♦ t trt♦♥ [r] t ♥str♠♥t strt♦♥ ♥rts ♥t θ⋆ ♦♥t♦♥② ♦♥ t rr♥t ♦ θ ♥ t ♥t s ♣tt ♣r♦t② λ[r] ♥ ②

λ[r] = ♠♥

p(θ⋆|x)q(θ[r];θ⋆)

p(θ[r]|x)q(θ⋆;θ[r]); 1

.


s ♦rt♠ s p(θ|①) s stt♦♥r② strt♦♥ trt♥ r♦♠ ♥♥t θ[0] ts trt♦♥ [r] s rtt♥ s

θ⋆ ∼ q(θ;θ[r])

θ[r+1] =

θ⋆ t ♣r♦t② λ[r]

θ[r] t ♣r♦t② 1− λ[r].

♦rt♠ tr♦♣♦sst♥s ♦rt♠

②r

♥ ❲♥ st♣ ♦ s s♠♣r s t t♦ ♣r♦r♠ t ②r♠♠ ♦rt♠s r ♦t♥ s ♠♦st ♣♦♣r ♣♣r♦ s t♦ s♠♣ sq♥♦ θ ♦r♥ t♦ tr♦♣♦st♥s s♠♣r ♥ ts ♣♣r♦ t tst♣s ♦ t s s♠♣r r r♣ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s♦rt♠ ♦r t stt♦♥r② strt♦♥ ♦ t r♦ ♥ st②s q t♦p(θ, z|x)

s ♦rt♠ ♣r♦r♠♥ t ♥r♥ ♦r t ♠①tr ♠♦s s p(θ|①)s ♠r♥ stt♦♥r② strt♦♥ trt♥ r♦♠ ♥ ♥t θ[0] tstrt♦♥ [r] s rtt♥ s

z[r] ∼ z|θ[r],x

θ⋆ ∼ q(θ;θ[r])

θ[r+1] =

θ⋆ t ♣r♦t② λ[r]

θ[r] t ♣r♦t② 1− λ[r],

r q(.;θ) s t ♥str♠♥t strt♦♥ ♦ t tr♦♣♦sst♥s st♣♥ r λ[r] s ts ♣t♥ ♣r♦t② ♥ ②

λ[r] = ♠♥

p(θ⋆|z[r],x)q(θ[r];θ⋆)

p(θ[r]|z[r],x)q(θ⋆;θ[r]); 1

.

♦rt♠ tr♦♣♦st♥s s♠♣r

❲ ♥♦ strt ts ♦rt♠ t t r♥♥♥ ①♠♣


trt♦♥ [r] ♦ ts ♦rt♠ s rtt♥ s

z[r] ∼ z|θ[r],x

π[r+1] ∼ π|z[r]

α⋆ ∼ q(α;α[r])

α[r+1] =

α⋆ t ♣r♦t② λ[r]

α[r] t ♣r♦t② 1− λ[r],

r q(.; .) s t ♥str♠♥t strt♦♥ ♦ t tr♦♣♦sst♥s st♣♥ r λ[r] s ts ♣t♥ ♣r♦t② ♥ ②

λ[r] = ♠♥

p(α⋆|z[r],x)q(α[r];α⋆)

p(α[r]|z[r],x)q(α⋆;α[r]); 1

.

♥♥♥ ①♠♣ tr♦♣♦st♥s s♠♣r

♦ st♦♥

♥ t ♠♦ st♦♥ ♥

♥t♦♥ ♦ t s ♦♥sr t ♥r ♥t ♠①tr ♠♦

p(xi;θ) =

g∑

k=1

πkp(xi;αk),

θ ∈ Θ r t ♣r♠tr s♣ Θ s ♥ ② t ♥♠r ♦ ♦♠♣♦♥♥ts ♥t ♥tr ♦ ♦♠♣♦♥♥t ♠♦ m r♦♣s t st ♦ t strt♦♥s♥ ② s♦

m = p(xi;θ) : θ ∈ Θ.

♠ ♠♦ m ♥s t ♥tr ♦ t ♦♠♣♦♥♥t strt♦♥s ♥ t♥♠r ♦ ♦♠♣♦♥♥ts s t s ♥r② ♥♥♦♥ t ♠♦ s t♦ ♥rr♦r♥ t♦ t t s ♥ ∆ s t st ♦ t ♠♦s ♦♥sr ②t ♣rtt♦♥r ♥ t ♠ s t♦ ♥ t st ♠♦ ♠♦♥ ∆

♦♦♦ ♥t♦♥ ♥ ♠ ♠♦s ♦♦ ♥t♦♥ ♥r② ♦s t♦ st♠t t st ♠♦ ♦r♥ t♦ t t ♦r ts♣♣r♦ ♥ ♥♦t rt② ♣♣ ♥ t ♠①tr ♠♦ ♦♥t①t ♥ ♥ s s ♦t ♦ ♠♦s r ♠ ♦r ♥st♥ ss♥ ♠①tr ♠♦ ttr ♦♠♣♦♥♥ts ②s ♦t♥s st ♦♦ s t♥ t ss♥ ♠♦st t♦ ♦♠♣♦♥♥ts s t st ♠♦ s t ♠♦ ♠s t sttr ♦ t♥ ts qt② ♦ st♠♥t t♦ t t ♥ ② ts ♦♦ ♥ ts ♦♠♣①t② ♥♠r ♦ ♣r♠trs

♦ st♦♥

♥♦r♠t♦♥ rtr ❲ s ♥ t♦♥ tt rst rtr ts♦♣ ♦ t ♦♦ ♥t♦♥ ♥ s t♦ st t ♥♠r ♦ sss s♣②♦r t ♦♠tr str♥ ♠t♦s ts ♣♣r♦s ♥ s t♦ st t♠♦ ♦ ♣r♦st ♠t♦ t s ♠♦r ♦♥♥♥t t♦ s ♥♦r♠t♦♥ rtr ♣r♦♣♦s ② t ♣r♦st r♠♦r s rtr rtr t r♦r♦s②♣r♦r♠ t ♠♦ st♦♥ ♦r♥ t♦ ♥ ♦t ♦ t st♠♥t rtr ♦r ♥ ♦t ♦ sst♦♥ rtr♦♥ ♥r② ts rtrrqr t ♠ rt t♦ ♠♦ ♥ ∆ s♥ t② ♥ ♦t♥ rtt♥ s ♣♥③t♦♥ ♦ t ♦♦♦ ♥t♦♥

m = L(θ;x)− h(νm),

r θ s t ♠ ♦ t ♠♦ m r νm s t ♣r♠trs ♥♠r ♦ t♠♦ m ♥ r h(.) s ♥t♦♥ ♥ ② t rtr♦♥ ♦t tt qt② ♦ ♥t ♦r t ♥♦r♠t♦♥ rtr♦♥ s t ♦♥sst♥② ♥ ♠♥s♦♥ssr♥ ♦♦ s②♠♣t♦t ♦r

♥t♦♥ ♦♥sst♥② ♥ ♠♥s♦♥ ♦r rtr♦♥ rtr♦♥ s ♦♥sst♥t♥ ♠♥s♦♥ t sts t s♠♣st tr ♠♦ t ♣r♦t② ♦♥ ♥ ts♠♣ s③ t♥s t♦ t ♥♥t②

♥♦r♠t♦♥ rtr ♦r t t st♠♥t

s st♦♥ s ♦t t♦ t ♠♦ st♦♥ st②s t ♣r♦♠ s❬❪ ♣tr ♦r t ♠①tr ♠♦s s♣② t st♦♥ ♦ t ss ♥♠r ♣r♥♣② s♥ t ♠♦s r ♠ ♥ s♥ t ♥♦r♠t♦♥ rtrr ♦♥② s②♠♣t♦t② tr

rq♥tst rtr♦♥

♥ rq♥tst r♠♦r t ♠ s t♦ ♥ t ♠♦ ♠♥♠③♥ t r r♥ ❬❪ ♦ t tr strt♦♥ rt t♦ t st♠t♦♥

♥t♦♥ r r♥ t xi ∈ Re t r

r♥ ♦ t ♣ f(xi) rt t♦ t ♣ g(xi) s

(f, g) =

∫

xi∈X

f(xi) ln f(xi)dxi −∫

xi∈X

f(xi) ln g(xi)dxi.

t f(xi) t ♣ ♦ t tr ♠♦ ♥ t ♠♦ ♠♥♠③♥ s q♥tt♦ ♥ t ♠♦ ♠♥♠③♥ t tr♠ ♦♥ t♥ s ♦ t ♣r♦s qt♦♥s ♦r ♠♦ m t ♠ s t♦ ♦♠♣t

η(xi; f,m, θ) =

∫

xi∈X

f(xi) ln p(xi; θ)dxi,

r p(xi; θ) s t ♣ ♦ t ♠♦ m ♣r♠tr③ ♥ ts ♠ θ s t strt♦♥ f s ♥♥♦♥ s t ♥tr st♠t♦r ♦ η(xi; f,m, θ) ♥ ②

η(xi; f ,m, θ) =1

nL(θ;xi).


♦r ts st♠t♦r srs r♦♠ t ♦♦♥ s

b = Ef

[

η(xi; f ,m, θ)− η(xi; f,m, θ)]

.

s t st ♠♦ ♠♦♥ ∆ ♠①♠③s t ♦rrt ♦♦♦

argmaxm∈∆

L(θ;x)− b.

❬❪ s♦ tt t ♦rrt tr♠ s s②♠♣t♦t② q t♦ t ♥♠r♦ ♣r♠trs

♥t♦♥ rtr♦♥ ♥♦r♠t♦♥ rtr♦♥ s♥ s

(m) = L(θ;x)− ν,

r θ s t ♠ ♦ t ♠♦ m ♥ ν ts ♥♠r ♦ ♣r♠trs

s t rtr♦♥ s ♥ st♠t♦r ♦ t ①♣tt♦♥ ♦ t ♠♥ ♦ t ♦♦♦ st② ♦ t rtr♦♥ ♣r♦♣rts ♥ ♦ ts ①t♥s♦♥ s ♥ ❬♦③❪ ♦r t ♦r ♦ t rtr♦♥ ♥ ♥♦♥sst♥t

Pr♦♣♦st♦♥ s ♥♦t ♦♥sst♥t s ♥♦t ♦♥sst♥t ♥ ♠♥s♦♥ ♥♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts r ♠

Pr♦♦ t ♠♦ m0 ♦s t ♠ ♦ ♠♥s♦♥ ν0 s ♥♦t ② θ0 ♥ t m1

♦s t ♠ ♦ ♠♥s♦♥ ν1 s ♥♦t ② θ1 s s m0 s t tr ♥♠r m0

♥ m1 r ♠ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts ♥ ν0 < ν1♥

2 ((m1)− (m0)) = 2(

L(x; θ1)− L(x; θ0))

− 2(ν1 − ν0)

D→ χ2ν1−ν0

− 2(ν1 − ν0).

s t rtr♦♥ s ♥♦t ♦♥sst♥t limn→∞

P ((m1) > (m0)) > 0)

s♥ P (χ2ν1−ν0

> 2(ν1 − ν0)) > 0 ♦t tt t ♠♦♥strt♦♥ ♥ ♥♦t ♣r♦r♠ t♦ st t ♥♠r ♦ sss ♥ ♥ s s t ♦♥r♥ t♦ t♦♦ rt♦ s ♥♥♦♥ s♥ t ♥♦s ②♦rs ♦♣♠♥t ♦♥ t ♦rr♦ t ♣r♠trs s♣ ♦r t s ♦t♥ ♥ ♦sr tt t rtr♦♥sts ♠♦r ♦♠♣t ♠♦s ♥ t tr ♦♥ s ♥ t st ♦ t ♠♦s❬❪

②s♥ rtr♦♥

t p(m) t ♣r♦r strt♦♥ ♦ m ∈ ∆ t ♣♦str♦r strt♦♥ ♦ ♥trsts ♥ ② s♥ t ②s r s ♦♦s

p(m|x) = p(x|m)p(m)

p(x).

♦ st♦♥

♥ ♥ ②s♥ ♣♦♥t ♦ t st ♠♦ m⋆ ♠①♠③s t ♣♦str♦rstrt♦♥

m⋆ = argmaxm∈∆

p(m|x) = argmaxm∈∆

p(x|m)p(m).

s t q♥tt② ♣r♦r♠♥ t ♠♦ st♦♥ s t ♥trt ♦♦ s♦♥♠ ♠r♥ ♦♦ ♦r ♥ ♥ ②

p(x|m) =

∫

θ∈Θ

p(x,θ|m)p(θ|m)dθ,

r t ♣r♠tr s♣ Θ ♣♥s ♦♥ m ts q♥tt② ♥ ♣♣r♦ ♠♥② ♠t♦s s t r ♦ ❬❲❪ t ♠♦st ss ♦♥ s t♦ st ♣♣r♦①♠t♦♥ ❬❪ ♣♣r♦①♠ts ln p(x|m) ② s♥ ♣♣♣r♦①♠t♦♥ ♥ ② r♣♥ t ♠♣ ② t ♠

♥t♦♥ rtr♦♥ ②s♥ ♥♦r♠t♦♥ rtr♦♥ s♥ s

(m) = L(θ;x)− ν

2lnn,

r θ s t ♠ ♦ t ♠♦ m ♥ r ν ♥♦ts ts ♥♠r ♦ ♣r♠trs

s rtr♦♥ ss♠s rrt② ♦♥t♦♥s ♦♥ t ♣ ♠② ♥♦t r② t ♠①tr ♠♦s rtr♠♦r ts ♣♣r♦①♠t♦♥ s ♦♥② s②♠♣t♦t② tr

Pr♦♣♦st♦♥ s ♦♥sst♥t rtr♦♥ s ♦♥sst♥t ♥ ♠♥s♦♥♥ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts r ♠

Pr♦♦ t ♠♦ m0 ♦s t ♠ ♦ ♠♥s♦♥ ν0 s ♥♦t ② θ0 ♥ t m1

♦s t ♠ ♦ ♠♥s♦♥ ν1 s ♥♦t ② θ1 s s m0 s t tr ♥♠r m0

♥ m1 r ♠ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts ♥ ν0 < ν1♥

2 ((m1)− (m0)) = 2(

L(x; θ1)− L(x; θ0))

− 2(ν1 − ν0) lnn

D→ χ2ν1−ν0

− 2(ν1 − ν0) lnn.

② s♥ t ♥♦tt♦♥ ∆ν = ν1− ν0 t ♦♦♥ rst s ♦t♥ ② ♣♣②♥ t②s ♥qt② ♥ n s r

P ((m1) > (m0)) ≤ P (|χ2∆ν

−∆ν | > ∆ν(−1 + lnn))

≤ 2∆ν

(∆ν(−1 + lnn))2n→∞→ 0,

s♥ E[χ2∆ν

] = ∆ν ♥ ❱r(χ2∆ν

) = 2∆ν s t rtr♦♥ s ♦♥sst♥t♥ ♠♥s♦♥ ♥ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts r ♠♦t tt t ♠♦♥strt♦♥ ♥ ♥♦t ♣r♦r♠ t♦ st t ♥♠r ♦ sss♥ ♥ s s t ♦♥r♥ t♦ t ♦♦ rt♦ s ♥♥♦♥ s♥ t♥♦s ②♦rs ♦♣♠♥t ♦♥ t ♦rr ♦ t ♣r♠trs s♣ ♦r t rtr♦♥ s ♠♦r r♦st t♥ t ♦♥ t ♥ ♦rst♠ts t ♥♠r♦ ♦♠♣♦♥♥ts ♥ t tr ♠♦ s ♥♦t ♥ ∆ ❬❪

♦t tt ② s♥ ♦② ♦♥ ♣r♠tr③t♦♥ ❬r❪ s♦s tt t rtr♦♥ s ♦♥sst♥t st♠t♦r ♦ t ♦rrt ♥♠r ♦ ♦♠♣♦♥♥ts ♥ t strt♦♥


rs ♠♣ ∆ s r ♦r t st♠t♦♥ ♦ t ♣r♠trs s ♦♠♣①t♥ t ①st ♣♣r♦ s ♥♦t ♦ ♥ ts ♣♣r♦ ♦♥ssts ♥ ♦♠♣tt♦♥ ♦ ♥ ♥♦r♠t♦♥ rtr♦♥ ♦r t ♠♦s s♦ t s t♠ ♦♥s♠♥rtr♠♦r t ♣rtt♦♥r ♦♥② ss t st♠t ss♦t t♦ t st ♠♦s t ♦tr st♠ts r ♥♦t s ♦r t t ♥②ss s r s♦ ② t ♣♣r♦ ♦ t rrs ♠♣ ❬r ❪ r t ♠♦ ♥t ♣r♠trs r s♠t♥♦s② st♠t ❯♥♦rt♥t② ts ♣♣r♦ ♥♦st ♦♠♣tt♦♥ ♦ t ♣r♦ts ♦ t ♠♦ tr♥st♦♥ ♥ ♦♠♣①♦r ts ♦t ♦♥ssts ♥ ♦♥ t st♠t♦♥ ♦ t ♣r♠trs ♦ t ♠♦s ♥ ∆ t ♥ ♦♠ ♠♥t♦r② ♥ t ♠♦ s♣ ♦♠s s Prt

♥♦r♠t♦♥ rtr♦♥ ♦r t ♣rtt♦♥ st♠♥t

♥ Pr♦①② t ♦♥sst♥② ♦ t ♥♦r♠t♦♥ rtr♦♥ ♥ r ♥ s t ♠♦s r r♦♥ t rtr♦♥ s②♠♣t♦t②♦rst♠ts t ♥♠r ♦ ♦♠♣♦♥♥ts ♦r♥ t♦ t ss s♣rt♦♥ ♦ r♥ ① ♥ ♦rt ❬❪ ♣r♦♣♦s t♦ ♥ sst♦♥♦t ♥ t ♥♦r♠t♦♥ rtr♦♥ ♥ s s t st ♠♦ ♠①♠③s t♥trt ♦♠♣tt ♦♦ s t t♦r z s ♥♥♦♥ t s r♣ ②ts ♠♣ ♥♦t ② z t t t ♠

♥t♦♥ ①t ♥trt ♦♠♣tt ♦♦ sss ♠♦ t sst♦♥ ♠ t s ♥ s

①(m) = ln p(x, z|m)

= ln

∫

θ∈Θ

p(x, z|m,θ)p(θ|m)dθ.

♦r ♥ t ♥tr ♥ ①♣t s ♦r ♥st♥ t st② ♥❬❪ ♦r t ♠ ♦ ♠t♥♦♠ strt♦♥s t s ♥♦t ♥r② t ss ♥ ♣♣r♦①♠t rs♦♥ ♦ ts rtr♦♥ s

♥t♦♥ ♥trt ♦♠♣tt ♦♦ ♥ ♣♣r♦①♠t ②

(m) = ln p(x, z|m, θ)− ν

2lnn.

♦♥s♦♥

♣♣t♦♥ ♦♥ r t st ♦ t ♥♦r♠t♦♥ rtr

❲ ♥ ①♠♣ ♦ t s♥ ♦ t ♥♦r♠t♦♥ rtr ♦♥ t t t st ♦r r♥t ♥♠rs ♦ sss t ♦rt♥ ♣rs♠♦♥♦sss♥ ♠①tr ♠♦s ❬❪ ♦♠♣♦s t st ♦ t ♦♥sr ♠♦s r s♣②s t s ♦ t ♥♦r♠t♦♥ rtr ♦r r♥t♥♠rs ♦ sss ♦♦♦ ♥t♦♥ s ♥rs♥ t t ss ♥♠r rtr♦♥ sts ♦r sss t rtr♦♥ sts tr sss♦r ♦t rtr stt t♦ st t ♥♠r ♦ sss s t♣rtt♦♥r s♥ rs♣t② ♥ s♦ ♥②s t ♣rtt♦♥ ♥tr sss rs♣t② t♦ sss ♥② t rtr♦♥ str♦♥②sts t ♣rtt♦♥ ♥ t♦ sss s ♣rtt♦♥ s♠s rst ♦r♥t♦ t sttr ♣♦t

t t st ♦ st♦♥ ② ♥♦r♠t♦♥ rtr

1 2 3 4 5 6

−12

50−

1200

−11

50

number of classes

crite

rion

valu

es

log−likeaicbicicl

♥♦r♠t♦♥ rtr

1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

5060

7080

90

eruptions

wai

ting

ttr ♣♦t t t ♣rtt♦♥ ♥ t♦ sssst ②

r rtr♦♥ s ♦r t st ♦ t ♦rt♥ ss♥ ♠①tr ♠♦s♦r r♥t ♥♠r ♦ sss t♦ str t t t st

♦♥s♦♥

s ♦r♣ ♣tr s ♣rs♥t t ♥r r♠♦r ♦ t ♠①tr ♠♦s ♥ s ♥ strt ② t ♦♠♣♦♥♥t ss♥ ♠①tr ♠♦s t s ♣♦♥t♦t tt t ♥r ♠ ♠♦ ♦s t♦ s② str ♦♠♣① t t♦r ♦r ♠① t t t rs ♦ s ♥ t t r♥trss ♦rrt

♦t st♠t♦♥ ♠t♦s rq♥tst ♥ ②s♥ ♥ ♣rs♥t ♥ tstss ♦r t rq♥tst ♣♣r♦ t♦ ♥r t ♣r♠trs s♥ t ♦s ♥♦t


rqr s♦♠ ♥♦r♠t♦♥ ♣r♦r ♦r t ♠ s ♥trt ♣r♦r♠t ♥r♥ ♥ t ②s♥ r♠♦r ♥ s s ♦r t ♦♥t ♣r♦rstrt♦♥s ♥ ♦rr t♦ s② st♠t t ♣r♠trs ②♣r♣r♠trs ♦t ♣r♦r strt♦♥s r st t♦ ② ♥♦r♠t

♠♦ st♦♥ s ♣r♦r♠ ② s♥ t rtr♦♥ s t ♠♦st ♦♠♠♦♥♥♦r♠t♦♥ rtr♦♥ ♦r t ♠①tr ♠♦s

♣r♣♦s ♦ t ♦♦♥ ♣trs s t♦ st② ♥ t♦ ♣r♦♣♦s ♥ ♠①tr♠♦s ♦♥ t♦ str ♦♠♣① t t♦t ss♠♥ t ♦♥t♦♥ ♥♣♥♥ t♥ t rs ❲ ♥♦ ♦s ♦♥ t t♦r t st str♥

Prt

♦s str♥ ♦r

t♦r t

s ♣rt ♦t t♦ t str ♥②ss ♦ t♦rt s s♣t ♥t♦ ♦r ♣trs rst ♦♥ ♣rs♥ts ♥ ♦r ♦ t str♥ ♣♣r♦s ♦t t♦ t t♦r t sts ❲ ♠♥②♦s ♦♥ tr ♠♥ ♠♦s ♣♣r♦s t ♦♥r ♠①tr ♠♦s t ♠①trs ♦ trs ♥ t ♠t t♥t ss ♠♦s s ♠♦s r strt♦♥ s♠ r t st s♦♥ ♥ t tr ♣trs ♣rs♥t ♦r ♦♥trt♦♥s t♦ ts r♠♦r ❲ ♣rs♥t t♦ ♥ ♠①tr♠♦s r ①t♥s♦♥s ♦ t ss t♥t ss♠♦ ♦r s ♠♦s t rs r r♦♣ ♥t♦♦♥t♦♥② ♥♣♥♥t ♦s s♣ strt♦♥s ♦ t ♦s ♠♦③ t ♥trss ♣♥♥ss rsts r ♣rt ♦ t♦ s♠tt rts st ♣tr s ♦t t♦ ♣r♦♣♦s ♠♦ ♦♠♣rs♦♥ strt ♦♥ t ①♠♣ ♦ t ♦r ♣tr s♦♥ ♣r♣♦s ♦ ts ♣tr s t♦ ♣rs♥t ♦r ♣s ♣r♦r♠♥ t ♥r♥ ♦ ♦t ♣r♦♣♦s♠♦s

♦s ♦♥② sr♠♥ ♦ tt t s t t t t trr♦s ♥ tr ♣s r t♥ ②♦trs ♦ r ♦♥♥ tt t

s t t ♦ t♦♥ t♥ ♦rt t

♦ ♦♥t♥ts

str ♥②ss ♦ t♦r t sts stt ♦ t rt

♥ ♦ str ♥②ss ♦r t♦r t


♦♥r ♠①tr ♠♦s

①trs ♦ trs

t t♥t ss ♠♦

♦♥s♦♥

♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s

♥tr♦t♦♥

①tr ♦ ♥trss ♥♣♥♥t ♦s

Prs♠♦♥♦s ♦ strt♦♥

①♠♠ ♦♦ st♠t♦♥ ♦rt♠

♦ st♦♥ ♦rt♠

♠r ①♣r♠♥ts ♦♥ s♠t t sts

♥②ss ♦ t♦ r t sts

♦♥s♦♥

♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♥tr♦t♦♥

①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠

♦ st♦♥ tr♦♣♦st♥s s♠♣r



♦♥s♦♥

♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

strt

♦♦s

♦ ♦♥t♥ts

♦♥s♦♥ ♦ Prt

♣tr

str ♥②ss ♦ t♦r t

sts stt ♦ t rt

♣r♣♦s ♦ ts ♣tr s t♦ ♣rs♥t t ♠♥ ♣♣r♦s t♦ str t♦r t sts rst st♦♥ s ♦t t♦ t ♦♠tr ♠t♦s♥ ♠♦r ♣rs② t♦ t ♠♥s ♦♥s ♦tr st♦♥s sr t tr ♠♥ ♠♦s♠t♦s ♦r ♣rs② t s♦♥ st♦♥ ♦ss♦♥ t ♦♥r ♠①tr ♠♦s t s♦ ts tss t♥t ss ♠♦ s s♣ ♠♦♦ t ♦♥r ♠①tr ♦♥ ss♠♥ t ♦♥t♦♥♥♣♥♥ t♥ t rs s ♠♦ s ♦♥trst ♦r s s♥ t ♣r♦♣♦s ♠♦s ♥tr♦ ♥t t♦ ♦♦♥ ♣trs ♦♥sst ♥ t♦ ①t♥s♦♥s♦ ts ♠♦ tr st♦♥ s ♦t t♦ t ♣rs♥tt♦♥ ♦ t tr ♠①tr ♠♦s st st♦♥♣rs♥ts t ♠t t♥t ss ♠♦ss ♠♦s r strt ♦♥ r t st tr♦♦t ts ♣tr

②s ♦ s♦r t ②♦ s ②♦♦ r♥ t t ②♦ t♦ ♣

②♦r ♠♦t str♥st ♠♥② ♦rt♣♣② ♦ r♥s ♦♠r

♥ ♦ str ♥②ss ♦r t♦r t

♥tr♦t♦♥ t♦r rs r ♦t♥ ♣rs♥t ♥ t t sts s♥t② r s② ss t② ♥♦ ② s rs s ♦ rst②t s ♥♦t ♦♥♥♥t t♦ s③ t♦r t ♥ tr ♥t s♣ ♦ ts s t♦ ♦♥tr♥ ② ♥ s② ♥tr♣rtt♦♥ ♦ t ♣rtt♦♥ ♦♥② t

♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt

♦♠♥t♦r ♣r♦♠s r qt♦s ♥ s♦♠ ♥trss ♣♥♥s t♦ ♠♦③ ♥♠r ♦ ♣r♠trs ♥ ♥♠r ♦ ♠♦s ♥ ♦♠♣tt♦♥ s t ♣♣rs ♠♣♦rt♥t ♦r s tt t ♠♦s s t♦ str rs♣t tt♦ ♦♦♥ ♦ts

♦ r ♦ts s ♦♥ t ♠♦s ♣rs♥t ♥ ts ♦r♣♣tr ♣t t t ♦♥ t ♦♦♥ r ♦ts

♦s t♦ ♣r♦ ♠♥♥ ♣r♠trs t♦ ♦♥tr♥ t ♦ s③t♦♥

♦s t♦ t ♥t♦ ♦♥t t ♥trss ♣♥♥s ② ♠t♥ t♦♠♥t♦r ♣r♦♠s rt t♦ ♦t ♦ t ♥♠r ♦ ♣r♠trs ♥ ♦t ♠♦ st♦♥

t r♦♦t ts ♣rt ♦♥sr t drt t♦r ♦ t♦rrs ♥♦t ② xi = (x1

i , . . . ,xdi ) ♥ ♥ ♥ s♣ X t♦r

r xji = (xjhi ;h = 1, . . . ,mj) smj ♠♦ts ♥ ss ♦♠♣t s♥t♦♥ s s xjhi = 1 ♥ i ts ♠♦t② h ♦r r j ♥ xjhi = 0♦trs

trtr ♦ ts ♣tr t♦♥ ♦ss ♦♥ t ♦♠tr ♠t♦s ♣r♠tt♥ t♦ str t♦r t sts ♦tr st♦♥s r ♦t t♦ ♣r♦st♠t♦s ♥ t♦♥ ♣rs♥ts t ♦♥r ♠①tr ♠♦ s t rr♥ t♦ str t♦r t ♥ ♣r♦st r♠♦r t♦♥ ♣rs♥tst ♠①tr ♦ ♣♥♥② trs t♦♥ ♣rs♥ts t ♠t t♥t♠♦s

♥♥♥ ①♠♣ r♥ ts ♣tr t r♥t ♠t♦s ♦♥ t♦ strt♦r t sts r strt ♦♥ t ♥♠♥s ♥tstr② t ❬❪ s ss t♦r t st

s ♥r② t st ♣rs♥t ♥ ♦♥ssts ♥ t ♥♦ss ♥② ♥tsts ♦r ♣r♠♦rs ♥ ♠♦rs ♠ s t♦ rtr③t ♦r ♦ t ♥tsts ♦r♥ t♦ tr ♥♦ss

♥♥♥ ①♠♣ ♥tstr② t st


t♦s ♦♥ t ♥t s♣

♠♥s ♦rt♠ ♦r t♦r t

♥ ♠♥s ♦rt♠ ♦r t♦r t ♣r♦♣♦s ② ♠♦♥r♥② ❬❪ ss t ♦♠♣t s♥t ♦♥ ♦ t t♦r rs


♥tst ♥tst rq♥② rq♥②

♦r♣ r♦ss♥♦ss ♦ ♠♦rs ♥ ♣r♠♦rs ② ♥tsts ❬❪ t r ♥♦s s s♦♥ ♦r r♦s

♥ ♦r s ♣♣r♦ xjhi s ♦♥sr s ♥r② r st♥s t♦ str s t sqr ♦♥ tt ♦ t

sqr st♥ sqr st♥ ts ♥t♦ ♦♥t t t ♦ ♠♦t② ♥ t st♥ ♦♠♣t ② ♥ ♠♦r ♠♣♦rt♥ t♦ t rr♠♦ts t♥ t♦ t ♠♦st ♦♠♠♦♥ ♦♥s

♥t♦♥ sqr st♥ t ① = (x1, . . . ,xn) t♦ t s♠♣ ♦♠♣♦s ② n ♥s xi sr ② d t♦r rs sqr st♥ t♥ xi1 ♥ xi2 t 1 ≤ i1, i2 ≤ n s ♥ ②

Dχ2(xi1 ;xi2) =d∑

j=1

mj∑

h=1

(xjhi1 − xjhi2 )2

njh,

r njh =∑n

i=1 xjhi

♦♠♠♥ts ♠♥ r ♦ ts ♣♣r♦ s tt t t♦r ♦ t ss♠♥sr s t♥ ③r♦ ♥ ♦♥♦s ♥♦t ♥t t rtrsts ♦t sss ♦t♥ ♣rtt♦♥ s s♦ ② ♥tr♣rt

♠♦s ♦rt♠ ♦r t♦r t

♥ ♠♦s ♦rt♠ s ♥ ♣r♦♣♦s ② ❩ ♥ ❬❪ t♦♦ t ♣r♦♠ rt t♦ t str ♥②ss ♦ r t♦r t sts s


♦rt♠ ①t♥s t ♠♥s ♦♥ ② s♥ s♠♣ ♠t♥ ss♠rt② ♠sr♦r t♦r rs t trt♦♥ t ♣ts t ♠♦s ② rq♥②s ♠t♦ ♥ ♦rr t♦ ♠♥♠③ ♦st ♥t♦♥

ss♠rt② ♥ ♠♦s

♥t♦♥ t♥ ss♠rt② t t♦ drt t♦r rs xi1♥ xi2 ♠t♥ ss♠rt② ♦♥ts t ♠s♠ts t♥ ♦t xi1 ♥xi2 s ss♠rt② s ♥ ②

D1(xi1 ,xi2) =d∑

j=1

δ(xji1 ,xji2) t δ(xji1 ,x

ji2) =

1 xji1 6= xji2

0 xji1 = xji2.

♥t♦♥ ♦ ❬❪ ♠♦ ♦ t s♠♣ ① = (x1, . . . ,xn) s t♦rµ ∈ X s s µ = (µjh; j = 1, . . . , d;h = 1, . . . ,mj) ♠♥♠③s

D(①;µ) =n∑

i=1

D1(xi;µ).

♦t tt µ s ♥♦t ♥ssr② ♥ ♠♥t ♦ ① ♥ tt t s ♥♦t ♥ssr② ♥q

♣t♠③ rtr♦♥ ❲♥ t ss♠rt② ♥ ② ♥t♦♥ s st♥ t ♠♦s ♦rt♠ ♦♣t♠③s t ♦♦♥ rtr♦♥

I(③,θ;①) =n∑

i=1

g∑

k=1

d∑

j=1

δ(xji ,µjk),

r θ = (µ1, . . . ,µg) ♥ r µk s t ♠♦ ♦ ss k

♦rt♠

trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ ss ♠♠rs♣ z

[r] = argminz

I(z,θ[r];x)

z[r]ik =

1 k = argmink′

D1(xi,µ[r]k′ )

0 ♦trs

♥tr♦ st♠t♦♥ θ[r+1] = argminθ

I(z[r],θ;x)

µ[r+1]k = argmin

µk

n∑

i=1

z[r]ikD1(xi;µk).

♦rt♠ ♠♦s ♦rt♠


♦♠♠♥ts ♠♦s ♦rt♠ t ♠♥s ♦♥ ♦♥rs t♦ ♦♠♥♠♠ ♦ t ♥t♦♥ I(③,θ;①) t s s♦ ♠♥t♦r② t♦ ♣r♦r♠ r♥t ♥t③t♦♥s ♥ ♦rr t♦ ♦♣ t♦ t t ♦ ♠♥♠♠ ♦ ts ♥t♦♥ ♥② ♥♦ttt t ♥tr♦ st♠t♦♥ st♣ s tt ② t ♥t♦♥ ♦ D1(., .) ♥ts ♦♣t♠③t♦♥ s ♣r♦r♠ ♦♦r♥ts ② ♦♦r♥ts

♣♣r♦ s ♦♥ t rtr♦♥ s ♦♠♣t ♦r r♥t ♥♠rs♦ sss t♦ ♣rtt♦♥s ♦ ♦ ♥trst

♥tr♣rtt♦♥ rst ♦♥ s♣ts t t ♥t♦ t♦ sss ♦s t♠♦s r ♥ ② ♦t ♠♦st ♣rs♥t ♥♦ss ♥tsts ♠ ttt t♦♦t s s♦♥ ♥ ♥tsts ♠ tt t t♦♦t s s♦♥ ①♣t tst ♥tst s♦♥ ♦♥ s♣ts t t ♥t♦ tr sss t st t t♦ ♣r♦s ♠♦s t ♥♦ss r ♥tsts ♠ tt tt♦♦t s r♦s ①♣t t rst ♥tst s② t ♠♦s ♣♣r♦♦s ♥♦t ♦ t♦ r② ♥rst♥ ts t st

♥♥♥ ①♠♣ ♠♦s str♥

t♦s ♦♥ t t♦r s♣

♥ ❲♥ ♠♥② rs r ♦rrt t② ♣r♦ s♦♠ r♥♥t ♥♦r♠t♦♥ s t ♥ ♥t t♦ ♣r♦r♠ st♦♥ ♦ rs ♦r rt♦♥♦ t s♣ ♠♥s♦♥ ❲♥ t rs r t♦r t♣ ♦rrs♣♦♥♥ ♥②ss ♥ s ♥ ♦rr t♦ r t s♣ ♠♥s♦♥ ♥ts ♠t♦ ♣r♦s ♥♠r ♦♦r♥ts ♦r ♥ r♦r t s♣♦ss t♦ s ss ♦♠tr ♣♣r♦ t♦ str ♥♠r t t ♠♥s ♦rt♠ ❲ ♣rs♥t t ♠t♦ ♦ ♥ ❲ ♦♥ ♥ ❨ ♥❬❪ ♦♠♥s ♥ ♠♥s ♦rt♠ ♥ ♥ r♠♦r

♦tt♦♥s ❲ r♠♥ tt t s♠♣ ① = (xi; i = 1, . . . , n) s ♦♠♣♦s t♥s sr ② d t♦r rs s s♥t ♦♥ tf ♥♦t♥ t ♠tr① ♦ s③ n× d r d ≤ mj ♦rrs♣♦♥s t♦ t d♠♥s♦♥r♣rs♥tt♦♥ ♦ t d t♦r rs t wj t ♠tr① ♦ ts ♦ s③mj × d ❲ ♦♥sr ③ s t ♠tr① ♦ s③ n× g r t r♦s ♦rrs♣♦♥ t♦ t♥s ♥ t ♦♠♥ t♦ t ss ❲ ♥♦t ② θ t ♠tr① ♦ t ♥tr♦s ♦ t str ♥ t t♦r s♣

♣t♠③ rtr♦♥ ♠ s t♦ ♦♠♥ ♥ ♠♥s ♦rt♠ s♦t ♣r♦♠ s q♥t t♦ t ♠♥♠③t♦♥ ♦ t ♦♦♥ rtr♦♥

Iα1,α2(③,θ;①) = α1

d∑

j=1

SS(f − ①jwj) + α2SS(f − ③θ),


r α1 > 0 α2 > 0 α1 + α2 = 1 r ①j s t ♠tr① r t ♠♥t (i, h)s q t♦ ♦♥ ♥ i ts ♠♦t② h ♦r r j ♥ s q t♦ ③r♦♦trs ♥ r SS(f) = tr(f ′f)

♠r ♥ t ♦♣ α1, α2) ❲♥ α1 = 1 t rtr♦♥ ♥ ② st st♥r ♦♥ ♦r ❲♥ α2 = 1 t rtr♦♥ ♥ ② s q♥tt♦ t st♥r ♦♥ ♦r t ♠♥s ♦rt♠ s ♦r ♦trs s ♦ (α1, α2)ts rtr♦♥ ♣r♦r♠s tr♦ t♥ t ♥ t ♠♥s ♦ts

♦rt♠ st♠t♦♥ ♦ (f ,wj, ③,θ) s ♣r♦r♠ ② t tr♥t♥st sqrs ♦rt♠ ♣r♦♣♦s ② ❬❪ s ♦rt♠ ♦♥rs t♦ ♦ ♠♥♠♠ ♦ t ♥t♦♥ Iα1,α2(③,θ;①) ♦ sr r♥t ♥t③t♦♥s ♦ts ♦rt♠ t♦ ♦♥ ♥ ♦rr t♦ ♦t♥ t st♠t♦rs ♠♥♠③♥ tsrtr♦♥

trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ ❲t ♠tr① ♥ ♥tr♦s ♦♣t♠③t♦♥

wj[r] = (①j′①j)−1①j

′f ♥ θ[r] = (③[r]

′③[r])−1③[r]

′f .

t♦r s♣ ♦♣t♠③t♦♥

f [r+1] = argmaxf

tr

(

f ′

[

α1

d∑

j=1

①j(①j′①j)−1①j

′+ α2③

[r](③[r]′③[r])−1③[r]

′

]

f

)

.

Prtt♦♥ ♦♣t♠③t♦♥

③[r+1] = argmin③

SS(f [r+1] − ③θ[r]).

♦rt♠ ♦rt♠ ♠♥♠③♥ Iα1,α2(③,θ;①) ❬❪

♦♠♠♥ts ♥ t♦♥ t♦ t ss ♠ts ♦ t ♦♠tr ♣♣r♦s tr♣r♦♠s r rs ② ts ♠t♦

rst ♣r♦♠ s ♦t t ♣r♠trs (α1, α2) r ① ② tsr ♥ tr s ♥♦ r ♥t② tr♠♥s t♠ tr♠♣ts ♦♥ t ♣rtt♦♥ r s♥♥t

s♦♥ ♣r♦♠ s ♦t t s③ ♦ t t♦r s♣ ♥ t s♣♠♥s♦♥ d s rtrr② ① ② t sr t rs ♦ ♦♦s♥ ♥♦r♠t♦♥

st ♣r♦♠ s ♦t t ss ♥tr♣rtt♦♥ s ♦♠♣① ♥t sss r s♠♠r③ ② t ♥tr♦s r ♥♦t ♥ ♥ t♥t s♣ t ♥ t t♦r s♣ rt ② ♦♠♥t♦♥s ♦ t ♦r♥rs

♦♥r ♠①tr ♠♦s

♦♥r ♠①tr ♠♦s

♥ ❲ s♥ ♥ ♣tr tt t ss♥ ♠♦ ♥ ss ♦♠♣♦♥♥t strt♦♥ ♥ t rs r ♥♠r ♥ t s♠ ② t♦♥r ♠♦ s t♦r t ♥②ss ② rst ❬r❪ s ♥tr②s s ♦♠♣♦♥♥t strt♦♥ ♥ t rs r t♦r ♦r t♦♠♣t ♦♥r ♠♦ st♠ts t ♣r♦t② ♦ t ♠♦t② r♦ss♥s ts s♦ ♠♥t♦r② t♦ ♠♣♦s ♦♥str♥ts ♦♥ ts ♠♦ t♦ t ♦♥r ♠①tr♠♦

trtr ♦ ts st♦♥ ss ♣♣r♦ ss♠s ♦♥t♦♥ ♥♣♥♥ t♥ t rs s ♠♦ s ♥♠ t♥t ss ♠♦ ♦r ♥ ②s❬♦♦❪ ♥ s ♥ t ♥ t♦♥ tr ♦♥r ♠①tr ♠♦s r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ r ♣rs♥t ♥ t♦♥

t♥t ss ♠♦

♦ ♣rs♥tt♦♥

♥ s ♠①tr ♠♦ ss♠s tt t rs r ♥♣♥♥t ♦♥t♦♥② ♦♥ ss ts ts ♦♠♣♦♥♥ts ♦♦ ♣r♦t ♦ ♠t♥♦♠ strt♦♥s

♥t♦♥ t♥t ss ♠♦ t xi t drt t♦r rs♥ s♥t ♦♥ xi rs r♦♠ t t♥t ss ♠♦ t g ♦♠♣♦♥♥tst♥ ts ♣ s rtt♥ s ♦♦s

p(xi;θ) =

g∑

k=1

πkp(xi;αk) t p(xi;αk) =d∏

j=1

mj∏

h=1

(αjhk )xjhi ,

r θ = (π,α) r π s ♥ ♦♥ t s♠♣① ♦ s③ g rα = (α1, . . . ,αg)

♥ r αk = (α1k, . . . ,α

dk) s s tt αj

k = (αjhk ;h = 1, . . . ,mj) s ♥ ♦♥t s♠♣① ♦ s③ mj ♦t tt αjhk ♥♦ts t ♣r♦t② tt ♥ ♥rs♥ r♦♠ ♦♠♣♦♥♥t k ts ♠♦t② h ♦r r j

s♣t ts s♠♣t② t t♥t ss ♠♦ s t♦ ♦♦ rsts ♥ ♣rt❬❨❪ ♦r r♥t rs t ♦r s♥s ❬+❪ ♦r ♥ ♠♥❬+❪

♦ ♥tt② ♥r ♥tt② ♦ t t♥t ss ♠♦ s♣r♦ ② ♠♥ ts ♥ ♦s ❬❪ ❲ ♥♦ ♣rs♥t trt♦r♠ rr ♥trst ② ts ♣r♦♦ ♥ rr t♦ t rt ❬❪

♦r♠ ♥r ♥tt② ♦ t t♥t ss ♠♦ ❬❪ t t♠♦ ♥ ② ♥t♦♥ t d ≥ 3 ♣♣♦s tr ①sts tr♣rtt♦♥ ♦t st S = 1, . . . , d ♥t♦ tr s♦♥t ♥♦♥♠♣t② ssts S1 S2 S3 s tt κb =

∏

j∈Sbmj t♥

min(g, κ1) + min(g, κ2) + min(g, κ3) ≥ 2g + 2.


♥ ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥ ♦r♦r t stt♠♥t r♠♥s ♥ t ♠①♥ ♣r♦♣♦rt♦♥s π r ① ♥♣♦st

♥s t t ♦♠tr ♣♣r♦s s s♦♥ ♥ ❬♦❪ t ♦♠tr♣♣r♦ ♦♦♥ ♦r t ♣rtt♦♥ ♥t♦ g sss ♠①♠③♥ t ♥♦r♠t♦♥ rtr♦♥♦r t χ2 rtr♦♥ s ♣♣r♦①♠t② q♥t t♦ ss♠ tt ♥s r r♥② t♥t ss ♠♦


♥r♥ ♦ t t♥t ss ♠♦ ♥ ♣r♦r♠ ♥ rq♥tst ♦r ♥ ②s♥ r♠♦r

♥ rq♥tst ♣♦♥t ♦ t st♠t♦♥ ♦ t ♠ ♥ ♣r♦r♠ ♥ ♠ ♦rt♠ ♣rs♥t ♦ ♦r ② ts ①t♥s♦♥s ♦t tt t ♦♦♥t♦♥ s ♣♣r♦♥ s♦ tr s ♥♦ ♥r② ♣r♦♠

♥ ②s♥ r♠♦r t st♠t♦♥ ♥ ♣r♦r♠ ② s s♠♣r♦t tt ② ♦♦s♥ t r②s ♥♦♥ ♥♦r♠t ♦♥t ♣r♦rs t ♣♦str♦rstrt♦♥s r ①♣t ♥ ♥ ①t ♥♦r♠t♦♥ rtr♦♥ ♥ ♦♠♣t

❲ ♥♦ t ♦t rq♥tst ♥ ②s♥ ♣♣r♦s

rq♥tst r♠♦r ♠ ♥ s② ♦t♥ ② t ♦♦♥ ♠

♦rt♠

trt♥ r♦♠ t ♥t ♦ θ[0] trt♦♥ [r] ♦ t ♠ ♦rt♠ srtt♥ s

st♣ t ♦♥t♦♥ ♣r♦ts

tik(θ[r]) =

π[r]k p(xi;α

[r]k )

p(xi;θ[r])

.

st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

π[r+1]k =

n[r]k

n♥ αjh[r+1]

k =

∑ni=1 tik(θ

[r])xjhi

n[r]k

,

r n[r]k =

∑ni=1 tik(θ

[r])

♦rt♠ ♦rt♠ ♦r t t♥t ss ♠♦

②s♥ r♠♦r ss ss♠♣t♦♥ ♦ t ♥♣♥♥ t♥ t♣r♦r strt♦♥s ♦ t ss ♣r♦♣♦rt♦♥s π ♥ ♦ t ss ♣r♠trs αj

k ♥♦s

♦♥r ♠①tr ♠♦s

tt

p(θ) = p(π)

g∏

k=1

d∏

j=1

p(αjk).

s t r②s ♥♦♥ ♥♦r♠t ♣r♦r ♦r ♠t♥♦♠ strt♦♥ s ♦♥trt ♦♥ t ♣r♦r strt♦♥ s rtt♥ s ♦♦s

p(π) = Dg

(

1

2, . . . ,

1

2

)

♥ p(αjk) = Dmj

(

1

2, . . . ,

1

2

)

.

♥r♥ s s♦ ♠ ② t ♦♦♥ s s♠♣r ♥rts sq♥ ♦ ♣r♠trs r♦♠ tr ♣♦str♦r strt♦♥s ♦t tt ts ♦rt♠s s② ♣r♦r♠ s♥ ♦♥t ♣r♦r strt♦♥s ♥♦ ①♣t ♣♦str♦r strt♦♥s

trt♥ r♦♠ t ♥t ♦ θ[0] trt♦♥ [r] ♦ t s s♠♣r♥ p(θ, ③|①) s stt♦♥r② strt♦♥ s rtt♥ s

∀i = 1, . . . , n z[r]i |xi,θ[r] ∼ Mg

(

ti1(θ[r]), . . . , tig(θ

[r]))

π[r+1]|z[r] ∼ Dg

(

1

2+ n

[r]1 , . . . ,

1

2+ n[r]

g

)

∀(k, j) αj[r+1]k |x, z[r] ∼ Dmj

(

1

2+ n

j1[r]k , . . . ,

1

2+ n

jmj [r]k

)

,

r ♥[r]k =

∑ni=1 z

[r]ik ♥ ♥jh[r]k =

∑ni=1 z

[r]ik x

jhi

♦rt♠ s s♠♣r ♦r t t♥t ss ♠♦

①t rtr♦♥ ② s♥ t ♣r♦♣rts ♦ t ♦♥t ♣r♦r strt♦♥s❬❪ ♣r♦♣♦s ♥ ①t rs♦♥ ♦ t rtr♦♥ ♦r t t♥t ss ♠♦♥ t ♥trt ♦♠♣tt ♦♦ ♦ ts ♠♦ ♥ ②

p(①, ③) =

∫

θ∈Θ

p(①, ③;θ)p(θ)dθ,

s ①♣t ② s♥ t ♣r♦r strt♦♥s ♥ ♥ ♦r ♥② ♦♣ (①, ③)t ♥trt ♦♠♣tt ♦♦ s q t♦

p(①, ③) =Γ(g

2)

Γ(12)g

∏gk=1 Γ(♥k +

12)

Γ(n+ g2)

g∏

k=1

d∏

j=1

Γ(mj

2)

Γ(12)mj

∏mj

h=1 Γ(♥jhk + 1

2)

Γ(♥k +♠j

2)

.

❱t♦r ③ s r♣ ♥ t ♦ qt♦♥ ② ts ♠①♠♠ ♦♦ st♠t ③② s♥ t ♠♣ r ♥ t ①t rtr♦♥ s ♥ s ♦♦s

① = ln p(①, ③).


♥ ❬❪ t t♦rs ♣r♦♣♦s t♦ s ♥ ♦rr t♦ ♦♠♣t t ♦♠♣tt ♦♦ ② s♥ ♠♣♦rt♥ s♠♣♥ ♣♣r♦ ② ♥r♥ ② tr♥♠r ①♣r♠♥ts tt t ①t rtr♦♥ ♦t♣r♦r♠s t ss s②♠♣t♦t♥♦r♠t♦♥ rtr ♥ s ♥ t ①t rtr r t② t♦ ♦r

♣♣r♦ ♠ r st♠t ♦r r♥t ♥♠rs ♦ sss ♥t rtr♦♥ s s t♦ st t st ♥♠r ♦ sss

♥tr♣rtt♦♥ st ♠♦ s t t♥t ss ♠♦ t tr ♦♠♣♦♥♥ts st♠t sss ♥ ♥tr♣rt s ♦♦s

♠♦rt② ss π1 = 0.72 r♦♣s t tt ♥♦s s s♦♥t str♦♥ ♣r♦t② ② t ♥tsts s ♣r♦t② s♣♣r t♥ ♦r t rst ♦r ♥tsts ♥ q t♦ ♦r tst ♦♥

s♦♥ ss π2 = 0.20 r♦♣s t tt ♠ s s♦♥ ②t rst ♦r ♥tsts t ♠♦r ♥rtt t♥ ♥ t ♣r♦s ss♣r♦t② t♥ ♥ t② r ♠ s r♦s② t st ♥tst t ♣r♦t②

tr ss π3 = 0.08 r♦♣s t tt ♠♥② r sr♦s s♣② ② t t ♥tst

♥♥♥ ①♠♣ t♥t ss ♠♦ str♥

Prs♠♦♥♦s rs♦♥s ♦ t t♥t ss ♠♦

♥♠r ♦ ♣r♠trs rqr ② t t♥t ss ♠♦ s q t♦

(g − 1) + gd∑

j=1

(mj − 1).

s ts ♥♠r s ♥r② str♦♥② s♠r t♥ t ♥♠r ♦ ♣r♠trsrqr ② t ♦♥r ♠♦ s q t♦

∏dj=1mj

♦r ttr sr♥ tr ♦ ♥ ♦t♥ ② r♥ t ♥♠r♦ ♣r♠trs ♦r t t♥t ss ♠♦ s ♣rs♠♦♥♦s rs♦♥s ♦ tt♥t ss ♠♦ s ♥tr♦ ② ① ♥ ♦rt ❬❪ ♦r ♥r②rs t♥ ts ♠♦s s ①t♥ t♦ t t♦r rs ❬♦❪ ♦♥str♥ts ♦♥ t ♣r♠tr s♣ rqr t ♥tr♦t♦♥ ♦ ♥ ♠♦♣r♠tr③t♦♥ ❲t ts ♥ ♣r♠tr③t♦♥ t ♠t♥♦♠ strt♦♥ ♦r j ♦r ♦♠♣♦♥♥t k s tr♠♥ ② ts ♥tr a

jk ♥♦t♥ t ♠♦rt②

♠♦t② ♥ ts s♣rs♦♥ ♣r♠tr εjk

♥t♦♥ tr♥t ♣r♠tr③t♦♥ ♦ t ♣rs♠♦♥♦s t♥t ss ♠♦ t♥t ss ♠♦ ♥ ♣r♠tr③ s ♦♦s

p(xi;θ) =n∑

k=1

πk

d∏

j=1

(

(1− εjk)ajhk (εjk/(mj − 1))1−a

jhk

)xjhi,

♦♥r ♠①tr ♠♦s

s t 0 < εjk < 1 t ♣rs♠♦♥♦s ♠♦ ss♠s tt ♦♥ ♠♦ ♦rrs♣♦♥♥ t♦ t ♠♦st ② ♠♦t② s rtrst ♦r ♠t♥♦♠ tr♠♥♥ ♣r♦t② ♠ss s ♥♦r♠② s♣r ♠♦♥ t ♦tr ♠♦ts s♠♦ rqrs (g−1)+gd ♣r♠trs ♦tr ♣rs♠♦♥♦s ♠♦s r ♦t♥② ss♠♥ t qt② ♦ εjk t♥ t ss ♦r t♥ t rs ♦r t♥t ss ♥ t rs

♠ts ♦ t t♥t ss ♠♦

t♥t ss ♠♦ ♠② sr r♦♠ sr ss ♥ t t r ♥trss ♦rrt ♦r ♥st♥ ♥ ♣♣t♦♥ ♣rs♥t ♥ ❬❱❪ s♦s tt t♥tss ♠♦ r♠t② ♦rst♠ts t ♥♠r ♦ sss ♥ t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ s ♦t ❲ ♥♦ ♣rs♥t tr tr♥t ♠①tr♠♦s r①♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♦t tt t rr st ♥♠r ♦ rs t r s t rs t♦ ♦sr ♦♥t♦♥② ♦rrtrs ♥ t st ♥ ♦♥sq♥t② t r s t rs t♦ ♥♦ s ss② s♥ t t♥t ss ♠♦

♦♥r ♠①tr ♠♦s t ♥trss ♣♥♥

s

♥ ♦♥r ♠♦s ❬r❪ ♣r♣♦s s t♦ ♠♦③ t ♥ ♦♣r♦t② ② st♥ ♥trt♦♥s t♥ rs s t ♦♥r ♠①tr♠♦ s ♥ s ♦r ♦♥ t♠ ❬r ❪ t♦ str t♦r t stt ♥trss ♣♥♥s ♦t tt s♦♠ ♦♥str♥ts t♦ ♠♣♦s ♦♥ ♦♥r ♠♦ ♥ ♦rr t♦ ♦t♥ t ♠♦ ♥tt②


♣♣r♦ s♣♥ ♥ ♥♠♥ ❬❪ ♣♣② ♦♥r♠①tr ♠♦ t♦ t t t ♦t tt t♦rs st♠t sr ♠♦s① ② ♥ ♦s t st ♦♥ ♦♥srs ♠①tr t ♦r♦♠♣♦♥♥ts

♥tr♣rtt♦♥ rst t♦ ♦♠♣♦♥♥ts t ♥t♦ ♦♥t t ♥trt♦♥s t♥ t ♥tsts ♥ st t♦ ♦♠♣♦♥♥ts r s♣s♥ tr ♦ ♦♥② ♦♥ ♠♦t② ♥trt♦♥ ♥ t ♥♦ss rrs♣t② r♦s ♥ s♦♥

♦♠♠♥ts ♦t tt ts ss♠♣t♦♥s r rqr ② t t♦rs t♦ tr rst ♥tr ♥ ts ♠♦ ts t t ttr t♥ t♠ ♠♦ ♥ t ♦tr ♥ ts ♥tr♣rtt♦♥ ♥s t ♥②ss ♦ ♦rsss s♦ t t s♠♠r② s ♠♦r ♦♠♣① ♥② ♦ rt③t ♥ ♦ t t♦ s♣ sss ♠♦♥ ♦♥② ♦♥ ♠♦t② r♦ss♥♥ ts sss ♣♣rs s rt② ♥ ♦rr t♦ ♠♦③ t♦♥t♦♥ ♣♥♥s

♥♥♥ ①♠♣ ♦♥r ♠①tr ♠♦ str♥ ❬❪

♦ st♦♥ ② ♦♥sr♥ ♥trss ♣♥♥② ♦ ♦rr ♦♥ t t♦rs♦ ❬❪ ♦t♥ ♦♦ rsts ♦r t str♥ ♦ r♦r♣ r♦ss♥♦stss t♦rs ♣r♦r♠ t ♠♦ st♦♥ ② s♥ ♦rr ♠t♦ tr♠♥s t ♥trss ♥trt♦♥ ♦r ♥♦t tt ts ♣♣r♦ s s♦♣t♠♥ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t ♥♦r♠t♦♥ rtr♦♥ s ② t ♣rtt♦♥r ♠♦ ♣rs♥t ♥ ❬❱❪ ♦♥srs t ♥trt♦♥s ♦ ♦rr t♦t r♥ ♥ t ♣♣t♦♥ tr r ♦♥② ♥trt♦♥s ♦ ♦rr ♦♥ rst♠t s ♦r t ♣r♦s rt t♦rs t♦ tr♠♥ ② ♥ t♥trss ♥trt♦♥s ♠♦ st♦♥ ♦r t ♦♥r ♠①tr ♠♦s s ♦♠♣① ♣r♦♠ s♥ t ♥♠r ♦ ♠♦s ♦♠s t t ♥♠r ♦rs

♦♦ ♠♥② ♣r♠trs ♥♠r ♦ ♣r♠trs rqr ② t ♦♥r♠①tr ♠♦ ♥rss t t ♥♠r ♦ ♠♦ts ♥ t t ♦♥sr♦rr ♦ ♥trt♦♥s s ts ♠♦ ♥ t t t t t ♠② ♥ t♦♦♠♥② ♣r♠trs ♦ tr s ♥ ♦rtt♥ rs ♥ t ♥tr♣rtt♦♥ ♦♠srr rtr♠♦r t ♣r♠trs ♥ ♣♦♦r② ♠♥♥ tr r t♦♦♥♠r♦s

♦♥s♦♥ ♦♥r ♠①tr ♠♦ s ♣♦r t♦♦ t♦ str t♦r t ♦r t s ♠♣♦rt♥t t♦ ♠♣♦s ♦♥str♥ts ♦♥ t ♣r♠trs s♣♥ ♦rr t♦ ♣r♦ ♠♥♥ ♠♦ ♦t ♠①tr ♠♦s ♣rs♥t ♥ ♣tr ♥ ♣tr ♥ ♥tr♣rt s ♦♥r ♠①tr ♠♦s t s♣

①trs ♦ trs

♦♥str♥ts ♦♥tr♦ t ♥♠r ♦ ♣r♠trs ♥ ♣r♦ ♠♥♥sss ♦t ♠♦s r ♥ t ♥ ♥t ♣♣r♦ t♦ ♣r♦r♠ t ♠♦st♦♥ ♥ ②s♥ r♠♦r

①trs ♦ trs

♣♥♥ trs

♥ s ♣♣r♦ ♣r♦♣♦s ② ♦ ♥ ❬❪ ♦♥ssts ♥♣♣r♦①♠t♥ srt ♠trt ♣r♦t② strt♦♥ t ♣♥♥ trs t ♣r♦t ♦ s♦♥♦rr strt♦♥s

♥t♦♥ P ♦ ♣♥♥ tr strt♦♥ t t tr T = E, V r E = 1, . . . , d ♥ V = (j, j′) : j ∈ E ♥ j′ ∈ E \ j r xi ss♠♣ r♦♠ ♣♥♥ tr strt♦♥ ♥ ② T t♥ ts ♣ s rtt♥s ♦♦s

p(xi;α) =

∏

(j,j′)∈V p(xji ,x

j′

i ;βjj′)

∏dj=1 p(x

ji ;α

j)vj−1,

r α = (αj,βjj′

; j = 1, . . . , d; j′ s s (j, j′) ∈ V ) r vj ♥♦ts t r♥ ♦ t ♥♦r ♦ j ♣ ♦ ♦♠♣♦♥♥t k s ♥ ②

p(xji ;αj) =

mj∏

h=1

(αjh)xjhi ♥ p(xji ,x

j′

i ;βjj′) =

mj∏

h=1

mj′∏

h′=1

(βjj′h′h′)x

jhi xj

′h′

i ,

t αj = (αjh;h = 1, . . . ,mj) ♥ βjj′

= (βjj′hh′ ;h = 1, . . . ,mj;h

′ = 1, . . . ,mj′) ♣r♠tr αjh ♥♦ts t ♣r♦t② tt r j ts ♠♦t② h ♥ t♣r♠tr βjj

′hh′ ♥♦ts t ♣r♦t② tt t ♦♣ ♦ rs (j, j′) tst ♦♣ ♦ ♠♦ts (h, h′)

st♠t♦♥ s s♦♥ ♥ ❬❪ t ♠①♠♠ ♦♦ st♠t ♥ rt②♦t♥ ② s♥ t rs ♦rt♠ st♠ts t tr ♦ ♠♥♠ ♥t❬r❪ ♦ t r♥ t t♥ t t♦ r♥♦♠ rs Xj ♥Xj′ s ♥ ② t ♠t ♥♦r♠t♦♥ ♥ s

I(Xj,Xj′) =

mj∑

h=1

mj′∑

h′=1

p(Xjh = 1, Xj′h′ = 1) lnp(Xjh = 1, Xj′h′ = 1)

p(Xjh = 1)p(Xj′h′ = 1).

r♦♠ ts ♥t♦♥ t ♠♣r ♠t ♥♦r♠t♦♥ s ♦r s♠♣ ①

♥t♦♥ ♠♣r ♠t ♥♦r♠t♦♥ ♠♣r ♠t ♥♦r♠t♦♥ t♥ ①j = (xji ; i = 1, . . . , n) ♥ ①j

′= (xj

′

i ; i = 1, . . . , n) ♦♠♣t r♦♠ ts♠♣ ① s ♥ s

I(①j,①j′

) =

mj∑

h=1

mj′∑

h′=1

f(xjh, xj′h′) ln

f(xjh, xj′h′)

f(xjh)f(xj′h′),

r f(xjh, xj′h′) = 1

n

∑ni=1 x

jhi x

j′h′

i ♥ f(xjh) = 1n

∑ni=1 x

jhi


♦♠♣t I(①j,①j′

i ) ∀(j, j′) ♥① t d(d − 1)/2 r♥s ♦r♥ t♦ tr t ♦ t

t bℓ s rtr t♥ ♦r q t♦ t t bℓ′ ♥r j < j′

t b1 ♥ b2

♦r ℓ = 3 t♦ d(d−1)/2 t r♥ bℓ t ♦s ♥♦t ♦r♠ ②t t st ♣r♦s② st

♦rt♠ st♠t♦♥ ♦ t ♣♥♥ tr

♠r ❯♥q s♦t♦♥ t ts r r♥t t♥ t s♦t♦♥♦ ♦rt♠ s ♥q

r ♠①tr ♠♦

♥ s ♣♣r♦ ♣r♦♣♦s ② ♥ ♦r♥ ❬❪ ♥r③s t ♣r♦st trs t♦ t ♠①tr ♠♦ r♠♦r t♦rs ss♠tt ♦♠♣♦♥♥t ♦♦s strt♦♥ ♣r ♣♥♥ tr ♥ ♥

st♠t♦♥ ♥ rq♥tst r♠♦r t ♠ s s② ♦t♥ ② ♥ ♠

♦rt♠ ♠ st♣ ♠①♠③s t ①♣tt♦♥ ♦ t ♦♠♣t ♦♦ ②s♥ ♦rt♠ r t ♠♣r ♠t ♥♦r♠t♦♥ s ♦♠♣t ♦r♥t♦ t ♦♥t♦♥ ♣r♦ts ♦ t ss ♠♠rs♣s ♥ ②s♥ r♠♦rt ♠♣ s s♦ ♦t♥ ② s♣ ♠ ♦rt♠ ♠①♠③♥ t ♣♦str♦rstrt♦♥

♣♣r♦ ❲ str t t st t ♠①tr ♠♦s ♦ ♣♥♥②trs t r♥t ♥♠rs ♦ sss ♥ s t rtr♦♥ t♦ stt st ♦♥

♥tr♣rtt♦♥ st ♠♦ s t ♦♠♣♦♥♥t ♦♥ ♦t tt ts♠♦ rqrs t st♠t♦♥ ♦ ♣r♠trs t t♥t ss ♠♦rqrs ♦♥② ♣r♠trs ts rtr♦♥ s ttr t♥ t♦♠♣♦♥♥t t♥t ss ♠♦ rs♣t② ♥ ts ♦rst s ♥♦t ttr s♥ t tr♦♠♣♦♥♥t t♥t ss ♠♦ ♦t♥s rtr♦♥ ♦ t rtr♦♥ s r rt② ♦st ♣rtt♦♥s r r♥t s s♦♥ ② t ♦♥s♦♥ ♠trs ♣rs♥t ♥

♥♥♥ ①♠♣ r ♠①tr ♠♦ str♥

t t♥t ss ♠♦

tr tr

tr trtr tr tr

♦♥s♦♥ ♠trs t♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♣♦♥♥t♠①tr ♦ trs ♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♣♦♥♥t t♥t ss♠♦ t tr♦♠♣♦♥♥t t♥t ss ♠♦

♦♥s♦♥ ♠♥ ♣r♦♠ ♦ ts ♠♦s s tt t② rqr t♦♦ ♦t♥ ♥♥trt ♥♠r ♦ ♣r♠trs rtr♠♦r t tr strtr s ♦t♥ ♥st♥ t t st s tt t ♥ t♥ t tr strtr ♥ r②r♥t s t ♥tr♣rtt♦♥ s ♦♥ ts strtr ♥ rr♥t ♥②♥♦t tt t ♠①trs ♦ trs r ♠♥♥ ♣r♥♣② ♥ t tr strtr①♣♥s s♦♠ s rt♦♥s♣s

t t♥t ss ♠♦

♥ s t♦ ♦♥sr t♦ t♥t rs rst ♦♥ s t♦r♥ s rt t♦ t ss ♠♠rs♣ s♦♥ ♦♥ s ♦♥t♥♦s ♥rt♦r ♠trt ♥ ♠♦③s t ♥trss ♣♥♥s

r ♦ ts ♠t♦s ❲♥ ♦rts r t ♦♥t♦♥♣♥♥s t♥ t t♦r ♦♥s ♥ ♠♦ ② ♦st ♥t♦♥❬♦r ❲❪ ② ss♠♥ tt ts ♦rts r ♥♦sr t ♠tt♥t ss ♠♦ ❬❱r ❱r❪ ♥tr② ♥♦r♣♦rts t ♥trss ♣♥♥ss ♠♦ s ♦♥♥t♦♥s t t ♣♣r♦ ♦ ❨ ♥ ♥ t♥r ❬❪ r t ♥trss ♣♥♥s r ♠♦ ② t♥t ♦♥t♥♦sr t ♣r♦t ♥t♦♥ ②r ♠♦ ❬t❪ ♥ ♦r ss t♦r ♥②ss ♠♦ s tt t♦ tr t♦r rs ♦r t♦ t♦s t♦r rs ♥ ♣♥♥s s ♠♦r ♥r ♣♣r♦ ♥t② ♦♥♥ r♣② ❬❪ ♣r♦♣♦s t ♠①tr ♠♦ ♦ t♥t trts ♥②③rs ss♠s tt t strt♦♥ ♦ t t♦r rs ♣♥s ♦♥ ♦t t♦r t♥t r t ss ♥ ♠♥② ♦♥t♥♦s t♥t trts rs ♥r♥ s s♦ t ♣♦♥t s s♦ rt♦♥ ♣♣r♦ ts ♠♦s ♦♥sr t ♥trss ♣♥♥s tr ♠♥ r s ttts ♣♥♥s t♦ ♥tr♣rt ♠♦♥ rt♦♥s t t♥t rs ♣rt♥♥t ♥tr♣rtt♦♥ ♥ t

♦s ♦♥ t ❬❪ ♣♣r♦ ♠t t♥t ss ♠♦ ♣r♦♣♦s♥ ❬❪ s ♥tr♦ t♦ ♥②③ ♥r② t sts t ①♣♥s t ♥trss♣♥♥s ② ♦t ♥t♦♥

♥t♦♥ ❬❪ ♠t t♥t ss ♠♦ ♣ ♦ ♦♠♣♦♥♥t k


s rtt♥ s

p(xi;αk) =

∫

R

d∏

j=1

Φ(akj + bkjt)xj1i (1− Φ(akj + bkjt))

1−xj1i dΦ(t),

r Φ(.) ♥♦ts t ♠t strt♦♥ ♥t♦♥ ♦ st♥r ♥♦r♠ r

♥ ♣rt ts ♣ s ♣♣r♦①♠t ② s♥ t ssr♠t qrtr ♠ s ♦t♥ ② s♥ ♥ ♠ ♦rt♠ ❲ ♥♦ ♣rs♥t t rst ♦ ts♠♦ ♦r t r♥♥♥ ①♠♣

♣♣r♦ s t st s♣ ♠♦ t ♦r sss ♣r♦♣♦s ♥❬❪ s♠s rt t t♦rs ♦ ❬❪ ♣rr t♦ s t r♥♦♠ts ♠♦s ♥ t♥t ss ♥②ss t t♦ sss ② ss♠ tt♦♥t♦♥ ♣♥♥s ♥ ♠♦③ ② s♥ ♦♥t♥♦s t♥tr rs ♠♦♥ t ♥s ♦r♥ t♦ t t♦rst t♥t ♦♥t♥♦s r ♥ rt t ♥♥ ♦ t ♦♥t♦♥ ♦♠s

♥tr♣rtt♦♥ ♦r♥ t♦ t t♦rs ♦♥ ss r♣rs♥ts t s♦♥tt ♥ t ♦tr r♣rs♥ts t r♦s ♦♥s r♥♦♠ t r♣rs♥ts t ♣t♥t s♣ ♥r♦r rtrsts ♦ t ①r② ♠sr ♠♦ ♦s ♥♦t rqr t t♦ t♦♥ rt sss str ♥tr♣rtt♦♥ s sr ♥ t s ♥♦t s② t♦ t t str♥t♦ t ♥trss ♣♥♥s

♥♥♥ ①♠♣ str♥ t t r♥♦♠ ts ♠♦

♦♥s♦♥ ② ♥ t♦ s ♦ t♥t rs t ♠t t♥t ss♠♦s ♣r♠ts t♦ ♦♥sr t ♥trss ♣♥♥s ♦r t s ♥♦t s② t♦rtr③ ts ♣♥♥s s♥ tr s ♥♦♥ ♣r♠tr rt♥ t str♥t♦ ts ♣♥♥s

♦♥s♦♥

ss t♥t ss ♠♦ s ♦t♥ s ♥ t s♠♣ s③ s r s ts ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s ♦t r♥t ♠t♦s ♦t♦ str t t ② t♥ ♥t♦ ♦♥t t ♥trss ♣♥♥s ♦rtr s ♥♦t ♥② ♠♦ ♣r♦s ♦♥ ♦♥t t♦ rtr③ t str♥t ♦ts ♣♥♥s

♥ ts ♦r ♥♦t s♣♦♥ ♦t t ♠①tr ♦ t♦r ♥②③rs s♥ ♥t t♦ ♦r t ♠♦s r s② ♥tr♣rt ♦ ♦s ♦♥♠♦s str t ♥s ② ♠♦♥ t strt♦♥ ♦ t rs♥ tr ♥t s♣

♦♥s♦♥

♦♥r ♠①tr ♠♦s s♠s t♦ t ♠♦st ♥r ♦♥ ❲ s♦ ♣r♦♣♦s♥ t t♦ ♦♦♥ ♣trs t♦ ♠①tr ♠♦s s♣ ♦♥str♥ts t♦ts ♥r ♠♦ s ♦t ♣r♦♣♦s ♠♦s ♦ t♦ s♠♠r③ t ♥trss♣♥♥s t ♣r♠trs

♣tr

♦s str♥ t ♦s

♦ ①tr♠ strt♦♥s

s ♣tr ♥tr♦s ♥ ①t♥s♦♥ ♦ t t♥tss ♠♦ s ♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s s♣ strt♦♥♦ t ♦s ♠♦③s t ♥trss ♣♥♥s ♥♣r♦s ♦♥ ♦♥t s♠♠r③♥ t str♥t ♦ts ♣♥♥s ♠①♠♠ ♦♦ ♥r♥ s ♣r♦r♠ ② ♠♦rt♠ t ♦♠♥t♦r ♣r♦♠s ♦ t ♠♦st♦♥ r ♦ ② ♠♠ ♦rt♠♠r ①♣r♠♥ts ♦♥ s♠t ♥ r t sts♥r♥ t ♠♥ rtrsts ♦ ts ♥ ♠①tr♠♦

♥ ♥r s♦s ♣r♦♠ t♦trt♥ t♥ ♠♦r

♦r r♥r

♥tr♦t♦♥

❲ ♣r♦♣♦s t♦ ①t♥ t ss t♥t ss ♠♦ ♦r t♦r t ② ♥ ♠①tr ♠♦ r①s t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ t♥t rs ❲ rr t♦ t ♣r♦♣♦s ♠♦ s t ♠①tr ♦ ①tr♠ ♣♥♥②strt♦♥s ♣r ♦s ♥♦t ② ♠

♠♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♥t ss ♠♥ ♥trss ♣♥♥s r ts ♥r♥ ② t r♣rtt♦♥♦ t rs ♥t♦ ts ♦s s ♣♣r♦ ♦♥ ♠♦♥ ♦ t ♠♥ ♦♥t♦♥ ♥trt♦♥s s rst ♣r♦♣♦s ② ♦r♥s♥ ♥ ♥t ❬ ❪♥ ♦rr t♦ str t sts t ♦♥t♥♦s ♥ t♦r rs ♦r t ♠♠♦ ♦ ♦♦s ♣rtr ♣♥♥② strt♦♥ ♦♥ssts ♥ ♦♠♣♦♥♥t ♠①tr ♦ t ♥♣♥♥ ♥ t ♠①♠ ♣♥♥② strt♦♥ ♦r♥ t♦ t r♠rs ❱ rtr♦♥ s s♣ strt♦♥ ♦ t ♦s

♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s

♣r♦s ♦♥ ♣r♠tr s♠♠r③♥ t str♥t ♦ t ♦♥t♦♥ ♣♥♥s ♦t rs s r ♣r♠tr s t ♣r♦♣♦rt♦♥ ♦ t ♠①♠♠ ♣♥♥②strt♦♥ rtr♠♦r t ♥tr ♦ t ♦♥t♦♥ ♣♥♥s s r♥ ♦t② t rt♦♥ ♥ ② t ♠①♠♠ ♣♥♥② strt♦♥ s t ♠♦♣ts t t ♦♥ t ♠♥ ♦♥t♦♥ ♣♥♥s ♥ tr str♥ts

♣r♦♣♦s ♠♦ ♥ ♥tr♣rt s t♦ ♣rs♠♦♥♦s rs♦♥ ♦ ♦♥r ♠①tr ♠♦ ♥ ts ♥ts r♦♠ ts ♥tr♣rtt ♣♦r rst ♥s t ♦♥sr ♥trt♦♥s ② r♦♣♥ ♥ t s♠ ♦ t rs r ♦♥t♦♥② ♣♥♥t str♥t ♦ ts ♣♥♥② s rt② t ♣r♦♣♦rt♦♥ ♦ t strt♦♥ ♦ ♠①♠♠ ♣♥♥② ♦♠♣r t♦ tt ♦t ♥♣♥♥ strt♦♥ s♦♥ ♦ s♣rst② s ♥ ② t s♠rt♦♥ ♦ t ♣r♠trs ♦ t ♠①♠♠ ♣♥♥② strt♦♥ ♦ t ♦s ♦r ♦♥r ♠①tr ♠♦s t st♦♥ ♦ t ♣rt♥♥t ♥trt♦♥s s ♦♠♥t♦r ♣r♦♠ r♦r ♣r♦♣♦s t♦ ♣r♦r♠ t ♠♦ st♦♥ ♠♠ ♦rt♠ ♥ ♦rr t♦ ♦ t ♥♠rt♦♥ ♦ t ♠♦s sts ♥r ♣♣r♦ ♦ s♦ st t ♥trt♦♥s ♦ ♠♦r ♥r ♦♥r♠①tr ♠♦

trtr ♦ ts ♣tr s ♣tr s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs t♦♥ ♣rs♥ts t ♥ ♠①tr ♠♦ t♥ ♥t♦ ♦♥t t ♥trss ♣♥♥s t♦♥ s ♦t t♦ t st♠t♦♥ ♦ t ♣r♠trs ② ♠①♠③t♦♥♦ t ♦♦ ♥ t s r t ss ♥♠r ♥ t ♦s ♦ rsr s♣♣♦s t♦ ♥♦♥ t♦♥ ♣rs♥ts ♠♠ ♦rt♠ ♦♥ ♦♠♥t♦r ts ♥r♥t t♦ ♦ st♦♥ t♦♥ ♣rs♥ts rsts ♦♥s♠t t t♦♥ strts t ♠ ♠♦ ♦♥ t♦ r str♥ ♥s ♦♥s♦♥ s ♥ ♥ t♦♥ ♦t tt tt♦r ♦ t ♣strt ♣r♦r♠♥ t ♠♦ st♦♥ ♥ t st♠t♦♥ ♦ t ♣r♠trs♦ ♠ s ♥ ♥ ♣tr ts rsts r ♣rt ♦ t rt ♦sstr♥ ♦r ♦♥t♦♥② ♦rrt t♦r t ❬❱❪


♥ ♠①tr ♠♦ ♦ ♥trss ♥♣♥♥t ♦s ♦♥srs tt♦♥t♦♥② ♦♥ ss k rs r r♦♣ ♥t♦ k ♥♣♥♥t ♦s ♥ ♦ ♦♦s s♣ strt♦♥

♣rtt♦♥ ♦ t rs ♣r ss r♣rtt♦♥ ♦ t rs ♥t♦♦s tr♠♥s ♣rtt♦♥ σk = (σk1, . . . ,σkk) ♦ 1, . . . , d ♥ k s♦♥t ♥♦♥♠♣t② ssts r σkb r♣rs♥ts t sst b ♦ rs ♥ t ♣rtt♦♥ σks ♣rtt♦♥ ♥s t t♦r ♦ t♦r rs xkb

i = xσkbi = (x

kbji ; j =

1, . . . , dkb) s t sst ♦ xi ss♦t t♦ σkb ♥tr dkb = r(σkb)

♣ strt s ♦♥ ♦r st t t ♦♦♥ r tt♣sr♦rr♣r♦t♦rr♦♣❴


s t ♥♠r ♦ rs t t♦ ♦ b ♦ ♦♠♣♦♥♥t k t♦r xkbji =

(xkbjhi ;h = 1, . . . ,m

kbj ) ♦rrs♣♦♥s t♦ r j ♦ ♦ b ♦r ♦♠♣♦♥♥t k ♥

ss ♦♠♣t s♥t ♦♥ r mkbj s t ♥♠r ♦ ♠♦ts ♦r t

r xkbji s xkbjhi = 1 ♥ i ts ♠♦t② h ♦r r x

kbji

♥ xkbjhi = 0 ♦trs

♠r r♥t ♥trss ♣♥♥s r♥t rs r♣rtt♦♥s ♥♦s r ♦ ♦r ♦♠♣♦♥♥t ♥ t② r r♦♣ ♥t♦ σ = (σ1, . . . ,σg)

♥t♦♥ ①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rst xi t♦ t drt t♦r r rs♥ r♦♠ ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs ♦s t ♣rtt♦♥ s ♥♦t ② σ ♥t ♣r♠trs ② θ ♥ ts ♣ s rtt♥ s ♦♦s

p(① i;σ,θ) =

g∑

k=1

πkp(① i;σk,αk) t p(① i;σk,αk) =

k∏

b=1

p(①kbi ;αkb),

r αk = (αk1, . . . ,αkk) ♥ r p(① kbi ;αkb) s t ♣ ♦ t ♦ b ♦ t

♦♠♣♦♥♥t k ♣r♠tr③ ② αkb

①♠♣ ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦rs t xi = (x1

i , . . . ,x5i ) t t♦r ♦ t♦r rs ♦♦♥

t ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♣rtt♦♥♦ t rs ♦ ts ♠♦ s σ = (σ1,σ2) t σ1 = (1, 2, 3, 4, 5) ♥σ2 = (1, 5, 2, 4, 3) r strts t ♥trss ♣♥♥s t♥♥t♦ ♦♥t ② t ♠♦ ♥ ♥ts tt t ♥trss ♦rrt♦♥ s♥t ♥ ♥ts tt ts ♦rrt♦♥ s t♥ ♥t♦ ♦♥t

X5

X4

X3

X2

X1

X1

X2

X3

X4

X5

x11

x12

ss

X3

X4

X2

X5

X1

X1

X5

X2

X4

X3

x21

x22

x23

ss

r ♥trss ♣♥♥s t♥ ♥t♦ ♦♥t ② t ♦♠♣♦♥♥t♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs t σ1 =(1, 2, 3, 4, 5) ♥ σ2 = (1, 5, 2, 4, 3)

♦t tt t ss t♥t ss ♠♦ t ♦♥t♦♥ ♥♣♥♥ ♦ r♣rs♥t ② t s ♦ t ♦♥ ♥ ♦♥ t ttr


t ♠♦s ♣♣r♦ ♣r ♦♥t♦♥② ♥♣♥♥t ♦s s r② ♥r s♥ ♥② strt♦♥ ♥ ♦s♥ ♦r ♦ strt♦♥ p(① kb

i ;αkb) ♠①tr ♠♦ ② ♦♥t♦♥ ♥♣♥♥t ♦s s ♣rs♠♦♥♦s rs♦♥♦ t ♦♥r ♠①tr ♠♦ ♥ t strt♦♥ ♦ ♦ tr♠♥s ♥trt♦♥s r ♦♥sr ♦t tt t ♦rr ♦ ts ♥trt♦♥s s tr♠♥ ② t ♥♠r ♦ t rs ♥t♦ t ♦ ♥② t ♥trt♦♥st♥ rs ♦ r♥t ♦s ③r♦ ♥ t♦s t♥ rs ♦ ts♠ ♦ ♥ ♠♦③ ② t s♣ strt♦♥ ♦ t ♦ ♠t♥s ♦ ts ♠♦ r k = d ♦r ss s q♥t t♦ t t♥t ss ♠♦

♥r ♥tt② ♥r ♥tt② ♦ t ♠①tr ♠♦s ♦r t♦r t ♥ t② ♣r♦ ♦r ② ♥ s♦♠ ♦♥str♥ts ♦♥t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥ s ♦r♠ ❬❪s t ♥r ♥tt② ♦ t ♠♦ s ♦t♥ ② s♥ ts ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ t♥ ♦s ♥r t♦ s♥t ♦♥t♦♥s

♦r♦r② ♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs q t♥ sss σ1 =, . . . ,= σg t 1 ≥ 3

♥ tt t ♦ strt♦♥s p(①kbi ;αkb) r ♥t ♥ υb rs ♦

r♦♠ t♥ s♣♣♦s tr ①sts tr♣rtt♦♥ ♦ t st S = σ11, . . . ,σ11 ♥t♦tr s♦♥t ♥♦♥♠♣t② ssts S1 S2 ♥ S3 s tt κu =

∏

j∈Suυj t♥


♥ ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥

Pr♦♦ r s t♦♥ r♦♠ xkbi t♦ x

kbi r x

kbi s t♦r r

♥ υb ♠♦ts r xkbi ♦♦s t t♥t ss ♠♦ s♦ ts ♥t

t② s ♥ ② ♦r♠ s ♦♥ t♦ t ♥r ♥tt②♦ t ♠♦ r♥ x

kbi

♦r♦r② ♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs tr ①sts tr♣rtt♦♥ ♦ σk qs ♦r k = 1, . . . , g ♥t♦ tr s♦♥t ♥♦♥♠♣t② ssts S1 S2 S3

∀k ∈ 1, . . . , g, ∀σkb ∈ σk, ∃u ∈ 1, 2, 3 s σkb ∈ Su,

♥ tt t ♦ strt♦♥s p(①kbi ;αkb) r ♥t ♥ υb rs

♦ r♦♠ s tt κu =∏

j∈Suυj t♥


♥ ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥

Pr♦♦ ♣r♦♦ s s♠r t♥ t ♣r♦♦ ♦ ♦r♦r② ♦t tt t ①st♥♦ xkb

i s ssr ② t qt② ♦ t tr♣rtt♦♥ ♦ t σk t♥ sss



♥ ♠ s t♦ ♥ ♣rs♠♦♥♦s strt♦♥ ♦r ♦ ttts ♥t♦ ♦♥t t ♣♥♥② t♥ rs rtr♠♦r t ♣r♠trs♦ t strt♦♥ ♥s ♦ ♠st ♠♥♥ ♦r t ♣rtt♦♥r ♥ ts♦♥t①t ♣r♦♣♦s t♦ ♠♦③ t strt♦♥ ♦ ♦ ② ♠①tr ♦ tt♦ ①tr♠ strt♦♥s ♦r♥ t♦ t r♠rs ❱ rtr♦♥ ♦♠♣t ♦♥ t ♦♣s ♦ rs ♠♦ rsts ♥ ♦♠♣♦♥♥t ♠①tr t♥ ♥♥♣♥♥ strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ ♥ s② ♥tr♣rt ② t sr

♠①♠♠ ♣♥♥② strt♦♥ s ♥tr♦ rst t♥ t ♠①tr♠♦ ♦ ①tr♠ ♣♥♥② strt♦♥s ♣r ♦s ♠ s s♦♥② t

♠r rr rs ❲t♦t ♦ss ♦ ♥rt② t rs r ♦♥sr s ♦rr ② rs♥ ♥♠r ♦ ♠♦ts ♥ ♦

∀(k, b) mkbj ≥ m

kbj+1 r j = 1, . . . , dkb − 1.

①♠♠ ♣♥♥② strt♦♥

♥ ♠①♠♠ ♣♥♥② strt♦♥ s ♥ s t ♦♣♣♦st strt♦♥ ♦ ♥♣♥♥ ♦r♥ t♦ t r♠rs ❱ rtr♦♥ ♦♠♣t ♦♥ t♦♣s ♦ rs ♥ t ♥♣♥♥ strt♦♥ ♠♥♠③s ts rtr♦♥ t ♠①♠♠ ♣♥♥② strt♦♥ ♠①♠③s t ❯♥r ts strt♦♥t ♠♦t② ♥♦ ♦ ♦♥ r ♣r♦s t ♠①♠♠ ♥♦r♠t♦♥ ♦♥ t ssq♥t rs

♠r ♦♥r♣r♦ ♥t♦♥ ♥ ♦t tt t s ♥♦♥r♣r♦ ♥t♦♥ ♥ t♥ rs ♥ x

kbi rss r♦♠ ts strt♦♥ t

♥♦ ♦ t r ♥ t rst ♥♠r ♦ ♠♦ts tr♠♥s ①t② t ♦trs t t rrs ♦s ♥♦t ♥ssr② ♣♣②

♠r ss srt♦♥s s strt♦♥ ♥s sss srt♦♥s r♦♠ t s♣ ♦ xkbji t♦ t s♣ ♦ xkbj+1

i t j = 1, . . . , dkb−1 rtt t rs r ♦rr ② rs♥ ♥♠r ♦ ♠♦ts ♥ ♦ ♥t t s r♣r♦ ♥t♦♥ ♥ ♦♥② ♥ mkb

j = mkbj+1

Pr♠tr③t♦♥ ♥ t rst r tr♠♥s t ♦tr ♦♥s ts strt♦♥ s ♥ ② ♣r♦t t♥ t ♠t♥♦♠ strt♦♥ ♦ t rstr ♣r♠tr③ ② t ♦♥t♥♦s t♦r

τ kb = (τhkb;h = 1, . . . ,mkb1 ) t τhkb ≥ 0 ♥

mkb1∑

h=1

τhkb = 1,

♥ t ♣r♦t t♥ t ♦♥t♦♥ strt♦♥s ♥ s s♣ ♠t♥♦♠strt♦♥s ♦ ♦♥t♦♥② ♦♥ x

kb1hi = 1 t♥ ♦r j = 2, . . . , dkb x

kbji


♦♦s ♠t♥♦♠ strt♦♥ ♣r♠tr③ ② t srt t♦r

δhjkb = (δhjh

′

kb ;h′ = 1, . . . ,mkbj ) t δhjh

′

kb ∈ 0, 1,m

kbj∑

h′=1

δhjh′

kb = 1 ♥m

kb1∑

h=1

δhjh′

kb ≥ 1.

♦t tt t ♦ ♦♥str♥ts ♥ t sss srt♦♥s ② ♥♦t♥ δkb =(δhjkb ;h = 1, . . . ,m

kb1 ; j = 2, . . . , dkb) t strt♦♥ ♦ ♠①♠♠ ♣♥♥②

♥ ♥♦ ♥

♥t♦♥ ①♠♠ ♣♥♥② strt♦♥ t xkbi t dkbrt

t♦r r ♦♦♥ t ♠①♠♠ ♣♥♥② strt♦♥ ♦s t srt ♣r♠trs r ♥♦t ② δkb ♥ ♦s t ♦♥t♥♦s ♦♥s r ♥♦t ②τ kb ♥ ts ♣ s rtt♥ s ♦♦s

p(xkbi ; τ kb, δkb) = p(x

kb1i ; τ kb)

dkb∏

j=2

p(xkbji |xkb1

i ; δhjkbh=1,...,mkb1

)

=

mkb1∏

h=1

(

τhkb

dkb∏

j=2

mkbj∏

h′=1

(δhjh′

kb )xkbjh′

i

)xkb1hi

.

①♠♣ rt ♥ trrt ♠①♠♠ ♣♥♥② strt♦♥s tt ♠①tr ♠♦ ♦s t ♦s ♦ rs ♦r t rst ♦♠♣♦♥♥t r ♥② σ1 = (1, 2, 3, 4, 5) strt♦♥s ♦ t ♦s r ♠①♠♠ ♣♥♥②♦♥s ♦s t ♣r♠trs r t ♦♦♥

δ11111 = δ21211 = δ31311 = δ41311 = δ1j112 = δ2j212 = 1,

τ 11 = (0.1, 0.3, 0.2, 0.4) ♥ τ 12 = (0.5, 0.5).

r s♣②s t ♣r♦ts ♦ t ♦♥t strt♦♥s ② t r ♦ r♦①s ♦t tt δkb ♥s t ♦t♦♥s r t ♣r♦ts r ♥♦♥ ③r♦ ♦t♦♥ ♦ r ♦①s ♥ τ kb ♥s t ♣r♦ts ♦ ts ♥♦♥ ③r♦ s r ♦t r ♦①s

♥tt② s♥t ♦♥t♦♥ ♦ ♥tt② s t♦ ♠♣♦s τhkb > 0 ♦r h = 1, . . . ,m

kb1 s strt♦♥ s r② ♠t ♥trst s t s s♦

♥rst tt t ♥ ♠♦st ♥r s ♦♥ ♥♦ ♣rs♥t ♦ t♦ s t ♥ ♠♦r ♥t ②

♦ strt♦♥ ♠①tr ♦ t♦ ①tr♠ str

t♦♥s

♥ ❲ ss♠ tt t ♦s ♦♠♣♦s ② t st t♦ rs ♦♦ ♦♠♣♦♥♥ts ♠①tr t♥ ♥ ♥♣♥♥ strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ t ♦ ♦♠♣♦s ② ♦♥ r ♦♦ ♠t♥♦♠strt♦♥


τ11

1

τ11

2

τ11

3τ11

4

x1111

x1112

x1113

x1114

x1123

x1122

x1121

rst ♦ ♦ ss ♦♥ ♦ ♦ ss

r ♦ ①♠♣s ♦ ♦ strt♦♥s ♦♦♥ ♠①♠♠ ♣♥♥②strt♦♥ r m11

1 = 4 m112 = 3 ♥ m12

1 = m122 = m

123 = 2

♥t♦♥ ♦s str♥ ♦ ♦s ♦ ①tr♠ strt♦♥s drt t♦r r xi s ♥rt ② ♠ ♠♦ t s r♥ ② ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦s t ♣ s rtt♥ s♦♦s

p(① i;σ,θ) =

g∑

k=1

πk

k∏

b=1

p(①kbi ;αkb).

♦r♦r t ♣ ♦ ♦ b ♦r ♦♠♣♦♥♥t k s rtt♥ s

p(①kbi ;αkb) =

(1− ρkb)p(①kbi ; ξkb) + ρkbp(①

kbi ; τ kb, δkb) dkb > 1

p(①kbi ; ξkb) ♦trs

r p(① kbi ; τ kb, δkb) s t ♣ ♦ t ♠①♠♠ ♣♥♥② strt♦♥ ♥

② ♥ r p(① kbi ; ξkb) s t ♣ ♦ t ♥♣♥♥ strt♦♥ ♥

② p(① kbi ; ξkb) =

∏dkb

j=1

∏mkbj

h=1 (ξjhkb )xkbjhi ♣r♠tr αkb = (ρkb, ξkb, τ kb, δkb)

r♦♣s t ♣r♠trs ♦ ♦ b ♦r ♦♠♣♦♥♥t k ♥② t r ρkb ∈ [0, 1] st ♣r♦♣♦rt♦♥ ♦ t ♠①♠♠ ♣♥♥② strt♦♥

♠r ♦ ♣r♠trs ♠ ♠♦ rqrs tt t♦♥ ♣r♠trs♦♠♣r t t ♠ ♠♦ ♥ ♦r ♦ t t st t♦ rst ♥♠r ♦ t♦♥ ♣r♠trs ♣♥s ♦♥② ♦♥ t ♥♠r ♦ ♠♦ts ♦t rst r ♦ t ♦ ♥ ♥♦t ♦♥ t ♥♠r ♦ rs ♥t♦ t ♦ ♥♠r ♦ ♣r♠trs ♦ ♠ ♥♦t ② ν♠ s s♦ ♥ ②

ν♠ = (g − 1) + gd∑

j=1

(mj − 1) +∑

(k,b)|dkb>1

mkb1 .

♥ t♦♥ t ♠ ♠♦ s s② ♥tr♣rt s ①♣♥ ♥ t ♥①t♣rr♣ ♦t tt t ♠t♥ s r ρkb = 0 ♥s t ♦ strt♦♥② t ♥♣♥♥ ♦♥ ♥ ts ♣rtr s t ♣r♠trs ♦ t ♠①♠♠♣♥♥② strt♦♥ r ♥♦ ♦♥r ♥


♥♥ ♦ strt♦♥ ❯♥r ts strt♦♥ t ♣r♦♣♦rt♦♥ ♦ t♠①♠♠ ♣♥♥② strt♦♥ rts t t♦♥ r♦♠ ♥♣♥♥ ♥rt ss♠♣t♦♥ tt t tr♥t strt♦♥ s t ♠①♠♠ ♣♥♥② strt♦♥ ♣r♠tr ρkb s ♥ ♥t♦r ♦ t ♥trr ♣♥♥②♦ t ♦ t s ♥♦t r ♣rs ♣♥♥② ♠♦♥ rs t ♣♥♥② t♥ rs ♦ t ♦ rtr♠♦r t st②s ♦♥ ♥ t♥♠r ♦ rs s rr t♥ t♦ t r♠rs ❱ s ♥♦♥ ♣♣r♦♥♥ ts s ♥trr ♣♥♥s r ♥ ② δkb str♥t ♦ts ♣♥♥s s ①♣♥ ② τ kb ♥ ts t♦r s t t ♦ t♦rr♣rs♥t ♠♦t② r♦ss♥s ♦♠♣r t t ♥♣♥♥ strt♦♥

Prs♦♠♥♦s ♦♥r ♠①tr ♠♦ ❲ ♥tr♣rt t ♠ ♠♦ s t♦ ♣rs♠♦♥♦s rs♦♥ ♦ t ♦♥r ♠①tr ♠♦ rst ♦♥ s♥ ② t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s tr♠♥♥ t ♦♥t♦♥♥trt♦♥s t♦ ♠♦ s♣ ♦ strt♦♥ s s♦♥ ♦ ♣rs♠♦♥② s♥ ♠♦♥ t ♥trt♦♥s ♦ ② ♦♠♣♦♥♥t ♦♥② t♦s♦rrs♣♦♥♥ t♦ t ♠①♠♠ ♣♥♥② strt♦♥ r ♠♦ t ♦tr♦♥s r ♦♥sr s ♥

♥tt② ♣r♦♣♦s strt♦♥ s ♥t ♥r t ♦♥t♦♥ ttt ♦ s ♦♠♣♦s ② t st tr rs dkb > 2 ♦r tt t ♠♦t②♥♠r ♦ t st r ♦ t ♦ s rtr t♥ t♦ mkb

2 > 2 s rst s♠♦♥strt ♥ ♣♣♥① ❲ r♠♥ tt t ♣r♠tr ρkb s ♥ ♥t♦r♦♥ t♦ ♠sr t ♣♥♥② t♥ rs ♥♦t ♠t t♦ ♣♥♥②t♥ ♦♣s ♦ rs ♦r dkb = 2 ♥ m

kb2 = 2 t♥ t ♦

strt♦♥ s ♥♦t ♥t s♦ ♥ ♦♥str♥t s ♥ ♦rr t♦ t♠♦st ♠♥♥ ♣r♠trs t ♦s♥ ♦ ρkb s t rst ♠①♠③♥t ♦♦♦ s t♦♥ ♦♥str♥t ♦s ♥♦t s② t ♥t♦♥ ♦ ρkbs ♥ ♥t♦r ♦ t ♣♥♥② str♥t t♥ t rs ♦ t s♠ ♦rtr♠♦r ts ♦♥str♥t s ♥tr s♥ ♦s t t st ♣♥♥sr ♥t ♦t tt ρkb s♠s t♦ ♦rrt t t r♠rs ❱ s strt② t ♦♦♥ ①♠♣

①♠♣ r♠rs ❱ ♥ ρkb t♦ ♠srs ♦ t ♣♥♥② r ♣rs♥ts t ♥ t♥ t r♠rs ❱ ♥ ρkb ♦♥ s♠t rt ♥r②rs ♦r s s♦ ♥ ♦sr ♥ ♠♥② ♦tr stt♦♥s

①♠♠ ♦♦ st♠t♦♥

♦rt♠

♠ t ① = (① 1, . . . ①n) t s♠♣ ♦♠♣♦s t n ♥♣♥♥t ♥ ♥t② strt ♥s ss♠ t♦ rs r♦♠ t ♠ ♠♦ r♦♠ tss♠♣ t ♠ s t♦ st♠t t ♠ ♦r ① ♠♦ m ♥ ② (g,σ)

①♠♠ ♦♦ st♠t♦♥ ♦rt♠

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Cramer’s V

ρkb

r ♦t♦♥ ♦ ρkb ♦♠♣t t t ♥tt② ♦♥str♥t ♦r♥t♦ t r♠rs ❱ ♦r t♦ ♥r② rs

♦♠♥t♦r ♣r♦♠ ❲ s♥ ♥ t♦♥ tt t ♥r♥ ♦r ♠①tr ♠♦ ♥ ♣r♦r♠ ♥ ♠ ♦rt♠ ♦r ♦♥ ♦ ts ①t♥s♦♥s t♠①♠③t♦♥ ♦ t ♦♠♣tt ♦♦♦ s s② ♦r t s ♥♦t t s♦r t ♠ ♠♦ s♥ t st♠t♦♥ ♦ t srt ♣r♠trs ♦ t ♠①♠♠♣♥♥② strt♦♥ s ♦♠♥t♦r ♣r♦♠ ♥ S(a, b) s t ♥♠r♦ ♣♦ss srt♦♥s r♦♠ st ♦ r♥ a ♥t♦ st ♦ r♥ b t♥ δkb s♥ ♥ t srt s♣ ♦ ♠♥s♦♥

∏dkb−1j=1 S(m

kbj ,m

kbj+1 ) ♦ ♥ ①st

♥♠rt♦♥ ♦r st♠t♥ t srt ♣r♠trs s ♥r② ♠♣♦ss ♥ ♦ ♦♥t♥s rs t ♠♥② ♠♦ts ♥♦r ♠♥② rs

①♠♣ ♦♠♥t♦r ♣r♦♠ ♥♦ ② t srt ♣r♠trs ♦ t tr rs ♥ mkb = (5, 4, 3) ♠♣s 51 840 ♣♦ssts ♦r δkb

st♠t♦♥ ♠♣ ♣r♠trs r st♠t ♠ ♦rt♠ ♦♥t ss ♣r♦♠ ♥♦ ② t ♥♥♦♥ ss ♠♠rs♣ t ts ♠ st♣t ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦ s ♥♣♥♥t② ♣r♦r♠ ♦♥ t ♣r♠trs ♦ ♦ s t t ♠ st♣ ♦ trt♦♥[r] t ♦♠♥t♦r ♣r♦♠ ♦ t srt ♣r♠tr st♠t♦♥ ♦r ♦ b ♦♦♠♣♦♥♥t k s ♦r♠ ② tr♦♣♦sst♥s ♦rt♠ ♦s t stt♦♥r②strt♦♥ s ♦s t♦ p(δkb|①kb, ③[r]) ♣r♦♣♦s strt♦♥ ♦ ts ♦rt♠r♥♦♠② s♠♣s t ♥t δ⋆kb t ♥t (ρ⋆kb, ξ

⋆kb, τ

⋆kb) s tr♠♥

st② ♦♠♣t ♥ ♦rr t♦ ♠①♠③ p(ρ⋆kb, ξ⋆kb, τ

⋆kb, δ

⋆kb|①kb, ③[r]) ♦t tt t

♦♥t♥♦s ♣r♠trs (ρ⋆kb, ξ⋆kb, τ

⋆kb) r ♦♥t♦♥② ♦t♥ ② ♥ ♠ ♦rt♠

② ♥tr♦♥ s♦♥ t♥t r ♥♦t♥ t ♠♠rs♣ ♦ t ♣♥♥②strt♦♥s ♦ t ♦ ♥♣♥♥ ♦r ♠①♠♠ ♣♥♥② strt♦♥


♦ ♦rt♠

♥ ♥r♥ ♦ ♣r♦r♠ ♥ ♠ ♦rt♠ ♦r♦♠♥t ♣r♦♠ ♦ t ss ♠♠rs♣ ♦r s t ♦♣t♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦ ♦♥ t srt ♣r♠trs s ♣r♦r♠ st♦st ♦rt♠ ♥ ♦♥② ssr t ♥rs ♦ t ①♣tt♦♥ ♦t ♦♠♣tt ♦♦♦ ♥ ♥♦t ts ♠①♠③t♦♥ ♦ t ♥r♥ s ♣r♦r♠ t ♦♦♥ ♠ ♦rt♠

trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ s st♣ t ♦♥t♦♥ ♣r♦ts

tik(θ[r]) =

π[r]k p(xi;σk,α

[r]k )

p(xi;σ,θ[r])

.

st♣ ♥rs ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

π[r+1]k =

n[r]k

n♥ α

[r+1]kb s s Lkb(α

[r+1]kb ;①, t[r]) ≥ Lkb(α

[r]kb ;①, t

[r]),

r Lkb(αkb;①, t[r]) =

∑ni=1 tik(θ

[r]) ln p(xkbi ;αkb) t t[r] = (tik(θ[r]); i =

1, . . . , n; k = 1, . . . , g) ♥ n[r]k =

∑ni=1 tik(θ

[r])

♦rt♠ ♠ ♦rt♠ t♦ ♦t♥ t ♠ ♠♦ ♠

s ♦rt♠ s st♦♣♣ tr rmax trt♦♥s ♦♣t♠③t♦♥ ♦♥ αkb s♥♣♥♥t② ♣r♦r♠ ♦r (k, b) t t ♠ st♣ ② t ♦♦♥ tr♦♣♦sst♥s ♦rt♠

♦s ♦♥ t st♣ ♦ t ♦rt♠

♥ tr♦♣♦sst♥s ♦rt♠ s ♥♣♥♥t② ①t ♦r (k, b) ♥ ♦rr t♦ ♣r♦r♠ t ♠ st♣ ♦ ♦rt♠ ♦r ① (k, b) ts♦rt♠ s stt♦♥r② strt♦♥ ♦s t♦ p(αkb|①, t[r]) ♥ t s ♣r♦r♠t trt♦♥ [r] ♦ t ♦ ♠ ♦rt♠ t s♠♣s sq♥ ♦ t ♦♣r♠trs (α[r,0]

kb , . . . ,α[r,smax]kb ) r smax s t ♥♠r ♦ trt♦♥s ① ② t

sr s t ♦rt♠ ♠s t ♥♥ t ♠①♠③♥ t ①♣tt♦♥ ♦ t♦♠♣tt ♦♦♦ ♣t

α[r+1]kb = argmax

s=1,...,smax

Lkb(α[r,s]kb ;①, t[r]).

❲ ♥♦ t t tr♦♣♦sst♥s ♦rt♠ t♥ t ts ♥str♠♥t strt♦♥ q(.;α[r,s]

kb )

①♠♠ ♦♦ st♠t♦♥ ♦rt♠

trt♥ r♦♠ t ♥t α[r,0]kb = α

[r]kb ts trt♦♥ [s] s rtt♥ s

α⋆kb ∼ q(αkb;α

[r,s]kb )

α[r,s+1]kb =

α⋆kb t ♣r♦t② λ[r,s]

α[r,s]kb t ♣r♦t② 1− λ[r,s].


♦s ♦♥ t ♣r♦♣♦s strt♦♥ ♥str♠♥t strt♦♥ q(αkb;α[r,s]kb )

s♠♣s t ♥t α⋆kb ♥ t♦ st♣s rst② t ♥♦r♠② s♠♣s t ♥t

δ⋆kb ♠♦♥ t ♥♦r♦♦ ♦ δ[r,s]kb ♥♦t ② ∆(δ

[r,s]kb ) s ♥♦r♦♦ s

♥ s t st ♦ t ♣r♠trs r t ♠♦st t♦ srt♦♥s r r♥t r♦♠t♦s ♦ δ[r,s]

kb ♦♥② t ♦♠♣ts t ♦♥t♥♦s ♣r♠trs ♦♥t♦♥② ♦♥∆(δ

[r,s]kb ) s s

(ρ⋆kb, ξ⋆kb, τ

⋆kb) = argmax

ρkb,ξkb,τkb

Lkb(ρkb, ξkb, τ kb, δ⋆kb;①, t

[r]).

♦t tt t ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦st②s ♥♦t strt♦rr ♥ ♥ t srt ♣r♠trs r ♥♦♥ ♦r② r♠r♥ tt t ♦ strt♦♥ s ts ♠①tr ♥tr♦ s♦♥t♥t r ♥t♥ t ♦ strt♦♥ ♠♠rs♣ ♥♣♥♥ ♦r ♠①♠♠ ♣♥♥② strt♦♥ s t ♦♥t♥♦s ♣r♠trs ♥ ② t♣r♦s qt♦♥ r ♦t♥ ② ♥ ♠ ♦rt♠ t ♥ t ♥①t st♦♥

①♠♣ ♦r♦♦ ♦ t srt ♣r♠tr r stts t♠♥ts ♦ ∆(δkb) t dkb = 2 mkb1 = 3 mkb2 = 2 ♥ t δ121kb = δ221kb =δ322kb = 1 ♥ δh2h

′

kb = 0 ♦trs

♦s ♦♥ t ♣t♥ ♣r♦t② ♥ ♦rr t♦ ♦♠♣t t ♥t♦♥ ♦t tr♦♣♦sst♥s ♦rt♠ ♣rs t ♣t♥ ♣r♦t② s♥ ②

λ[r,s] = min

p(①kb, t[r];α⋆kb)

p(①kb, t[r];α[r,s]kb )

|∆(δ⋆kb)||∆(δ

[r,s]kb )|

; 1

,

|∆(δ[r,s]kb )| ♥♦t♥ t r♥ ♦ ∆(δ

[r,s]kb )

♠r ①st ♣♣r♦ s st♦st ♦♥ ❲♥ t s♣ ♦ ♣♦ssδkb s s♠ ♦r ①♠♣ ♥ t ♦ r♦♣s s♠ ♥♠r ♦ ♥r② rs♥ ①st ♣♣r♦ ♦t♥s t s♠ rsts s t ♣r♦♣♦s ♦rt♠ t ss♦♠♣tt♦♥ t♠ s t rt♥ ♣♣r♦ ①st ♦r st♦st ♣♥s♦♥ t ♥♠r ♦ rs ♥ ♠♦ts


xkb11

xkb12

xkb13

xkb22

xkb21

xkb11

xkb12

xkb13

xkb22

xkb21

xkb11

xkb12

xkb13

xkb22

xkb21

xkb11

xkb12

xkb13

xkb22

xkb21

xkb11

xkb12

xkb13

xkb22

xkb21

r ♦r t r♦ h′ ♥ t ♦♠♥ h ♥ts tt δh2h′

kb = 1♥ t tt δh2h

′

kb = 0 δkb r t ♠♥ts ♦ ∆(δkb)

tr♠♥t♦♥ ♦ (ρ⋆kb, ξ⋆kb, τ

⋆kb) ② t ♣r♦♣♦s str

t♦♥

s♦♥ t♥t r t rst t♥t t♦r ③ ♥ts t ss ♠♠rs♣ s♦♥ t♥t t♦r ♥♦ts t ♦ strt♦♥ ♠♠rs♣ t s♥♦t ② ② = (y

kbi ; i = 1, . . . , n; k = 1, . . . , g; b = 1, . . . ,k) r ykbi = 1

①kbi rss r♦♠ t ♠①♠♠ ♣♥♥② strt♦♥ ♦r ♦ b ♦ ss k ♥

ykbi = 0 ① kb

i rss r♦♠ t ♥♣♥♥ strt♦♥ ♦r ♦ b ♦ ss k

♦♠♣tt ♦♦♦ ♦ ♠①tr ♠♦ strt♦♥♦rrs♣♦♥s t♦ t ♠r♥ strt♦♥ ♦ t r♥♦♠ r ❳ ♦t♥ r♦♠t tr♣t strt♦♥ ♦ t r♥♦♠ rs (❳,❨,❩) ♥ t ♦s r♥♣♥♥t ♦♥t♦♥② ♦♥ ❩ t ♦♠♣tt ♦♦♦ ♦t ♥ ❨♥ ❩ s ♥ s

L(θ;x,②, ③) =

g∑

k=1

♥k ln πk +g∑

k=1

k∑

b=1

Lkb(αkb;x,②, ③),

①♠♠ ♦♦ st♠t♦♥ ♦rt♠

r Lkb(αkb;x,②, ③) ♥♦ts t ♦♠♣tt ♦♦♦ ♦ ♦ b ♦r♦♠♣♦♥♥t k ♥ ②

Lkb(αkb;x,②, ③) =

n∑

i=1

zik

(

(1−ykbi ) ln

(

(1−ρkb)p(①kbi ; ξkb)

)

+ykbi ln

(

ρkbp(①kbi ; τ kb, δkb)

)

)

.

♦♥t♦♥ st♠t♦♥ ♦ t ♦♥t♥♦s ♣r♠trs t trt♦♥ [s] ♦♦rt♠ ♣r♦r♠ t trt♦♥ [r] ♦ ♦rt♠ t srt ♥t♣r♠tr δ⋆kb s s♠♣ ♥ t ♦♥t♥♦s ♥t ♣r♠trs r ♥s ♦♦s

(ρ⋆kb, ξ⋆kb, τ

⋆kb) = argmax

ρkb,ξkb,τkb

Lkb(ρkb, ξkb, τ kb, δ⋆kb;①, t

[r]).

♦ ♦♥t♦♥② ♦♥ (δ⋆kb,①, t[r]) t ♦♥t♥♦s ♣r♠trs r ♦t♥ ② t

♦♦♥ ♠ ♦rt♠

trt♥ r♦♠ ♥ ♥t (ρ[0]kb ,α

[0]kb , τ

[0]kb) trt♦♥ [ℓ] s rtt♥ s

st♣ t t ♦♥t♦♥ ①♣tt♦♥ ♦ ykbi

ui(α[ℓ]kb) =

ρ[ℓ]kbp(①

kbi ; τ

[ℓ]kb, δ

⋆kb)

(1− ρ[ℓ]kb)p(①

kbi ; ξ

[ℓ]kb) + ρ

[ℓ]kbp(①

kbi ; τ

[ℓ]kb, δ

⋆kb),

st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

ρ[ℓ+1]kb =

n[ℓ]kb

n[r]k

, ξjh[ℓ+1]kb =

njh[ℓ]kb

n[r]k − n

[ℓ]kb

♥ τh[ℓ+1]kb =

nh[ℓ]kb

n[ℓ]kb

,

t n[ℓ]kb =

∑ni=1 tik(θ

[r])ui(α[ℓ]kb) n

h[ℓ]kb =

∑ni=1 tik(θ

[r])ui(α[ℓ]kb)x

kb1hi

♥ njh[ℓ]kb =∑n

i=1 tik(θ[r])(1− ui(α

[ℓ]kb))x

kbjhi .

♦rt♠ ♠ ♦rt♠ t♦ ♦t♥ (ρ⋆kb, ξ⋆kb, τ

⋆kb)

♦♥tr ♥ ♦♣t♠♠ r♥ ♦r ①♣r♠♥ts ♠♣r② ♥♦t tt t ♦♦♦ ♥t♦♥ ♦ t ♠①tr t♥ t ♥♣♥♥ ♥ t♠①♠♠ ♣♥♥② strt♦♥s ♥q ♦♣t♠♠ ❲ ♦♥tr tt ts♥t♦♥ s ♥ ♥q ♠①♠♠

♠r ♥ ♠ ♦rt♠ t srt ♣r♠trs r ♥♦♥ ♥ ts♣ s r δkb r ♥♦♥ ♦r (k, b) t ♦♥t♥♦s ♣r♠trs ♦ st♠t ♥q ♠ ♦rt♠ t trt♦♥ [r] ♦ ts ♦rt♠ t st♣♦ ♦♠♣t ♦t ①♣tt♦♥s ♦ ③[r] ♥ ②[r] t ♠ st♣ ♦ st♠t t ♦♥t♥♦s ♣r♠trs ♠①♠③♥ t ①♣tt♦♥ ♦ t ♦♠♣tt♦♦♦


♦ st♦♥ ♦rt♠

♠ ♠ s t♦ st t ♠♦ ♥ ② (g, σ) ttr t t ts ♥ ②s♥ r♠♦r t ♠ s t♦ ♥ t ♠♦ ♥ t rst♣♦str♦r ♣r♦t②

Pr♦r strt♦♥s ❲ ♦♥sr tt p(g) = 1gmax

g ≤ gmax ♥ 0 ♦trsr gmax s t ♠①♠♠ ♥♠r ♦ sss ♦ ② t sr ♥ ss♠tt p(σ|g) ♦♦s ♥♦r♠ strt♦♥

P♦str♦r strt♦♥s st ♠♦ ♠①♠③s ts ♣♦str♦r strt♦♥♦r♥ t♦ t ♣r♦r strt♦♥s t s ♥ s

(g, σ) = r♠①g

[

r♠①σ

p(①|g,σ)]

r p(①|g,σ) ∝∫

θ∈Θ

p(①|θ, g,σ)p(θ)dθ.

♦ ♥ (g, σ) ♠♠ ♦rt♠ s s ♦r st♠t♥ r♠①σ p(①|g,σ) ♦r ♦ g ∈ 1, . . . , gmax s ♠t♦ ♠ts t ♦♠♥t♦r ♣r♦♠♥♦ ② t tt♦♥ ♦ t ♦ strtr ♦ rs s♥ t ♣r♦s r♥♦♠ ♠♦♥ t σ ♦ ♥trst

♠r ♥ t rrs ♠♣ rrs ♠♣ ♠t♦ ♦ s❬❪ ♦r ts ♣♣r♦ s rr② ♣r♦r♠ t ♠① ♣r♠trs ♦♥t♥♦s ♥ srt ♥ ♥ s s t s t t♦ ♥ ♠♣♣♥t♥ t ♣r♠trs s♣ ♦ t♦ ♠♦s ♦ ♣r♦♣♦s t♦ s ♥ sr ♠♠♦rt♠ ♥ p(σ|①, g) s stt♦♥r② strt♦♥

①♣♦rt♦♥ ♦ t s♣ ♦ t ♠♦s ②

♦rt♠

♥ s ♦rt♠ tr♥ts t♥ t♦ st♣s t ♥rt♦♥ ♦ ♥♦r♦♦ ♦♥t♦♥② ♦♥ t rr♥t ♠♦ ② ♣r♦♣♦s strt♦♥ ♥ t ♥rt♦♥ ♦ ♥ ♠♦ ♦♥♥ t♦ ts ♥♦r♦♦ ♦r♥ t♦ ts ♣♦str♦r♣r♦t②

♦ st♦♥ ♦rt♠

s ♠♠ ♦rt♠ s p(σ|①, g) s stt♦♥r② strt♦♥ trt♥r♦♠ ♥ ♥t ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s σ[0] tstrt♦♥ [q] s rtt♥ s

♦r♦♦ st♣ s♠♣♥ ♦ st♦st ♥♦r♦♦ Σ[q]

Σ[q] ∼ q(Σ;σ[q]).

♦ st♣ s♠♣♥ ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦sσ[q+1]

σ[q+1] = p(σ|①, g,Σ[q]) r p(σ|①, g,Σ[q]) ∝

p(①|g,σ) σ ∈ Σ[q]

0 ♦trs

♦rt♠ ♦rt♠ t♦ ①♣♦r t ♠♦s

❲ ♥♦ t ♦t st♣s ♦ t ♦ ♠♠ ♦rt♠

ts ♦ t ♦r♦♦ st♣ tr♠♥st ♥♦r♦♦ ♦ σ[q] ♦ ♥ s t st ♦ ♠♦s r t ♠♦st ♦♥ r s t ♦r ♦♥♦♠♣♦♥♥t ♥ ♥♦tr ♦ ♣♦sst② t♦ ♥ ♦

σ : ∃!(k, b, j) j ∈ σ[q]kb ♥ j /∈ σkb

∪

σ[q]

.

♦r s ts tr♠♥st ♥♦r♦♦ ♥ r② r ♦r ♣r♦♣♦sstrt♦♥ ♦s r♥ t t♦ st♦st ♥♦r♦♦ Σ[q] ② ♠t♥ t♥♠r ♦ (k, b) r σkb ♦ r♥t t♦ σ

[q]kb s t s♠♣♥ ♦r♥

t♦ q(.;σ[q]) s ♣r♦r♠ ② t tr ♦♦♥ st♣s ♦♠♣♦♥♥t s♠♣♥

k[q] ∼ U [1, . . . , g].

♥ ♦ s♠♣♥

b[q]from ∼ U [1, . . . , B[q]

k[q]].

rr♥ ♦ s♠♣♥

b[q]to = b[q], B[q]

k[q]+ 1 r b[q] ∼ U [1, . . . , B[q]

k[q] \ b[q]from].

st♦st ♥♦r♦♦ Σ[q] s t♥ ♥ s

Σ[q] =

σ : ∃!(k, b, j) j ∈ σ[q]kb , j /∈ σkb ♥ j ∈ σkb′ t k = k[q], b = b

[q]r♦♠, b

′ ∈ b[q]t♦

∪

σ[q]

.

❲ ♥♦t ② σ[q+ε(e)] t ♠♥ts ♦ Σ[q] r ε(e) = e

|Σ[q]|+1♥ e = 1, . . . , |Σ[q]|

①♠♣ ♥♦r♦♦ Σ[q] r s♦s ♥ strt♦♥ ♦ ts ♥

t♦♥ ♦ t ♥♦r♦♦ Σ[q] ♥ σ[q]k = (1, 2, 3, 4)


X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

X4

X3

X2

X1

X1

X2

X3

X4

r ①♠♣ ♦ t s♣♣♦rt ♦ Σ[q] ♥ t s ♦ ♦r rs trs ♦ t jt r♦ ♥ ♦ t j

′t ♦♠♥ r ♥ t s♠ ♦ t♥ t (j, j′) s ♣♥t ♥ s s ♣♥t ♥ t ♦trs ♠♥ts♦ Σ[q] b[q]from = 1 ♠♥ts ♦ Σ[q] b[q]from = 2

ts ♦ t Pttr♥ st♣ t t ♥rt♦♥ ♣ttr♥ st♣ t ♦rt♠ ♥st ♦ p(①|g,σ) ∀σ ∈ Σ[q] t♦ ♠♣♠♥t ♦rt♠ ② s♥ t ♣♣r♦①♠t♦♥ ts ♣r♦t② s ♣♣r♦①♠t ②

ln p(①|g,σ) ≃ L(θ;①, g,σ)− ν♠2

log(n),

θ ♥ t ♠①♠♠ ♦♦ st♠t♦r ♦t♥ ② t ♠ ♦rt♠ ♣r♦s② sr ♥ t♦♥ s t trt♦♥ [q] ♦r e = 1, . . . , |Σ[q]| tst♠t♦r θ

[q+ε(e)]ss♦t t♦ t ♠♥t σ[q+ε(e)] s ♦♠♣t ② ♦rt♠

♥t③t♦♥ ❲tr t ♥t st ♦r σ[0] t ♦rt♠ ♦♥rst♦ t s♠ stt♦♥r② strt♦♥ ♦r ts ♦♥r♥ ♥ r② s♦♥ t ♥t③t♦♥ s ♣♦♦r ♥ ♦s ♦♥sst ♥ t ♠♦st ♦rrt rs rr s♥♥t sst♦♥ s ♣♣ ♦♥ t ♠tr① ♦ r♠rs❱ st♥s ♦♥ t ♦♣s ♦ rs ❲ st ♦r σ[0]

k t ♣rtt♦♥ ♣r♦② t ♠♥♠③s t ♥♠r ♦ ♦s ♥ ①s t ♦s♦♥sst♥ ♦ ♠♦r t♥ ♦r rs ♦t tt t ♥♠r ♦ t rs t♥t♦ ♦ s ♠t t♦ ♦r ♦r t ♥t③t♦♥ s r② ♦s ♥♠♦r t♥ ♦r rs r ♦sr r♥ ♦r ①♣r♠♥ts ♦s② t♠♠ ♦rt♠ ♥ t♥ ♦t ts ♥t ♦♥str♥t ♥ssr②

t♦♣♣♥ rtr♦♥ ♦rt♠ s st♦♣♣ ♥ qmax sss trt♦♥s ♥♦t s♦r ttr ♠♦


♦♥sq♥s ♦ t ♠♦ st♦♥ ♦♥ t ♦

rt♠

♥ t trt♦♥ [q] ♦ t ♠♠ ♦rt♠ ♣r♦r♠♥ t ♠♦ st♦♥

♦rt♠ t ♠ ♦rt♠ ♦rt♠ st♠ts θ[q+ε(e)]

ss♦t t♦ t ♠♦ σ[q+ε(e)] ♦r e = 1, . . . , |Σ[q]| ♥ ts ♠♦s r ♦s t♦

σ[q] tr ♠①♠♠ ♦♦ st♠ts s♦ ♦s t♦ θ[q]

Pr♠trs ♦ t ♥♦♥♠♦ ♦s ♠ ♦rt♠ ♥t③t♦♥ s

s♦ ♦♥ ② t ♦ θ[q]

♦r t ♥♦♥ ♠♦ ♦s s ♥ s s

σ[q+ε(e)]kb = σ

[q]kb θ

[q+ε(e)][0]kb = θ

[q]

kb

Pr♠trs ♦ t ♠♦ ♦s ♦r t ♦tr ♦s t ♦♥t♥♦s ♣r♠trs r r♥♦♠② s♠♣ ♥ ♦rr t♦ ♦ t ♦♠♥t♦r ♣r♦♠s s sq♥t ♠t♦ t♦ ♥t③ δ[q+ε(e)][0]

kb srt♦♥s r♦♠ xkb1i t♦ x

kbji

r s♠♣ ♦r♥ t♦ ① ♥ t ♦♥t♥♦s ♣r♠trs ♣r♦s② s♠♣(ρ

[q+ε(e)][0]kb ,α

[q+ε(e)][0]kb , τ

[q+ε(e)][0]kb ) ♦r j = 2, . . . , dkb s ♦♦s

δ.j[q+ε(e)][0]kb ∝

n∏

i=1

p(xkb1i , x

kbji ; ρ

[q+ε(e)][0]kb ,α

1[q+ε(e)][0]kb ,α

j[q+ε(e)][0]kb , τ

[q+ε(e)][0]kb , δ.jkb)

z[q]ik ,

r δ

.j[q+ε(e)]kb = (δ

hj[q+ε(e)]kb ;h = 1, . . . ,m

kb1 ) ♥ r z[q]ik = E

[

Zik|xi,θ[q]]

♠r ♦t t ♥♠r ♦ trt♦♥s ♦ t ♦rt♠ rmax ss ♥ t♦♥ t ♦rt♠ s st♦♣♣ tr ① ♥♠r ♦ trt♦♥srmax t ♦rt♠ s st♦♣♣ ♦r ts ♦♥r♥ t ♣r♦♣♦s ♥t③t♦♥♠ts t ♣r♦♠s ♥ t ♠♦ s ♣♦str♦r ♣r♦t② t st② ♥ t ♥♦r♦♦ Σ[q] r♥ s♦♠ sss trt♦♥s s♦ ts ♦♦♦ ♥rs s ts ♦rt♠s r ♥tr♦ t ♥♠r ♦ trt♦♥s ♦ ♦rt♠ t ♠♦st ♥tr♥ ♦rt♠ s s♠ ❲♥ t st ♠♦ s st② ♦rt♠ ts ttr st② ♥ ts ♠♦ r♥ ♠♥② trt♦♥s s♦ ttr♦♣♦sst♥s ♦rt♠ ♥ t ♠ ♦rt♠ ♦rt♠ r♣r♦r♠ ♦ts ♦ t♠s s t s ♥♦t ♥ssr② t♦ r ♥♠r ♦ trt♦♥ss st♦♣♣♥ rtr♦♥


♣rs♥ts t st♠♥t ♣r♠trs s s ♦r t s♠t♦♥s

♦rt♠s ♠♠ ♠ tr♦♣♦sst♥s ♠

rtr qmax = 20× d rmax = 10 smax = 1 tmax = 5

❱s ♦ t r♥t st♦♣♣♥ rtr


t② ♦ t ♦rt♠ ♦r t δkb st♠t♦♥

♠ ♥ ts st♦♥ strt t ♣r♦r♠♥ ♦ t tr♦♣♦sst♥s ♦rt♠ st♠t♥ δkb s t♦♥ ♥ t r♥ ♦ ts ♥t③t♦♥ ♥② ♥ ts ♦rt♠ s ♥tr♦ ♥ t ♠♠ ♦rt♠ ♥ ♥ t ♠♦rt♠ rs♣t② st♠t t ♠♦ ♥ t ♣r♠trs ♥ t t♦♦♥r q② t s s♦♥ ♥ t ♦♦♥ s♠t♦♥s tt t ♦rt♠ st②sr♥t ♣ t♦ s① ♠♦ts ♣r r ♥ ♣ t♦ s① rs ♣r ♦ s♦♥t♦♥s ♦ ♥ ♠♦st stt♦♥s

①♣r♠♥t ♦♥t♦♥s ♠♣s ♦ s③ sr ② rs ♥ ts♠ ♥♠r ♦ ♠♦ts r ♥rt ② ♠①tr t♥ ♥ ♥♣♥♥strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ ♣r♠trs r s♦ st♠t ② t tr♦♣♦sst♥s ♦rt♠ sr ♥ t♦♥ s♥ ♦♥②♦♥ ss s ♥rt ♥t③t♦♥s ♦ t srt ♣r♠trs r ♣r♦r♠♦r♥ t♦ qt♦♥ t zi1 = 1 ♦r i = 1, . . . , 200

sts r s♦s t ♦①♣♦ts ♦ t ♥♠rs ♦ trt♦♥s rqr ②t tr♦♣♦sst♥s ♦rt♠ ♥ ♦rr t♦ ♥ t tr ♥s t♥ ♠♦ts ♠①♠③♥ t ♦♦ ♦r♥ t♦ ts s♠t♦♥s ♦♥ ♦srs ttt rsts ♦ ts ♦rt♠ r ♦♦ t♥s t♦ ts ♥t③t♦♥ ♦s s♥♥t② r♥ t ♥♠r ♦ trt♦♥s ♥ ♥ ♦rr t♦ ♥ t ♠①♠♠♦♦ st♠t♦rs

t② ♦ t ♦rt♠ ♦r ♠♦ st♦♥

♠ ♥ ♦rr t♦ strt t ♥② ♦ t ♦rt♠ ♦r t ♠♦ st♦♥♥ s♦ t ♥ st♠t♦♥ ♣r♦ss ♥t t♦ st② t ♦t♦♥ ♦ tr r♥ ♦r♥ t♦ t ♥♠r ♦ rs ♥ t♦ t s③ ♦t t st

①♣r♠♥t ♦♥t♦♥s ♥ ♠♥② stt♦♥s s♠♣s r ♥rt ♦r♥ t♦ t ♠ ♠♦ t t♦ ♦♠♣♦♥♥ts ♦t tt t ♣r♠tr u s♥tr♦ ♦r ♦♥tr♦♥ t ♦r♣♣♥ ♦ sss ♥ t s ♦s t♦ ♦♥ t♥t sss r s♦t② ♦r♣♣ s ♣r♠tr ① t rr♦r rt t♦ ♦r st stt♦♥

σkb = (d/b, 1 + d/b) ρkb = 0.6(1− u) τ kb = (0.60, 0.20, 0.20),

δh2h′

1b = 1 h = h′ δ1221b = δ2231b = δ3211b = 1 αj1b = (0.20, 0.20, 0.60),

α12b = α1

1b(1− u)+ (0.075, 0.850, 0.075)u ♥ α22b = α2

1b(1− u)+ (0.850, 0.075, 0.075)u.

♥ t t ♦rt♠ s st♦♣♣ s s♦♦♥ s t ♥s srt st♠t ♥♦♥ ♦♦r t♥ ♦r q t♦ t ♦♦ ♦t♥ t t tr srt ♣r♠trs s ♦r ts♠t♦♥


r ♦①♣♦ts ♦ t ♥♠r ♦ trt♦♥s rqr ② t tr♦♣♦sst♥s ♦

rt♠ ♥ ♦rr t♦ ♥ t st ♥s t♥ ♠♦ts ♦r♥ t♦ t ♥♠r ♦ ♠♦ts

♥ tsts r s♠t t ♣r♦♣♦rt♦♥ ♦ ♠①♠♠ ♣♥♥② strt♦♥ q t♦

r rs t t ♣r♦♣♦s ♥t③t♦♥ r ♠♦ts ♣r rs t t

♣r♦♣♦s ♥t③t♦♥ r rs t r♥♦♠ ♥t③t♦♥ r ♠♦ts ♣r

rs t r♥♦♠ ♥t③t♦♥

sts s♦s t ♠♥ ♥ t st♥r t♦♥ ♦ t r r♥ t♥ t ♣r♠trs s ♦r t t st ♥rt♦♥ ♥ tst♠t ♣r♠trs ♦r♥ t♦ t ♥♠r ♦ rs ❲♥ n ♥rss tr r♥ ♦♥rs t♦ ③r♦ t ♦♥r♠s t ♦♦ ♦r ♦ t♣r♦♣♦s ♦rt♠


d \ n

♠♥ st♥r t♦♥ ♦ t r r♥


♦♥tr♣t ♠t♦ ♦

t s t st s sst ♦ t t♦♥ ♥♦♥s ♦♥tr♣tPr♥ r② ❬❪ t s ♦♠♣♦s t ♠rr ♦♠♥ ♦ rtr ♥♦t ♣r♥♥t ♦r ♦ ♥♦t ♥♦ t② r t t t♠ ♦ ♥tr ♦r♥♣r♦♠ s t♦ ♣rt t rr♥t ♦♥tr♣t ♠t♦ ♦ ♥♦ s ♦♥tr♠♠t♦s ♦r s♦rt tr♠ ♠t♦s ♦ ♦♠♥ s ♦♥ r ♠♦r♣ ♥ s♦♦♦♥♦♠ rtrsts ♦♠♥ s sr ② ♥♥ rs ♥♠r ♦r♥ r ♦r♥ ♥ ♠♦r s ❲ ♥ ss ♥ ♠♦r s t♦♥ ❲ ♦ s♥st♦♥ ♦ s♥s ♦♣t♦♥ st♥r ♦ ♥ ♥① ♦ s r♦♥ ❲ ♦♥s♠♦r s♠ s ♥♦ ♦r♥ ❲❲♦ ②s ♦r ♥♦ ♥ ♠ ①♣♦sr ♦♦ ♦r♥♦t ♦♦ ♦r t ♥②ss t ♦♥tr♣t ♠t♦ s s ♥ ♥ ♦rr t♦♦r ♥ str♥ ♦♥t①t

♦ st♦♥ ♣rs♥ts t s ♦ t rtr♦♥ ♦r t ♠♥ t ♠ ♠♦s ❯♥t ♦r sss t rsts ♦ t ♠ ♠♦ r ttrt♥ t♠ ♦ t ♠ ♠♦ st♦♥ ♦ ss ♥♠r s ttr ♦r t ♠♠♦ s♥ t sts t tr ♥♠r ♦ sss t ♠ ♠♦ ♦rst♠tst

g ♠ ♠

❱s ♦ t rtr♦♥ ♦t♥ ② ♦t ♠♦s t r♥t♥♠rs ♦ sss st s ♦r♥ t♦ t rtr♦♥ r ♥ ♦

♦ ♥tr♣rtt♦♥ r s♠♠r③ t rsts ♦ t st♠♠♦♦r♥ t♦ t rtr♦♥ t ♦s t♦ sr t sss ② tr ♠♥ trs ♣r♦♣♦rt♦♥s ♥trss ♦rrt♦♥s ♥ ♦r♥ts t st♠t sss rr♣rs♥t t rs♣t t♦ tr ♣r♦♣♦rt♦♥ ♥ rs♥ ♦rr ♦t tt tr♦rrs♣♦♥♥ r ♣♥s ♦♥ tr ♣r♦♣♦rt♦♥ ♠t ♣r♦♣♦rt♦♥s r


♥t ♦♥ t t s ♥ sss tr ♥t♦♥s r ♥ rst ♦♥ st ♥trrs ♦rrt♦♥s ρkb ♦r t ♦s ♦ t ss ♦rr ② trstr♥t ♦ ♦rrt♦♥ ♥ rs♥ ♦rr s♦♥ ♦♥ s t ♥trrs♦rrt♦♥s τ kb ♦r ♦ r♥ ♦r♥ t♦ t str♥t ♦ tr ♣♥♥s ♥ rs♥ ♦rr tr s t rs r♣rtt♦♥ ♣r ♦s ♥ts tt t r s ss♥ t♦ t ♦ ♥ t ♥tstt ♦♥t♦♥② ♦♥ ts ss t r s ♥♣♥♥t ♦ t rs ♦ ts♦ ♦r ①♠♣ ts r s♦s tt t rst ss s ♣r♦♣♦rt♦♥ ♦ 0.49♥ tt t rs r s♣t ♥t♦ tr ♦s

ss ②♦♥ ♠s ♥r ts ss ♣r♦♣♦rt♦♥ s q t♦ r r t♦ ♣♥♥②

♦s ♥ ♦♥ ♦ ♦ ♥♣♥♥ ♦ ♥ ts ss t ♦♠♥ ♥ tr r♥ ♥♠r r

♦rrt ρkb t ♣rs♥ ♦ ♦t ①tr♠ stt♦♥s ②♦♥ ♦♠♥t♦t ♥ ♦ ♦♠♥ t ♦ts ♦ r♥ ①♣♥ ② ♦t δkb♥ τ kb

♦ t t♦♥ ♦ ♦t ♠♠rs ♦ ♦♣ r ♦s δkb♥ t♦♥ s ♠♦st ♣rs♥t τ kb

♦ t ♣rt ♦ s♠ s ♥r ♦♣ ♠♠rs ♦s♥ t♦r② t♦ ♥ tr ♥ tr ♥ ♥① st②s ♦ αkb

ss ♦ ♥ ♥♦t ♣rt♥ s♠ ♥r ts ss ♣r♦♣♦rt♦♥ s q t♦ r r t♦ ♣♥♥②

♦s ♥ ♦♥ ♦ ♦ ♥♣♥♥ ♦ tr s str♦♥ ♦rrt♦♥ t♥ t ♥ ♦ t s♥s

♦♣t♦♥ ♥ t s r♦♥ ρkb ♥ ts ss t ♦♠♥ ♣rt♥s♠ ♥r② s♥ t t ♦♣t♦♥s δkb ♥ τ kb

♦ ts ♦ s♦s ♥ t♥ t ♥♠r ♦ r♥ ♥ t ♦ t ♦♠♥ ♦r r t ♦♠♥ t ♠♦r r♥ t② δkb

♦ ♥ ts ss ♦t ♠♠rs ♦ t ♦♣ ♦♥ stsαkb

ss ♣♦♦r ♥ r ♠s ♥r ts ss ♣r♦♣♦rt♦♥ s q t♦ r s ♦♥ ♦ ♦

♥♣♥♥ ♦ ts s ss r t ♥♠r ♦ r♥ s r② % ♦

♦♠♥ t st r♥ t ♦♥ssts ♠♦st② ♦ rtr ♦ ♦♠♥t ♦ s ♦ t♦♥ s s tr s♥s ② ♦r ♥r♦♣s ♥ ♣rt ♦ s♠ s ♥r ♦♥ ♥ ts t♦r② ♥s ♥♦t ①♣♦s t♦ t ♠ αkb

♦♥s♦♥ t s ♥♦t tt t ♠ ♠♦ s ♠♦r r♥t ♦r ts t st♥ t ♥♠r ♦ sss s ♠t ♥ t② r ♥tr♣rt ♥ t♦♥ tss♠♣t♦♥ ♦ ♦♥t♦♥ ♥♣♥♥ t♥ rs s♠s t♦♦ str♥♥t ♦rs♦♠ ♦♣s ♦ r rt♦♥s♣ t♥ ♥ ♥♠r ♦ r♥ rt♦♥st♥ t t♦♥ ♦ ♦t ♠♠rs ♦ ♦♣ ♥ ♦♥tr② r sts②st♠ s ♣rs♥t


1 0.75 0.5 0.25 00 0.25 0.5 0.75 1

ρkb τkb σkb

0

0.49

0.86

1

Class 1

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

Class 2

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

Class 3

Chi

WAg

WEd

HEd

HOc

Liv

WRe

WWo

Med

r ♠♠r② ♦ t st ♠ ♠♦ ♦r♥ t♦ t rtr♦♥ ♦rt ♦♥tr♣t ♠t♦ ♦ t st

s str♥

t ♥s s♦♥ ♦♠♣♥② s ♦t ♥♦r♠t♦♥ r♦♠ tr♥ rrs ♥ ♦rr t♦ str s st s r sr② ♥♥ rs ♦ ♦r ♣tt ♦r s♥ ♣t ♦r ♦ t ♠♦trst ♦r t ♥ s♦ ♥ t rt trt♠♥t ♥st ♦♠♣t rs♣rt♦r② ss ♥ rr ♠s s♥t♦♥ s ♠s♠♣t②♥ ♠♣ ♠♦tr ♣r♥t trt♠♥t ♥st rs♣rt♦r② ss ♥rr

♥♦r♠t♦♥ rtr s♣②s t rtr♦♥ s ♥ t ♥♠r♦ ♣r♠trs rqr ② t ♠ ♥ t ♠ ♠♦s rtr♠♦r t ♦♠♣t♥ t♠ ♥ ♠♥ts ♦t♥ t ♣r♦ss♦r ♥t ♦r t♦ st♠tt ♠ ♠♦ ② strt♥ ♠♠ ♥s t st♦♣♣♥ rtr♦♥ ♦ qmax = 180 t ♠ ♠♦ ♥s s t t ♣ ①♠♦ ❬+❪

♦r t ♠ ♠♦ t rtr♦♥ sts ♥♠r ♦ sss s♥ tst t sss ♥tr♣rtt♦♥ ♦ t strs s s♦ t ♥ ♥ ss♠ tt t qt② ♦ t st♠t s r② ♣♦♦r r ♣s t♥tr♣rtt♦♥ ♦r t ♠ ♠♦ t ♦♠♣♦♥♥ts st ♠♦ ♦r♥ t♦t rtr♦♥ ts ♥tr♣rtt♦♥ s t s♠ s t ♥tr♣rtt♦♥ ♦ r ♦r ①♠♣ ts r s♦s tt t rst ss s ♣r♦♣♦rt♦♥ ♦ 0.29 ♥ t s♦♠♣♦s ♦ ♦r ♦s ♠♦st ♦rrt ♦ ♦ t rst ss s ρkb ≃ 0.80


g ♠

ν♠

♠ ν♠

t♠ ♠♥

sts ♦r t ♠ ♥ t ♠ ♠♦s ♦r♥ t♦ r♥t ♥♠rs♦ sss ♦r ♦t ♠♦s rst r♦ ♦rrs♣♦♥s t♦ t rtr♦♥ s ♥ ts♦♥ r♦ ♥ts t ♥♠r ♦ ♦♥t♥♦s ♣r♠trs st rsts ♦r♥t♦ t rtr♦♥ r ♥ ♦ ♦♠♣t♥ t♠ ♦r t ♠ ♠♦ st♠t♦♥ s♥ ♥ ♠♥ts

♥ t str♥t ♦ t st ♠♦ts ♥ s ♦s t♦ 0.85 s ♦ ♦♥ssts♥ t rs ♥

1 0.75 0.5 0.25 00 0.25 0.5 0.75 1

ρkb τkb σkb

0

0.29

0.56

0.76

0.9

1

Class 1

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Class 2

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Class 3

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Class 4

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Class 5

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

Apt

TOC

TRC

TDC

Iso

Dis

Emp

TRM

TDM

r ♠♠r② ♦ t st ♠ ♠♦ ♦r♥ t♦ rtr♦♥ ♦r ts t st

♥tr♣rtt♦♥ ♦ ss r s ♥♦ ♣♦ss ♥tr♣rtt♦♥ ♦ ss ♥♦ttt t ♦trs sss r s♦ ♠♥♥ s ts ♥ ❬❱❪

♥r ts ss s ♣r♦♣♦rt♦♥ q t♦ ♥ ♦♥ssts ♦ tr♦s ♦ ♣♥♥② ♥ ♦♥ ♦ ♦ ♥♣♥♥

♦ tr s str♦♥ ♦rrt♦♥ ρ11 t♥ t rs rrtrt♠♥t ♦ t ♥ ♠♦tr ♣r♥t trt♠♥t ♥st rs♣rt♦r②


ss s♣② t♥ t ♠♦t② ♥♦ trt♠♥t ♥st t rr ♥ t s♥ ♦ ♣r♥t trt♠♥t ♥st rs♣rt♦r② ss ♦ts ♠♦tr τ 11 ♥ δ11

♦ tr s str♦♥ ♦rrt♦♥ ρ12 t♥ t rs trt♠♥t ♥st rs♣rt♦r② ♥ss ♦ t ♥ ♠♦tr ♣r♥t trt♠♥t ♥st rr s♣② t♥ t ♠♦t② ♣r♥t trt♠♥t♥st rs♣rt♦r② ♥ss ♦ t ♥ t ♣rs♥ ♦ rr ♣r♥ttrt♠♥t ♦ ts ♠♦tr τ 12 ♥ δ12

♦ tr ①sts ♥♦tr str♦♥ ♥ t♥ t ♦r ♦ t♠♦tr t ♠♣t②♥ ♦ t ♠ ♥ ts s♥t♦♥ τ 13 ♥ δ13

♦ ts ♦ s rtr③ ② ♥ s♥ ♦ ♣r♥t trt♠♥t♥st ♦♠♣t ♥ ♦♥t♥s 50% ♦ t s ♥t ② ts ♥ss α14

♦♥s♦♥

② s♥ t ♦ ①t♥s♦♥ ♦ t ♠ ♠♦ ♥ ♠①tr ♠♦ t♠ ♠♦ s ♥ ♣r♦♣♦s t♦ str t♦r t ② t♥ ♥t♦ ♦♥tt ♥trss ♣♥♥② ♦ strt♦♥ ♦ t ♠ ♠♦ s ♥ s ♠①tr t♥ ♥ ♥♣♥♥ strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ s s♣ strt♦♥ st②s ♣rs♠♦♥♦s ♥ ♦s r♥t s ♦♥tr♣rtt♦♥ rst s ♥ ② t ♦s ♦ rs r♥ ♦t t♦♥t♦♥ ♣♥♥s t♥ rs ♥ ② t ♣r♦♣♦rt♦♥s ♦ t ♠①♠♠♣♥♥② strt♦♥s rtr③ t str♥t ♦ ts ♣♥♥s s♦♥ s ♠♦r ♣rs s♥ t ♣r♠trs ♦ t ♦ strt♦♥ rt t♥s t♥ ♠♦ts ♥ tr str♥ts ♠ ♠♦ s ♥ ♦♠♣rt♦ t t♥t ss ♠♦ ♦♥ t♦ r t sts

♣r♠tr ♥ t ♠♦ r s♠t♥♦s② st♠t ♠♠ ♦rt♠ s ♦rt♠ ♦s t♦ r t ♦♠♥t♦r ♣r♦♠s ♦ t ♦strtr tt♦♥ ♥ t ♥s t♥ ♠♦ts sr ♦r t st♠t♦♥ ♦t ♠①♠♠ ♣♥♥② strt♦♥ rsts r ♦♦ ♥ t ♥♠r ♦♠♦ts s s♠ ♦r r ♦r ♠♦r t♥ s① ♠♦ts t tt♦♥♦ ♦tr ♥s ♠ts s♦♠ ♣rsst♥t ts ♦ t ♦rt♠ ♥ s♦ ♥ts s ♣r♦♣♦s ♣♣r♦ t♦ st♠t t ♦ strtr s ♥♦t ♣t ♦rt sts t ♦ts ♦ rs

♠♥ r ♦ ts ♦rt♠ s ts ♥ t♦ ♦♠♣t t ♠ ss♦tt♦ ♥t ♠♦ s st♠t♦♥ s t♠ ♦♥s♠♥ ♥ ♦♥② t ♠

ss♦t t♦ t st ♠♦ s ♥tr♣rt s ♣r♦♣♦s ♥ t ♥①t ♣tr ♥ ♠①tr ♠♦ ♦♥ ts r s♥ ts ♥trt ♦♠♣tt♦♦ s ①♣t s ♣r♦♣rts ♦s t ♥s t♦ s t ♠ t♦ ♣r♦r♠t ♠♦ st♦♥

♥② t ♣r♦♣♦s ♠♦ ♥ s② ①t♥ t♦ t s ♦ ♦r♥ t♦r ts s♦♠ t♦♥ ♦♥str♥ts ♦♥ t ♣♥♥② strtr ♦ strt♦♥ ♦ ♠①♠♠ ♣♥♥② ♥ t♦ ♦t tt ts ♦♥str♥ts s♦♠t t ♦♠♥t♦r rsr ♦ t ♣♥♥② strtrs

♣tr

♦s str♥ t

♦♥t♦♥ ♣♥♥② ♠♦s

s ♣tr ♣rs♥ts ♦r s♦♥ ♦♥trt♦♥ t♦ t♠♦s r♠♦r ♣r♠tt♥ t♦ str t♦r t s ♦♥trt♦♥ ♦♥ssts ♥ ♠①tr ♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ ♦♦s ♣rs♠♦♥♦s ♠t♥♦♠ strt♦♥ r t r ♣r♠trs ♦rrs♣♦♥ t♦ ts ♠♦s ♥r♥ s s② ♣r♦r♠ ♥ ♠ ♦rt♠ t ♥ ♦ t ♠♦ st♦♥ s tt② ♥ ♥t ♣♣r♦①♠t♦♥ ♦ t ♥trt ♦♠♣tt ♦♦♠r ①♣r♠♥ts ♦♥ s♠t ♥ r t sts♥r♥ t ♠♥ rtrsts ♦ ts ♥ ♠①tr♠♦

r♠ ♣♦ rrst ts ♠♥ ts ♥ ♠ts

③③s rs t♥ r♦

♦ r♠ P♦

♥tr♦t♦♥

♥ ts ♣tr ♣rs♥t s♣rs ♠①tr ♠♦ r①s t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ ♥ ♦rr t♦ ♦r♦♠ t ss s ② t t♥t ss♠♦ ♥ ♦r♦♠s t ♠♥ r ♦ t ♠ ♠♦ ♥ t♠♦ st♦♥ ♥ s② ♥ ♥t② ♣r♦r♠ ② ♦♥ t ♦♠♥t♦r♣r♦♠s s st♣ ♦s ♥♦t rqr t ♠ s♥ t ♥trt ♦♠♣tt♦♦ ♥ ♣rs② ♣♣r♦ rst② t ♠♦ st♦♥ ♥ ♣r♦r♠② ♠♠ ♦rt♠ r t ♠ s ♥♦t rqr ♦♥② t ♣r♠trs r♦♥② st♠t ♦r t st ♠♦ s♥♥t② ♠ts t ♦♠♣tt♦♥ t♠

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

s ♥ ♠♦ ♥♠ ♦♥t♦♥ ♦s ♦ rrr ♥ ts rt ②♠♠ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦r ♦♥sr♥ t♠♥ ♦♥t♦♥ ♣♥♥s ♦r♦r t s♣ strt♦♥ ♦ t ♦ s ♠t♥♦♠ strt♦♥ ♣r ♠♦s s strt♦♥ ss♠s tt ♠♦t②r♦ss♥s ♥♠ ♠♦s r rtrst ♥ tt t ♦tr ♦♥s ♦♦ ♥♦r♠strt♦♥ s t ss♦t ♠t♥♦♠ strt♦♥ s ♣rs♠♦♥♦s s♥ts r ♣r♠trs r ♠t t♦ t ♣r♠trs ♦ t ♠♦s

s s♠♣ ♠①tr ♠♦ ♠♠ s ♦♦ ♥r ♥ t ♦♥ ♥ t♠♠ ♠♦ ♥s t ♠①tr ♠♦ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♠ s♥ t ♦s ♠♥② ss tr♦ ♠♦③♥ ♦ t ♠♥ ♦♥t♦♥♣♥♥s ♥ t ♦tr ♥ t ♥s t ♠①tr ♠♦s r①♥ tsss♠♣t♦♥ s♥ ts ① strt♦♥ ♦ t ♦ rqrs ♣r♠trs ♦ttt s t ♠ ♠♦ t ♠♠ ♠♦ ♥ ♥tr♣rt s ♣rs♠♦♥♦srs♦♥ ♦ t ♦♥r ♠①tr ♠♦ ♥ t r♣rtt♦♥ ♦ t rs♥t♦ ♦s ♥s t ♦♥sr ♥trt♦♥s t strt♦♥ ♣r ♠♦s ♥t♦♦s ♥s s♣ strt♦♥ ♦r ♥trt♦♥ rtr♠♦r rst♥sss r ♠♥♥ s♥ t ♥trss ♣♥♥s r r♦t ♦t ② t♦♦♠♣♠♥tr② s t ♦ r ♥trt♦♥ ♥ t ss♦t ♠♦♥trt♦♥ tr♦ ♦t♦♥s ♥ ♣r♦ts ♦t tt t ♠♠ ♠♦ s ♦♠♣r♥s ♣♣r♦ s♥ t ♥s t ♠ ♠♦ ♥ ♣rt ♦ ts ♣rs♠♦♥♦s rs♦♥s ♣rs♥t ♥ t♦♥

♦r ① ♠♦ ♥♠r ♦ sss r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥♥♠rs ♦ ♠♦s t ♠①♠♠ ♦♦ st♠t s ♦t♥ ♥ ♠ ♦rt♠ ♠♦ s st tr♦♣♦st♥s ♦rt♠ ♥ ts♦rt♠ s s s♠♣r ♥rts ♥ r♣rtt♦♥ ♦ t rs ♥t♦♦s ♥ ♥ ♥♠r ♦ ♠♦s ② ♦♥ tr♦♣♦sst♥s st♣ t s ♣r♦r♠♦r ① ♥♠r ♦ sss ♥ ♦s ♦♠♥t♦r ♣r♦♠s ♥♦ ② t st♦♥ ♦ t ♦s ♦ rs ♥ ② t st♠t♦♥ ♦ t ♥♠rs ♦ ♠♦ss ♦rt♠ s s ♦♥ t t tt t ♥trt ♦♠♣tt ♦♦ rqr ♦r t ♣t♥ ♣r♦t② ♦♠♣tt♦♥ ♦ t tr♦♣♦sst♥s ♥st s s♠♣r s ss ♥ ♥♦♥ ♠♦s tr♦ ② ♥♦r♠t ♦♥t ♣r♦r ♥② ts ♣♣r♦ s t♦ ♠♥ ♥ts t ♣r♠ts t♦ rt s ♦ t ♣♣r♦ t s ♠♥t♦♥ tt t ♦rst♠t♦♥ ♦ t♥♠r ♦ ♠♦s ② ts ♣♣r♦ s strt r♥ ♦r ♥♠r ①♣r♠♥tsrtr♠♦r t ♦s s t♦ ♣r♦r♠ ♥ ♥t ♠♦ st♦♥ ♥ rs♦♥♦♠♣tt♦♥ t♠ s♥ t ♣r♠trs r ♦♥② st♠t ♦r t ♥q st♠♦ s ts ♣♣r♦ s ♣♦ss ♥sr t♦ t ♦♠♥t♦r ♠♦ st♦♥♣r♦♠ s ♥♦♥ t♦ r ♥ ♦r ♦♥r ♠①tr ♠♦

trtr ♦ ts ♣tr s ♣♣r s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♦♥t♦♥ ♦s ♦ t♦♥ s ♦t t♦ ♠①♠♠ ♦♦ st♠t♦♥ ♥ ♠ ♦rt♠ t♦♥ ♣rs♥ts t tr♦♣♦st♥s s♠♣r ♣r♦r♠♥ t ♠♦ st♦♥ tr♦ t ♥trt ♦♠♣tt♦♦ ♥ t♦♥ s♦ tt t ♣r♦♣♦s ♣♣r♦ ♦r ♦♠♣t♥ t ♥trt ♦♠♣tt ♦♦ rs t ss ♦ t ♣♣r♦ ♦r

①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

♦r ♥♠r② ♠♣s③ t ♦♦ ♦r ♦ t tr♦♣♦st♥ss♠♣r ♥ t ①t② ♦ t ♠♠ ♠♦ ♦♥ s♠t t t♦♥ ♣rs♥tst♦ str ♥②ss ♦ ♦♦ t sts ♣r♦r♠ ② t ♣ ♦♦s ♦♥s♦♥ s r♥ ♥ tr ①t♥s♦♥s r sss ♥ t♦♥ ts rsts r ♣rt ♦ t rt ♥t ♠①tr ♠♦ ♦ ♦♥t♦♥ ♣♥♥s ♠♦st♦ str t♦r t ❬❱❪

①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r

♠♦s

♥ ♣r♦♣♦s ♠♦ rrr s ♦♥t♦♥ ♦s ♦ ♠♠ss♠s tt t rs ♥♣♥♥t② r♦♠ ♠①tr ♦ g ♦♠♣♦♥♥ts ♦ ♦♥t♦♥② ♥♣♥♥t ♦s r t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s s qt♥ sss ♦ ♦♦s ♠t♥♦♠ strt♦♥ ♣r ♠♦s s ♠t♥♦♠ strt♦♥ ♥ r ♣r♠trs ♦rrs♣♦♥♥ t♦ t ♠♦s♦ t strt♦♥ ♦r ♣rs② t ♠♦s r ♥ s t ♦t♦♥s ♦ trst ♣r♦ts t ♦tr ♣r♠trs r q

♦♥t♦♥② ♥♣♥♥t ♦s q t♥ sss

r♣rtt♦♥ ♦ t rs s ss♠ t♦ q t♥ sss s♦ s t ♥♦tt♦♥s ♦ t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♥♥ t♦♥ ② ♦♠tt♥ t ♥①t♦♥ ♦♥ k

♣rtt♦♥ ♦ t rs qs t♥ sss r♣rtt♦♥ ♦ td t♦r rs xi = (①1i , . . . , ①

di ) ♥t♦ ♦s tr♠♥s ♣rtt♦♥ σ =

(σ1, . . . ,σ) ♦ 1, . . . , d ♥ s♦♥t ♥♦♥♠♣t② ssts s ♣rtt♦♥ ♥s♥ ♥rt t♦r rs xb

i = xσbi = (x

bhi ;h = 1, . . . ,mb) ♦t♥

② t ♦♥t♥t♦♥ ♦ t sst ♦ xi ss♦t t♦ σb r mb =∏

j∈σbmj

s t ♥♠r ♦ t ♠♦t② r♦ss♥s ♥t♦ ♦ b r xbi ss

s♥t ♦♥ s♥ xbhi = 1 ♥ i ts ♠♦t② h ♦r t ♥t♦r r t ♠♦t② r♦ss♥ h ♦ t ♥t rs t t♦t ♦ b ♥ xbhi = 0 ♦trs

r♣t ♥ ♠♦ s♣ ♠♦ s ♥ ② t ♥♠r ♦ ♦♠♣♦♥♥ts t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥ t ♥♠r ♦ ♠♦s ♦r ♠t♥♦♠ strt♦♥ ♦ t s ♥ ② t tr♣t ω = (g,σ, ℓ) rℓ = (ℓ1, . . . , ℓg) r♦♣s t ♥♠rs ♦ ♠♦s t ℓk = (ℓk1, . . . , ℓk) ♥ rℓkb s t ♥♠r ♦ ♠♦s ♦ xb

i ♦r ss k (t 0 < ℓkb < mb)

♥t♦♥ ①tr ♦ ♦♥t♦♥② ♥♣♥♥t ♦s q t♥ sss t♦r r xi s r♥ ② ♠♠ ♠♦ ♥ ② ω ♥ ♣r♠tr③

♦♥♦ t tt♣sr♦rr♣r♦t♦rr♦♣❴

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

② θ ts ♣ s rtt♥ s

p(xi;θ,ω) =

g∑

k=1

πkp(xi;αk,σ, ℓk) t p(xi;αk,σ, ℓk) =∏

b=1

p(xbi ;αkb, ℓkb),

r θ = (π,α) ♥♦ts t ♦ ♠①tr ♣r♠trs r π = (π1, . . . , πg)s t t♦r ♦ ss ♣r♦♣♦rt♦♥s t 0 < πk ≤ 1 ♥

∑gk=1 πk = 1 ♥ r

α = (α1, . . . ,αg) s t t♦r r♦♣s t ♣r♠trs ♦ t ♠t♥♦♠strt♦♥s ♣r ♠♦s t αk = (αk1, . . . ,αk)

t♥♦♠ strt♦♥ ♣r ♠♦s

❲ ♥♦ s♣② t ♠t♥♦♠ strt♦♥ ♣r ♠♦s ♦ t s ♥tr♦ ts♣r♠tr s♣ ♦r t♦ ♥ ts ♣

♥t♦♥ Pr♠tr s♣ ♦ ♠t♥♦♠ strt♦♥ ♣r ♠♦s tαkb = (αhkb;h = 1, . . . ,mb) t t♦r ♦ s③ mb ♥ t τkb t ♠♣♣♥r♦♠ 1, . . . ,mb t♦ 1, . . . ,mb ♦rr♥ t ♠♥ts ♦ αkb ② rs♥ s αkb ♥♦ts t ♣r♠trs ♦ t ♠t♥♦♠ strt♦♥ ♣r ℓkb ♠♦st♥ t s ♥ ♥ t ♦♥str♥ s♠♣① S(ℓkb,mb) ♥ s ♦♦s

S(ℓkb,mb) =

αkb : 0 ≤ αhkb ≤ 1,mb∑

h=1

αhkb = 1, α(ℓkj+1)

kb = . . . = α(mb)kb

.

r s t s♦rtr ♥♦tt♦♥ α(h)kb = α

τkj(h)

kb s♦ α(h)kb ≥ α

(h+1)kb (1 ≤ h < mb)

♥t♦♥ t♥♦♠ strt♦♥ ♣r ♠♦s ♥rt t♦rt x

bi s mb ♠♦ts ♥ ♦♦s ♠trt strt♦♥ ♣r ℓkb ♠♦s

ts ♣ s s♦ rtt♥ s

p(xbi ;αkb, ℓkb) =

mb∏

h=1

(

αhkb)x

bhi ,

r αkb = (αhkb;h = 1, . . . ,mb) ∈ S(ℓkb,mb) ♥ αhkb s t ♣r♦t② tt

♥ i ts ♠♦t② h ♦ t ♦♥t♥t t♦r r xbi

①tr ♠♦ ♦ ♦♥t♦♥ ♠♦s

♥t♦♥ ①tr ♠♦ ♦ ♦♥t♦♥ ♠♦s t♦r r xis r♥ ② ♠♠ ♠♦ ♥ ② ω ♥ ♣r♠tr③ ② θ ts ♣ s ♥ ②

p(xi;θ,ω) =

g∑

k=1

πk

∏

b=1

mb∏

h=1

(

αhkb)x

bhi .

♠r ♠ ♠♦ s ♥ ♥ t ♠ ♠♦ s♥ t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ t♥ t ♥t rs s ♥ ② ♣tt♥ ♦♥r ♣r ♦ s♦ d = ♥ σ = (1, . . . , ) ♥ ② ①♥ t ♥♠r ♦♠♦s s t ♥♠r ♦ ♠♦ts ♦ t rs ♠♥s ♦♥ ℓkj = mj − 1

①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

Pr♦♣rts ♦ t ♠①tr ♠♦ ♣r ♦♥t♦♥ ♠♦s

♦ s ♦ ♥tr♣rtt♦♥ ♠♠ ♠♦ s t♦ s ♦ ♥tr♣rtt♦♥rst② t ♥trss ♣♥♥s ♦ rs q t♥ sss r ♠♣s③ ② t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥ ② σ ♦♥② t♥trss ♥ ♥tr♦ ♣♥♥s ♦ ♠♦ts ♣♦ss② r♥t t♥sss r s♠♠r③ ② t ♠♦s ♦t♦♥s ♥ ♣r♦ts

♦ ♦♠♣t tr♠s s♦rtr s♠♠r② ♦r strt♦♥ s s♦ ② s♥ t ♦♦♥ ♦♠♣t tr♠s κkb ♥ ρkb ♥ ♦♥ [0, 1] ②

κkb =ℓkb

mb − 1♥ ρkb =

ℓkb∑

h=1

α(h)kb .

② rt rs♣t② t ♦♠♣①t② ♥ t str♥t ♦ t ♥trss ♥ ♥tr♦ ♣♥♥s ♦r ♥st♥ t s♠r s κkb ♥ t rr s ρkb t ♠♦r♠ss ♥ rtrst ♠♦t② r♦ss♥s s t strt♦♥ ♥ t♠♦s r ♥tr♣rt s ♥ ♦r♦♥trt♦♥ t t ♥♦r♠ strt♦♥ ♠♦♥ t ♠♦t② r♦ss♥s

♥t② ♦t tt t r♣rtt♦♥ ♦ t rs r♥ts t ♠♦♥r ♥tt② s♥ t s q t♥ sss ♥ t ts ♦♥str♥tt rsts ♦ ❬❪ ♥ ♣♣ t♦ ♣r♦ t ♥r ♥tt② ♦ t♠♠ ♠♦ ts r ♥ ♥ ♣♣♥① s♣t t ♦♥str♥t t♦ ♦t s♠ r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♦r t sss t ♠♦ st②s① s ♦ t s♣ ♦ strt♦♥

♠r ♦ ♣r♠trs ♠♥ ♦ t ♦r♠r ♣rs♠♦♥♦s rs♦♥s ♦ t♠♠♦ ♦♥t♦♥ ♥♣♥♥ ♦ ♣r♦♣♦s ♥ ❬❪ s t♦ ♦♥sr ♦♥②♦♥ ♠♦ ♦r ♠t♥♦♠ strt♦♥ ♦ t ♥t r s t♦♥ r♥t ♦♥str♥ts ♦ qt② r t♥ t♥ t rs ♥♦r sss♥ t ♠♥② ♦ ts ♠♦s r ♥ ♥t♦ t ♠♦ ♠② ♦ ♠♠ ② ♣tt♥ = d ♥ ℓkb = 1 ♥ t♦♥ t ♠♠ ♠♦s ♥ ν♠♠ ♣r♠trs ♥ ②

ν♠♠ = (g − 1) +

g∑

k=1

∑

b=1

ℓkb.

s ♠♦ ♦ t ♠♠ ♠② ♥ rqr ss ♣r♠trs t♥ ♠ ♠♦t ν♠ = (g−1)+g×∑

b=1(mb−1) ♣r♠trst♦ t ts ♥t♦ ♦♥t

t ♦♥t♦♥ ♣♥♥s

♣r♠tr③t♦♥ ♦ t ♦ strt♦♥

♥ ♣rs♠♦♥♦s rs♦♥s ♦ t ♠ ♠♦ ♥tr♦ ♥ ❬❪r ♠♥♥ s♥ ♠t♥♦♠ strt♦♥ s ①♣rss t t♦ t②♣s ♦♣r♠trs srt ♦♥ tr♠♥s t ♦t♦♥ ♦ t ♠♦ ♦ t strt♦♥♥ ♦♥t♥♦s ♦♥ s ts ♣r♦t② s t♦♥ ② s♥ t s♠

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♣r♦♣♦s ♥ ♣r♠tr③t♦♥ ♦ t ♦ strt♦♥ ♥♦t ② (δkb,akb)s ♣r♠tr③t♦♥ tts t ♥tr♣rtt♦♥ ♥ t rt♥ ♦ t ♣r♦r ♥♣♦str♦r strt♦♥s rt t♦ t ♦ ♣r♠trs s t♦♥

♣r♠tr③t♦♥ srt ♣r♠tr δkb = δhkb;h = 1, . . . , ℓkb tr♠♥s t ♠♦ ♦t♦♥s s♥ δhkb ♥ts t ♠♦t② r♦ss♥ r t ♠♦h s ♦t t δhkb ∈ 1, . . . ,mb ♥ δhkb 6= δh

′

kb h 6= h′ ♦♥t♥♦s♣r♠tr akb = (ahkb;h = 1, . . . , ℓkb + 1) tr♠♥s t ♣r♦t② ♠ss ♦ t ℓkb♠♦s ② ts rst ℓkb ♠♥ts ahkb t h = 1, . . . , ℓkb) ♥ t ♣r♦t② ♠ss♦ t ♥♦♥♠♦ ② ts st ♠♥t aℓkb+1

kb ♣r♠tr akb s ♥ ♦♥ t♦♦♥ tr♥t s♠♣①

St(mb) =

akb : 0 ≤ ahkb ≤ 1, ∀h ≤ ℓkb + 1 ♥ ahkb ≥aℓkb+1kb

mb − ℓkb, ∀h ≤ ℓkb

.

♣r♠tr αkb ♥ t ♦♣ (δkb,akb) r rt ②

αhkb =

ah′

kb ∃h′ s tt δh′

kb = haℓkb+1

kb

mb−ℓkb♦trs

①♠♠ ♦♦ st♠t♦♥ ♥

♦rt♠

♠ t ① = (x1, . . . ,xn) t s♠♣ ♦♠♣♦s t n ♥♣♥♥t ♥ ♥t② strt ♥s ss♠♥ t♦ r♥ ② t ♠♠ ♠♦ r♦♠ tss♠♣ t ♠ s t♦ st♠t t ♠ ♦r ① ♠♦ ♥ ② ω

❲♥ ω s ♥♦♥ t ♠♠ ♠♦ ♥ ♥tr♣rt s ♠ ♠♦ ♣♣ ♦♥t ♦♥t♥t rs x

bi r ♦♥str♥ts r t♥ ♣r♠trs

s t ♠ ♥ s② ♦t♥ ② t ♦♦♥ ♠ ♦rt♠


trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ ♦♥t♦♥ ♣r♦ts ♦♠♣tt♦♥

tik(θ[r]) =

π[r]k p(xi;α

[r]k ,σ, ℓk)

∑gk′=1 π

[r]k′ p(xi;α

[r]k′ ,σ, ℓk′)

.

st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

π[r+1]k =

n[r]k

n♥ α(h)[r+1]

kb =

n(h)[r]kb

n[r]k

(1 ≤ h ≤ ℓkb)

1−∑ℓkj

h′=1α(h′)[r+1]kb

mb−ℓkb♦trs

② s♥ t ♥♦tt♦♥s n[r]

k =∑n

i=1 tik(θ[r]) ♥ nh[r+1]

kb =∑n

i=1 tik(θ[r])x

bhi

♦rt♠ ♠ ♦rt♠ t♦ ♦t♥ t ♠ ♦ ♠♠ ♠♦

♠r ♥ t ♥t♦♥ τkb ♦t tt t t ♠ st♣ ♦ trt♦♥ [r] t♥t♦♥ τkb s r♥ s t rs♥ ♦rr♥ ♥t♦♥ ♦ t nh[r+1]

kb ♥ ♦ss t♦ ♥ n(h)[r+1]

kb t n(h)[r+1]kb ≥ n

(h+1)[r+1]kb

♦ st♦♥ tr♦♣♦st♥s

s♠♣r

Pr♦r strt♦♥s ❲ ss♠ tt p(g) = 1gmax

♦r g = 1, . . . , gmax ♥ ttp(σ) r♠♥ tt g ♥ σ r ♥♣♥♥t ♥ p(ℓ|g,σ) ♦♦ ♥♦r♠ strt♦♥s

♠ ♠ s t♦ ♦t♥ t ♠♦ ω = (g, σ, ℓ) s t rst ♣♦str♦r♣r♦t②

ω = argmaxg,σ,ℓ

p(①|g,σ, ℓ) = argmaxg,σ,ℓ

p(g,σ, ℓ|①).

t gmax ♠♦s ♥♦t ② ω(g) = (g,σ(g), ℓ(g)) ♦r g = 1, . . . , gmax r

(σ(g), ℓ(g)) = argmaxσ,ℓ

p(①|g,σ, ℓ) = argmaxσ,ℓ

p(σ, ℓ|①, g).

st ♠♦ s s♦ ♥ s

ω = argmaxg

p(ω(g)|①),

♥ s ♦♥ ② ♣♣②♥ t ♣♣r♦①♠t♦♥ ♠♦♥ t♦s gmax st ♠♦s

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♥ ♥ ①st sr strt② s ♥♦t ♦ ♦r t♦ ♦rrt rs♦♥srst② t ♥♠r ♦ ♦♣s (σ, ℓ) ♥ ①ss② ♥ s♦♥② tst♠t♦♥ ♦ t ♠ ♦r ♦ t♠ s ♥ ♥♥ssr② st ♦ ♦♠♣tt♦♥t♠ tr♦♣♦st♥s s♠♣r strt② ♦r♦♠s ts t♦ rst t s♠ t♠ s ♥♦ sr

♦r ① ♦ g t ♦♣ (σ(g), ℓ(g)) s st♠t ② t ♦♦♥ tr♦♣♦st♥s s♠♣r ❬❪ ♥ p(σ, ℓ|g,①) s stt♦♥r② strt♦♥

s ♦rt♠ s p(σ, ℓ|g,①) s ♠r♥ stt♦♥r② strt♦♥ trt♥r♦♠ ♥ ♥t (σ[0], ℓ[0]) trt♦♥ [s] s rtt♥ s

θ[s+1] ∼ θ|ω[s],①, ③[s]

③[s+1] ∼ ③|ω[s],①,θ[s+1]

(σ[s+1], ℓ[s+1]) ∼ σ, ℓ|ω[s],①, ③[s+1],

r ω[s] = (g,σ[s], ℓ[s])

♦rt♠ tr♦♣♦st♥s s♠♣r t♦ ♦t♥ ω(g)

♠r ♥ t ♠♦ s♠♣♥ rt s♠♣♥ r♦♠ s ts st♣ s s♦ ♣r♦r♠ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠♦s t stt♦♥r② strt♦♥ s p(σ, ℓ|g,①, ③[r+1]) ♦r ts r ♥ ♥t♦♥

♠♣♥ ♦ t ss ♠♠rs♣s s t ♦sr t r ♥♣♥♥tt ♦♥t♦♥ strt♦♥ ♦ ③ s ss ♥ s rtt♥ s

p(③|ω,①,θ) =n∏

i=1

p(zi|ω,xi,θ) t p(zi|ω,xi,θ) =g∏

k=1

(tik(θ))zik .

♥ ts st♦♥ rst② t t ♦♥t♦♥ strt♦♥s s♠♣♥ t ♣r♠trs ♥♦t ② ♥str♠♥t ♠♥ts ② s♥ t ♦ ♣r♠tr③t♦♥♥ ♥ t♦♥ ♥ s♦♥② t t s♠♣♥ ♦ (σ, ℓ) ♦♥sr st ♥trst ♠♥ts

♠♣♥ ♦ t ♥str♠♥t ♠♥ts

❲ ♥♦ t t s♠♣♥ r♦♠ ♥ ② p(θ|ω[s],①, ③[s])

Pr♦r ss♠♣t♦♥ ❲ ss♠ t ♣r♦r ♥♣♥♥ t♥ t ss ♣r♦♣♦rt♦♥s ♥ t ♣r♠trs ♦ t ♦ strt♦♥s ♦ t ♣r♦r ♦ t ♦♣r♠tr s rtt♥ s ♦♦s

p(θ|ω) = p(π|ω)

g∏

k=1

mb∏

b=1

p(αkb|ω).


♦t tt ts ♣r♦♣rt② ♦ ♦♥t♦♥ ♥♣♥♥ s ♣t ② t strt♦♥ ♦ θ♦♥t♦♥② ♦♥ (ω,①, ③) s♥

p(θ|ω,①, ③) = p(π|ω,①, ③)g∏

k=1

mb∏

b=1

p(αkb|ω,①, ③).

Pr♦r ♥ ♣♦str♦r strt♦♥s ♦ π r②s ♥♦♥ ♥♦r♠t ♣r♦rstrt♦♥ ♦r ♠t♥♦♠ s ♦♥t rt strt♦♥ ❬♦❪ ♦t ♣r♦r ♥ t ♣♦str♦r strt♦♥s ♦ π ❬❪ r rs♣t② ♥ ②

π|ω ∼ Dg

(1

2, . . . ,

1

2

)

♥ π|ω,①, ③ ∼ Dg

(1

2+ ♥1, . . . ,

1

2+ ♥g

)

,

r ♥k =∑n

i=1 zik

Pr♦r strt♦♥ ♦ αkb ❲ ♥♦ s t ♣r♠tr③t♦♥ ♦ t ♦ strt♦♥ (δkb,akb) ♥ ♥ t♦♥ ❲ ss♠ t ♥♣♥♥ t♥t ♣r♦r ♦ δkb ♥ ♦ akb s♦

p(αkb|ω) = p(δkb|ω)p(akb|ω).

❲ s ♥♦r♠ strt♦♥ ♠♦♥ t ♠♦ ♦t♦♥s ♥ ♦♥t tr♥t rt strt♦♥ s ♣r♦r ♦ akb s♦

p(δkb|ω) =

(

mb

ℓkb

)−1

♥ akb|ω ∼ Dtℓkb+1

(

γ1kb, . . . , γℓkb+1kb ;mb

)

,

r t γhkb r t ♣r♠trs ♦ t tr♥t rt strt♦♥ s♦ ttakb|ω ∈ St(mb) ❲ ♥♦ ① γhkb = 1 stt♦♥ s ♥ ♥ ♣♣♥① ♣r♦♣♦s ♣r♦r s s♦ ② ♥♦r♠t s♥ t s ♥ ♥♦r♠ strt♦♥

P♦str♦r strt♦♥ ♦ αkb ♣♦str♦r strt♦♥ ♦ αkb s rtt♥ s

p(αkb|ω,①, ③) = p(δkb|ω,①, ③)p(akb|ω, δkb,①, ③).

strt♦♥ ♦ δkb|ω,①, ③ s ♠t♥♦♠ ♦♥ t t♦♦ ♠♥② s t♦ ♦♠♣t t δkb = δhkb;h = 1, . . . , ℓkb t st ♦♥t♥♥ t ♥s ♦ tℓkb rst s ♦ ♥hkb =

∑ni=1 zikx

bhi ♦rr

∀h ∈ 1, . . . , ℓkb − 1, ♥δhkbkb ≥ ♥

δh+1kbkb .

❲ ss♠ tt t r♥ t♥ t ♠♦ ♣r♦ts ♥ t ♥♦♥♠♦♣r♦ts r s♥♥t ♦ ♥ ♣♣r♦①♠t t ♦♥t♦♥ strt♦♥

p(akb|ω) ∝∏ℓkb+1

h=1(ahkb)

γh

kb−1

1

ah

kb≥

aℓkb+1

kb

mb−ℓkb

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♦ δkb ② r ♥ δkb s ♣♣r♦①♠t♦♥ s str♥t♥ ② t st ♦♥r♥ s♣ ♦ t srt ♣r♠trs ❬❪ ♦♥r♥♥ ♥♦ akb s ts ♣r♦r s♦♥t ts ♦♥t♦♥ strt♦♥ s ①♣t② ♥ s

akb|ω, δkb,①, ③ ∼ Dtℓkb+1

(

1 + ♥(1)kb , . . . , 1 + ♥

(ℓkj)

kb , 1 + ♥ℓkbkb ;mb)

,

r ♥(h)kb s t ht rr ♦ t st ♥hkb;h = 1, . . . ,mb ♥ ♥ℓkbkb =

♥k −∑ℓkb

h=1 ♥(h)kb

♠♣♥ ♦ ♥ ♠♦ (σ[s+1], ℓ[s+1])

♥ s♠♣♥ ♦ ω[s+1] = (g,σ[s+1], ℓ[s+1]) ♦r♥ t♦ s ♣r♦r♠ ② ♦♥ trt♦♥ ♦ t ♦♦♥ ♠♠ ♦rt♠ ♦s t stt♦♥r② strt♦♥ s p(σ, ℓ|g,①, ③[r+1]) s ♦rt♠ s ♥ t♦ st♣s rst② ts♠♣s ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♥ r♣rtt♦♥ ♦t rs ♥t♦ ♦s ♥ t ♠♦ ♥♠r ♦ t ♠♦ ♦s ♥♦t rs♣t② ② σ[s+1] ♥ ℓ[s+1/2] ♦♥② t s♠♣s t ♠♦ ♥♠r ♦ ♦② ♦♥ ♠♠ trt♦♥ s t s♠♣♥ ♦ ω[s+1] s ♦♠♣♦s ♥t♦ t t♦♦♦♥ st♣s

s ♦rt♠ s p(σ, ℓ|g,①, ③[s+1]) s stt♦♥r② strt♦♥ t ttrt♦♥ [s] ♦ ♦rt♠ t s♠♣♥ ♦ ω[s+1] s ♣r♦r♠ ♦r♥t♦ ♦t ♦♦♥ st♣s

(σ[s+1], ℓ[s+1/2]) ∼ σ, ℓ|ω[s],①, ③[s+1]

ℓ[s+1] ∼ ℓ|ω[s+1/2],①, ③[s+1],

r ω[s+1/2] = (g,σ[s+1], ℓ[s+1/2])

♦rt♠ ♦rt♠

tr♦♣♦sst♥s ♦rt♠ t♦ s♠♣ ω[s+1/2]

s♠♣♥ ♦ ω[s+1/2] s ♣r♦r♠ ② ♦♥ trt♦♥ ♦ t tr♦♣♦sst♥s♦rt♠ ♥t♦ t♦ st♣s rst② t ♥str♠♥t strt♦♥ q(.;ω[s])♥rts ♥t ω⋆ = (g,σ⋆, ℓ⋆) ♦♥② ω[s+1] s s♠♣ ♦r♥ t♦ t♣t♥ ♣r♦t②

♥str♠♥t strt♦♥ ♥str♠♥t strt♦♥ q(.;ω[s]) s♠♣s ω⋆

♥ t♦ st♣s rst st♣ ♥s t ♦ tt♦♥ ♦ ♦♥ r ♥ ♣rtσ⋆ s ♥♦r♠② s♠♣ ♥ V (σ[s]) = σ : ∃!b s b ∈ σ

[s]j ♥ b /∈ σj s♦♥

st♣ ♥♦r♠② s♠♣s t ♠♦ ♥♠rs ♠♦♥ ts ♣♦ss s ♦r t♠♦ ♦s ℓ⋆kj = ℓ

[s]kj ♦r ♥♦♥♠♦ ♦s j s tt σ[s]

j = σ⋆j


♣t♥ ♣r♦t② ♣t♥ ♣r♦t② λ[s] s ♥ ②

λ[s] = min

p(①, ③[s+1]|ω⋆)

p(①, ③[s+1]|ω[s])

q(ω[s];ω⋆)

q(ω⋆;ω[s]); 1

.

♦♠♣tt♦♥ ♦ λ[s] ♥♦s t♦ ♦♠♣t t ♥trt ♦♠♣tt ♦♦ ❲ ♥♦ sr ♦ t♦ s♦ ts ♣r♦♠ t♦t s♥ t s

♣♣r♦①♠t♦♥ ♦r s♥ t♦♦ ♠ t♠ ♦♥s♠♥ ♠♠ ♠t♦s s♠♣♥♦ ω[s+1/2] s s♦ ♣r♦r♠ ② t ♦♦♥ tr♦♣♦sst♥s ♦rt♠

s ♦rt♠ s p(σ, ℓ|g,①, ③[s+1]) s stt♦♥r② strt♦♥ trt♥r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s

ω⋆ ∼ q(ω;ω[s])

ω[s+1/2] =

ω⋆ t ♣r♦t② λ[s]

ω[s] t ♣r♦t② 1− λ[s].


♦rt♠ t♦ s♠♣ ℓ[s+1]

s st♣ ♦s s t♦ ♥rs ♦r rs t ♠♦ ♥♠r ♦ ♦ ②♦♥ t trt♦♥ ♦ ℓ[s+1]

kb s s♠♣ r♦♠ p(ℓkb|ω[s+1/2],①, ③[s+1]) ♥ ②

p(ℓkb|ω[s+1/2],①, ③[s+1]) ∝

p(①b|③[s+1], ℓkb) |ℓkb − ℓ[s+1/2]kb | < 2

♥ ℓkb /∈ 0,mb.0 ♦trs

r ①b = (xbi ; i = 1, . . . , n) s ts ♦rt♠ rqrs t ♦♠♣tt♦♥ ♦

p(①b|③, ℓkb) ♥ ②

p(①b|③, ℓkb) =∫

S(ℓkb,mb)

mb∏

h=1

(αhkb)♥hkbdαkb

♥ tt t ♥♦

♥trt ♦♠♣tt ♦♦

♥trt ♦♠♣tt ♦♦ s ♥ s

p(①, ③|ω) = p(③|ω)

g∏

k=1

∏

b=1

p(①b|③, ℓkb).

♦t tt t q♥tts p(①, ③|ω) ♥ p(①j|③, ℓkb) r rs♣t② rqr t♦ ♦♠♣t t ♣t♥ ♣r♦t② ♦ t tr♦♣♦sst♥s ♦rt♠ ♥ ② ♥ t♦ s♠♣ t ♥♠r ♦ ♠♦s r♦♠ t ♥ t ②

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♣♣r♦①♠t♦♥s ♦r ♥st♥ t ♥trt ♦♠♣tt ♦♦ s♣♣r♦①♠t ②

ln p(①, ③|ω) = ln p(①, ③|θ⋆,ω)− ν

2lnn+O(1),

r θ⋆ s t ♠①♠♠ ♦♠♣tt ♦♦ st♠t ♦r ts ♥ ♦♣♣r♦①♠t♦♥ s ♦♥② s②♠♣t♦t② tr ♥ ♥ ♦rst♠t t ♠♦ ♥♠rss t♦♥ s ③|ω ♦♦s ♥♦r♠ strt♦♥ ♠♦♥ t ♣♦ss♣rtt♦♥s ♣r♦♣♦s t♦ ♦♠♣t p(①b|③, ℓkb) t♦ ♦t♥ p(①, ③|ω) s ♦♠♣tt♦♥ s ♥♦t s② s♥ αkb s ♥ ♦♥ S(ℓkb;mb) ♥ ♥♦t ♦♥ t ♦ s♠♣①♦ s③ ℓkb ①♣t ♥ ℓkb = mb − 1 ♥ s s ♥ s t ♣♣r♦ ♦t ♠ ♠♦ ❬❪ ♥ ①♣t ♦r♠ s ♥ ♥ t ♦♦♥ ♣r♦♣♦st♦♥② ♣r♦r♠♥ ♥ ①t ♦♠♣tt♦♥ ♦ t ♥tr ♦r t ♦♥t♥♦s ♣r♠trs♥ ♥ ♣♣r♦①♠t♦♥ ♦♥ t srt ♦♥s ♦r t ♣r♦♦ s ♥ ♣♣♥①

Pr♦♣♦st♦♥ ♥trt ♦♠♣tt ♦♦ s ♣♣r♦①♠t ② ♥t♥ t s♠ ♦r t srt ♣r♠trs ♦ t ♠♦s ♦t♦♥s ♥ ② ♣r♦r♠♥ t ①t ♦♠♣tt♦♥ ♦♥ t ♦♥t♥♦s ♣r♠trs s♦

p(xb|z, ℓkb) ≈(

1

mb − ℓkb

)nℓkbkb

ℓkb∏

h=1

Bi(

1mb−h+1

; n(h)kb + 1; nhkb + 1

)

mb − h,

r Bi(x; a, b) = B(1; a, b)−B(x; a, b) ♥ r B(x; a, b) s t ♥♦♠♣t t♥t♦♥ ♥ ② B(x; a, b) =

∫ x

0wa(1− w)bdw

r♦♠ t ♣r♦s ①♣rss♦♥ t s strt♦rr t♦ ♦t♥ p(①, ③|ω)


♥trt ♦♠♣tt ♦♦ ♦♠♣rs♦♥ ♦ ♦t

♣♣r♦s

♠ r♥ ts ①♣r♠♥t t t ss ♦ t rtr♦♥ ♦r tst♦♥ ♦ t ♥♠r ♦ ♠♦s ♥ t ♥ ♥ ② t ♣r♦♣♦s ♦♠♣tt♦♥♦ t ♥trt ♦♠♣tt ♦♦

t ♥rt♦♥ ❲ ♥t t♦ ♦♠♣r ♦t ♣♣r♦s ♦r t st♦♥ ♦ t♥♠r ♦ ♠♦s ♦ s♠t s♠♣s ♦♠♣♦s t n ♥s rs♥r♦♠ ♠t♥♦♠ strt♦♥ ♣r ♠♦s Ms(r, r, r,

1−3rs−3

, . . . , 1−3rs−3

) t s ♠♦ts ♥ tr ♠♦s ♥ ♣r♦t② r ♦r r♥t s③s ♦ s♠♣ 105

s♠♣s r ♥rt t r♥t s ♦ (r, s)

sts r s ♦♠♣rs♦♥ t♥ t ♣r♦♣♦s ♣♣r♦ ♥ t ♣♣r♦①♠t♦♥ ♦r t st♦♥ ♦ t ♥♠r ♦ ♠♦s ♣r♦♣♦srtr♦♥ ♦t♥s ttr rsts t♥ t rtr♦♥ ♥ t ♦r st stt♦♥s♦r t r s♠♣ s③s rtr♠♦r t ♦s t♦ ♥r ♦rst♠ts t ♥♠r


0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

sample size

prob

abili

ty

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

sample size

prob

abili

ty

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

sample size

prob

abili

ty

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

1.0

sample size

prob

abili

ty

r Pr♦t② tt t rtr♦♥ r♣rs♥t ♥ ♥ ♥s ♥t ♣r♦♣♦s ♣♣r♦ r♣rs♥t ♥ ♦ r ♥s st t tr ♥♠r ♦♠♦s r♣rs♥t ♥ ♣♥ ♥ ♥ ♦rst♠t t r♣rs♥t ♥ ♦tt ♥ r s r s r s r s

♦ ♠♦s ♥② ts rt② s s♠r t♥ t rtr♦♥ ♦♥ ❲ ♥tr ♥♦♥t♦ ♠♦r s♣ ♦♠♠♥ts

♥ s ♠♦s r ♣r♦t② ♠ss ♥ t② r s② tts♥ tr r ♠♦ts s ♦t rtr t s♠ ♦r s♥t② ♥ t tr ♥♠r ♦ ♠♦s t ♣r♦t② ♦s t♦ ♦♥ ♥ ♦r s♠s♠♣s

❲♥ t ♠♦ ♣r♦ts rs s t s ♠♦r t t♦ ♥t②t♠ ♥ s s t rtr♦♥ ttr ♥s t tr ♥♠r ♦ ♠♦st♥ t ♣r♦♣♦s ♣♣r♦ ♦r t s♠ s♠♣s s③ ♦r t♥ ♦rt rtr♦♥ s ♠♦rt rs t♦ ♦rst♠t t ♥♠r ♦ ♠♦s

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

t ♣r♦♣♦s ♣♣r♦ ♥rst♠ts ts ♥♠r ♥ t s r♦♥ ❲♥ ts♠♣ s③ s rr t♥ t ♣r♦♣♦s ♣♣r♦ ♦t♥s ttr rsts ♥t ♥s tr ♥♠r ♦ ♠♦s ♠♦st ②s t rtr♦♥ ♣s ♥♦rst♠t♦♥ rs

t ♥♠r ♦ ♠♦ts ♥rss s t♥ t ♣r♦♠ ♦♠s rr♥ t ♣r♦♣♦s ♣♣r♦ s♦ s♦s ts ♥trst s♥ t rtr♦♥ s str♦♥②s ♥ s s rtr♦♥ ♣s ts s ♥ ♦r r t st t ♣r♦♣♦s ♣♣r♦ ♠♦st ②s ♥s t tr ♥♠r ♦ ♠♦s ♥ ts♠♣ s③ s rr t♥

♥② ♥♦t tt ♥ t ♠♦r ♦♠♣① stt♦♥s ♥ s ♣r♦t② ♠ss ♦r t ♠♦s ♥ r ♥♠r ♦ ♠♦ts t ♣r♦♣♦s ♣♣r♦♥rst♠ts t ♥♠r ♦ ♠♦s ♥ t s♠♣ s③ s s♠ t♥ ♦♥rst♦ t tr ♠♦ s ♥ t s♠♣ s③ ♥rss ♦t tt ♥ s st s ♦ t rtr♦♥ st②s s♥♥t ♥ ♦r r t st

s ♦♥ ts ①♣r♠♥t t ♣r♦♣♦s rtr♦♥ ♣♣rs s t ♠♦st r♥ts♥ ts s②♠♣t♦t ♦r s ttr t♥ t s②♠♣t♦t ♦r ♦ t

rtr♦♥ ♥ t ♥r ♦rst♠ts t ♥♠r ♦ ♠♦s ♥ ts rt② ss♠r t♥ t rt② ♦ t rtr♦♥

♠t♦♥ t s♣ ♠♦

♠ r♥ ts ①♣r♠♥t t t ♦♦ ♦r ♦ t ♦rt♠s♠ ♦rt♠ ♥ tr♦♣♦st♥s s♠♣r ♦r ♣r♦r♠♥ t st♠t♦♥♦ t ♠ ♥ t ♠♦ st♦♥ ♦ t r ♥rt ♦r♥ t♦ ♠♠

♠♦ t♥ t ♠♦ ♥ t ♠ r st♠t qt② ♦ t st♠t♦♥s tr♠♥ ② t r r♥ ❲ s♦ tt ts q♥tt② ♦♥rs t♦ ③r♦ ♥ t s♠♣ s③ ♥rss ♦ ♦♥ t♦ t ♦♦ ♦r♦ ♦t ♦rt♠s

t ♥rt♦♥ t st ♦ s① rs t tr ♠♦ts s ♥rt♦r♥ t♦ ♦♠♣♦♥♥t ♠♠ ♠♦ t t ♦♦♥ ♣r♠trs

σ = (1, 2, 3, 4, 5, 6), ℓkj = 2, π = (0.5, 0.5), αkj = (0.4, 0.4, 0.2/7).

♠♦s r ♦t t r♥t ♠♦t② r♦ss♥s ♦r ♦t sss

sts ♦r r♥t s ♦ n = (50, 100, 200, 400, 800) s♠♣s r ♥rt r r♥ s ♦♠♣t t♥ t tr ♥ tst♠t ♣r♠trs ♣rs♥ts t ♠♥ ♦ ts r♥

s t r r♥ ♦♥rs t♦ ③r♦ ♥ t s♠♣ s③ ♥rss ♠ tt t st♠t strt♦♥ ♦♥rs t♦ t tr ♦♥ s ♦♥ t♦ t ♦♦ ♦r ♦ t st♠t♦♥ ♦rt♠

♠t♦♥ t ♠ss♣ ♠♦

♠ r♥ ts ①♣r♠♥t ♥r♥ tt t ①t② ♦ t ♠♠ ♠♦♦s t t♦ ♣ ♦♦ rsts ♥ t ♠♦ s ♠ss♣ s s♠t


n ♠♥

s

♥ ♥ st♥r t♦♥ ♦ t r r♥ ♦♠♣t t♥ t tr ♣r♠trs ♦ t s♣ ♠♦ ♥ t ♠①♠♠ ♦♦ st♠ts ss♦t t♦ t ♠♦ st ② t tr♦♣♦st♥s♦rt♠ ♦r r♥t s♠♣ s③s

s♠♣s ♦r♥ t♦ ♦♠♣♦♥♥t ♠①tr ♠♦ r t ♥trss ♣♥♥s r r♥t ♦r ♦t ♦♠♣♦♥♥ts t♥♥ ♣r♠tr ♦s s t♦ ♠♦②t str♥t ♦ t ♥trss ♣♥♥s ♥ t ss ♦r♣♣♥ rsts♦ t ♠♠ ♠♦ r ♦♠♣r t♦ t♦s ♦ t ♠ ♠♦

t ♥rt♦♥ t st ♦ s③ s s♠♣ r♦♠ t ♦♦♥ ♦♠♣♦♥♥t♠①tr ♠♦ ♦ ♠♥s♦♥ s①

p(x;θ) = 0.53∏

h=1

p(x2h−1,x2h;θ) + 0.5 p(x1;θ)p(x6;θ)2∏

h=1

p(x2h,x2h+1;θ),

t p(xj,xj+1;θ) = p(xj;θ)(

λ1xj=xj+1 + (1− λ)p(xj+1;θ))

♥ t p(xj;θ) =∑3

h=1(1/3)xjh s ♥ λ = 0 t s♠♣ s ♥rt ② ♥♦r♠ strt♦♥

♥ sss r ♦♥s rr s t t♥♥ ♣r♠tr λ t rr r t♥trss ♣♥♥s ♥ t ss s♣rt♦♥ ♦t tt ♠♠ s ♥♦t t tr♠♦ s♥ t ♦♥t♦♥② ♦rrt rs r ♥♦t t s♠ ♥ ♦t sss

sts ♦r r♥t s ♦ λ = (0.2, 0.4, 0.6, 0.8) s♠♣s r ♥rt r r♥ ss♦t t♦ t ♠♦ t t st ♥♠r ♦sss st ② t rtr♦♥ ♠♦♥ g = 1, ..., 4 s ♦♠♣t ♣rs♥ts t rsts ♦t♥ ② t ♠♠ ♥ t ♠ ♠♦s

λ ♠♠ ♠

r r♥ ♥ ♠♥ ♦ t ♥♠r ♦ sss ♦t♥② ♠♠ ♥ ♠

rr s λ t rr s t r r♥ ♦r ♦t ♠♦s♦r t ①t② ♦ t ♠♠ ♠♦ ♦s t♦ ♣ ♥ ♣t ♦ tr r♥ ts r♥ r♦s r♠t② str t t♠ ♠♦ rtr♠♦r ♥ t sss r s♣rt r ♦ λ t♠♠ ♠♦ ♥s ♠♦r ♦t♥ t tr ♥♠r ♦ sss t♥ t ♠ ♠♦

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s


♦r ♦t ♣♣t♦♥s t st♠t♦♥ ♦ t ♠♠ ♠♦ s ♣r♦r♠ ② t ♣ ♦♦s ♦t t sts r ♥ ♦♦s ♦♣ ② t t♦rs

rs str♥

t ❲ st② ♦♦ t st sr♥ ♣♥s srs ② ♣♠ ♥ ①tr♥ ♠♦r♣♦♦ rtrsts ♣rs♥t ♥ ❬r❪s srs r ♥t♦ tr ss♣s r♦s rs r♠♥r rs ♥ srs rs

rs mj ♠♦ts♦r ♥♦♥ ♦♥t♥♦s

②r♦s ♥♦♥ r② ♣r♦♥♦♥s t ♥ t ♥ t

♦rr ♥♦♥ ♠♥②♥r ♠ ♠

Prs♥tt♦♥ ♦ t ♣♠ ♥ ①tr♥ ♠♦r♣♦♦ rssr♥ t ♣♥s

①♣r♠♥t stt♥s ss♣s ♠♠rs♣s ♦ t ♥s r ♥♦r g = 1, . . . , 6 t ♠ ♦ t ♠ ♠♦ s ♦t♥ ② ♥t③t♦♥s ♦ ♥ ♠♦rt♠ ♥s ♦ trt♦♥s r ♣r♦r♠ ♦r t ♠♦ st♦♥♦ t ♠♠ ♠♦ ♦♦ ② ♥t③t♦♥s ♦ ♠ ♦rt♠ t♦ ♥ t ♠

sts ♣rs♥ts t s ♦ t rtr♦♥ ♦r ♦t ♠♦s ♥r♥t ♥♠rs ♦ sss ♥ ♦t ♠♦s st t♦ ♦♠♣♦♥♥ts t s♦ t rtr♦♥ r ttr ♦r t ♠♠ ♠♦ t♥ ♦r t ♠ ♠♦ ♦r t♥♠rs ♦ sss s t ♠♠ ♠♦ ttr ts t t t♥ t ♠ ♠♦

g ♠♠ ♠

❱s ♦ t rtr♦♥ ♦r r♥t ♥♠rs ♦ sss ♦t♥ ②t ♠♠ ♥ t ♠ ♠♦s ♦ ♥ts t st s ♦ ts rtr♦♥

♦r♥ t♦ s♣②♥ t ♦♥s♦♥ ♠tr① t♥ t st♠t♣rtt♦♥s ♥ t ss♣s ♠ tt t rs r ♠♦r r♥t t♥t t♦ ♦tr ss♣s ♥ ♦t ♠♦s t t rs ♥ ss t st♠t ♣rtt♦♥s ② ♦t ♠♦s r s♠r r♠r tt t ♠♠ ♠♦ts ss ♦tr ss♣s ♥ ts ss t♥ t ♠ ♠♦


♠♠ ♠

ss ss ss ss r♦s

r♠♥r rs

♦♥s♦♥ ts t♥ t ss♣s ♥ st♠t ♣rtt♦♥ ♥t♦t♦ sss

r s♣②s t srs sttr♣♦t ♦♥ t rst ♦rrs♣♦♥♥ ♥②ss ♣♥ ♥ ♥ts t ss♣s ❲ ♥♦t tt t rs r ♥ t s♠♦t♦♥ ♦tt♦♠ t ♦r t rst ♣r♥♣ ♦rrs♣♦♥♥ ♠♣ ❲ s♣② t♣rtt♦♥ ♦rrs♣♦♥♥ t♦ t st ♠♦ t ♠♠ ♠♦ t t♦ ♦♠♣♦♥♥ts♥ r ♦t tt ♦r ♦t ♠♦s t rst ♣r♥♣ ♦rrs♣♦♥♥ ①s♦s t♦ ♥ sst♦♥ r

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

1.5

2.0

First principal correspondence analysis

Sec

ond

corr

espo

nden

ce a

naly

sis

Dichrous Lherminieri Subalaris

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

1.5

2.0


Sec

ond

corr

espo

nden

ce a

naly

sis

Class 1 cmm Class 2 cmm

r rs ♦♥ t rst ♣r♥♣ ♦rrs♣♦♥♥ ♥②ss ♠♣ tt ss♣s ♥ t t st ♠♠ ♠♦ st♠t ♣rtt♦♥ ♦tr♥s ♥t t ♥s t ♥ ss ♦r t ♠♠ ♠♦ ♥ ♥ ss ♦r t ♠ ♠♦ ♥ ♥♦r♠ ♥♦s ♦♥ [0, 0.1] s ♦♥ ♦t ①s♦r ♥ ♥ ♦rr t♦ ♠♣r♦ s③t♦♥

❲ ♥♦ sr t st ♦♠♣♦♥♥t ♠♠ ♠♦ ♥ t st♠t♠♦ ss♠s ♦♥t♦♥ ♥♣♥♥ t♥ rs ts ♠♦ s ♦ ♥trsts ♦ ts s♣rst② ♥ t s ♠♦r ♣rs♠♦♥♦s t♥ t ♠ ♠♦ s♥ s♠ ♥♠r ♦ ♠♦s s st♠t s s♦♥ ② t s♠♠r② ♣r♦♣♦s ② κkj♥ ρkj ♥ ♥ ♥ ♣rs♥t ♥ s t rst rs rrtr③ ② ♠♦ts t ♣r♦t② s t rs r ♦♥t♦♥② ♥♣♥♥t κkj ♥ts t ♥♠r ♦ ♠♦ts ♥ ♣r♦t②♣♣r t♥ t ♥♦r♠ strt♦♥ ♦r ①♠♣ t ♠t♥♦♠ strt♦♥ ♦t r s s t♦ ♠♦s ♦r ♦t sss s♦ κkj = 2/3

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♦r ②r♦s s ♦rr ♥rss ss

♠♠r② ♦ t ♠♠ ♠♦ t tr sss κkj s s♣② ♥ ♣♥♥ ρkj s s♣② ♥ ♣r♥tss

♠①♠♠ ♦♦ st♠ts ♦ t ♦♠♣♦♥♥t ♣r♠trs r ♣rs♥t② r sr ♦rrs♣♦♥s t♦ ♦ ♦ r ts ♥♦t♥ tt t st♠t ♠♦ ss♠s t ♦♥t♦♥ ♥♣♥♥ ♦r ♦ ♦ rs t ♠♦t② r♦ss♥s r ♦♥ ♠♦ s st♠t ♦r t st♦♥ ♦♠♣♦♥♥t r ♦s ♦r ts ♠♦t② r♦ss♥s s♣② tr ♠t♣r♦t② ♠sss ♦r ♦♠♣♦♥♥t t ♦♠♣♦♥♥t r ♥t ② r♥t♦♦rs s ♠♦t② r♦ss♥s r ♣rs♥t ② rs♥ ♦rr ♦ ♠t♣r♦t② ♠ss

♦t tt t ♠♦ ♦t♦♥s r sr♠♥t s♥ t ♠♦t② rs♣t s ♣r♦t② ♦ rs♣ ♦r ss t ♠♦t② trs♣ ♥ t s ♣r♦t② ♦ rs♣

0.0

0.2

0.4

1 3 2 4

collar.

0.0

0.4

0.8

1.2

3 2

eyebrows.

0.0

0.4

0.8

white black BLACK_white

sub−caudal.

0.0

0.5

1.0

1.5

none few

border.

0.0

0.4

0.8

male

gender.

r ss ♣r♠trs ♦ t ♦♠♣♦♥♥ts ♠♠ ♠♦ st♠t ♦♥ trs t ♦♦r rs♣t② t r② ♦♦r ♦rrs♣♦♥s t♦ t♣r♦t② ♠ss ♦ t ♠♦s ♦r ss rs♣t② t♦ ss

♥② t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s♠s rst s♥ t ♦♥t♦♥ r♠rs ❱ ♠srs ♣rs♥t ♥ r s♠ ❲ s♦ ♣r♦r♠ ♦♦tstr♣ tst ♦ t ♦ ♥t② ♦ t r♠rs ❱ ② ♥rt♥ s♠♣s ❲ ♦t♥ ♣ ♦ s♦ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ st


ss

ss

tr① ♦ t r♠rs ❱ ♠srs ♦♠♣t ♦r♥ t♦ t st♠tsss

t ♥♠♠t♦♥s str♥

t ❲ ♥t t♦ str ♣t♥ts ❬❩❪ sr ② ♥r② rs♦rr♥ ♦ ♥s ♠r ♣♥ ♠ r♥ ♣s♥ Ps ♠trt♦♥♣♥s ♥ r♥♥ ♦ rtr r ♥ ② ♦♥ r ♥ tr ♠♦ts t♠♣rtr ♦ t ♣t♥t ♠ T < 37C 37C ≤ T < 38C ♥ 38C ≥ T ❲ ♥♦ tt s♦♠ ♣t♥ts ♦♥ ♦ t ♦♦♥ sss ♦ t r♥r② s②st♠♥♠♠t♦♥ ♦ r♥r② r ♥ ♣rts ♦ r♥ ♣s ♦r♥

①♣r♠♥t ♦♥t♦♥s ❲ s t s♠ ①♣r♠♥t ♦♥t♦♥s s t rsstr♥

sts ♣rs♥ts t s ♦ t rtr♦♥ ♦r ♦t ♠♦s ♥r♥t ♥♠rs ♦ sss ♦r ♥♠r ♦ sss t rtr♦♥ ♦t ♠♠ ♠♦ s ttr t♥ t♦s ♦ t ♠ ♠♦ rtr♠♦r t ♠♠ ♠♦sts tr sss t ♠ ♠♦ sts ♦r sss s ♣♥♦♠♥♦♥ ♥ t♦ t ♦t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♦ t ♠ ♠♦

g ♠♠ ♠

❱s ♦ t rtr♦♥ ♦r r♥t ♥♠rs ♦ sss ♥ ♦r t♠♠ ♥ t ♠ ♠♦s ♦ ♥ts t st s ♦ ts rtr♦♥

♦t tt t st♠t strt♦♥s ♦ t ♠ ♥ t ♠♠ ♠♦s r r♥t ♦t♥ ♣rtt♦♥ r s♦ r♥t s♣②s t ♦♥s♦♥♠trs t♥ t st ♠♠ ♠♦ ♥ t ♠ ♠♦s t tr ♥ ♦rsss s ♥s ♦♥sttt r♦♣ s s♣rt r♦♠ t♦tr ♥s ss ♦r t tr ♠♦s t ♦tr ♥s ss♠♠rs♣ tr♠♥ ② t st ♠♦

r s♣②s t ♥s ♦♥ t ♣r♥♣ ♦rrs♣♦♥♥ ♥②ss♠♣ r t st♠t sss r s♣rt

♠♠ ♠♦ t tr sss s t ♦♦♥ r♣rtt♦♥ ♦ t rs♥t♦ ♦s σ = (♠♣ Ps r, , ♠) s s♦♥ ② t s♠

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♠♠

♠ ♠ ♠

♠♠

♠ ♠ ♠ ♠

♦♥s♦♥ ♠trs t♥ t st ♠♠ ♠♦ ♥ t ♠ ♠♦st tr ♥ ♦r sss

−0.5 0.0 0.5 1.0

−0.

8−

0.6

−0.

4−

0.2

0.0

0.2

0.4

0.6

Axe 1 of the multiple correspondance analysis

Axe

5 o

f the

mul

tiple

cor

resp

onda

nce

anal

ysis

r ♥s ♦♥ t ♣r♥♣ ♦rrs♣♦♥♥ ♥②ss ♠♣ t tst ♠♠ ♠♦ st♠t ♣rtt♦♥ ♥ ♥♦r♠ ♥♦s ♦♥ [0, 0.1] s ♦♥ ♦t ①s ♦r ♥ ♥ ♦rr t♦ ♠♣r♦ s③t♦♥ ♦♦rs♥ s②♠♦s ♥t t ss ♠♠rs♣

♠r② ρkj ♥ κkj s♣② ♥ t tr sss r ♦♥♥trt ♥ ♠♦t② r♦ss♥s ♦r t ♦ ♦♥ ♥ ♥ ♦♥ ♦t♦♥ t ♣r♦t② ♦s t♦♦♥ ♦r t t♦ ♦tr ♦s

♠♣ ♠ Ps rss ss ss

♠♠r② ♦ t ♠♠ ♠♦ t tr sss κkj s s♣② ♥♣♥ ♥ ρkj s s♣② ♥ ♣r♥tss

♦♦♥ ss ♥tr♣rtt♦♥ s s ♦♥ t ss ♣r♠trs s♣② ②r ♦t tt t rs r♥ ♣s♥ ♥ r♥♥ ♦ rtr r t♠♦st sr♠♥t ♦♥s

♠♦rt② ss r♦♣s ♥s ♥ ♥♦ ♥s ♥ ♥♦ ♠r♣♥

♦♥s♦♥

s♦♥ ss r♦♣s ♥s ♥ ♥♦ ♥s t ♠r ♣♥ tr ss r♦♣s ♥s ♥ ♥s ♥ ♠r ♣♥

rtr♠♦r ts ♥s s♦♠ r ♥ ♠trt♦♥ ♣♥

0.0

0.2

0.4

fiever fiever fiever fiever cold normal cold normalnormalnormal fiever normalyes yes no yes no no yes yes yes yes no yesno yes yes yes no no yes yes yes no no noyes no no yes no no yes no yes no no yes

Tem−Pus−Mic−Bur.

0.0

1.0

2.0

no yes

Nau.

0.0

1.0

2.0

yes no

Lum.

r st♠t ♣r♠trs ♦ t tr♦♠♣♦♥♥t ♠♠ ♠♦ s♣② ②t r♣♦t ♥t♦♥ ♦ t ♣ ♦♦s ♦♦r ♦rrs♣♦♥s t♦ ss r② ♦♦r ♦rrs♣♦♥s t♦ ss ♥ ♣ r② ♦♦r ♦rrs♣♦♥s t♦ ss

♦♥s♦♥

♥ ts ♣tr ♣rs♥t ♥ ♠①tr ♠♦ ♠♠ t♦ str t♦r t ts str♥t s t♦ r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♥t♦ st② ♣rs♠♦♥♦s s♠♠r② ♦ t strt♦♥ s ♥ ② κkj ♥ ρkj ss ♥ s♠♠r③ ② t ♠♦ ♦t♦♥s s s♦♥ ♦♥ t rs♣♣t♦♥ t ♠♠ ♠♦ ♥ ♦t♣r♦r♠ t ss t♥t ss ♠♦ ♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s tr t♥s t♦ ts s♣rst②

♦♠♥t♦r ♣r♦♠s ♥♦ ② t ♦ tt♦♥ ♥ ② t st♦♥♦ t ♥♠rs ♦ ♠♦s r ♦ ② tr♦♣♦st♥s ♦rt♠ s♦rt♠ ♥ s s t ♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt♦♦ ♥ ♥t② ♣♣r♦①♠t s ts ♣♣r♦ ♥ s t♦st t ♥trt♦♥s ♦ t ♦♥r ♠①tr ♠♦ ♣r ♦

♦r t ♠♦ s r② st♠t t t st s r ♥♠r ♦rs ♦♠ ♦♥str♥ts ♦♥ t ♦ rs r♣rtt♦♥ ♦ s♦ ♦r ♥st♥ t ♥♠r ♦ rs ♥t♦ ♦s ♦ ♠t t♦ tr rs♥♦tr s♦t♦♥ ♦ t♦ st♠t t ♠♦ ② ♦rrr strt②t t s ♥♦♥ tt ts ♠t♦s r s♦♣t♠

♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s

♥② ♠♣♦s t qt② ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s♦r t sss s ♣r♦♣rt② ♦s s t♦ ♣r♦ t ♥r ♥tt② ♦t ♠♠ ♠♦ s ♦ ①t② s ♦♥tr♥ ② ① ♦ strt♦♥ ♦r ♦♥ ♦ tr② t♦ r① t ssqt② ♦ σ t t ♠♦♥♦♥tt② rs

♣tr

♦ ♦♠♣rs♦♥ ♣r♦r♠ ②

tr ♣s

s ♣tr ♠ s t♦ strt t ♣sstrt ♥ ♦♦s rs♣t② ♣r♦r♠t ♥r♥ ♦ t ♠ ♥ t ♠♠ ♠♦s♥ ♦rr t♦ ♠ ♠♦♥strt♦♥ ♦ ♦t ♣s s t♠ t♦ ♣r♦r♠ t str ♥②ss ♦ t r♥♥♥①♠♣ ♣rs♥t ② ♦ ♣tr ❲ r♠♥ tt ts t st s♣②s t t♦♥ s♦♥♦r r♦s ♦ ♥t ①r②s tt ♠② s♦ ♥♣♥t rs ♣r♦r♠ ② ♥tsts♦t tt ts ♣tr ♥ s♦ s s tt♦r ♦♦t ♣s ♥ t ♣r♦s ♣rs♥tt♦♥ ♦ tr♠♥ ♥t♦♥s ♥ ♠♥② sr♣ts ♦♥ t♦ ♣r♦r♠ tstr ♥②ss

♥♠s r q t s♦♠♥♠s r ♠♦r q t♥ ♦trs

♦r r ♥♠ r♠

strt

strt ♦r

Prs♥tt♦♥ ♣ strt ♣r♦r♠s t str♥ ♦ t♦rt ♦r♥ t♦ t ♠ ♠♦ ts ♠♥ ♥t♦♥s r ♠♣♠♥t ♥ ❲ r♠♥ tt ♥ ts ♠♦ rs r r♦♣ ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♥ ♦rr t♦ ♦♥sr t ♠♥ ♥trss ♦rrt♦♥s ♥trss♣♥♥② t♥ rs r♦♣ ♥s t s♠ ♦ s t♥ ♥t♦ ♦♥t② ♠①♥ t♦ ①tr♠ strt♦♥s r rs♣t② ♥♣♥♥ ♥ ♠①♠♠ ♣♥♥②

♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

♦♥♦ strt ♣ s rr♥t② ♦♥ ♦r t t ♦♦♥ r tt♣sr♦rr♣r♦t♦rr♦♣❴ ♥stt♦♥ ♥t ♦♥ ♦ strt ♥ ♣r♦r♠ ② s♥ t ♦♦♥ sr♣ts

♥st ♦♠♠♥> ♥st♣sstrt

r♣♦stt♣♦r♣r♦t♦r

strt sr♣t strt ♥stt♦♥

strt ♦♥> rqrstrt

strt sr♣t strt ♦♥

st♠t♦♥ ♣r♠tr st♠t♦♥ ② ♠①♠♠ ♦♦ s ♣r♦r♠ ♠ ♦rt♠ s ♦rt♠ s ♦r t ♠♦ st♦♥ ♦s t♦♠♥t♦r ♣r♦♠s ♥ ② t ♦ strtr sr

♥ ♥t♦♥s

♥t♦♥s ♦♠♣♦s t strt ♣ ♥ ♥t♦♥ ♣r♦r♠s tstr ♥②ss ♦ t t ts t♥♥ ♣r♠trs ♥ s♣ ② t sr ②♥ s♣ ♥t♦♥ tr st ♥t♦♥s r ♠♣♠♥t ♥ ♦rr t♦r♥② ♣rs♥t t ♣r♠trs ② ♣r♦♥ ♥♠r ♦r r♣ s♠♠rs

str♥ ♥t♦♥ str ♥②ss ♥ ♣r♦r♠ t t ♥t♦♥strt t♥ ♦r r♠♥ts

> strtt ♥❴str

♠♦ strt② strt②tt

strt sr♣t str♥ ♥t♦♥

s ♥t♦♥ s t♦ ♠♥t♦r② r♠♥ts t r♠ t t♦ str ♦s t ♦♠♥s r ♥♦♥③r♦ ♥trs ♦r

t♦rs ♥ ♥tr t♦r ♥❴str s♣②♥ t ♥♠r ♦ sss

t s r t♥ ♦r t r♠♥ts ♠♦ ♥ strt② r♠♥t ♠♦ s t♦r ♥ t ♠♦t② ♥♠r ♦r r r♠♥t strt② s ♥ ♥st♥ ♦ t strt②t ss ♦♥

t♥s t st♠♥ts ♥♣ts ♣r♠trs rt t♦ t st♠t♦♥ ♦rt♠s

strt

♥t♦♥ strt rtr♥s ♥ ♥st♥ ♦ t stt ss ♦♥t♥s t ♦t♣ts

t♥♥ ♥t♦♥ st♠♥ts ♣r♠trs ♦ t st♠t♦♥ ♦rt♠s♦♥t♥ ♥ t strt②t ss ♥ s♣ ② t ♥t♦♥ strt②t

t♥ ♦r r♠♥ts

> strt②tt ♥❴♥t st♦♣❴rtr♦♥

♥♦t ♣rtt♦♥ ♣rtt♦♥tt

strt sr♣t t♥♥ ♥t♦♥

♥♣t t ♠tr① t s ♠♥t♦r② ♥ t tr ♦trs ♥♣t ♣r♠trs♦ t♦ t♥ t ♦rt♠s

r♠♥t ♥❴♥t sts t ♥♠r ♦ t♠s r ♠♠ ♥ sstrt ② t ♠♠ ♥s r ♥t③

r♠♥t st♦♣❴rtr♦♥ s t ♥tr ♦rrs♣♦♥♥ t♦ t ♥♠r♦ sss trt♦♥s ♦ t ♠♠ ♥ r ♥♦ ttr ♠♦ s ♥t♥ t ♦rt♠ s st♦♣♣ ② t t ts t s ♦ 20× d

r♠♥t ♣rtt♦♥ s t ♥t ♦ t r♣rtt♦♥ ♦ t rs♥t♦ ♦s σ[0] ② t t s q t♦ t ♣rtt♦♥ ♣r♦ ② t ♠♥♠③♥ t ♦ ♥♠r t♦t ♦ ♦♥sst♥ ♦ ♠♦r t♥♦r rs

r t♦♦ ♥t♦♥s strt ♣ s♦ ♣r♦s t♦♦ ♥t♦♥s s♠♠r② s♠♠r②❴♣♥♥s ♥ ♣♦t rs♣t② t♦ s♠♠r③ rsts t♦ ♣rs♥t t ♠♥ ♦♥t♦♥ ♣♥♥s ♥ t♦ s③ t ♣r♠trs

strt t♦ str t ♥tsts t st

str♥ t ♠ ❲ ♥♦ ♣rs♥t t rsts ♦ t ♠ ♠♦ ♦t♥② t ♣ strt ② s♥ t ♦♦♥ sr♣t

♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

t st ♦♥> t♥tst

♥t♦♥ ♦ t t♥♥ ♣r♠trs ♦r t st♠t♦♥ ♦rt♠> st strt②t♥tst ♥❴♥t st♦♣❴rtr♦♥

st♠t♦♥ ♦ t ♦♠♣♦♥♥t ♠ ♠♦> rs strt♥tst ♠♦ r♣ strt②

st

strt sr♣t ♥tsts t st str♥

♦ st♦♥ rtr♦♥ sts t♦ sss t ♦ t ♠s tt t ♠ ♠♦ ttr ts t t t♥ t ♠♦ ♣rs♥t ♥❬❪ s♥ ts rtr♦♥ s rtr♦♥ s ♦r t ♠♥ t ♠ ♠♦s r s♣② ♥ ❲ ♥t t ♦♠♣t♥ t♠♥ s♦♥s ♦t♥ t ♣r♦ss♦r ♥t ♦r t♦ st♠t t ♠♠♦ r ♠♠ ♥s r strt t st♦♣♣♥ rs qmax = 100 t ♠ ♠♦ ♥s ss t♥ s t t ♣ ①♠♦ ❬+❪

g ♠ ♠

t♠ s

rtr♦♥ s ♦r t ♠ ♥ t ♠ ♠♦s ♦r♥ t♦r♥t ♥♠rs ♦ sss ♦r t ♥tstr② t st st s r ♥ ♦

❲ ♥♦t tt t ♠ ♠♦ ♦t♥s ttr s ♦r t rtr♦♥ t♥t ♠ ♠♦ ♥ g = 1, 2 ❲♥ t ♥♠r ♦ sss s rr g ≥ 3 t st♠ ♠♦ ss♠s t ♦♥t♦♥ ♥♣♥♥ t♥ rs

♦♠♣rs♦♥ ♦ t rsts rtr♦♥ sts t♦ sss ♦r t ♠♠♦ s rst s ♦r♥t t s♣tt♥ ♦ t tt t♥ t s♦♥ ♥t r♦s ♦♥s rtr♠♦r t t♦ ♠♥ rtrsts ♦ t ♦♥r ♠①tr♠♦ ♠♣♦s ♥ ❬❪ r t♦♠t② tt ② t ♠♦ ♠♣♦rt♥ ♦t t♦ ♠♦t② r♦ss♥s r t ♥tsts t s♠ ♥♦ss ♥ ♣♥♥② t♥ t ♥♦ss ♦ t ♥tsts ♥ s t st♠t♠♦ s ♦r♥t t t ♠♣♦s ♠♦ ♣rs♥t ♥ ❬❪ ♥♦ ♥♦r♠t♦♥s ♥ ♣r♦r

strt

st ♠♦ ♥tr♣rtt♦♥ ♠♦ ♥tr♣rtt♦♥ s tt ② t♦ t♦♦s♥t♦♥s ♥t♦♥ s♠♠r② ♣r♦s ♥r ♦r ♦ t ♠♦ ♥♦r♠t♦♥ rtr ♣r♦♣♦rt♦♥s ♦s ♦ rs ♥ ♥trss ♣♥♥sρkb

♥t♦♥ ♣r♦♥ ♠♦ ♦r> s♠♠r②rs

♠r ♦ sss ♦♦♦ Pr♦♣♦rt♦♥s ♦s r♣rtt♦♥ ♦ t rs ♦r t ss

❱rs ♦♦

♦s r♣rtt♦♥ ♦ t rs ♦r t ss ❱rs ♦

♦ ♦

strt sr♣t ♦ ♦r

♥t♦♥ s♠♠r②❴♣♥♥s ♦ss ♦♥ t ♥trss ♣♥♥② ♣r♠trs

♥t♦♥ ♣r♦♥ s♠♠r② ♦ t ♥trss ♣♥♥s> s♠♠r②❴♣♥♥srs

♦s r♣rtt♦♥ ♦ t rs ♦r t ss

♦ ♦♥t♥s t rs t ♦

s♦♥ s♦♥ s♦♥ s♦♥ s♦♥ r♦s r♦s r♦s r♦s r♦s♦s r♣rtt♦♥ ♦ t rs ♦r t ss

♦ ♦♥t♥s t rs t ♦

s♦♥ r♦s r♦s s♦♥

♦ ♦♥t♥s t rs t ♦

strt sr♣t ♠♠r② ♦ t ♥trss ♣♥♥s

♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

♦r♥ t♦ t ♣r♦s ♦t♣ts t tt ♠♦ ♥ ♥tr♣rt s ♦♦s

♠♦rt② ss π1 = 0.86 ♠♥② trs t s♦♥ tt r s str♦♥ ♣♥♥② t♥ t ♥♦ss σ1 = (1, 2, 3, 4, 5) ♥ρ11 = 0.35 ♣♥♥② strtr ♦ t ♠①♠♠ ♣♥♥② strt♦♥ ♥ts ♥ ♦r ♦♥trt♦♥ ♦ ♦t ♠♦t② ♥trt♦♥s rt ♥tsts t s♠ ♥♦ss s♣② ♥ t② ♠ tt tt♦♦t s s♦♥ τ ❴s♦♥

11 = 0.95 ♥ τ ❴r♦s11 = 0.05

♠♥♦rt② ss π2 = 0.14 r♦♣s ♣r♥♣② t r♦s tt r s ♣♥♥② t♥ t ♥tsts ♥ ♣r♦s ♦♣♣♦st ♥♦ss t ♥♦ss ♦ t ♦tr ♦♥s r ♥♣♥♥t ♥ t ss σ2 =(3, 4, 1, 2, 5) ρ21 = 0.25 ♥ ρ22 = 0

st ♠♦ s③t♦♥ ♥② t ♥t♦♥ ♣♦t ♣r♦s r♣ s♠♠r② ♦ t ♣r♠trs

♥t♦♥ ♣r♦♥ t r♣ s♠♠r② ♥ ② r > ♣♦trs

strt sr♣t r♣ s♠♠r② ♦ t ♣r♠trs

♥ ♦r♥ts t st♠t sss r r♣rs♥t t rs♣t t♦ tr ♣r♦♣♦rt♦♥ ♥ rs♥ ♦rr ♦t tt tr ♦rrs♣♦♥♥ r ♣♥s ♦♥ tr♣r♦♣♦rt♦♥ ♠t ♣r♦♣♦rt♦♥s r ♥t ♦♥ t t s ♥ ssstr ♥t♦♥s r ♥ rst ♦♥ s t ♥trrs ♦rrt♦♥s ρkb♦r t ♦s ♦ t ss ♦rr ② tr str♥t ♦ ♦rrt♦♥ ♥ rs♥♦rr s♦♥ ♦♥ s t ♥trrs ♦rrt♦♥s τ kb ♦r ♦ r♥♦r♥ t♦ t str♥t ♦ tr ♣♥♥s ♥ rs♥ ♦rr tr ♦♥s t rs r♣rtt♦♥ ♣r ♦s ♥ts tt t r sss♥ ♥t♦ t ♦ ♥ t ♥ts tt ♦♥t♦♥② ♦♥ ts sst r s ♥♣♥♥t ♦ t rs ♦ ts ♦ ♦r ①♠♣ ts rs♦s tt t rst ss s ♣r♦♣♦rt♦♥ ♦ 0.86 ♥ tt t rs rss♥ ♥t♦ t s♠ ♦

♦♦s

♦♦s ♦r

Prs♥tt♦♥ ♣ ♦♦s ♣r♦r♠s t str♥ ♦ t♦rt ♦r♥ t♦ t ♠♠ ♠♦ ❲ r♠♥ tt ♥ ts ♠♦ rs rr♦♣ ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s q t♥ sss ♥ tt ♦ ♦♦s ♠t♥♦♠ strt♦♥ ♣r ♠♦s t ♥t♦♥s ♦ ts ♣r ♠♣♠♥t ♥ ♦ t② s♦ ♠♣♠♥t ♥ ♥ ♦rr t♦ ♥rst ♦♠♣tt♦♥ s♣

♦♦s

1 0.75 0.5 0.25 00 0.25 0.5 0.75 1ρkb τkb

0

0.86

1

Class 1

de1

de2

de3

de4

de5

Class 2

de1

de2

de3

de4

de5

de1

de2

de3

de4

de5

r ♠♠r② ♦ t st ♠ ♦r♥ t♦ ♦r t ♥tsts t st

♦♥♦ ♦♦s ♣ s rr♥t② ♦♥ ♦r t t ♦♦♥ r tt♣sr♦rr♣r♦t♦rr♦♣❴ ♥stt♦♥ ♥t ♦♥ ♦ ♦♦s ♥ ♣r♦r♠ ② s♥ t ♦♦♥ sr♣ts

♥st ♦♠♠♥> ♥st♣s♦♦s

r♣♦stt♣♦r♣r♦t♦r

♦♦s sr♣t ♦♦s ♥stt♦♥

♦♦s ♦♥> rqr♦♦s

♦♦s sr♣t ♦♦s ♦♥

st♠t♦♥ ♣r♠tr st♠t♦♥ ② ♠①♠♠ ♦♦ s ♣r♦r♠ ②♥ ♦rt♠ s ♦rt♠ s s ♦r ♠♦ st♦♥ t♦ ♦ ♦♠♥t♦r ♣r♦♠s ♥♦ ② t ♦ strtr sr ♦t tt t s♦rt♠ ss ♥ ♥t ♣♣r♦①♠t♦♥ ♦ t ♥trt ♦♠♣tt ♦♦

♥ ♥t♦♥s

♦r ♥t♦♥s ♦♠♣♦s t ♦♦s ♣ ♦♥ ♥t♦♥ ♣r♦r♠s t str♥②ss ♥ tr ♥t♦♥s ♣ ♦r t rst ♥tr♣rtt♦♥

♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

str♥ ♥t♦♥ str ♥②ss ♥ ♣r♦r♠ t t ♥t♦♥♦♦sstr t♥ s♥ r♠♥ts

> ♦♦sstr① s❴♥t s❴tr

s❴ ❴♥t ❴t♦

♦♦s sr♣t ♥tsts t st str♥

s ♥t♦♥ s t♦ ♠♥t♦r② r♠♥ts t r♠ ① t♦ ♥②③ ♦s ♦♠♥s r t♦rs ♥ ♥tr s stt♥ t ♥♠r ♦ sss

t s r t♥ ♦r t t♥♥ r♠♥ts ♥♠r ♦ ♠♠ ♥s ♣r♦r♠ t♦ st t st ♠♦ s st ② t

r♠♥t s❴♥t t s ♥♠r ♦ trt♦♥s ♦ t s s♠♣r s st ② t r♠♥t s❴tr

t s ♥♠r ♦ trt♦♥s ♦ t r♥♥ ♦ t s s♠♣r s st ② t

r♠♥t s❴ t s ♥♠r ♦ r♥t ♥t③t♦♥s ♦ t ♠ ♦rt♠ st♠t♥ t

♠ ♦r t st ♠♦ ♦r♥ t♦ t s s♠♣r s st ② t r♠♥t❴♥t t s

♠ ♦rt♠ s st♦♣♣ ♥ t ♥rs ♦ t ♦♦ s s♠rt♥ ❴t♦ t s

r t♦♦ ♥t♦♥s ♦♦s ♣ s♦ ♣r♦s t♦♦ ♥t♦♥s s♠♠r② r♣♦t ♥ ♣♦t rs♣t② s♠♠r② ♦ t ♠♦ r♣ s♠♠r② ♦ t ♣r♠trs ♥ sttr♣♦t ♦ t ♥s ♥ t♠t♣ ♦rrs♣♦♥♥ ♠♣

♦♦s t♦ str t ♥tsts t st

str♥ t ♠♠ ❲ ♥♦ s♣② t rsts ♦ t ♠♠ ♠♦ st♠tt t ♣ ♦♦s ② s♥ t ♦♦♥ sr♣t

> rs ♦♦sstr♥tst s❴♥t

s❴tr ❴♥t


♦ st♦♥ s ♦ t rtr♦♥ ♦t♥ ② t tr ♠♦s♠ ♠ ♥ ♠♠ r ♣rs♥t ♥

♠♦ tt♥ t st t t s t ♦♠♣♦♥♥t ♠ ♠♦ ♠♠♠♦ ts t t ttr t♥ t ♠ ♠♦ ♦r ♥♠r ♦ sss s♠r t♥

♦♦s

g ♠ ♠ ♠♠

rtr♦♥ s ♦r t ♠ t ♠ ♥ t ♠♠ ♠♦s ♦r♥ t♦ r♥t ♥♠rs ♦ sss ♦r t ♥tstr② t st st s r ♥♦

tr ♦t tt ♥ t ss ♥♠r s ♣♣r ♦r q t♦ tr ♦t ♠ ♥♠♠ ♠♦s r q♥t t♦ t ♠ ♠♦

♣♦ss rs♦♥ ①♣♥♥ t ♣♦♦r ♣r♦r♠♥ ♦ t ♠♠ ♠♦ ♦ ts ♦♥str♥t ♦ t qt② t♥ ss ♦ t r♣rtt♦♥ ♦ t rs ♥t♦♦s ❲ r♠♥ tt ts ss♠♣t♦♥ s ♥♦t ♠ ② t ♠ ♠♦

♥ ♦rr t♦ strt t t♦♦s ♥t♦♥s ♦ ♦♦s ♥♦ ♥②③ t ♦♠♣♦♥♥t ♠♠ ♠♦

♦♠♣♦♥♥t ♠♠ ♥tr♣rtt♦♥ ♠♦ ♥tr♣rtt♦♥ ♦ t ♠♠♠♦s tt ② tr t♦♦s ♥t♦♥ ♥t♦♥ s♠♠r② ♣r♦s ♥r♦r ♦ t ♠♦ ♥♦r♠t♦♥ rtr ♥♠rs ♦ ♠♦s τkb κkb

♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

♥t♦♥ ♣r♦♥ ♠♦ ♦r> s♠♠r②rs

♠r ♦ rs ♠r ♦ ♥s ♠r ♦ ♠♦ts ss ♥♠r ♦♦♦

♦ ♥♠r

ss ss

♥①

ss ss ♣♣ ♥①

ss ss


♥t♦♥ r♣♦t ♣r♦s r♣ s♠♠r② ♦ t ♣r♠trs ♥ t ♣♦ts r♣♦t rt♥ t ♣r♦t② ♦ t ♠♦s ♣r ss ♦r ♦② ♦rr♥ t ♠♦t② r♦ss♥s ♦r♥ t♦ tr ♣♦str♦r ♣r♦t②

r♣♦t ♦ t ♣r♠trs ♣rs♥t ② r > r♣♦trs


♠♦rt② ss s♣② ♥ r② s ♠♥② ♦♠♣♦s t t s♦♥ ♥♦ss s♦♥ ss s♣② ♥ s ♦♠♣♦s t tt ♥♦s sr♦s ② s♦♠ ♥tsts s♣② t t ♦t tt t ♥tst ♠♥② ♥♦ss t tt s s♦♥ s♥ ts ♦rrs♣♦♥♥ r s ♠♦ ♥ ts ♦t♦♥♦r ♦t sss

♦♠♣♦♥♥t ♠♠ s③t♦♥ ♥t♦♥ ♣♦t ♣r♦s sttr♣♦t♦ t ♥s ② ♥t♥ tr ss ♠♠rs♣ ♦r♥ t♦ t ♠♣ r♥ ♦rrs♣♦♥♥ ♥②ss ♠♣ r t ①s r ♦s♥ ② t sr

♦♦s

0.0

0.4

0.8

sound sound carious sound sound carioussound carious carious carious sound soundsound sound carious carious carious sound

de1−de2−de3.

0.0

0.4

0.8

1.2

sound

de4.

0.0

0.4

0.8

carious sound

de5.

r ♠♠r② ♦ t ♠♠ ♣r♠trs

ttr♣♦t ♦ t ♥s ♣rs♥t ② r > ♣♦trs


♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s

−0.5 0.0 0.5 1.0 1.5 2.0 2.5

−0.

50.

00.

51.

01.

5

Axe 1 of the multiple correspondence analysis

Axe

2 o

f the

mul

tiple

cor

resp

onde

nce

anal

ysis

r ttr♣♦t ♥ t rst ♦rrs♣♦♥♥ ♥②ss ♠♣ ♥s t t♦ t ♠♦rt② ss r s♣② ② r tr♥s t♦s t t♦t ♠♥♦rt② ss r s♣② ② rs ♥ ♥♦r♠ ♥♦s ♦♥ ❬ ❪s ♥ t♦♠t② ♦♥ ♦t ①s ♦r ♥ ♥ ♦rr t♦ ♠♣r♦s③t♦♥

♦♥s♦♥ ♦ Prt

❲ s♥ tt t ss t♥t ss ♠♦ ♦s t♦ t t s♠t sts t♥s t♦ ts s♣rst② ♥ ② ts ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♦r s t sts ♠♦r ♦♠♣① ♠♦s t♥ ♥t♦ ♦♥t t ♥trss♣♥♥s r rr♥t s♥ t ♥♦r♠t♦♥ ♦ ♦♥t♦♥ ♣♥♥② s ♥♦t♣rs♥t

❲♥ t ♥♠r ♦ ♥s s s♥t② r ♦r♥ t♦ t ♥♠r ♦rs t t♥t ss ♠♦ ♥ s ♥ ts ♦♥t♦♥ ♥♣♥♥ss♠♣t♦♥ s ♦t ❲ ♣rs♥t tr ♠♥ ♠♦s ♦ t ♦r♣②r①♥ ts ss♠♣t♦♥ t t② r ♥ t r♥t ts ♠♦ st♦♥ ♥stt② ♦r ♥ ♥tr♣rtt♦♥ ♦ sss s ♣r♦r♠ tr♦♦t♥♦tr t♥t r

❲ s♦ ♣r♦♣♦s t♦ ♠①tr ♠♦s r s♣ rs♦♥s ♦ t♦♥r ♠①tr ♠♦ ② ♦♥sr t ♠♥ ♥trss ♣♥♥s t♥st♦ ♦♠♣♦♥♥t strt♦♥ ♣r ♥♣♥♥t ♦s r ♠♥ str♥t s tt♦t ♠♦s ♥ s♠♠r③ ② ♠♥♥ ♣r♠trs ♥ t ♠♠♦ ♣r♦s ♦♥ ♦♥t ♥ ♦♥ ♣♥♥② rt♦♥s♣ ② ♦s ♦ rs t ♠♠ ♠♦ ♣r♦s rtrst ♠♦s ♥ t♦ ♥t♦rs♦ t ♣♥♥② str♥t ♣r ♦ ♦t tt ♦t ♠♦s ♥ ♦♥sr ♥trt♦♥s ♠♦♥ ♠♦r t♥ t♦ rs t s ♦♥s t ♥t♦ ♦♥t t♥trt♦♥s ♦ ♦rr ♦♥ ♦r t♦

s ♠♦s ♦ t ♦♥r ♠①tr ♠② ♦t ♣r♦♣♦s ♠♦s r ♥t ♦♠♣① ♥ ♦r t ♠♦ st♦♥ s t② s ♦ s♥ t♥♠r ♦ ♦♠♣t♥ ♠♦s ♥ rtr♠♦r t ♥♦r♠t♦♥ rtrr ♥r② s②♠♣t♦t s♦ t② r ♥ ♥ t ♥♠r ♦ ♠♦s s r♦r♥ t♦ t s♠♣ s③ ♦r ♥st♥ t rtr♦♥ s s t♦ st t♠♦ ♥♠r ❲ ♣r♦♣♦s ♠♠ ♦rt♠ ♣r♦r♠♥ r♥♦♠ ♠♦♥ t ♠♦s ♥ ♦rr t♦ r ts r ♦r t ♦♠♣tt♦♥t♠ ♥rss t t s③ ♦ t ♠♦ s♣ s ♣♥♦♠♥♦♥ s str♦♥♦st t♦ t ♥②ss ② t ♣r♦♣♦s ♠♦s ♦ t sts t r ♥♠r♦ rs s t ♠ ♠♦ s rsr ♦r t str ♥②ss ♦ t stst rs ♠♠ ♠♦ s ss ♦♠♣① ♥ ts ♠♦ st♦♥ ♦s ♥♦trqr ♥② ♣r♠tr st♠t ♦ t ♠♠ ♠♦ ♥ str ♠♦r ♦♠♣① tsts ♦r ♦r s t♦ s ts ♠♦s ♦♥ t sts ♥ t ♠♦st rs ♥ ♥ t ♥♠r ♦ rs s r tr r t♦♦ ♠♥② ♠♦s♥ ♦♠♣tt♦♥ ♦ t s ♦rt♠ rqrs t♦♦ ♠♥② trt♦♥s t♦ s♠♣♦r♥ t♦ ts stt♦♥r② strt♦♥ ♥ s s ♣r♠t ♣♣r♦ ♦♦♥ssts ♥ t st♠t♦♥ ♦ ♦♦ ♠♦ t ♥♦t t st ♦♥ s t ♠♦

♣tr

st♦♥ ♦ ♣r♦r♠ ② tr♠♥st t s♦♣t♠ ♣♣r♦ t♦rr ♠t♦

♥② t ♣♣r♦s ♣r♦r♠♥ t ♠♦ st♦♥ ♦t♥ rqr t♦ ♥rt ♣r♠trs ♦r ♥t ♠♦ t ♦♥② ♥tr♣rt ♦♥s r t♦srt t♦ t st ♠♦ r ♦ ♦♥ssts ♥ ♣r♦r♠♥ t ♠♦st♦♥ t♦t ♥♥ t ♣r♠trs ♦ t ♠♦ ♥ts ♥ tr t♦ ♥rt ♣r♠trs ♦♥② ♦r t st ♠♦ ♥ ♠♦s ♥ ts ♣r♦♣rt② s♠♣② t ♥ ♦ t ♠♦ st♦♥

Prt

♦s str♥ ♦r ♠①

t

s ♣rt ♦t t♦ t str ♥②ss ♦ ♠① ts s♣t ♥ tr ♣trs rst ♦♥ ♣rs♥ts ♥ ♦r ♦ t str♥ ♣♣r♦ s♣ t♦ t ♠① t sts ❲ ♠♥② ♦s♦♥ t t♦ ♠♥ ♠①tr ♠♦s r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♥ ♥ ♣rt ♦t ♦ ♠trt strt♦♥s ♦r ♠① t s♦♥ ♣tr ♣rs♥ts ♠①tr ♠♦ t♦ str ♠① t sts t ♦♥t♥♦s ♥ t♦r rs s ♠♦ rs r♦♠ t ♠t t♥t ss♠♦ ♦♣ ♦r t t♦r t ♥②ss tr ♣tr ♣rs♥ts ♦♥ ♦ t ♠♥ ♦♥trt♦♥♦ ts tss t ♦♥ssts ♥ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t♦ str t sts t ♥②♥ ♦ rs ♠tt♥ ♠t strt♦♥♥t♦♥ s rsts r ♣rt ♦ s♠tt rt

❯♥ ♦s ♥st ♣s ♦t♠♥ ① rs

♥ tr♦s r r

♦♥ ♠s sà ♣rtr ♦♠♥

étt ♦t♠té♣♥ r♦♦t ❱♦②s ♥

sr

♦ ♦♥t♥ts

str ♥②ss ♦ ♠① t sts stt ♦ t rt

♥ ♦ str ♥②ss ♦r ♠① t r ♦ s♠♣ ♠t♦s t♦ str ♠① t ①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s ❯♥r♥ ss♥ ♠①tr ♠♦ ♦♥s♦♥

♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s

♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠ ♦ st♦♥ ♦rt♠ ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥

♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♦♣s ②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ tr r t sts ♦♥s♦♥

♦♥s♦♥ ♦ Prt

♣tr

str ♥②ss ♦ ♠① t sts

stt ♦ t rt

♣r♣♦s ♦ ts ♣tr s t♦ ♣rs♥t t ♠♥ ♣♣r♦s t♦ str ♠① t stsrst② ♠♣s③ t s♣t② ♦ t ♠① t♦r t str ♥②ss♦♥② ♣rs♥t s♦♠ ♥ ♣♣r♦s t♦ ♣r♦r♠t str ♥②ss ♦ s t♥② t t t♦ ♠♦st r♥t ♣♣r♦s t♦str ♠① t sts

é♥étq ♥ ♥♦èrs r♦♠♦s♦♠s ♥s t♠♦s♣èr

s t①s ♣♦r s ①st ♠♦♥ t♣s ♦♥t

♦r ésr ♥t ♥♦s ♣♦rtr

♥ ♦ str ♥②ss ♦r ♠① t

♥tr♦t♦♥ ♦②s ♠♥② t sts r ♦t♥ ♦♠♣♦s t ♠① rs r♥t ♥s ♦ rs ♥ t t st ♦ t s ss♥t t♦ t♦str s t sts t② ♥r♥t t♦ t str ♥②ss ♦ ♠① t♣r♦r♠ ② ♠①tr ♠♦s s t ♦ ♠trt strt♦♥ ♦r s t♥ t ss♥ rs♣t② t P♦ss♦♥ ♥ t ♠t♥♦♠ strt♦♥s r rr♥ t♦ str ♦♥t♥♦s rs♣t② ♥tr ♥ t♦r ttr s ♥♦ rr♥ strt♦♥ ♦r ♠① t ♦ t sst ♣♣r♦ ♦♥ssts♥ ss♠♥ t ♦♥t♦♥ ♥♣♥♥ t♥ t rs ♥ ♥ st♥ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♦r ts♣♣r♦ ♥ t♦ ss ♦r♦r s♦♠ ♠♦s ♣♣r♦ t strt♦♥ ♦♥trss ♦rrt ♠① t t t② r ♥♦t s② ♠♥♥ ♥ t♦♥♠♥s♦♥ ♠r♥s ♦ ♦♠♣♦♥♥t ♦ ♥♦t ♦♦ ss strt♦♥s st ♠①tr ♠♦s r s t♦ str ts ♦t ♣♣rs t♦ r ♦r ss♥ t ♣r♦s ♠♥♥ ♠♦s

♣tr str ♥②ss ♦ ♠① t sts stt ♦ t rt

♦ r ♦ts s ♣tr ♣r♦♣♦ss ♥ ♦r ♦ t str♥ ♣♣r♦s ♦r ♠① t t ♣ts t t ♦♥ t ♠♣♦rt♥ t♦ ♣r♦ ♠trt♠①tr ♠♦s rs♣t ♦t ♦♦♥ ♦ts

♦ ♣r♦ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t

♦ ♠♦③ t ♥trss ♣♥♥s

s t♦ ♦ts ♠ t♦ s♠♣② t ♠♦ ♥tr♣rtt♦♥ ♥ t ♣rtt♦♥r s ♥ s r♠♦r ♥ t ♦♥♠♥s♦♥ ♠r♥s ♦ t ♦♠♣♦♥♥tsr ss ♦r♦r ts ss ♥tr♣rtt♦♥ s ♠♦r ♣rs ♥ t ♥trss♣♥♥s r ♠♦③ s t ♠①tr ♠♦s ♣rs♥t ♥ t ♦♦♥♣trs ♠ t rs♣t♥ ♦t ♦ts

t r♦♦t ts ♣rt ♦♥sr t ert t♦r ♦ ♠① rs ♥♦t ② xi = (x1i , . . . , x

ci ,x

c+1i , . . . ,xei ) ❲ ♥♦t ② x

i = (x1i , . . . , xci)

ts sst ♦ t c ♦♥t♥♦s rs ♥ t s♠ ② ♥♦t ② x

i =(xc+1

i , . . . ,xei ) ts sst ♦ t d srt rs r c + d = e ♦t ttt tr♠ srt s♣ ♦r ♣rs♥t ♠♦ ❲ ♥♦t ② m t♥♠r ♦ ♠♦t② r♦ss♥s ♦ t st ♦ t t♦r rs ♦ x

i

trtr ♦ ts ♣tr t♦♥ rst② ♣rs♥ts tr ♥ ♠t♦s r ♥♦t r♥t s♥ t② ♦ ♥♦t ♦♥sr r ♥ ts ♥t s♣ ♦♥②ts st♦♥ ♦r♠ts t♦ ①t♥s♦♥s ♦ ss ♠t♦s t♦ ♣r♦r♠ t str♥②ss ♦ ♠① rs t♦ ♦♦♥ st♦♥s ♣rs♥t t t♦ ♠♦st r♥t♣♣r♦s t♦ str ♠① t t♦♥ ts t ♠①tr ♦ ♦t♦♥ ♠♦s♥ ts ①t♥s♦♥ ♣r ♦ t♦♥ ♥tr♦s t ♥r♥ ss♥ ♠①tr♠♦ ♦♥s♦♥ s ♥ ♥ t♦♥

r ♦ s♠♣ ♠t♦s t♦ str ♠①

t

♠t♦s

♥ ♦rr t♦ ♠♣s③ t ts ♥r♥t t♦ t ♠① t str♥ ♥♠rt tr ♥ t ♥♦t ♥t ♠t♦s ♣r♠t t♦ str s t♦r ♦ ts ♠t♦s ♠♥ r ♥ tr t② ♦ ♥♦trs♣t t ♥ ♦ r tr t② ♦ ♥♦t ♦♥sr r ♥ ts♥t s♣

❲♦ ♦♥t♥♦s ♠t♦ ♥ ♠② t♠♣t t♦ str t srt rss t② r ♦♥t♥♦s s ts ♠t♦ ♦♥ssts ♥ ♦♥rt♥ t t♦r♥ ♦r♥ ttrt s t♦ ♥♠r s ♠t♦ s♣ t♦ t ♦♥t♥♦st s t♥ ♣♣ t♦ ♣r♦r♠ t str ♥②ss s ♣♣r♦ ♠s r②str♦♥ ss♠♣t♦♥ ♦r t ♦r♥ rs ♥ t ss♠s tt tr s ts♠ ♣ t♥ t ♦♣s ♦ sss ♠♦ts ♦r♦r ts ♣♣r♦

r ♦ s♠♣ ♠t♦s t♦ str ♠① t

s t♦ rr ♥ t ♣rs♥ ♦ t♦r rs ♥ t ss♠s ♠♥♥ss ♦rr t♥ t ♠♦ts ♦r ♥st♥ t s ♥♦t ♣♦ss t♦ ♥♠r rs t♦ t♦r s ♦♦r ♦r ♦②

❲♦ srt ♠t♦ s ♠t♦ ♦♥ssts ♥ srt③t♦♥ ♦ t ♦♥t♥♦s rs s ♠t♦ s♣ t♦ t t♦r rs s s t♦ ♣r♦r♠t str ♥②ss ♦s ♦ t ♥♠rs ♦ ♠♦ts ♥ ♦ t ♦♥♦t♦♥s r t ♦r ts ♦s r r s♥ t② ♥♥ trsts ♦ t str ♥②ss ♦r♦r tr r t ♥♠rs ♦ t♦rs tsrt③t♦♥ ♣r♦ss s t♦ ♦ss ♦ ♥♦r♠t♦♥ ♥② t ♥trss ♣♥♥s r t② ♠♦③ s♥ t rs r ♦♥sr s t♦rs Prt

t♦r ♣♣r♦ t♣ t♦r ♥②ss ♠t♦ ♣r♠ts t♦ ♣r♦t♥s sr ② t♦r rs ♥ ♦♥t♥♦s s♣ s ②r♣♥ t srt rs ② tr t♦r ♦♦r♥ts ♥② ♠t♦ s♣ t♦t ♦♥t♥♦s rs ♥ s t♦ str t t st ♦r ♥♦t tt♥ ts s♣ s ♦♥t♥♦s ♥s ♥ t ♦♥② ♥t ♥♠r ♦ ss ♣♥♦♠♥♦♥ s♦ ♥rss t rs ♦ ♥r② ♦r♦r t rsts rss ♠♥♥ s♥ t rs r ♥♦t str ♥ tr ♥t s♣ ♥t ♥tr♣rtt♦♥ ♦ t sss s ♦♥ ② t ♣r♠trs ♦ t t♦r s♣

tr ♠t♦s ♣rs♥t ♦ r ♥♦t ♥t s♥ t② ♦ ♥♦t rs♣tt ♥tr ♦ r s ♥ t ♦♦♥ t st ♠t♦s ♠♦③t strt♦♥ ♦ t rs ♥ tr ♥t s♣

①t♥s♦♥ ♦ ss ♠t♦s ♦r ♠① t

♠♥s ♦rt♠ ♠♥s ♦rt♠ ♦♥② rqrs ♥t♦♥ ♦ st♥ t♥ t ♥s t♦ str ♥② ♥ ♦ t r♥t st♥ ♠srs ♥ st ♦r ♠① t s ♦r ♥st♥ t sst♦♥s ♦ ❩ ♥❬❪ ♥ ♦ ♠ ♥ ② ❬❪ ♦s② ts ♣♣r♦ ♣s trs ♦ t ♦♠tr ♠t♦s sss ♥ t♦♥

♦♥t♦♥ ♥♣♥♥ ♠①tr ♠♦ ♦ rr♥ strt♦♥♦r ♠① t s ♣r♦♠ t♦ ♣r♦r♠ t str ♥②ss t ♠①tr ♠♦ss ♣r♦♠ s s② ♦ ② t ♦♥t♦♥ ♥♣♥♥ ♠♦ s t♦♥ ♥ ♦♠♣♦♥♥t strt♦♥ s ♥ ② t ♣r♦t ♦ ♥rt strt♦♥s s ss strt♦♥s ♥ s s t ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts s ♣r♦♣♦s ② r ❬❪ ♥ ② ♦st ♥ P♣♦r♦ ❬P❪ ♦♥ssts ♥ stt♥ t ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ② ss strt♦♥s s ♠♦r ♥♦t♦♥ ♦r t ♦♥t♦♥ ♥♣♥♥ ♠♦ s s ♥ ts ss♠♣t♦♥ s ♦t♦ ♣rs♥t t♦ ♠♥ ♣♣r♦s r① t ♦♥t♦♥ ♥♣♥♥ss♠♣t♦♥


①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥

♣r ♦s

t ♦t♦♥ ♠①tr ♠♦ ♥tr♦ ② ❲ r③♥♦s ❬r③❪♦s t♦ str t sts t ♦♥t♥♦s ♥ t♦r rs

♥ t ♦♥t♥ts t ♦ t♦r rs ♥t♦ s♥ ♦♥ ♦♦s ♠t♥♦♠ strt♦♥ ♦r♦r t ss♠s tt t ♦♥t♥♦srs ♦♦ ♠trt ss♥ strt♦♥ ♦♥t♦♥② ♦♥ t ss ♥♦♥ ♠♦t② r♦ss♥ ♦r ♣rs② ts ♠♥s ♣♥ ♦♥ ♦t ss ♥t♦r rs s t ♦♥t♦♥ ♣♥♥② t♥ t ♦ rss t♥ ♥t♦ ♦♥t

♦t♦♥ ♠♦

♠ ♥ ♥ t ❬❪ ♥♦t tt t rs♥ r♦♠ ①♣r♠♥tt♦♥s ♥♣s②♦♦② ♦t♥ ♦♥t♥ ♦t srt ♥ ♦♥t♥♦s rs ♦ t② ♥tr♦t ♦t♦♥ ♠♦ t♦ ♠srs ♦ ss♦t♦♥ t♥ t rs ♦ st sts

♥ ♦t♦♥ ♠♦ ♥s t ♠t♥♦♠ strt♦♥ ♦♥ t ♦t♦r rs ♥ ♠trt ss♥ strt♦♥ ♦♥ t ♦♥t♥♦srs ♦♥t♦♥② ♦♥ t t♦r ♦♥s ♦r ♣rs② t st ♦ t t♦r rs x

i s ♦♥sr s ♦♥ t♦r rs ♦♦s r♠t♥♦♠ strt♦♥ Mm(λ1, ..., λm) s λh ♥♦ts t ♣r♦t② tt x

i

ts t ♠♦t② r♦ss♥ h ♦r♦r ♦♥t♦♥② ♦♥ x

i t♥ t ♠♦t②r♦ss♥ h t crt ♦♥t♥♦s r x

i ♦♦s crt ss♥ strt♦♥ Nc(µ

h,Σ) s t t♦r rs ♥♥ t ♠♥ ♦ t ♦♥t♥♦srs t ♥♦t tr s♣rs♦♥

♦tt♦♥s s t st ♦ t t♦r rs x

i s ♦♥sr ② t ♦t♦♥♠♦ s ♦♥ t♦r rs s ♦♠♣t s♥t ♦♥ s sxhi = 1 t ♥ ts t ♠♦t② r♦ss♥ h ♥ xhi = 0 ♦trs

♥t♦♥ ♦t♦♥ ♠♦ t♦r ♦ ♠① rs xi = (x

i ,x

i ) sr♥ ② ♦t♦♥ ♠♦ ts ♣ s rtt♥ s ♦♦s

p(xi;α) =m∏

h=1

(

λhφc(x

i ;µh,Σ)

)xhi ,

r α r♦♣s t s♣rs♦♥ ♠tr① Σ ♥ t t♦r (λh,µh;h = 1, . . . ,m)

①tr ♦ ♦t♦♥ ♠♦s

♦t♦♥ ♠♦ s ①t♥ t♦ t ♠①tr r♠♦r ♥ t ♠①tr♦ ♦t♦♥ ♠♦s s s ♥ sr♠♥♥t ♥②ss ② ❲ r③♥♦s ❬r③❪

①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s

♥ ♥ str ♥②ss ② r♥ ♥ ❲ r③♥♦s ❬❪ ♥ ♦rr t♦t ♥t♦ ♦♥t t ♥trss ♣♥♥s

♥t♦♥ ①tr ♦ ♦t♦♥ ♠♦s t♦r ♦ ♠① t xi =(x

i ,x

i ) s r♥ ② ♠①tr ♦ ♦t♦♥ ♠♦s t ♣ ♦ ts ♦♠♣♦♥♥t ks rtt♥ s ♦♦s ♦r k = 1, . . . , g

p(xi;αk) =m∏

h=1

(

λhkφc(x

i ;µhk,Σ)

)xhi ,

r αk r♦♣s t s♣rs♦♥ ♠tr① Σ ♥ t t♦r (λhk,µhk;h = 1, . . . ,m)

♥trss ♣♥♥s ♠①tr ♦ ♦t♦♥ ♠♦s ♦♥srs t ♥trss ♣♥♥s ♣r ♦♣ ♦ rs s ♦♦s

♦t rs r t♦r tr ♥trss ♣♥♥s r ♠♦③② t ♠t♥♦♠ strt♦♥s

♦t rs r ♦♥t♥♦s tr ♥trss ♣♥♥s r ♠♦③② t rt ss♥ strt♦♥s

♦♥ r s t♦r ♥ ♦♥ r s ♦♥t♥♦s tr ♥trss♣♥♥s r ♠♦③ ② t ♥♥ ♦ t t♦r r ♦♥ t♠♥s ♦ t ss♥ strt♦♥s ♦ t ♦♥t♥♦s r

♥tt② s ♣♦♥t♦t ② ❲s ♥ ♦ ❬❲❪ t ♠①tr ♦♦t♦♥ ♠♦s s ♥♦t ♥t s ♦ t ♥tr♠♥② ♦ ss ♠♠rs♣st ♦t♦♥ ♥ ♦rr t♦ ♦r♦♠ ts ♦ ♥tt② ts t♦rs s♦♠ ♦♥str♥ts ♦♥ t ♠♥ ♣r♠trs ♦ t ss♥ strt♦♥s

♥♠♥s♦♥ ♠r♥ strt♦♥s ❲ ♠♣s③ tt t ♦♥t♦♥ ♥♣♥♥ ♠♦ s ♠♥♥ s♥ ts ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ ts ♦♠♣♦♥♥ts r ss ♦r ♥st♥ t② ♦♥sst ♥ ♠t♥♦♠ ♦r ss♥ strt♦♥s ♦r t ♠①tr ♦ ♦t♦♥ ♠♦s t ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦ t t♦r rs ♦r ♦♠♣♦♥♥t r ss s♥t② r ♠t♥♦♠ strt♦♥s ♦r t ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ t ♦♥t♥♦s rs ♦r ♦♠♣♦♥♥t r ♥♦t ss ♥ t②♦♥sst ♥ ♠①tr ♦ ♦♠♦sst ss♥ t m ♦♠♣♦♥♥ts

Pr♠tr st♠t♦♥ ♥r♥ ♥ s② ♣r♦r♠ ♥ ♦t rq♥tst♥ ②s♥ r♠♦rs ♥ t t♦rs ♦♥② ♣rs♥t t ♥ t rq♥tst ♦♥ ♠ ♥ ♦t♥ ② t ♦♦♥ ♠ ♦rt♠


trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t ♦♥t♦♥ ♣r♦ts

tik(θ[r]) =

π[r]k p(xi;α

[r]k )

p(xi;θ[r])

.

st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

π[r+1]k =

n[r]k

n, λ

h[r+1]k =

1

n[r]k

n∑

i=1

tik(θ[r])xhi ,

µh[r+1]k =

1

n[r]k

n∑

i=1

tik(θ[r])xhi x

i ,

Σ[r+1] =

1

n[r]

g∑

k=1

mj∑

h=1

n∑

i=1

tik(θ[r])xhi (x

i − µh[r+1]k )′(x

i − µh[r+1]k ),

r n[r]

k =∑n

i=1 tik(θ[r])

♦rt♠ ♠ ♦rt♠ ♦r t ♠①tr ♦ ♦t♦♥ ♠♦s

♠♣ ♥ s② ♦t♥ ② st♥ s ♣r♦r s ♣r♦r ss♠s♥♣♥♥ t♥ t ♣r♠trs ♥ sts ♦♥t strt♦♥s ♦r ♣r♠trs t s s♦ s② t♦ s s♠♣r s♥ t ♣r♠trs ①♣t ♣♦str♦r strt♦♥s

①tr ♦ ♦s ♦ ♦t♦♥ ♠♦

♥ ♥♠r ♦ ♣r♠trs rqr ② t ♠①tr ♦ ♦t♦♥ ♠♦s♥rss t t ♥♠r ♦ t♦r rs ♥ t t ♥♠r ♦ tr♠♦ts s ♦r♥s♥ ♥ ♥t ❬ ❪ ♣r♦♣♦s ♥ ①t♥s♦♥ ♦ts ♠♦ ♥ tr ①t♥s♦♥ t rs r s♣t ♥t♦ ♦♥t♦♥② ♥♣♥♥t♦s s tt ♦ s ♦♠♣♦s t t ♠♦st ♦♥ t♦r r♦r♦r ♦ ♦ rs ♦♦s ♦t♦♥ ♠♦

♥t♦♥ ①tr ♦ ♦s ♦ ♦t♦♥ ♠♦ t♦r xi ♦♠♣♦st ♦♥t♥♦s ♥ t♦r rs rss r♦♠ ♠①tr ♦ ♦s ♦ ♦t♦♥♠♦ t ♣ ♦ ts ♦♠♣♦♥♥t k s rtt♥ s ♦♦s ♦r k = 1, . . . , g

p(xi;αk) =∏

b=1

p(xbi ;αkb),

r αk = (αkb; b = 1, . . . ,) ♥ t ♣ ♦ ♦ b ♦r ♦♠♣♦♥♥t k s rtt♥

①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s

s ♦♦s ♦r b = 1, . . . ,

p(xi;αkb) =

φdkb(xbi ,µkb,Σkb) xb

i s ♦♥t♥♦smb∏

h=1

(λkh)xbhi xb

i s t♦r

mb∏

h=1

(

λhkbφdkb−1(xbi ;µh

kb,Σkb))x

bhi

xbi s ♠①

r x

bi ♥ x

bi r rs♣t② t ♦♥t♥♦s ♥ t t♦r rs

♦ ♦ b

t ♠♦s tr r ♦♥② ♦♥t♥♦s rs c = e ♥ d = 0 ♥ t

rs r t ♥t♦ t s♠ ♦ x1i = xi t♥ t ♠♦

s q♥t t♦ t tr♦sst ss♥ ♠①tr ♠♦ tr r ♦♥② ♦♥t♥♦s rs c = e ♥ d = 0 ♥

♦ s ♦♠♣♦s t ♦♥② ♦♥ r xbi = xbi ♦r b = 1, . . . , c

t♥ t ♠♦ s q♥t t♦ t ss♥ ♠①tr ♠♦ t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ Σk s ♦♥

tr r ♦♥② t♦r rs c = 0 ♥ d = e t♥ t ♠♦s q♥t t♦ t t♥t ss ♠♦ s t♦♥

Pr♠tr st♠t♦♥ ♥r♥ s s② ♣r♦r♠ ♥ t ♦♥t♦♥ ♥♣♥♥ t♥ t ♦s ♦s t♦ ♣t t ♠ ♦rt♠ ♣rs♥t♥ ♦rt♠ ♥ ♦rr t♦ ♦t♥ t ♠ ♦ t ♠①tr ♦ ♦s ♦ ♦t♦♥♠♦ ②s♥ ♥r♥ ♦ ♣r♦r♠ ② s s♠♣r t ♣r♦rsr ss♠ t♦ ♥♣♥♥t ♥ ♦♦ ♦♥t strt♦♥s

♦ st♠t♦♥ ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s t♦rs st♠t t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ② ♥ s♥♥ ♠t♦t ① ♥♠r ♦ sss ♥ ♥ ①st ♣♣r♦ s ♥♦t ♦ st♦♥ ♠ ♦ ts s♥♥ ♠t♦ s t♦ ♦♣t♠③ ♥ ♥♦r♠t♦♥ rtr♦♥ s ♠t♦ s ♥t③ ② t ♦② ♥♣♥♥t ♠♦ ♥ r♥t♠♦s r ♣r♦♣♦s ② s♥ t ♥trss ♣♥♥s ♦♠♣t t t rr♥t ♠♦

Pr♦r♠♥s ♦ t ♠①tr ♦ ♦s ♦ ♦t♦♥ ♠♦ s ♣rs♥t ♥❬❪ t ♠①tr ♦ ♦s ♦ ♦t♦♥ ♠♦ ♥ ♦t♣r♦r♠ t ♦② ♥♣♥♥t ♠♦ ♦r ts ♠♦ s t♦ ♠♥ rs rst ♦♥ s♦t t ss ♥tr♣rtt♦♥ s♥ t ♦♥♠♥s♦♥ ♠r♥ strt♦♥ ♦ ♦♠♣♦♥♥t s ♥♦t ss t r s ♦♥t♥♦s s♦♥ ♦♥ s ♦tt t② t♦ ♣r♦r♠ ♠♦ st♦♥ ♥ t ♣r♦♣♦s ♣♣r♦ ♥ s♦♣t♠ t♦ st t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s rtr♠♦r t♦ ♦ t ♦rrt♦♥ ♦♥t t♥ ♦♥t♥♦s r ♥ t♦r


♦♥ s st ♦r ts ♦ s r s♥ t tr♠♥s t ♥tsr♥ t ♠♦ st♠t♦♥

❯♥r♥ ss♥ ♠①tr ♠♦

♥ ♥r♥ ss♥ ♠①tr ♠♦ ♥tr♦ ② rtt❬❪ ♣r♦r♠s t str ♥②ss ♦ t sts t ♦♥t♥♦s ♥ ♦r♥ rs ts ♠♥ ss♠♣t♦♥ s tt t ♦sr ♦r♥ ♥ ♥r② rs r♥rt r♦♠ ♥r②♥ ♥♦sr ♦♥t♥♦s rs ♦r♥ t♦ t s ♦ st ♦ trs♦s

♠r t♦r rs r ♥♦t ♦ t♦r rs①♣t t ♥r② ♦♥s ♥♥♦t ♠♦③ ② t ♥r♥ ss♥ ♠①tr♠♦ ♥ ts ♠♦ ss♠s ♥ ♦rr t♥ t ♠♦ts s ♥♦t♣rs♥t ♦r s rs

sr♣t♦♥ ♦ t ♥r♥ ss♥ ♠①tr ♠♦

ss♥ r ❲ ♦♥sr t t♦r yi = (x

i ,y

i ) r y

i s ♦♥t♥♦st♦r ♦ s③ d t♦r yi s ss♠ t♦ r♥ ② t ♦♠♦sst ss♥♠①tr ♠♦ ♦s t ♣ s

p(yi;θ) =

g∑

k=1

πkφe(yi;µk,Σ).

ss♥ t♥t r ♥ ♣rt t rs yi r ♥♦t ♦sr ♦rt② r rt t♦ t st ♦ t ♦sr srt rs x

i s ♦♦s

∀j = c+ 1, . . . , e, xjhi = 1 bjhk < yji ≤ bjh+1k ,

r bjhk < bjh+1k ♦r j = c + 1, . . . , e ♥ h = 1, . . . ,mj ♥ r bj1k = −∞ ♥

bjmj+1k = ∞ ♦♥s bjhk tr♠♥ t ♦sr srt rs r♦♠ tt♥t ♦♥t♥♦s ♦♥s s ♦t♥ tt t ♦sr rs xi = (x

i ,x

i ) t ♦♦♥ ♣

p(xi;θ) =

g∑

k=1

πk

∫

Sk(x

i )

φe(yi,µk,Σ)dyi ,

r Sk(x

i ) s t ♦♠♥ ♦ t ♥trt♦♥ ♦ t♥t ss♥ rs yi rtt♦ t ♦sr srt rs x

i ♦r ♣rs② Sk(x

i ) = Sc+1k (xc+1

i )× . . . ×Sek(xei ) r t ♥tr Sjk(xji ) s ♥ ♦r j = c + 1, . . . , e s s Sjk(xji ) =

]bjhk , bjh+1k ] xjhi = 1

♦♥s♦♥

tr♥t ♦r♠ ♦ t ♣ ♥ tr♥t ♥ ♠♦r r♥② ♦r♠ ♦ t♣ ♥ ② s ♦t♥ ② ♥♦t♥ tt t ♦♥t♦♥ strt♦♥ ♦ x

i

♥ x

i s ♥ ♥r♥ ss♥ ♦♥ s strt♦♥ s t ♠♥ µ|k ♥

t ♦r♥ ♠tr① Σ| r ♥ ②

µ|k = µ

k +ΣΣ−1(x

i − µ

k) ♥ Σ| = Σ −ΣΣ

−1Σ,

r µ

k ♥ µ

k r rs♣t② t ♠♥s ♦ x

i ♥ ♦ x

i ♥ r t ♦

r♥ ♠tr① Σ =

[

Σ Σ

Σ Σ

]

s ♦♠♣♦s ♥t♦ s♠trs ♦r ♥st♥

Σ s t s♠tr① ♦ Σ ♦♠♣♦s ② t r♦s ♥ t ♦♠♥s rt t♦ t♦sr ♦♥t♥♦s rs s tr♥t ♦r♠ ♦ t ♣ ♦s s t♦ ♥t ♥r♥ ss♥ ♠①tr ♠♦

♥t♦♥ ❯♥r♥ ss♥ ♠①tr ♠♦ t xi t t♦r ♦ e ♠①rs r♥ t ♥r♥ ss♥ ♠①tr ♠♦ ts ♣ s rtt♥ s♦♦s

p(xi;θ) =

g∑

k=1

πkφc(x

i ;µ

k,Σ)

∫

Sk(x

i )

φd(y

i ;µ|k ,Σ|)dyi .

r ♦♥t♦♥ ♦r ♠♦ ♥tt② ♥ t t♥t t♦r yi s ♥♦t♦sr tr s ♥♦ ♥♦r♠t♦♥ ♦♥ ts ♠♥ ♥ r♥ s t ♠♦ ss♠stt t ♠♥ts ♦ µ

k r ♥ ♥ tt t ♦♥ ♠♥ts ♦ Σ r q t♦♦♥

st♠t♦♥ ♦ t ♥r♥ ss♥ ♠①tr ♠♦

♥r♥ ② ♠①♠③t♦♥ ♦ t ♦♦♦ ♥t♦♥ s ♥♦t s② s♦ t ♣rs♥ ♦ d♠♥s♦♥ ♥trs ♥ ♥♦ ①♣t ♦r♠ ♥ Σ s ♥♦t♦♥

rtt ♣r♦♣♦ss t♦ ♣r♦r♠ t ♥r♥ ② s♥ s♠♣① ♠t♦ ♦ttt ts ♣♣r♦ ♠ts t ♦r t ♥♠r ♦ srt rs

♦♥s♦♥

❲ ♣♦♥t ♦t t ♠♣♦rt♥ t♦ ♦♥sr t rs ♥ tr ♥ts♣ ♦r t ♠① t r ♥♦t s② str ② ♠① ♠♦s s♦ t ♦ st♥r ♠trt strt♦♥ ♦r s rs s t ♠s t♦ ♣r♦♣♦s r♥t ♠trt strt♦♥ ♦r ♠① t ❲ ♣t tt ♦♥ t ♠♣♦rt♥ tt t ♦♥♠♥s♦♥ ♠r♥s ♦ ts strt♦♥s rss ♥ tt t ♣♥♥s r ♠♦③

♠♦ ♣rs♥t ♥ ♣tr ♦s t♦ ♣r♦r♠ t str ♥②ss ♦ tsts t ♦♥t♥♦s ♥ t♦r rs ② ♥ ts ♦ts ♦ttt ts stt♦♥ t st t ♦♥t♥♦s ♥ t♦r rs s t ♠♦stst ♦♥


♥ t ♦r♣② t t♦rs ♦ ♥♦t st② t s ♦ ♠① rs t♥tr s s t ♠♦ ♣rs♥t ♥ ♣tr s ♦ ♥trst ♥ t sr② ♥r s♥ t ♦s t♦ str t sts t ♥② ♥ ♦ rs ♠tt♥

♣tr

♦s str♥ ♦ ss♥

♥ ♦st strt♦♥s

s ♣tr ♥tr♦s s♣rs ♠①tr ♠♦ ♦r tsts t ♦♥t♥♦s ♥ t♦r rs ♦♠♣♦♥♥t strt♦♥s ♦ t ♦♥t♥♦s rs rss♥ ♦r♦r t t♦r rs r ss♠ t♦ ♥♣♥♥t ♦♥t♦♥② ♦♥ t ss ♥♦♥ t ♦♥t♥♦s rs ♥② ♦♥t♦♥② ♦♥t ♦♥t♥♦s rs t ♦♠♣♦♥♥t strt♦♥s ♦t t♦r rs r ♥r ♦st strt♦♥sr ♣r♠trs r ♥♦t ③r♦ ♥ t rrss♦♥ ♠①♠♠ ♦♦ ♥r♥ ♥ t ♠♦ st♦♥ r s♠t♥♦s② ♣r♦r♠ ② ♠ ♦rt♠♦r ① ♥♠r ♦ sss♠r ①♣r♠♥ts strt t ♠♦ r♥ ♥ str ♥②ss ♥ ♥ s♠s♣rs sst♦♥♥ ♥s r

str♠ t♦ts rà ♠♦t t♠♣s ♥♦ ♣♦t rr

r q t♦♠ t♦♠ t♦♠♥ ♦r sr

t strs ♦rt ♣r q ♦ str♦ ♦rt ♣r

r q t♦♠ t♦♠ t♦♠ ♥s ♣♦r♠ rrs st

♥tr♦t♦♥

❲ ♣rs♥t ♠①tr ♠♦ t♦ str t sts t ♦♥t♥♦s ♥ t♦rrs s ♠♦ s ♦ ♦t t♦ ♣r♦ ss ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♥ t♦ ♠♦③ t ♥trss ♣♥♥s

♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s

♦r s ♠♦ t ♦♥t♥♦s rs ♦♦ ♠trt ss♥ strt♦♥ ♦r ♦♠♣♦♥♥t ♦♥t♦♥② ♦♥ t ss ♥ ♦♥ t ♦♥t♥♦srs t t♦r rs r ss♠ t♦ ♥♣♥♥t② r♥ ② ♥r♦st strt♦♥s rst♥ ♠♦ s s♦ ♥♠ ♠①tr ♠♦ ♦ ss♥♥ ♦st strt♦♥s

♥r ♦st rrss♦♥s r ss② s ② ♠①tr ♠♦s ♦r t♦r t ❬♦r❪ ♦r♦r t ♠t t♥t ss ♠♦ ❬❱r❪ ss t♥t♦♥t♥♦s rs t♦ ♠♦③ t ♥trss ♣♥♥s t♥ t t♦r rs ♦ t s ♥tr t♦ s ♠①tr ♠♦ ♦ ss♥ ♥ ♦ststrt♦♥s ♥ ♦t ♦ t ♦♥t♥♦s ♥ t♦r ♥s ♦ rs r♦sr

♣rs♠♦♥♦s rs♦♥ ♦ ts ♠♦ s ♥tr♦ ② ♥ s♦♠ ♦♥str♥ts ♦♥t ♦st ♣r♠tr s♣ ♦ t rst♥ ♠♦ s ♠♦r s② ♥tr♣rt ♥♥ ♣r♦r♠ ttr tr ♦ t♥ t s ♥ t r♥ ♦r ① ♥♠r♦ sss t st♦♥ ♦ t ♣rs♠♦♥♦s ♠♦ ♥ t ♣r♠tr st♠t♦♥r s♠t♥♦s② ♣r♦r♠ ② ♠ ♦rt♠ ♦♣t♠③s ♥ ♥♦r♠t♦♥rtr♦♥ r♥ ♦r ♥♠r ①♣r♠♥ts strt t r♥ ♦ t rtr♦♥ ♦♠♣r t♦ t rtr♦♥

♥② ♥ t ♠♦ s ♥tr♦ t♦ ♣r♦r♠ str ♥②ss t ♥ ♣♣ ♦r s♠s♣rs sst♦♥ ❬❩+❪ ♥ t s ♥♦♥ ttt ♥rt ♣♣r♦s ♥ ♦t♣r♦r♠ t ♠t♦s s♣ ♦ t sst♦♥♥ s ♣♥♦♠♥♦♥ s ♣rtr② ♦sr ♥ ♦srt♦♥s r ❬❪ ♥ ts ♠t♦s ①♣♦t t ♥♦r♠t♦♥s ♣rs♥t ♥ ♦t ♥ ♥ t sr♠♥t ♣♣r♦s t ♦♥② ♥t♦ ♦♥tt t ♥♦r♠t♦♥ ♥ t② r♥ sst♦♥ r ♦♥② ♦♥ t t

trtr ♦ ts ♣tr s rt s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦r t stst ♦♥t♥♦s ♥ t♦r rs s ♠♦ ♣r♦r♠s t str ♥②ss ② ♣r♦♥ ss ♦♥♠♥s♦♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♥ ②♠♦③♥ t ♥trss ♣♥♥s ♥ str♥ r♠♦r t♦♥ s♦t t♦ t ♠①♠♠ ♦♦ st♠t♦♥ t♦♥ ♣rs♥ts t ♠ ♦rt♠ s♠t♥♦s② ♣r♦r♠s t st♠t♦♥ ♦ ♦t ♠♦ ♥ ♣r♠trs② ♦♣t♠③♥ ♥ ♥♦r♠t♦♥ rtr♦♥ t♦♥ ♣rs♥ts r♥t ♥♠r ①♣r♠♥ts ② strt t r♥ ♦ t rtr♦♥ t♦ ♣r♦r♠ t ♠♦st♦♥ ♦r♦r t② s♦ t ♣r♦r♠♥s ♦ t st♠t♦♥ ♦rt♠ ♥t ♠♦ r♦st♥ss t♦♥ ♣rs♥ts ♦♥ ♣♣t♦♥ ♥ str♥ ♥ ♦♥♣♣t♦♥ ♥ s♠s♣rs sst♦♥ ♦♥ t♦ r t sts ♦♥s♦♥ s♥ ♥ t♦♥

t t xi = (x1i , . . . , xci ,x

c+1i , . . . ,xei ) t ert t♦r ♦ ♠①

rs rst c rs r ♦♥t♥♦s ♥ ts sst ♦ rs s ♥♦t② x

i st d rs r t♦r rs s♥ s♥t ♦♥ ♥ts sst ♦ rs s ♥♦t ② x

i ♦t tt c+ d = e

①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s

①tr ♠♦ ♦ ss♥ ♥ ♦st str

t♦♥s

♠ ♠♦ ♣r♦r♠s t str ♥②ss ♦ ♦♥t♥♦s ♥ t♦r tst t ♦ ♦t t♦ ♣r♦ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♥ t♦ ♠♦③ t ♥trss ♣♥♥s

♥ ♣ ♦ ♦♠♣♦♥♥t s ♥ ② t ♣r♦t t♥ t♣ ♦ t ♦ ♦♥t♥♦s rs ♥ t ♣ ♦ t ♦ t♦r rs♦♥t♦♥② ♦♥ t ♦ ♦♥t♥♦s rs ♦r ♣rs② ♦♥r♥♥ t♦♥t♥♦s rs t② ♦♦ ♠trt ss♥ strt♦♥ ♦r ♦♠♣♦♥♥t ♦♥r♥♥ t t♦r rs t ♠♦ ss♠s tr ♥♣♥♥♦♥t♦♥② ♦♥ ♦t t ss ♠♠rs♣ ♥ t ♦♥t♥♦s rs

♥t♦♥ ①tr ♠♦ rt t♦ ♥ t♦r xi s r♥ ② ♠①tr ♠♦ rs♣t♥ ♥ t ♣ ♦ ts ♦♠♣♦♥♥t k s rtt♥ s♦♦s ♦r k = 1, . . . , g

p(xi;αk) = p(x

i ;αk)p(x

i |x

i ;αk)

= φc(x

i ;µk,Σk)e∏

j=c+1

p(xji |x

i ;βkj),

r αk = (µk,Σk,βk) ♥♦ts t ♦ ♦♠♣♦♥♥t ♣r♠trs r t t♦rµk ∈ R

d ♥♦ts t ♠♥ ♦ t ♦♥t♥♦s rs ♦r ♦♠♣♦♥♥t k ♥ r t♠tr① Σk ♥♦ts tr ♦r♥ ♠tr① t♦r βk = (βkj; j = c+ 1, . . . , e)♥♦ts t ♦ ♣r♠trs ♦ ♦♠♣♦♥♥t k r rt t♦ t t♦rrs ♥ t t♦r βkj r♦♣s t ♣r♠trs rt t♦ t strt♦♥♦ t t♦r r x

ji ♦r ♦♠♣♦♥♥t k

♥ ♠♦ ss♠s tt ♦r ♦♠♣♦♥♥t t strt♦♥ ♦ t♦r r s ♥r ♦st rrss♦♥ ♦s t ①♣♥t♦r② rs r t ♦♥t♥♦s ♦♥s

♥t♦♥ ①tr ♠♦ ♦ ♦st rrss♦♥s ❲t t ♥♦tt♦♥ x0 = 1t ♦♠♣♦♥♥t strt♦♥s ♦ xji ♦r j = c+1, . . . , e r ♥ ② t ♦♦♥♣ ♦r k = 1, . . . , g

p(xji |x

i ;βkj) =

mj∏

h=1

exp(

∑cj′=0 β

j′hkj x

j′

i

)

∑mj

h′=1 exp(

∑cj′=0 β

j′h′

kj xj′

i

)

xjhi

,

r t ♣r♠trs βkj = (βj′hkj ; j

′ = 0, . . . , c;h = 1, . . . ,mj) ∈ Rc+1 ♥♦ts t

♦st ♣r♠trs ♦ t t♦r r xji ♦r ss k ♥ ♦rr t♦ ♥sr t

♠♦ ♥tt② ♣t ∀(k, j, j′), βj′1kj = 0 ♥② ♥♦t tt t ♣r♠tr

β0hkj s t ♥tr♣t ♦ t ♦st rrss♦♥ t ♦tr ♣r♠trs βj

′hkj ♦r

j′ = 1, . . . , c r t s♦♣ ♣r♠trs


P♦t♥t r ♥♠r ♦ ♣r♠trs ♠♦ ♥ rqr r ♥♠r♦ ♣r♠trs ♥ t ♥♠r ♦ ♣r♠trs ♥♦ ② t ♠♦ s q t♦

(g − 1) + g

(

c(c+ 3)

2

)

+ ge∑

j=c+1

(mj − 1)(c+ 1).

♥ ♦rr t♦ ♦t♥ ttr sr♥ tr♦ ♥tr♦ ♣rs♠♦♥♦srs♦♥ ♦ t ♠♦ ② r♥ t s♣ ♦ t ♦st ♦♥ts

♣rs ♦st ♥t♦♥s ♦r t ♠① ♦♥t♦♥ ♣♥♥s s♣rst② ♦ t ♠♦ s ♥ ② t srt ♣r♠trs δkj = (δj

′

kj; j′ = 0, . . . , c)

♥ δj′

kj = 1 t♦r r j s ♦♥t♦♥② ♣♥♥t ♦♥ ♦♥t♥♦s

r j′ ♦r ♦♠♣♦♥♥t k ♥ δj′

kj = 0 ♦trs ♦t tt δ0kj = 0 ♥♦s ♥♥tr♣t ♥ t ♦st rrss♦♥ ♦ t♦r r j ♦r ♦♠♣♦♥♥t k sδkj ①s s♦♠ ♦st ♣r♠trs t♦ ③r♦ s♥ βkj ∈ S(δkj) t

S(δkj) =

βkj : ∀(k, j, j′) s s δj′kj = 1, t♥ ∀h βj′hkj = 0

.

♦♥tr♦ ♥♠r ♦ ♣r♠trs ♥♠r ♦ ♣r♠trs rqr ② t♥r ♠♦ ♥ ② (g, δ) s q t♦

(g − 1) + g

(

c(c+ 3)

2

)

+

g∑

k=1

e∑

j=c+1

c∑

j′=0

δj′

kj(mj − 1).

♥♥ ♠♦ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♣r♦s ♠♥♥ sss ♥ ss ♥ s♠♠r③ ② ♣r♠trs♠♥ ♥ r♥ ♦r t ♦♥t♥♦s rs ♥ ♣r♦t② ♦ ♠♦t② ♦r t♦r r q t♦

p(xjh = 1|zik = 1) =

∫

Rc

φc(x

i ;µk,Σk)exp

(

∑cj′=0 β

j′hkj x

j′

i

)

∑mj

h′=1 exp(

∑cj′=0 β

j′h′

kj xj′

i

)dx

i .

t♦ ts ♥tr s ♥♦t ①♣t t s s② ♣♣r♦①♠t ② ♠♠ ♠t♦rtr♠♦r ♦r ss t ♣♥♥s t♥ t ♦♥t♥♦s rs r♠♦③ ② t ♦rrt♦♥ ♠tr① t t♦r r xji s ♦♥t♦♥②♣♥♥t t t ♦♥t♥♦s ♦♥ xj

′

i δj′

kj = 1

♣♥♥s ♥t♦r r s ♥ ①♠♣ ♦ t ♣♥♥s t♥ rs t♥ ♥t♦ ♦♥t ② t ♠♦ ♥ t♥ rs ♥♦ts ♣♥♥② t♥ rs ♥ ♥ s♥ ♦ ♥ ♥♦ts ♦♥t♦♥ ♥♣♥♥ ♦t tt t ♦sr rs r ♥ t zi tr s qt♥ t ♦♥t♥♦s rs t t♦r rs r ♥♦t ♥ t♦tr♥ t ♥trss ♣♥♥② t♥ t ♦♥t♥♦s ♥ t t♦r rsr ♥ ② t srt ♣r♠trs δ

①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠

zi

x1i

x2i

x3i

x4i

r ①♠♣ ♦ t ♣♥♥s t♥ ♥t♦ ♦♥t ② t ♠♦ rx

i = (x1i , x2i ) ♥ x

i = (x3i ,x

4i ) t δ

1k3 = δ2k3 = δ1k4 = 1 ♥ δ2k4 = 0

♥r ♥tt② ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥ss ♥r② ♥t ts ♦ t ♣r♦♦ r ♥ ♥ ♣♣♥① ♠♦♥strt♦♥ s s♣t ♥ t♦ ♣rts rst② s♠ ♦r t ♣♦ss s ♦x

i t♦ ♦t♥ ♠①tr ♦ ss♥ strt♦♥s ♥ t♦ s ts ♥tt② rsts❬ ❨❪ ♦♥② s♦ t ♥tt② ♦ t ♣r♠trs ♦ ♦st♥t♦♥

①♠♠ ♦♦ st♠t♦♥ ♥

♦rt♠

♥ ❲ ♦♥sr t s♠♣ ① = (x1, . . . ,xn) ♦♥ssts ♦ n ♥s ss♠ ♥♣♥♥t② r♥ ② ♠①tr ♠♦ ♦ ss♥ ♥ ♦ststrt♦♥s ♠ s s② ♦t♥ ② t ♦♦♥ ♠ ♦rt♠ s ♦rt♠ tt t ♦ s ♣r♦r♠ ♦r ① ♠♦ ♥ ② t ♦♣(g, δ)

♥r♥ ♦ t ♦st ♣r♠trs t t ♠ st♣ t ♠①♠③t♦♥s ♦♥ t♣r♦♣♦rt♦♥s ♥ ♦♥ t ss♥ ♣r♠trs r s② ♣r♦r♠ ♦r tst♠t♦♥ ♦ t ♣r♠trs rt t♦ t ♦st ♥t♦♥s ♥♦ t♦ s♦ ♥♦♥①♣t qt♦♥s ♦ t② r ss② ♦t♥ ② t♦♥♣s♦♥ ♠t♦♥ t ♠ s st t♦ st♠t t ♦st rrss♦♥ ♣r♠trs r ♥s r♥t ts


trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t ♦♥t♦♥ ♣r♦ts

tik(θ[r]) =

π[r]k p(xi;α

[r]k )

p(xi;θ[r])

.

st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦

π[r+1]k =

n[r]k

n, µ

[r+1]k =

1

n[r]k

n∑

i=1

tik(θ[r])xi,

Σ[r+1]k =

1

n[r]k

n∑

i=1

tik(θ[r])(xi − µ

[r+1]k )′(xi − µ

[r+1]k ),

β[r+1]kj = argmax

βkj∈S(δkj)

n∑

i=1

tik(θ[r]) ln p(xji |x

i ;βkj),

r n[r]k =

∑ni=1 tik(θ

[r])

♦rt♠ ♠ ♦rt♠

♦ st♦♥ ♦rt♠

♠ ♠♦ st♦♥ s ♦♠♥t♦r ♣r♦♠ s ♦ t st♠t♦♥ ♦t srt ♣r♠tr δ s ♥t t♦ ♦t♥ t ♣r♠tr δ ♠①♠③st ♥♦r♠t♦♥ rtr♦♥ ♦r ① ♥♠r ♦ sss

♥ t s ♥r② ♠♣♦ss t♦ ♥ t ♠ ♦r δ s♥ t ♥♠r ♦ ♦♠♣t♥ ♠♦s s 2g(c+1)d s t rtr♦♥ s ♣♥③t♦♥ ♦ t♦srt ♦♦♦ ♥ s ♥ ♠ ♦rt♠ ♠①♠③♥ t ♣♥③♦srt ♦♦♦ ❬r❪ s t♦♥ s t ♠ st♣ ♦♥ssts♥ ♠①♠③♥ t ①♣tt♦♥ ♦ t ♣♥③ ♦♠♣tt ♦♦

♦t♦♥ ♦ t ♠ st♣ t trt♦♥ [r] t ♠ st♣ ♦ t ♠ ♦rt♠♠s t tr♠♥♥ (δ

[r+1]kj ,β

[r+1]kj ) s s

(δ[r+1]kj ,β

[r+1]kj ) = argmax

δkj∈0,1c+1

argmaxβkj∈S(δkj)

n∑

i=1

tik(θ[r]) ln p(xji |x

i ;βkj)−νkj2

lnn,

r νkj =∑c

j′=0 δj′

kj ♥ts t ♥♠r ♦ ♣r♠trs rqr ② t ♦strrss♦♥ rt t♦ t♦r r j ♦r ♦♠♣♦♥♥t k


♠ ♦rt♠ t♦ ♦ t ♦♠♥t♦r ♣r♦♠s s♣ ♦ δs r s♦ ♥ ①st ♣♣r♦ t♦ tr♠♥ δ

[r+1]kj ♦r♥ t♦ s ♥♦t

♦ s ♣rr t♦ s ♠ rs♦♥ ♦ ts ♦rt♠ ♥ ts ♦rt♠t ①♣tt♦♥ ♦ t ♣♥③ ♦♠♣tt ♦♦ s st ♥rs t t ♠st♣ ♦ t trt♦♥ [r] ♦ t ♠ st♣ s s♥♥t ♥ s♥♥t ♠t♦s♥t③ ② (δ

[r]kj ,β

[r]kj)

♥ t ♠♣♦rt♥ ♦ sr ♥t③t♦♥s ♦t tt ts ♣♣r♦ ♣st ss ♣r♦♣rts ♦ t ♠ ♦rt♠ ♦ ts tr♠♥st ♦rt♠ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t ♣♥③ ♦srt ♦♦ ♣♥s♦ t ♥t ♦ (δ[0],θ[0]) s t s ♠♥t♦r② t♦ ♣r♦r♠ ts ♦rt♠t r♥t ♥t③t♦♥s t♦ ssr t ♦♥r♥ t♦ ♦ ♠①♠♠ ♦ t♣♥③ ♦srt ♦♦


♠ ❲ ①♣r♠♥t② st② t ♦r ♦ t ♠ ♦rt♠ ♥ t ♣r♦r♠s t ♠♦ st♦♥ s♥ ♦t ss ♥♦r♠t♦♥ rtr ♥ rsts ttst t♦ t ♦♦ ♦r ♦ t ♠ ♦rt♠ ♦r t s♠t♥♦sst♠t♦♥ ♦ t ♠♦ ♥ t ♣r♠trs ♦r♦r t② s♦ tt t rtr♦♥ ♦t♣r♦r♠s t rtr♦♥ ♥ ts ttr ♦rst♠ts t ♥♠r♦ ♦♠♣♦♥♥ts ♥ t ♦♥t♦♥ ♣♥♥s t♥ ♠① t

trtr ♦ ts st♦♥ rst② t r s♠t ♦r♥ t♦ t ♠①tr♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦♥② t r s♠t ♦r♥t♦ ♦tr ♠♦s

♠t♦♥s ② t s♣ ♠♦

♥♦♥ ♥♠r ♦ sss

t ♥rt♦♥ t r s♠♣ ♦r♥ t♦ t ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s t t♦ ♦♠♣♦♥♥ts ♥ q ♣r♦♣♦rt♦♥s ♥s rsr t t♦ ♦♥t♥♦s rs ♥ t♦ ♥r② ♦♥s ♠♦ ♣r♠trs r t ♦♦♥

µk = (ε(k − 2), ε(k − 1)), Σk =

[

1 k − 1.5k − 1.5 1

]

, δk1 = (1, 1, 0),

δk2 = (1, 0, 1), βj′2kj = ε,

r t ♣r♠tr ε ♦s t♦ ① t sss ♦r♣s rr s ε t ttrs♣rt r t sss ♦r t♦ sss ♦r♣s ♥ ♦r r♥t s③s ♦ s♠♣s(n = 50, 100, 200, 400) t sts r ♥rt


st♠t♦♥ ♦♥t♦♥s ♦r t st tr ♠♦s r ♥ ♦♠♣tt♦♥ ttr ♠♦ t ♠♦ ♠①♠③♥ t rtr♦♥ ♥ t ♠♦ ♠①♠③♥t rtr♦♥ st♠t♦♥ ♦ t ♣r♠trs rt t♦ t tr ♠♦ s♣r♦r♠ ② t ♠ ♦rt♠ ♠①♠③♥ t ♦♦ ♦r ① ♠♦ ♦rt♠ t♦ ♦tr ♣r♠tr st♠t♦♥s r ♣r♦r♠ ② t ♠ ♦rt♠ ♠①♠③♥ ♥ ♥♦r♠t♦♥ rtr♦♥ ♥ s r♥♦♠ ♥t③t♦♥s♦ t st♠t♦♥ ♦rt♠ r ♦♥ ♦t tt ♥ ♥ ts s♠♣ s t s ♥♦t♦ t♦ ♣r♦r♠ ♥ ♠ ♦rt♠ ♦r ♠♦ s♥ tr r 212 ♠♦s ♥♦♠♣tt♦♥

sts ♥ ♣rs♥t t r r♥ ♦ t tr ♠♦m ♥ ♦ t st ♠♦ ♦r♥ t rs♣t② rtr♦♥ ♥♦t ②m rs♣t② m ❲ ♦♥ t♦ t ♦r ♦ t ♦rt♠ s♦r t ♣r♠tr st♠t♦♥ ♥ t r r♥ t♥s t♦ ③r♦♥ n r♦s ♦r t tr ♣♣r♦s rtr♠♦r ♦r ♥t s♠♣ s③ t♥♦r♠t♦♥ rtr ♦ t♦ r t r r♥ ② st♥ ss♦♠♣① ♠♦s

r♣ 10% 20%n m

m m

♥s ♦ t r r♥s ♥ s♣ ♠♦stt♦♥ r t ♥♠r ♦ sss s ♥♦♥

♠r ♦ sss ♥♥♦♥

t ♥rt♦♥ t r s♠♣ ♦r♥ t♦ ♠♦ sr ♥ t ♣r♦s s♠t♦♥ ♦r tr ♦r♣♣ sss ♥ ♦r r♥t s♠♣ s③s (n =50, 100, 200, 400) t sts r ♥rt

st♠t♦♥ ♦♥t♦♥s ♦r t st t st ♠♦s ♦r♥ t♦ t ♥ t rtr r st♠t ♦r g = 1, . . . , 4 ♥ s r♥♦♠ ♥t③t♦♥s ♦ t ♦rt♠ r ♦♥

sts s♣②s t ♠♥ ♦ t st ♥♠r ♦ sss ♥ t str♥ ♥① ❬❪ ♦♠♣t t t st♠t ♣rtt♦♥ ♦ t st ♥♠r ♦sss ♦r ♦t ♥♦r♠t♦♥ rtr ♦r ♦ t rtr♦♥ s ttrs♥ t ♥rst♠ts t ♥♠r ♦ sss ♥ t s♠♣ s③ s s♠ ♥♥ sss ♦r♣ rtr♠♦r ts ♦♥r♥ t♦ t tr ♥♠r ♦ ssss str t♥ ♦r t rtr♦♥ ♥ ts ttr ♦rst♠ts t ♥♠r ♦sss ♥ t t st s r st ♥ ♥① rt t♦ t ♠♦ ♦♣t♠③ t rtr♦♥ s s♠ s♥ ts ♥① s q t♦ ③r♦ ♥


g = 1 ♦t tt ts ♥① ts s ♦r t rtr♦♥ ♥ ts ttr♦rst♠ts t ♥♠r ♦ sss s ♥ ♠ tt ♥ t rtr♦♥♦rst♠ts t ♥♠r ♦ sss t s♣ts tr ss ♥t♦ t♦ sss

r♣ 05% 10% 20%n

♥s ♦ t st ♥♠r ♦ sss ♥ ♣♥ ♥ st ♥♥s ♥ ♣r♥tss ♦♠♣t ♦r ♦t ♥♦r♠t♦♥ rtr

♦♥t♦♥ ♥♣♥♥ stt♦♥

t ♥rt♦♥ t r s♠♣ ♦r♥ t♦ t ♦♠♣♦♥♥ts ♠♦ rt t♦r rs r ♦♥t♦♥② ♥♣♥♥t t♦ t ♦♥t♥♦s ♦♥s ts♣r♠trs r

µk = (k − 2, k − 1), Σk =

[

1 k − 1.5k − 1.5 1

]

, δk1 = (1, 0, 0),

δk2 = (1, 0, 0), β02kj = (−1)k/2.

st♠t♦♥ ♦♥t♦♥s ♦r t st t st ♠♦s ♦r♥ t♦ t ♥ t rtr r st♠t ② ♠ ♦rt♠ r♥♦♠② ♥t③ t♠st g = 2

sts s♣②s t ♦♥t r t ♦st ♥tr♣ts r ♥♦t♥ ♥ t ♦♥t r t ♦♥t♦♥ ♣♥♥② rt♦♥s♣ t♥ t♦r r ♥ ♦♥t♥♦s ♦♥ s ③r♦

n

δ0kj = 1 δ(1,2)kj = 1 δ0kj = 1 δ

(1,2)kj = 1

♦♥t r t ♦st ♥tr♣ts r ♥♦t ♥ (δ0kj = 1) ♥ ♦♥t r t ♦♥t♦♥ ♣♥♥② rt♦♥s♣ t♥ t♦r r♥ ♦♥t♥♦s ♦♥ s st♠t (δ

(1,2)kj = 1) ♦r ♦t ♥♦r♠t♦♥ rtr

❲ ♥♦t tt t rtr♦♥ s ttr ♦r s♥ t rtr♦♥♦rst♠ts s♦♠ rt♦♥s♣s t♥ rs ♥ t ♦♥t♦♥ ss♠♣t♦♥s


♠t♦♥s ② ♠ss♣ ♠♦s

t ♥rt♦♥ t sts ♦ s③ r s♠♣ ♦r♥ t♦ t ♦♦♥♦♠♣♦♥♥ts ♦rrt ss♥ ♠①tr ♠♦

πk = 0.5, µ1j = 0.5, µ2j = −µ1j, Σ1 = Σ2 =

1 ε 0.5 εε 1 ε 0.50.5 ε 1 εε 0.5 ε 1

,

r ε s ♥ st♠♥t ♣r♠tr ε = 0 ♠♥② rs r ♦♥t♦♥②♥♣♥♥t ♥ ε s t rs r ♦♥t♦♥② ♣♥♥t t♦ st rs r srt③ t♦ ♦t♥ t♦r t t ♦r s]−∞,−1] ]− 1, 0] ]0, 1] ♥ ]1,∞[

st♠t♦♥ ♦♥t♦♥s st ♦♠♣♦♥♥t ♠♦ ♦r♥ t♦ ♥♦r♠t♦♥ rtr♦♥ s st♠t ② t ♠ ♦rt♠ ♥t③ t♠s

sts s♣②s t ♦♥t ♦ t ♥♦♥ ♦♥ts ♥ t ♦strrss♦♥ ♦r t ♦rrt rs ♥ t ε♦rrt rs ♦r♥t♦ ♦t rtr ♦r r♥t s ♦ ε ❲♥ ε = 0 t ♦♥t ♦ t ♦st rrss♦♥ t♥ t ε♦rrt rs t♦ s t

rtr♦♥ ♦rst♠ts t ♦♥t♦♥ ♣♥♥s ♦r t tts ttr t♣♥♥s t♥ t ♦tr rs

d

♦rrt ε♦rrt ♦rrt ε♦rrt

♦♥t r t ♦♥t♦♥ ♣♥♥s r ♠♦ ② ♦t♥♦r♠t♦♥ rtr

s ♦ t s ♦ t rtr♦♥ s♦♥ r♥ t ♥♠r ①♣r♠♥ts ♦r s t♦ s t rtr♦♥ t♦ ♣r♦r♠ t ♠♦ st♦♥ ♥ ts ttr ♥ ♥t s♦♠ ♦♥t♦♥ ♣♥♥s t♥ ♠① rs


ss t str♥

t st sr♣t♦♥ ♥ rt ss t ❬t❪ sr ♣t♥ts ♣r rs ♦♥t♥♦s t t♦r ♥ ♦♥ ♣rt ttrt ♥ s ♥ t ❯ ♠♥ r♥♥ r♣♦st♦r② ♣rtttrt s ♥r② r ♥t♥ t ♣rs♥ ♦ rt ss ❲ ♥ ts


♥♦r♠t♦♥ r♥ ♦r str♥ rtr♠♦r t s① ♥s ♥ ♠ss♥s r ♦♠tt

♦♠♦♥♦s ♠♦s str♥ ♦t tt ♦t ♥s ♦ rs r♠♣♦rt♥t ♦r t str ♥②ss ♥ t ♥②ss ♣r♦r♠ ♦♥ t ♦♥t♥♦srs ② ♠①tr ♦ ss♥ strt♦♥s sts ♦r sss t ♥②ss♣r♦r♠ ♦♥ t t♦r ♦♥s ② t t♥t ss ♠♦ sts tr sssr s♣②s t st♠t ♣rtt♦♥ ② t ss♥ ♠①tr ♠♦ ♥ trst ♦♠♣♦♥♥t ♠♣ r ♥ ② t ♠t♥♦♠ ♠①tr ♠♦ ♥ trst ♦rrs♣♦♥♥ ♠♣ r s s tt sss ♦r♣ ♥ trst t♦r ♠♣s ♥ ♦ ♥♦t ♥ t♦r ♠♣ r t st♠t sssr s♣rt

−3 −2 −1 0 1 2 3

−3

−2

−1

01

23

4

First principal component analysis

Sec

ond

prin

cipa

l com

pone

nt a

naly

sis

Class 1 Class 2Class 3Class 4

−3 −2 −1 0 1 2 3

−3

−2

−1

01

23

4


Sec

ond

prin

cipa

l cor

resp

onde

nce

anal

ysis

Class 1 Class 2Class 3

r Prtt♦♥s st♠t ② t ♦♠♦♥♦s ♠♦ ♣rtt♦♥ ♦ tss♥ ♠①tr ♠♦ r♥ ♥ t rst ♦♠♣♦♥♥t ♠♣ ♣rtt♦♥ ♦ tt♥t ss ♠♦ r♥ ♥ t rst ♦rrs♣♦♥♥ ♠♣

s s♦♥ ② t ♦♥s♦♥ ♠tr① ♣rs♥t ♥ t ♣rtt♦♥s ♦t♥ ② t ♠♦ ♦r ♦♠♦♥♦s rs r r② r♥t r♦♠ t ♣rtttrt ♥ t ♦ t st r♥ ♥① ♦♠♣t t♥ t ♣rtt♦♥ ♦ t ss♥ ♠①tr ♠♦ ♥ t ♣rt ttrt s q t♦ t s q t♦ ♥ t s ♦♠♣t t♥ t ♠t♥♦♠ ♠①tr ♠♦ ♥t ♣rt ttrt st g = 2 t rr♦r rts r ♦r t ♦♥t♥♦ss ♥ ♦r t t♦r s ♥② ♦t ♣rtt♦♥s ♦ t ♦♠♦♥♦s♠♦s r r♥t s♥ tr st r♥ ♥① s r q t♦

tr♦♥♦s ♠♦s str♥ ❲ ♣r♦r♠ t str ♥②ss ♦♥t ♦ t st ② s♥ t♦ ♠♦s t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♥t ♠①tr ♦ ss♥ ♥ ♦st strt♦♥s rsts s♣② ♥ ♠ tt t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ttr


rt ♦♥t♥♦s rs t♦r rsss ss ss ss ss ss ss ss s♥ ♣rs♥

♦♥s♦♥ ts t♥ t ♣rt ttrt ♥ t t♦ ♣rtt♦♥sst♠t ② t ♦♠♦♥♦s ♠①tr ♠♦s

♣♣r♦s t t strt♦♥ ♥ ts ♠♦ sts t♦ sss s♥ ts rtr♦♥ s r t ♦♥ ♦♠♣♦♥♥t ♥ t tr ♦♠♣♦♥♥ts ♦t tt t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♦rst♠ts t ♥♠r ♦sss s♥ t rtr♦♥ sts tr sss t ♦ ② ♦♥sr♥ t ♦ t st t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s♦t♥s ♠♦r ♠♥♥ ♠♦ s♥ t s ss sss ♥ ss ♣r♠trs

♦♥ ♥♣ ♣r♦♣♦s ♠♦ rtr♦♥

♦♦♦ Pr♠trs

❱s ♦ t rtr♦♥ ♥ ♦ t ♦♦♦ ♥t♦♥ ♥ ♥♠r♦ ♣r♠trs ♦r ♦t ♦♠♣♦♥♥t ♠♦s ♥ ♦♠♣tt♦♥

st ♠♦ ♥tr♣rtt♦♥ ♠♦rt② ss 70% r♦♣s ♥s t♥t s♠st s ♦r t ♦♥t♥♦s rs ①♣t ♦r t r t ♦r♦r tr r♥s r s♠r t♥ t♠ ♦ t ♠♥♦rt② ss 30% t♦rrt♦♥ t♥ t ♦♥t♥♦s rs r str♦♥r s s♣② ♥ r ss ♦♥t♥♦s rs ♥♥ t t♦r ♦♥s ♥ ss r t♥♥ ss r r t♥ ♠♣ts t ♠♦st t♦r rs ♥ ss t s t r ♦♣s ♠♣t t ♠♦st t♦rr ♥ ss

r s♣②s t ♣rtt♦♥ ♦t♥ ② t ♦♠♣♦♥♥t ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s ♥ t rst ♣♥ ♦ t P♠①t ❬❪❲ ♥ s tt t s♦♥ ①s s sr♠♥t t♥ ♦t st♠t sss

♦♠♣rs♦♥ t ♦tr ♣♣r♦s rr♦r rt ♦t♥ ② t ♠①tr♠♦ ♦ ss♥ ♥ ♦st strt♦♥s s 38% ♦r♥ t♦ ❬❪ t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♥ t♠① ♦t♥ ♥ rr♦r rt ♦ 23% ♥②t trt♦♥ rr str♥ ♠t♦s ♠ssst♦♥ rt r♥♥t♥ 22% ♥ 46% ♦t tt t ♠♥s ♣♣r♦ ❬❪ ♦t♥s ♥ rr♦rrt ♦ 15% t t st♦♥ ♦ t ♥♠r ♦ sss s ♠♦r t


age

trestbps

chol

thalach

oldpeach

sex

cp

fbs

restcy

exang

slope

ca

tha

ss

age

trestbps

chol

thalach

oldpeach

sex

cp

fbs

restcy

exang

slope

ca

tha

ss

r ♣♥♥s t♥ t ♦♥t♥♦s rs tr♥s ♦♥ t t♥ t t♦r rs r ♦♥ t rt ♣r ss ♥ ♥♦s ♣♥♥② s♦ ♥♦ ♥ ♦♥t ♥ t ♦st rrss♦♥ ♥ t ss

−2 −1 0 1 2

−1

01

2

First principal component analysis mixte

Sec

ond

prin

cipa

l com

pone

nt a

naly

sis

mix

te

Class 1 Class 2

r Prtt♦♥ r♥ ♥ t rst ♦♠♣♦♥♥t ♠♣ ♦ t P♠①t

♥♦♠ s♠s♣rs sst♦♥

t st sr♣t♦♥ ♥♦♠ s ♥r ♦ s♥ t st srs ♣t♥ts ❬❪ ② ♦r ♦♥t♥♦s rs ♦♠♣♥ t♠ ♥ ②s ♥②rs ②r ♦ ♦♣rt♦♥ ♥ t t♠♦r t♥ss ② t♦ ♥r② rs s① ♥♣rs♥s♥ ♦ r ♥ ② ♦♥ stts r tt ♦t♦♠③ r♦♠ ♠♥♦♠ ♦r ♥♦t

sr♠♥t ♥ ♥rt ♣♣r♦s ♦♠♣rs♦♥ ❲ strt ttt ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♥ ♦t♣r♦r♠ ss


♠t♦s ♦ s♠s♣rs sst♦♥ s♣② ♥ ♦srt♦♥s r ♦ r♥♦♠② ♥ ♣rt ♦ t s ❲ ♦♠♣r t rr♦r rt ♦ t♥ ♣rtt♦♥ ♦t♥ ② ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥st♦ t rr♦r rt ♦t♥ t ♦st rrss♦♥ ♣rs♥ts t ♠♥♦ t ♠sss②♥ rt ♦♠♣t ♦♥ r♥♦♠② ♥ ♣rtt♦♥ ♦r r♥t♣r♥ts ♦ ♠ss♥ s

% ♠ss♥ s

♣r♦♣♦s ♠♦ ♦st

♥ ♦ t ♠sss②♥ rt ♦♠♣t ♦♥ t ♥s ♥ ♠ss♥ ♠♠rs♣

♦ r♥t ♣♣r♦s ♦r t♦ r♥t ♦ts ❲ r♠♥ tt t♦st rrss♦♥ ♠ s t♦ rt② ♠♦③ t ♦rr t♥ t sss ♥ts ♠t♦s s ♦♣ s♣② ♦r t s♠s♣rs sst♦♥ ♠ ♦ t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♠♦s s ♠♦r♠t♦s ♥ t ♠♦③s t ♦ strt♦♥ ♦ t t

♦♠♠♥ts ♣rs♥t rsts r s ①♣t ❲♥ ♠♦rt② ♦ t ♥s s t ♦st rrss♦♥ ♦t♥s ♦r ♠sss②♥ rt ♦r ♥ ♠♦rt② ♦ t ♥s s ♥ t ♠♥ ♥♦r♠t♦♥ s ♦♥t♥ ② ts ♥s s r♥ ts ①♣r♠♥t ♦sr tt t♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦t♣r♦r♠s t ♦st ♠♦♦r ♥ rt ♦ ♠ss♥ s rtr♠♦r ts rt r♠t② r♦s ♦rt ♦st rrss♦♥ ♥ r② ♥s r ts st②s st♦r t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♥ s 27.8% ♦t ♥s s r♦♠ ♠♥♦♠ t ♦st rrss♦♥ s ♦s t♦ t ♦rsrr♦r rt ♥ 95% ♦ t s r ♠ss♥

♥♥ sss ♥ str ♥②ss ❲ ♥♦ ♥tr♣rt ♦t sss ♦t♥♥ t ♥s r ♥ ♠♦rt② ss 60% r♦♣s ♥s♥ ♦♥ ♦♠♣♥② t♠ ♥ r♥t② trt ss s ♠♥② ♦♠♣♦s② ②♦♥ ♦♠♥ ♥ s♠ t♠♦r ♥ ts ss t r s t t♠♦r trr s t r rs ♥ t ♠♥♦rt② ss 40% ♥ t ♣t♥ts rt ♦♠♣♥② t♠ s s♦rtr ♦r ♦♠♣♥♠♥t st♦♣♣ ♥ r ttrt♠♥t s ♦ s ♣t♥ts r ♦r ♥r② t t♠♦r ♠♥② ♣rs♥t♦r t ♠♥ ♥ ♥rs♥ t r rs ss ♥tr♣rtt♦♥ s s ♦♥ t♠r♥ ♣r♠trs ♣rs♥t ♥ r

♦♥s♦♥ ♠tr① s♣② ♥ s♦s tt t st♠t ♣rtt♦♥s ♦s t♦ t sr stts ♠♦rt② ss ♥♦s s♠ rs ♦ t r♦♠♥♦♠ ts rs s r ♥ t ♠♥♦rt② ss

♦♥s♦♥

−2000 0 2000 4000 60000.00

000

0.00

020

density of time

C 1 C 2

0 50 100

0.00

00.

010

0.02

0

density of age

C 1 C 2

1960 1965 1970 1975 1980

0.00

0.05

0.10

0.15

density of year

C 1 C 2

−10 −5 0 5 10 15 20

0.00

0.10

0.20

density of thickness

C 1 C 2

female male

0.0

0.4

0.8

barplot of sex

C 1 C 2 C 1 C 2absence presence

0.0

0.4

0.8

barplot of ulcer

C 1 C 2 C 1 C 2

r P♦tt♥ ♦ t ♠r♥ ♣r♠trs ♦ t ♦♠♣♦♥♥t ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s st♠t ♦♥ t ♥♦♠ t sts

♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s♠♦rt② ss ♠♥♦rt② ss

♥♦t ♦r♠ ♠♥♦♠ ♦r♠ ♠♥♦♠

♦♥s♦♥ ♠tr① t♥ t ♣rtt♦♥ st♠t ② t ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s ♥ t sr stts

♦♥s♦♥

♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s s ♥ ♥t ♣♣r♦t♦ str t sts t ♦♥t♥♦s ♥ t♦r rs ♦ t s ♦♦ ♥r t♦ t ♥♦♥ ♠♦s ♥ ②s ♥ ♠①tr ♦ ♦t♦♥ ♠♦s ♥ts rs ts rst ♥t s t♦ t ♥t♦ ♦♥t t ♥trss ♣♥♥st♥ t rs s t ♣r♦♣♦s ♠♦ ♦s t ss ♥♦ ②t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ts s♦♥ ♥t s t♦ ♣ ssstrt♦♥s ♦r t ♦♥♠♥s♦♥ ♠r♥s ♦ ♦♠♣♦♥♥t ♥ t ♣rtt♦♥r s② s♠♠r③s ss ② t ♣r♠trs ♦ t ss strt♦♥s♥ ② t ♦st ♥t♦♥s

♣rs♠♦♥♦s rs♦♥s ♦ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦ t♦ ♠♦ t ♠♥ ♦♥t♦♥ ♣♥♥s t♥ ♠① rss t ss ♥tr♣rtt♦♥ s sr ♠♦ st♦♥ ♥ t ♣r♠tr


st♠t♦♥ r s♠t♥♦s② ♣r♦r♠ ♠ ♦rt♠ ♠①♠③♥ ♥ ♥♦r♠t♦♥ rtr♦♥ ♦r♥ t♦ ♦r ①♣r♠♥ts ♦r s t♦ s t

rtr♦♥ s t ♥♦r♠t♦♥ rtr♦♥ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♥ s ♥ s♠

s♣rs sst♦♥ ♦r♥ t♦ ♦r ♣♣t♦♥ t ♥ ♦♦ ♥r t♦t ss sr♠♥t♦♥ ♣♣r♦s s♣② ♥ ♥s r

♣tr

♦s str♥ ♦ ss♥

♦♣s ♦r ♠① t

♠①tr ♠♦ ♦ ss♥ ♦♣s s ♣rs♥t t♦str ♠① t r ♥② ♥s ♦ rs r♦ t② ♠t ♠t strt♦♥ ♥t♦♥ s ♣♣r♦ ♦s t♦ strt♦rr② ♥s♠♣ ♠trt ♥trss ♣♥♥② ♠♦s ♣rsr♥ ♥② ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t ♦ ♥trst ♦r t sttst♥ ②♣②♥ ts ♦r t ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t r ss ♣r♠tr ♦♥s ♥ ♦rr t♦ ttt ♠♦ ♥tr♣rtt♦♥ ♥ t♦♥ t ♥trss ♣♥♥s r t♥ ♥t♦ ♦♥t ② t ss♥ ♦♣s ♣r♦ ♦♥ ♦rrt♦♥ ♦♥t ♥ ♦♦♣r♦♣rts ♣r ♦♣ ♦ rs ♥ ♣r sss ♠♦ ♥r③s r♥t ①st♥ ♠♦s ♥♦r ♦♠♦♥♦s ♥ ♠① rs ♥r♥ s♣r♦r♠ tr♦♣♦st♥s s♠♣r ♥ ②s♥ r♠♦r ♠r ①♣r♠♥ts strtt ♠♦ ①t② ♥ ts r♥

s ❩♦rq♥ t ♠ts ♥ ♦♣ s♦

t q t rss♠st♦s s r②♦♥s sr ♥ s ♣♦♥t

♣♦♥tà ♣r♥ ♥tôt P♦rq♦

Pr q ♦r s♦♥ sst ♣s é♣r♣é

sst rss♠ésr ♥ s ♣♦♥t

♠ê♠ s♣rt ♦♠♠♥ t s ♠rs

♥ ♦♥♥tr♥t s♦♥ s♣rtsr ♥ s t ♠ê♠ ♦s

í♦s ③♥t③ás①s ❩♦r

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

♥tr♦t♦♥

♠ ♦ ts ♣tr s t♦ ♣rs♥t ♠♦s str♥ ♦r ♠① t♦ ♥② ♥s ♦ rs ♠tt♥ ♠t strt♦♥ ♥t♦♥ s ♠♦s ♦ ♦t t♦ ♣rsr ss strt♦♥s ♦r ts ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t ♥ t♦ ♣rs♠♦♥♦s② ♥ ♠♥♥②♠♦③ t ♥trss ♣♥♥s

s ♦t ♥ ♥tr② ② t s ♦ ♦♣s ❬♦ ❪ ♥ ♦♣s ♠trt ♠♦ ② stt♥ ♦♥ t ♦♥ ♥ t♦♥♠♥s♦♥ ♠r♥s ♥ ♦♥ t ♦tr ♥ t ♣♥♥② ♠♦ t♥rs ♦r ♣rs② t t strt♦♥ s ♣♣r♦ ② ♣r♠tr♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t r ss ♥ ♦s t ss♥ ♦♣s ❬♦ ❲❪ ♠♦③ t♥trss ♣♥♥s ♦t tt ❬ ❪ r② s ♦♥ ss♥♦♣ t♦ ♥ strt♦♥ ♦ ♠① rs ♣r♦♣♦s ♠♦ s s♦ ♥r③t♦♥ ♦ ts ♣♣r♦ t♦ t ♥t ♠①tr ♠♦ r♠♦r

♥ ♠①tr ♠♦ s ♠♥♥ s♥ t ♣r♠ts tr s♠ ♦s r♥② ♥tr♣rtt♦♥ t ♣r♦♣♦rt♦♥s ♥t t ss ts t ♦♥♠♥s♦♥ ♠r♥ ♣r♠trs ♦ ♦♠♣♦♥♥ts r♦② sr t sss t ♦rrt♦♥ ♠trs r♥ ts sr♣t♦♥ ♥② ② s♥ t ♦♥t♥♦s t♥t strtr ♦ t ss♥ ♦♣s Pt②♣ s③t♦♥ ♣r ss♦s t♦ s♠♠r③ t ♠♥ ♥trss ♣♥♥s ♥ ♣r♦s sttr♣♦t ♦t ♥s ♦r♥ t♦ t ss ♣r♠trs

♦t tt ♦s♠s ♥ rs ❬❪ r♥t② s♠tt ♥ rt ♣r♦♣♦ss t♦ s ♠①tr ♦ ♦♣s t♦ ♣r♦r♠ str ♥②ss t♦rsst② r♥t ♦♣s ♠♦♥ t♠ t ss♥ ♦♣s r ♦♥sr r♠♦ s ♦s t♦ t ♣♣r♦ ♦♣ ♥ ts ♣tr ♦r t♦ ♠♣♦rt♥tr♥s t♦ ♠♥t♦♥ rst② ♣r♦♣♦s ②s♥ ♥r♥ tt♦rs ♣r♦♣♦s ♥ ♣♣r♦ ② ♠①♠♠ ♦♦ ♥r ♦♥str♥ts ♦♥②s♦♠ s③t♦♥s t♦♦s r ♣rs♥t r

trtr ♦ ts ♣tr s ♣♣r s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♥tr♦ t♦ str ts ♥s tt ①st♥ ♠♦s ♥ ts ♦♥trt♦♥ t♦ t s③t♦♥ ♦ ♠① rs t♦♥ s ♦t t♦ t ♣r♠tr st♠t♦♥ ♥ ②s♥ r♠♦r s♥ t♠①♠♠ ♦♦ st♠t s ♥tt♥ ❬P❪ t♦♥ strts t♦r ♦ t ♦rt♠ ♣r♦r♠♥ t ♥r♥ ♥ s♦ t ♠♦ r♦st♥ss ♦♥♥♠r ①♣r♠♥ts t♦♥ ♣rs♥ts tr ♣♣t♦♥s ♦ t ♥ ♠①tr♠♦ ② str♥ tr r t sts t♦♥ ♦♥s ts ♦r tsrsts r ♣rt ♦ t rt ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠①t ❬❱❪

①tr ♠♦ ♦ ss♥ ♦♣s

①tr ♠♦ ♦ ss♥ ♦♣s

♥t ♠①tr ♠♦

t

t♦r ♦ e ♠① rs s ♥♦t ② xi = (x1i , . . . , xei ) ∈ R

c × X te = c+ d ts rst c ♠♥ts r t st ♦ t ♦♥t♥♦s rs ♥ ♦♥ ts♣ R

c ♥ rtr ♥♦t ② x

i ts st d ♠♥ts r t st ♦ t srtrs ♥tr ♦r♥ ♦r ♥r② ♥ ♦♥ t s♣ X ♥ rtr ♥♦t ②x

i ♦t tt xji s ♥ ♦r♥ r t mj ♠♦ts t♥ t ss ♥♠r

♦♥ 1, . . . ,mj

♦tt♦♥ ❲ r♠♥ tt s t ♥r ♥♦tt♦♥ P (.; .) ♦r t ♠tstrt♦♥ ♥t♦♥s ♥ p(.; .) ♦r t ♣r♦t② strt♦♥ ♥t♦♥ ♣

Pr♦t② strt♦♥ ♥t♦♥

♥t♦♥ ♥t ♠①tr ♠♦ ♦ ♣r♠tr strt♦♥s t xi rs♣♣♦s t♦ r♥ ② t ♠①tr ♠♦ ♦ g ♣r♠tr strt♦♥s ♦s t♣ s rtt♥ s ♦♦s

p(xi;θ) =

g∑

k=1

πkp(xi;αk),

r θ = (π,α) ♥♦ts t ♦ ♣r♠trs t♦r π = (π1, . . . , πg)r♦♣s t ♣r♦♣♦rt♦♥s ♦ ss k ♥♦t ② πk ♥ rs♣t♥ t ♦♦♥♦♥str♥ts 0 < πk ≤ 1 ♥

∑gk=1 πk = 1 t t♦r α = (α1, . . . ,αg) r♦♣s

t ♣r♠trs ♦ ss k ♥♦t ② αk

Pr♦♣rt② t♥t r ♥t ♠①tr ♠♦ ♥ ①♣rss ② s♥t t♥t r zi s t♦r r ♥ts t ss ♠♠rs♣② s♥ ♦♠♣t s♥t ♦♥ ♥ ♦♦s t ♠t♥♦♠ strt♦♥Mg(π1, . . . , πg) s ♥ ♥tr♣rt s t ♠r♥ strt♦♥ ♦ xis ♦♥ t strt♦♥ ♦ t ♦♣ (xi, zi)

ss♥ ♦♣ ♦r ♠① t

♦♠♣♦♥♥t strt♦♥s ♦♦♥ ss♥ ♦♣s

♦♣s ♦ t♦ ♠trt ♠♦ ② stt♥ ♦♥ t ♦♥ ♥ t♦♥♠♥s♦♥ ♠r♥s ♥ ♦♥ t ♦tr ♥ t ♣♥♥② ♠♦ t♥rs ❲ ♥♦ ♣rs♥t t ♠r♥ strt♦♥ ♦ t ♦♠♣♦♥♥ts t♥ ♦s♦♥ t ss♥ ♦♣ s ♦ ♥trst ♦r s s♥ t ♣r♦s ♦♥ ♦rrt♦♥♦♥t ♣r ♦♣ ♦ rs ♥ s♥ t ♦s ♥ s② ♣r♠tr st♠t♦♥

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

♥♠♥s♦♥ ♠r♥s ♦ t ♦♠♣♦♥♥ts

♦r ♦♠♣♦♥♥t ss♠ tt t ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t♦♥s t♦ t ①♣♦♥♥t ♠② ♥ ♦rr t♦ ♣r♦ ♠♥♥ sss

♥t♦♥ ♥♠♥s♦♥ ♠r♥s ♦ t ♦♠♣♦♥♥ts ♠r♥ strt♦♥ ♦ t r xji ♦r ♦♠♣♦♥♥t k ♦♥s t♦ t ①♣♦♥♥t ♠② ♥s p(xji ;βkj) ♦r ♣ ♥ P (x

ji ;βkj) s ♦r ♣rs②

xji s ♦♥t♥♦s ts ♠r♥ ♦ ♦♠♣♦♥♥t k ♦♦s ss♥ strt♦♥t ♠♥ µkj ♥ r♥ σ2

kj xji |zik = 1 ∼ N1(µkj, σ2kj) ♥ βkj =

(µkj, σ2kj) ∈ R× R

+∗ xji s ♥tr ts ♠r♥ ♦ ♦♠♣♦♥♥t k ♦♦s P♦ss♦♥ strt♦♥

xji |zik = 1 ∼ P(βkj) ♥ βkj ∈ R+∗

xji s ♦r♥ ts ♠r♥ ♦ ♦♠♣♦♥♥t k ♦♦s ♠t♥♦♠ strt♦♥ xji |zik = 1 ∼ Mmj

(βkj) βkj ♥ ♥ ♦♥ t s♠♣① ♦ s③ mj

♣♥♥② ♠♦ ♦ t ♦♠♣♦♥♥ts

♠①tr ♠♦ ♦ ss♥ ♦♣s ss♠s tt ♦♠♣♦♥♥t k ♦♦s ss♥ ♦♣ ♦s t ♦rrt♦♥ ♠tr① ♦ s③ e× e s ♥♦t ② Γk ❲♥♦t Φe(.;Γk) t ♦ t ert ♥tr ss♥ strt♦♥ t ♦rrt♦♥♠tr① Γk ♥ Φ−1

1 (.) t ♥rs ♠t strt♦♥ ♥t♦♥ ♦N1(0, 1) s ♦t♥ t ♦♦♥ ♥t♦♥ ♦ t ♦♠♣♦♥♥t

♥t♦♥ ♠t strt♦♥ ♥t♦♥ ♦ t ♦♠♣♦♥♥ts ♦r t ♠①tr ♠♦ ♦ ss♥ ♦♣s t ♦ ♦♠♣♦♥♥t k s rtt♥ s

P (xi;αk) = Φe(Φ−11 (u1k), . . . ,Φ

−11 (uek);0,Γk),

r ujk = P (xji ;βkj) ♥ r αk = (βk,Γk) ♥♦ts t ♦ ♣r♠trs ♦♦♠♣♦♥♥t k t βk = (βk1, . . . ,βke)

Pr♦♣rt② t♥r③ ♦♥t ♦ ♦rrt♦♥ ♣r ss ss♥♦♣ ♣r♦s ♦♥t ♦ ♦rrt♦♥ ♣r ♦♣ ♦ rs s ♦♦♣r♦♣rts ♥ ♥ ♦t rs r ♦♥t♥♦s t s q t♦ t ♣♣r♦♥ ♦ t ♦♥ts ♦ ♦rrt♦♥ ♦t♥ ② t ♠♦♥♦t♦♥ tr♥s♦r♠t♦♥s♦ t rs ❬❲❪ rtr♠♦r ♥ ♦t rs r srt t s qt♦ t ♣♦②♦r ♦♥t ♦ ♦rrt♦♥ ❬s❪

Pr♦♣rt② ♦♥ t♥t r ♠①tr ♠♦ ♦ ss♥ ♦♣s♥♦s s♦♥ t♥t r t♦ t ss ♠♠rs♣ ♦♥ssts♥ ♥ ert ♦♥t♥♦s r ♥♦t ② yi = (y1i , . . . , y

ei ) ∈ R

e ♦♥t♦♥② ♦♥ t ss ♠♠rs♣ ts r ♦♦s ♥ ert ♥tr ss♥strt♦♥ ♥ yi|zik = 1 ∼ Ne(0,Γk) ♥

xji = P−1(Φ1(yj);βkj), ∀j = 1, . . . , e,

t♥ ♦♠♣♦♥♥t k s ss♥ ♦♣ ♦s t s P (xi;αk)

①tr ♠♦ ♦ ss♥ ♦♣s

①tr ♠♦ ♦ ss♥ ♦♣s ♦r ♠① t

❲ ♥tr♦ t ♥t♦♥ Ψ(x

i ;αk) =(xji−µkj

σkj; j = 1, . . . , c

)

♥ t s♣ ♦

t ♥t♥ts ♦ x

i ♦r ss k s ♥♦t Sk(x

i ) = Sc+1k (xc+1

i )× . . .× Sek(xei ) ♥tr Sjk(xji ) =]b⊖k (x

ji ), b

⊕k (x

ji )] s ♥ ♦r j = c + 1, . . . , e ♥ ts ♦♥s r

b⊖k (xji ) = Φ−1

1 (P (xji − 1;βkj)) ♥ b⊕k (x

ji ) = Φ−1

1 (P (xji ;βkj)) ❲ ♥♦ ♥ t ♣♦ t ♦♠♣♦♥♥ts ♦r♥ t♦ s ♣r♦♣♦s ♥ ❬❪

♥t♦♥ ①tr ♠♦ ♦ ss♥ ♦♣s t xi ♦♦s ♠①tr♠♦ ♦ ss♥ ♦♣s ts ♣ s t ♥t ♠①tr ♠♦ ♥ ♥ ♦s t ♣ ♦ ♦♠♣♦♥♥t k s rtt♥ s

p(xi;αk) = p(x

i ;αk)p(x

i |x

i ;αk)

=φc(Ψ(x

i ;αk);0,Γk)∏c

j=1 σkj

∫

Sk(x

i )

φd(u;µ

k ,Σ

k)du,

r Γk =

[

Γk Γk

Γk Γk

]

s ♦♠♣♦s ♥t♦ s♠trs ♦r ♥st♥ Γk s t

s♠tr① ♦ Γk ♦♠♣♦s ② t r♦s ♥ t ♦♠♥s rt t♦ t ♦sr♦♥t♥♦s rs ♦r♦r µ

k s t ♦♥t♦♥ ♠♥ ♦ yi ♥ ② µ

k =ΓkΓ

−1kΨ(x

i ;αk) ♥ Σ

k s ts ♦♥t♦♥ ♦r♥ ♠tr① ♥ ② Σ

k =Γk − ΓkΓ

−1kΓk

Pr♦♣rt② ♥rt ♠♦ ♠①tr ♠♦ ♦ ss♥ ♦♣s ♥♦st ♥rt ♠♦ s♣t ♥t♦ t ♦♦♥ tr st♣s

ss ♠♠rs♣ s♠♣♥ zi ∼ Mg(π1, . . . , πg) ss♥ ♦♣ s♠♣♥ yi|zik = 1 ∼ Ne(0,Γk) sr t tr♠♥st ♦♠♣tt♦♥ xi s ♦t♥ r♦♠

♠rs

♦♠♦sst ♠♦s ❲♥ t s♠♣ s③ s s♠ t tr ♦ t♥t s ♥ t r♥ ♦ t st♠t ♠② ttr s♦♠ ♦♥str♥ts♦♥ t ♣r♠tr s♣ r s ♣r♦♣♦s ♣rs♠♦♥♦s rs♦♥♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ② ss♠♥ t qt② t♥t ♦rrt♦♥ ♠trs s♦

Γ1 = . . . = Γg.

♦t tt ts ♠♦ s ♥♠ ♦♠♦sst s♥ t ♦r♥ ♠trs♦ t t♥t ss♥ rs r q t♥ sss

♠r ♦ ♣r♠trs tr♦sst rs♣t② ♦♠♦sst♠①tr ♠♦ ♦ ss♥ ♦♣s ♥s ν rs♣t② ν♦ ♣r♠trsr

ν = (g−1)+g

(

e(e− 1)

2+

d∑

j=1

νj

)

♥ ν♦ = (g−1)+e(e− 1)

2+g

d∑

j=1

νj,

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

r νj ♥♦ts t ♥♠r ♦ ♣r♠trs ♦ t ♠r♥ strt♦♥ ♦r j ♦r ♦♥ ♦♠♣♦♥♥t ♦r ♣rs② t t s♣ ♠r♥ strt♦♥ ♦ t ♦♠♣♦♥♥ts νj s q t♦

νj =

2 xj s ♥♠r1 xj s srt

mj − 1 xj s ♦r♥

♦ ♥tt② ♠①tr ♠♦ ♦ ss♥ ♦♣s s ♥t♥ t s♥s ♥ ♥ ❬ ❨❪ t st ♦♥ r s ♦♥t♥♦s♦r ♥tr ♣r♦♦ s ♥ ♥ ♣♣♥①

tr♥ts ♦ t ♠①tr ♠♦

t ♠♦s

♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t♦ ♥r③ ♠♥② ss♠♦s str♥s ♠♦♥ t♠ ♦♥ ♥ t t ♦♦♥ ♦r

♦s② t ♦rrt♦♥ ♠trs r ♦♥ Γk = I ∀k =1, . . . , g t♥ t ♠①tr ♠♦ ♦ ss♥ ♦♣s s q♥t t♦ t♦♥t♦♥ ♥♣♥♥ ♠①tr ♠♦

t rs r ♦♥t♥♦s c = e ♥ d = 0 t♥ t ♠①tr♠♦ ♦ ss♥ ♦♣s ♦♠s ♠trt ss♥ ♠①tr ♠♦t♦t ♦♥str♥t t♥ t ♣r♠trs ❬❪

♠①tr ♠♦ ♦ ss♥ ♦♣s s ♥ t♦ t ♥♥ ss♥♠①tr ♠♦ ♦r ♥st♥ t s q♥t ♥ t r ♦r♥ t♦ t♠①tr ♠♦ ♦ ❬♦❪ ♥ s s ts ♠♦ s st ② s♦♥ ♦♠♦ts

❲♥ t rs r ♦t ♦♥t♥♦s ♥ ♦r♥ t ♠①tr ♠♦ ♦ss♥ ♦♣s s ♥ ♣r♠tr③t♦♥ ♦ t ♠①tr ♠♦ ♣r♦♣♦s② rtt ❬❪ s t♦♥ ♦r rtt st♠ts rt② ts♣ Sk(x

i ) ♦♥t♥♥ t ♥t♥ts ♦ x

i ♥ ♥♦t t ♠r♥ ♣r♠trss t ♠①♠♠ ♦♦ ♥r♥ s s♦ ♣r♦r♠ s♠♣① ♦rt♠ r♠t② ♠t♥ t ♥♠r ♦ ♦r♥ rs ♦t tt ♦r♣♣r♦ ♦r t ♥r♥ ♦s ts r s ts ♥ t♦♥

t s③t♦♥ ♣r ss ②♣r♦t ♦ ss♥ ♦♣s

❲ ♥ s t ♠♦ ♣r♠trs t♦ ♦t♥ s③t♦♥ ♦ t ♥s ♣rss ♥ t♦ r♥ ♦t t ♠♥ ♥trss ♣♥♥s s ♦r ss k rst②♦♠♣t t ♦♦r♥ts q t♦ E[yi|xi, zik = 1;αk] ♥ s♦♥② ♣r♦t t♠♦♥ t ♣r♥♣ ♦♠♣♦♥♥t ♥②ss s♣ ♦ t ss♥ ♦♣ ♦ ♦♠♣♦♥♥t k♦t♥ ② t s♣tr ♦♠♣♦st♦♥ ♦ Γk

♥s r♥ ② t ♦♠♣♦♥♥t k ♦♦ ♥tr ss♥ strt♦♥♥ t t♦r ♠♣ s♦ t② r ♦s t♦ t ♦r♥ ♦s r♥ ② ♥♦tr♦♠♣♦♥♥t ♥ ①♣tt♦♥ r♥t r♦♠ ③r♦ s♦ t② r rtr t♦ t ♦r♥♥② t ♦rrt♦♥ r s♠♠r③s t ♥trss ♦rrt♦♥s ♦♦♥①♠♣ strts ts ♣♥♦♠♥♦♥

①tr ♠♦ ♦ ss♥ ♦♣s

①♠♣ ①tr ♠♦ ♦ ss♥ ♦♣s ♥ s③t♦♥ ♣r ss tt ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦♠♣♦s t tr rs♦♥ ♦♥t♥♦s ♦♥ ♥tr ♥ ♦♥ ♥r② ♥ ts ♦rr t

π = (0.5, 0.5), β11 = (−2, 1), β12 = 5, β13 = (0.5, 0.5),β21 = (2, 1), β22 = 15,

β23 = (0.5, 0.5), Γ1 =

1 −0.4 0.4−0.4 1 0.40.4 0.4 1

♥ Γ2 =

1 0.8 0.10.8 1 0.10.1 0.1 1

.

−4 −2 0 2 4

510

1520

25

x1

x2

−6 −4 −2 0 2

−2

−1

01

23

first principal component axis

seco

nd p

rinci

pal c

ompo

nent

axi

s

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

inertia: 60.8 %

iner

tia: 3

2.5

%

continuous

integer

binary

r ①♠♣ ♦ s③t♦♥ sttr♣♦t ♦ t ♥s sr② tr rs ♦♥ ♦♥t♥♦s sss ♦♥ ♥tr ♦r♥t ♥ ♦♥ ♥r②s②♠♦ ♥s sttr♣♦t ♥ t rst ♦♠♣♦♥♥t ♠♣ ♦ ss rs r♣rs♥tt♦♥ ♥ t rst ♦♠♣♦♥♥t ♠♣ ♦ ss ♦♦r ♥tst ss ♠♠rs♣s

s③t♦♥ ♦ ss s ♣rs♥t ♥ r ♦♥r♥♥ t ♥st sttr♣♦t s♦s ♥tr ss t r ♦♥ ♥ s♦♥ ss t ♦♥ ♦t ♦♥ t t s ♦♥r♥♥ t rs t r♣rs♥tt♦♥ ♣♦♥ts ♦t② str♦♥ ♥trss ♦rrt♦♥ t♥ t ♦♥t♥♦s ♥ t ♥tr rs

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

②s♥ ♥r♥ tr♦♣♦st♥s

s♠♣r

♠ ❲ ♦sr t s♠♣ ① = (x1, . . . ,xn) ♦♠♣♦s t n ♥♣♥♥t ♥s xi ∈ R

c×X ss♠ t♦ r♥ ② ♠①tr ♠♦ ♦ ss♥ ♦♣s ♠ s t♦ ♥r t ♣r♠trs ♦r♥ t♦ t t

rq♥tst ♦♥t①t ♥r♥ ② ♠①♠♠ ♦♦ s t ♣r♦♠♦r t ♣r♠tr ♦♣s ♥ t ♠r♥ ♣r♠trs r ♥♥♦♥ ♦ ts ♦t♥ r♣ ② t ♥r♥ ♥t♦♥ ♦r r♥s ♠t♦ ♣r♦r♠♥ t♥r♥ ♥ t♦ st♣s s ♣tr ♦ ❬♦❪ rst st♣ st♠ts t♠r♥ ♣r♠trs ② ♠①♠③♥ ♥rt ♦♦ t s♦♥ st♣st♠ts t ♦rrt♦♥ ♣r♠trs ② ♠①♠③♥ t ♦♦ ♦♥t♦♥② ♦♥t ♠r♥ ♣r♠trs s ♣♣r♦ s s ♥ ❬❪ ♦r t ♠①♠♠♦♦ st♠t ♥ ss♥t② ♦t♥ ♥ t rs r ♦♥t♥♦s② s♥ t ①♣♦♥t ♦rt♠ ♣r♦♣♦s ② ❬❪ ♥ ts ♣♣r♦ ♥♥♦t ①t♥ t♦ t ♠① t stt♥ s ♥ ♠ ♦rt♠ ♥ ♥♦t ♠♣♠♥t t♦ ♦t♥ t ♠①♠♠ ♦♦ st♠ts ♦ ♠①tr ♠♦ ♦ss♥ ♦♣s ♥ t ♠① t s rtr♠♦r ♥ t ♠ st♣ ♦ ①♣t t st♣ ♦ t♦♦ ♠ t♠ ♦♥s♠♥ t srt rsr ♥♠r♦s s ♦ t ♦♠♣tt♦♥ ♦ t ♥tr ♦ ♠♥s♦♥ d ♥ ♥

②s♥ ♦♥t①t ♥ ♦rr t♦ ♦ ♦t ♣r♦s ♣r♦♠s ♣rr t♦ ♦r♥ ②s♥ r♠♦r ❲ rst② ♥ t ♣r♦r strt♦♥s ♥ s♦♥②♣rs♥t t s s♠♣r ♣r♦r♠♥ t ♥r♥

①♠♠ ♣♦str♦r st♠t

Pr♦r strt♦♥s

♥♣♥♥ ss♠♣t♦♥ ss ss♠♣t♦♥ s t♦ s♣♣♦s t ♥♣♥♥ t♥ t ♣r♦r strt♦♥s ts

p(θ) = p(π)

g∏

k=1

(

p(Γk)d∏

j=1

p(βkj)

)

.

Pr♦♣♦rt♦♥s ss ♦♥t ♣r♦r strt♦♥ ♦ t ♣r♦♣♦rt♦♥ t♦rs t r②s ♥♦♥ ♥♦r♠t ♦♥ s rt strt♦♥ ♦s t ♣r♠trs r q t♦

π ∼ Dg

(

1

2, . . . ,

1

2

)

.

r♥ ♣r♠trs ♣r♦r strt♦♥ ♦ t ♠r♥ ♣r♠trs r tss ♦♥t ♦♥s ♦r ♣rs②

②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r

xj s ♦♥t♥♦s t♥ βkj ♥♦ts t ♣r♠trs ♦ ♥rt ss♥strt♦♥ s♦ p(βkj) = p(µkj|σ2

kj)p(σ2kj) t

σ2kj ∼ G−1(c0, C0) ♥ µkj|σ2

kj ∼ N1(b0, σ2kj/N0),

r G−1(., .) ♥♦ts t ♥rs ♠♠ strt♦♥ ❲t ♥ ♠♣r②s♥ ♣♣r♦ t ②♣r♣r♠trs (c0, C0, b0, N0) r ① s ♣r♦♣♦s ② ❬❪ s♦ c0 = 1.28 C0 = 0.36❱r(①j) b0 = 1

n

∑ni=1 x

ji ♥

N0 =2.6

r♠① ①j−r♠♥ ①j xj s ♥tr βkj ♥♦ts t ♣r♠tr ♦ P♦ss♦♥ strt♦♥ ♥

βkj ∼ G(a0, A0).

♦r♥ t♦ ❬❪ t s ♦ ②♣r♣r♠trs a0 ♥ A0 r ♠♣r②① t♦ a0 = 1 ♥ A0 = a0n/

∑ni=1 x

ji

xj s ♦r♥ βkj ♥♦ts t ♣r♠tr ♦ ♠t♥♦♠ strt♦♥ ♥ts r②s ♥♦♥ ♥♦r♠t ♦♥t ♣r♦r ♥♦s tt

βkj ∼ Dmj

(

1

2, . . . ,

1

2

)

.

♦rrt♦♥ ♠trs ♦♥t ♣r♦r ♦ ♦r♥ ♠tr① s t ♥rs❲srt strt♦♥ ♥♦t ② W−1(., .) ♦ t s ♥tr t♦ ♥ t ♣r♦r ♦t ♦rrt♦♥ ♠tr① Γk r♦♠ t ♣r♦r ♦ t ♦rrt♦♥ ♠tr① Λk s♥ Γk|Λk str♠♥st ❬♦❪ ♦

Λk ∼ W−1(s0, S0) ♥ ∀1 ≤ h, ℓ ≤ e, Γk[h, ℓ] =Λk[h, ℓ]

√

Λk[h, h]Λk[ℓ, ℓ],

r (s0, S0) r t♦ ②♣r♣r♠trs ♦r t ss ♣♣r♦ ♦♥sst♥♥ tt♥ t ②♣r♣r♠trs tr♦ ♥ ♠♣r ②s♥ ♣♣r♦ s ♥♦t ♣♦ss s♥ yi s ♥♦t ♦sr ❲ ts ♣t s0 = e + 1 ♥ S0 q t♦ t ♥tt②♠tr① s♥ ♥ ts s t ♠r♥ strt♦♥ ♦ ♦rrt♦♥ ♦♥t s♥♦r♠ ♦♥ ]− 1, 1[ ❬❪

P♦str♦r strt♦♥

②s♥ ♥r♥ s ♣r♦r♠ ② s♠♣♥ sq♥ ♦ ♣r♠trs r♦♠tr ♣♦str♦r strt♦♥ ♥ ♣rt s s s♠♣r s t ♠♦st♣♦♣r ♣♣r♦ t♦ ♣r♦r♠ ②s♥ ♥r♥ ♦ ♠①tr ♠♦ s♥ t ss tt♥t strtr ♦ t t ♥ t tr♥t② s♠♣s t ss ♠♠rs♣s♦♥t♦♥② ♦♥ t ♣r♠trs ♥ ♦♥ t t ♥ t ♣r♠trs ♦♥t♦♥② ♦♥ t ss ♠♠rs♣s ♥ ♦♥ t t ♥ ts stt♦♥r② strt♦♥s p(θ, ③|①) t sq♥ ♦ t ♥rt ♣r♠trs s r♥ ② t ♠r♥♣♦str♦r strt♦♥ p(θ|①) s ♦rt♠ rs ♦♥ t♦ ♥str♠♥t rst ss ♠♠rs♣ ♦ t ♥s ♦ ① ♥♦t ② ③ = (z1, . . . , zn) ♥ tss♥ t♦r ♦ t ♥s ♥♦t ② ② = (y1, . . . ,yn)

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

s s♠♣r

trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ s

③[r],②[r−1/2] ∼ ③,②|①,θ[r−1]

β[r]kj ,y

j[r][rk] ∼ βkj,y

j[rk]|①,y

[r][rk], ③

[r],β[r]k ,Γ

[r−1]k

π[r] ∼ π|③[r]

Γ[r]k ∼ Γk|②[r], ③[r],

♦rt♠ s s♠♣r

r y[rk] = yi:z

[r]i =k

y[r]i = (y1[r]i , . . . , y

j−1[r]i , y

j+1[r−1/2]i , . . . , y

e[r−1/2]i ) ♥ β

[r]k =

(β[r]k1, . . . ,β

[r]kj−1,β

[r−1]kj+1, . . . ,β

[r−1]ke )

♠r s♠♣♥ ♦ t ss♥ r ss♥ r② s t ♥rt r♥ ♦♥ trt♦♥ ♦ t s s♠♣r t ♦♦s② tsstt♦♥r② strt♦♥ st②s ♥♥ s t s♠♣♥ s ♠♥t♦r② s♦ t str♦♥ ♣♥♥② t♥ ② ♥ ③ ♥ t♥ y

j[rk] ♥ βkj

♠r ♥ t tr♦♣♦st♥s s♠♣r t s♠♣♥s r♦♠ ♥ r ss t t♦ ♦tr ♦♥s r ♠♦r ♦♠♣① ♥ ts♠♣♥ r♦♠ ♥♦s t♦ ♦♠♣t t ♦♥t♦♥ ♣r♦ts ♦ t ss♠♠rs♣s s♦ t♦ ♦♠♣t t ♥tr ♥ ♥ t ♥♠r ♦ srt rs s r ts ♦♠♣tt♦♥ s t♠ ♦♥s♠♥ ♦r t s♠♣♥ r♦♠ ♥ ♥t② ♣r♦r♠ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♥ p(zi,yi|xi, t(r−1)) s stt♦♥r② strt♦♥ ♦♥r♥♥t s♠♣♥ ♦r♥ t♦ t s ♣r♦r♠ ♥ t♦ st♣s rst② t ♠r♥♣r♠tr s s♠♣ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♥p(βkj|①,y↑j(r)

[rk] , ③(r),β

↑j(r)k ,Γk) s stt♦♥r② strt♦♥ ♦♥② t t♥t s

s♥ t♦r s s♠♣ r♦♠ ts ♦♥t♦♥ strt♦♥

♠r s♠♣♥ ♦ t ss♥ r ss♥ r② s t ♥rt r♥ ♦♥ trt♦♥ ♦ t s s♠♣r t ♦♦s② tsstt♦♥r② strt♦♥ st②s ♥♥ s t s♠♣♥ s ♠♥t♦r② s♦ t str♦♥ ♣♥♥② t♥ ② ♥ ③ ♥ t♥ y

j[rk] ♥ βkj

❲ ♥♦ t t ♦r st♣s ♦ t s s♠♣r ♥ ♣♦♥t ♦t t tst♦ s♠♣ r♦♠ ♥ s ♦t st♣s r ♠♦ t♦ ♦t♥ ttr♦♣♦st♥s s♠♣r t ♥ t ♥①t st♦♥

ss ♠♠rs♣ ♥ ss♥ t♦r s♠♣♥

♠ s t♦ s♠♣ r♦♠ ② s♥ t ♥♣♥♥ t♥ t ♥s t t♦rs (③,②) r s② s♠♣ ♦♥t♦♥② ♦♥ (①,θ[r−1]) ♦r♥t♦

p(③,②|①,θ[r−1]) =n∏

i=1

p(zi|xi,θ[r−1])p(yi|xi, zi,θ[r−1]).


❲ ♥♦ t ♦t strt♦♥s ♦ t rt s ♦ t ♦ qt♦♥ z

[r]i s ♥♣♥♥t② s♠♣ r♦♠ t ♦♦♥ ♠t♥♦♠ str

t♦♥zi|xi,θ[r−1] ∼ Mg(ti1(θ

[r−1]), . . . , tig(θ[r−1])),

r tik(θ[r−1]) =

π[r−1]k p(xi;α

[r−1]k )

p(xi;θ[r−1])

s t ♣♦str♦r ♣r♦t② tt xi s ♥

r♥ ② ♦♠♣♦♥♥t k t t ♣r♠trs θ[r−1] y

[r−1/2]i s ♥♣♥♥t② s♠♣ ② r♠r♥ tt t rst c ♠♥ts

♦ yi ♥♦t ② yi r tr♠♥st ♦r ① tr♣t (xi, zi,θ[r−1]) t

zik = 1 s s yi = Ψ(x

i ;α[r−1]k ) ts st d ♠♥ts ♥♦t ②

yi r s♠♣ ♦r♥ t♦ drt ss♥ strt♦♥ Nd(0,Γ[r−1]k )

tr♥t ♦♥ t s♣ Sk(x

i )

p(yi |xi, zi,θ[r−1]) ∝g∏

k=1

(

φd(y

i ;µ[r−1]k ,Σ

[r−1]k )1yi ∈Sk(x

i )

)zik,

r µ[r−1]k = Γ

[r−1]k Γ

−1[r−1]k Ψ(x

i ;α[r−1]k )

♠r ts t♦ ♦♠♣t tik(θ[r−1]) ♦t tt t ♦♠♣tt♦♥ ♦

tik(θ[r−1]) ♥♦s t♦ ♦♠♣t t ♥tr ♥ ♥ ♥ t♦♦ ♠

t♠ ♦♥s♠♥ d s r d > 6 s t s♠♣♥ ♦r♥ t♦ ss♦ ♣r♦r♠ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♦♥ tst② ♥ t ♥ t ♥①t st♦♥

r♥ ♣r♠tr ♥ ss♥ t♦r s♠♣♥

♠ s t s♠♣♥ r♦♠ ♥ ♦♠♣♦s s ♦♦s

p(βkj,yj[rk]|①,y

[r][rk], ③

[r],β[r]k ,Γ

[r−1]k ) = p(βkj|①,y[r][rk], ③

[r],β[r]k ,Γ

[r−1]k )

× p(yj[rk]|①,y[r][rk], ③

[r],β[r]k ,βkj,Γ

[r−1]k ).

❲ ♥♦ t ♦t strt♦♥s ♦ t rt s ♦ t ♦ qt♦♥ ♦♥t♦♥ strt♦♥ ♦ βkj s ♥ t ♥ ♥♥♦♥ ♥tr♣t

s s

p(βkj |①,y[r][rk], ③[r],β

[r]k ,Γ

[r−1]k ) ∝ p(βkj)

n∏

i=1

(

p(xji |y↑j[r]i , z

[r]i ,Γ

[r−1]k ,βkj)

)z[r]ik

.

♦♥t♦♥ strt♦♥ ♦ xji |y↑j[r]i , z

[r]i ,Γ

[r−1]k t z[r]ik = 1 s ♦♥ t

rt s ♦ t ♦ qt♦♥ s ♥ ②

p(xji |y↑j[r]i , z

[r]i ,Γ

[r−1]k ,βkj) =

φ1(xji−µkjσkj

; µi, σ2i )/σkj 1 ≤ j ≤ c

Φ1(b⊕(xji )−µi

σi)− Φ1(

b⊖(xji )−µiσi

) ♦trs,

r t r µi = Γ[r−1]k [j, ]Γ

[r−1]k [, ]−1y

↑j[r]i s t ♦♥t♦♥ ♠♥ ♦

yji Γk[j, ] ♥ t r♦ j ♦ Γk ♣r ♦ t ♠♥t j ♥ Γk[, ] ♥

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

t ♠tr① Γk ♣r ♦ t r♦ ♥ t ♦♠♥ j ♥ r σ2i s t

♦♥t♦♥ r♥ ♦ yji ♥ ② σ2i = 1−Γ

[r−1]k [j, ]Γ

[r−1]k [, ]−1

Γ[r−1]k [, j]

② t ♥♣♥♥ t♥ t ♥s t ♦♥t♦♥ strt♦♥♦ yj[rk] s ①♣t② ♥ s

p(yj[rk]|①,y[r][rk], ③

[r],β[r]k ,βkj,Γ

[r−1]k ) =n∏

i=1

(

p(yji |xji ,y↑j[r]i , z

[r]i ,βkj,Γ

[r−1]k )

)z[r]ik

.

xj s ♦♥t♥♦s r 1 ≤ j ≤ c ♥ z[r]i = k t

♦♥t♦♥ strt♦♥ ♦ yji s tr♠♥st s s

yj[r]i =

xji − µ[r]kj

σ[r]kj

.

xj s srt r c + 1 ≤ j ≤ e ♥ z[r]ik = 1 t

♦♥t♦♥ strt♦♥ ♦ yji s tr♥t ss♥ strt♦♥ s s

p(yji |xji ,y↑j[r]i , z

[r]i ,β

[r]kj ,Γ

[r−1]k ) =

φ1(yji ; µi, σ

2i )

p(xji ;β[r]kj)

1yji∈[b

⊖[r]k (xji ),b

⊕[r]k (xji )]

,

r b⊖[r]k (xji ) = P (xji − 1;β

[r]kj) ♥ b

⊕[r]k (xji ) = P (xji ;β

[r]kj)

♠r ts t♦ s♠♣ t ♠r♥ ♣r♠trs s♠♣♥ ♦ βkjs ♥♦t s② ♣r♦r♠ s♥ t ♥♦r♠③♥ ♦♥st♥t ♥ ♥ s ♥♥♦♥s st♣ s t♥ r♣ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ st ♥ t ♥①t st♦♥ ♦r ♥♦t tt t s♠♣♥ ♦ yj[rk] r♦♠ ss② ♣r♦r♠

❱t♦r ♦ ♣r♦♣♦rt♦♥s s♠♣♥

♠ s t s♠♣♥ r♦♠ s ss ♦r t ♠①tr ♠♦ ♦♥t r②s ♥♦♥ ♥♦r♠t ♣r♦r ♥♦s tt

π|③[r] ∼ Dg

(

♥[r]1 +1

2, . . . , ♥[r]g +

1

2

)

,

r ♥[r]k =∑n

i=1 z[r]ik

♦rrt♦♥ ♠tr① s♠♣♥

♠ s t s♠♣♥ r♦♠ ❲ s t ♣♣r♦ ♣r♦♣♦s ② ❬♦❪♥ t s ♦ s♠♣r♠tr ss♥ ♦♣ s ♥t♦ t♦ st♣srst② ♦r♥ ♠tr① s ♥rt ② ts ①♣t ♣♦str♦r strt♦♥ ♥s♦♥② t ♦rrt♦♥ ♠tr① s ② ♥♦r♠③♥ t ♦r♥ ♠tr①❲♥ (②, ③) r ♥♦♥ r ♥ t ♥♦♥ s ♦ ♠trt ss♥


♠①tr ♠♦ t ♥♦♥ ♠♥s s t s♠♣♥ ♦r♥ t♦ Γk|②[r], ③[r] s♣r♦r♠ ② t t♦ ♦♦♥ st♣s

Λk|②[r], ③[r] ∼ W−1

s0 + ♥[r−1]

k , S0 +∑

i:z[r]i =k

y[r]Ti y

[r]i

∀1 ≤ h, ℓ ≤ e, Γk[h, ℓ] =Λk[h, ℓ]

√

Λk[h, h]Λk[ℓ, ℓ].

♠r ♠♣♥ ♦ t ♦rrt♦♥ ♠trs ♦r t ♦♠♦sst ♠♦s t ♦♠♦sst ♠♦ ss♠s t qt② t♥ t ♦rrt♦♥ ♠trs♥ s s ♦♥② s♠♣ ♦♥ Λ s♦ s r♣ ②

Λ|②[r], ③[r] ∼ W−1

(

s0 + n, S0 +n∑

i=1

y[r]Ti y

[r]i

)

,

♥ ♣t Λk = Λ ♦r k = 1, . . . , g

♦r♥ t♦ ♦t ♠rs ♥ t rst t♦ st♣s ♦ t s s♠♣r♥♦ ts ♦ ② t ♦♦♥ ②r ♠♠ ♦rt♠

tr♦♣♦st♥s s♠♣r

❲♥ s♦♠ st♣s ♦ s s♠♣r ♥♥♦t s② s♠t t ♠② st♦ ♣r♦r♠ t ♥r♥ ②r ♠♠ ♦rt♠ ❬❪ s s ttr♦♣♦st♥s s♠♣r r♣s ♦t s♠♣♥ r♦♠ ③,②|①,θ[r−1]

♥ βkj|①,y[r][rk], ③[r],β

[r]k ,Γ

[r−1]k ♥ ② ♥ ② ♦♥ trt♦♥ ♦ t♦

tr♦♣♦sst♥s st♣s tt ♥♦ t

ss ♠♠rs♣ ♥ ss♥ t♦r s♠♣♥

st♣ s ♣r♦r♠ ♦♥ trt♦♥ ♦ t tr♦♣♦sst♥s ♦rt♠ s ♦rt♠ s ♥♣♥♥t② ♣r♦r♠ t♦ s♠♣ ♦♣ (zi,yi)s♥ t ♥s r ♥♣♥♥t ts stt♦♥r② strt♦♥ s

p(zi,yi|xi,θ[r−1]) ∝g∏

k=1

(

π[r−1]k φe(yi;0,Γ

[r−1]k )1

yi=Ψ(xi ;α[r−1]k )

1yi ∈Sk(x

i )

)zik.

tr♦♣♦sst♥s ♦rt♠ s♠♣s ♥t (z⋆i ,y

⋆i ) ② t ♥str

♠♥t strt♦♥ q1(.|xi,θ[r−1]) ♥♦r♠② s♠♣s z⋆i t♥ s♠♣sy⋆i |z⋆i s ♦♦s ♦♥t♦♥② ♦♥ z⋆ik⋆ = 1 ts ♥str♠♥t strt♦♥ s tr♠♥st ♦r t rst c ♠♥ts ♦ y⋆i ♥♦t ② y⋆i s s y⋆i = Ψ(x

i ;α[r−1]k⋆ )

t s♠♣s t st d ♠♥ts ♦ y⋆i ♥♦t ② y⋆i ♦r♥ t♦ ♠trt♥♣♥♥t ss♥ strt♦♥ tr♥t ♦♥ Sk⋆(x

i ) s

q1(zi,yi|xi,θ[r−1]) =

g∏

k=1

(

1

g

φd(y

i ;0, I)∏e

j=c+1 p(xji ;β

[r−1]kj )

1yi=Ψ(xi ;α

[r−1]k )

1yi ∈Sk(x

i )

)z⋆ik

.

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

♥t s ♣t t t ♣r♦t②

ρ[r]1i = min

∏gk=1

(

πkφe(y⋆i ;0,Γ

[r−1]k )

)z⋆ik

∏gk=1

(

πkφe(y[r−1]i ;0,Γ

[r−1]k )

)z[r−1]ik

q1(z[r−1]i ,y

[r−1]i |xi)

q1(z⋆i ,y⋆i |xi)

; 1

.

s t trt♦♥ [r] ♦ t ♦rt♠ t s♠♣♥ ♦r♥ t♦ s♣r♦r♠ ♦♥ trt♦♥ ♦ t ♦♦♥ tr♦♣♦sst♥s ♦rt♠

s ♦rt♠ s p(zi,yi|xi,θ[r−1]) s stt♦♥r② strt♦♥ ts s rtt♥ s ♦♦s

(z⋆i ,y⋆i ) ∼ q1(zi,yi|xi)

(z[r]i ,y

[r−1/2]i ) =

(z⋆i ,y⋆i ) t ♣r♦t② ρ[r]1i

(z[r−1]i ,y

[r−1]i ) t ♣r♦t② 1− ρ

[r]1i .

♦rt♠ tr♦♣♦sst♥s

r♥ ♣r♠tr s♠♣♥

st♣ s ♣r♦r♠ ♥ t♦ st♣s rst② t s♠♣♥ ♦ β[r]kj ♦r♥

t♦ s ♣r♦r♠ ♦♥ trt♦♥ ♦ t tr♦♣♦sst♥s ♦rt♠ ♦st stt♦♥r② strt♦♥ s p(βkj|①,y[r][rk], ③

[r],β[r]k ,Γk) ♦♥② t s♠♣♥ ♦

yj[r][rk] s ♣r♦r♠ ♦r♥ t♦ ts ♦♥t♦♥ strt♦♥ ♥ ②

♥str♠♥t strt♦♥ ♦ t tr♦♣♦sst♥s ♦rt♠ q2(.|①, ③) s♠♣s ♥t β⋆kj ♦r♥ t♦ t ♣♦str♦r strt♦♥ ♦ βkj ♥r t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ ts strt♦♥ s ①♣t s♥ t ♦♥t ♣r♦r strt♦♥s r s ♦

q2(.|①, ③) = p(βkj|①, ③,Γk = I).

s ♦r♥ t♦ t ♥t β⋆kj s ♣t t t ♣r♦t②

ρ[r]2 = min

p(β⋆kj)q2(β[r−1]kj |①, ③)

p(β[r−1]kj )q2(β

⋆kj |①, ③)

∏

i:z[r]i =k

p(yji |xji ,y

↑j[r]i , zi,β

⋆kj ,Γ

[r−1]k )

p(yji |xji ,y

↑j[r]i , zi,β

[r−1]kj ,Γ

[r−1]k )

; 1

.

s t trt♦♥ [r] ♦ t ♦rt♠ t s♠♣♥ r♦♠ s ♣r♦r♠ ♦♥ trt♦♥ ♦ t ♦♦♥ tr♦♣♦sst♥s ♦rt♠


s ♦rt♠ s p(βkj|x[rk],y[r][rk], ③,β

[r]k ,Γk) s stt♦♥r② strt♦♥ t

s rtt♥ s ♦♦s

β⋆kj ∼ q2(βkj|①, ③)

β[r]kj =

β⋆kj t ♣r♦t② ρ[r]2

β[r−1]kj t ♣r♦t② 1− ρ

[r]2 .

♦rt♠ tr♦♣♦sst♥s

♠r ♥str♠♥t strt♦♥s ♦t tt t s♠r r t ♥trss ♣♥♥s ♦ t r xi t ♦sr ♦ t stt♦♥r② strt♦♥s rt ♥str♠♥t strt♦♥s ♦ ♦t tr♦♣♦sst♥s ♦rt♠s

st♥ ♣r♦♠

st♥ ♣r♦♠ s ♥r② s♦ ② s♣ ♣r♦rs ❬t❪♦r s ♦♥ t r♠♥t ♦♣ ♥ ❬P❪ ts t♥qs r ♣r♥♣② ♠♣t♥ ♥ g s ♥♦♥

❲♥ t ♠♦ s s t♦ str t ♥♠r ♦ sss s ♥♥♦♥ ♥ t♠♦ st♦♥ s ♣r♦r♠ ② t rtr♦♥ s♠t♥♦s② ♦s t st♥ ♣♥♦♠♥♦♥ ♥ ♦♥ t ♦♥ ♥ ts rtr♦♥ sts qts♣rt sss ♥ t s♠♣ s③ s s♠ s♦ t st♥ s ♥♦t ♣rs♥t♥ ♣rt s ♦ t ss s♣rt② ♥ t ♦tr ♥ ♥ t ♥ st♠♦r sss ♥ t s♠♣ s③ ♥rss t st♥ ♣r♦♠ s stts♥ ts ♣♥♦♠♥♦♥ ♥ss s②♠♣t♦t②

♦s② ♥ t ♥♠r ♦ sss s ① ♥ t s③ ♦ s♠♣ s s♠t st♥ ♣r♦♠ ♥ ♦r ♥ s s ♦r s ♥tr② t♦ st ♣r♦rs t ♥ ❬t❪


♥ ♦rr t♦ strt t ♣r♦♣rts ♦ t ♠♦ t♦ ♥♠r ①♣r♠♥tsr ♣r♦r♠ rst ♦♥ ♦♥ssts ♥ s♠t♥ t ♦r♥ t♦ t ♣r♦♣♦s♠♦ ♥ t♦ st② t ♦♥r♥ ♦ t st♠ts s♦♥ ♦♥ ♦♥ssts ♥s♠t♥ t ♦r♥ t♦ ♠①tr ♦ P♦ss♦♥ strt♦♥s ❬❪ ♥ ♦rr t♦s♦ t r♦st♥ss ♦ t ♣r♦♣♦s ♠♦ st♠t s ♦♠♣t ② r♥t ♣r♠trs s♠♣ ② t s ♦rt♠

①♣r♠♥t ♦♥t♦♥s

♦r stt♦♥ s♠♣s r ♥rt ♥ t ♦rt♠ s ♥t③t t ♠①♠♠ ♦♦ st♠t ♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦ r♥♥ s ♣r♦r♠ r♥ trt♦♥s ♥ t ♣r♠tr ♥t③t♦♥ sr♥t ♥ t ♥trss ♣♥♥s r s♠ ♦rt♠ s st♦♣♣ tr

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

trt♦♥s ♠①♠♠ ♣♦str♦r st♠t s ♣♣r♦①♠t ② t ♠♥♦ t s♠♣ ♣r♠trs r r♥ s ♣♣r♦①♠t trt♦♥s ♦ ♦♥tr♦ ♠t♦

♠t♦♥ ① rs ♦♥ ♦♥t♥♦s ♦♥ ♥tr ♥ ♦♥ ♥r②❲ ♦♥sr t ♠①tr ♠♦ ♦ ss♥ ♦♣s t ♥ ①♠♣ ♥♦♠♣♦s t ♦♥ ♦♥t♥♦s r ♦♥ ♥tr r ♥ ♦♥ ♥r② rr strts t rs♥ ♦r ♦ t r r♥ ♦t ♠♦ t t ♠①♠♠ ♣♦str♦r st♠t r♦♠ t ♠♦ t t tr ♣r♠trs ♦r♥ t♦ t s♠♣ s③ ♥ t ♠① s s s♠t♦♥ strtst ♦♦ ♦r ♦ t tr♦♣♦st♥s ♦rt♠ rtr♠♦r t ♣♣r♦①♠t♦♥ ♦ t ♠①♠♠ ♣♦str♦r st♠t ② t ♠♥ ♦ t ♣r♠trss♠♣ ② ts ♦rt♠ s ♥t

100 200 400 800 1600

0.00

0.05

0.10

0.15

0.20

size of sample

Kul

lbac

k−Le

ible

r di

verg

ence

r rs ♦ t r r♥ ♦ t ♠♦ t t♠①♠♠ ♣♦str♦r st♠t r♦♠ t ♠♦ t t tr ♣r♠tr

♠t♦♥ ♦st♥ss ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s r♥ts ①♣r♠♥ts t r s♠♣ ♦r♥ t♦ rt P♦ss♦♥ ♠①tr ♠♦❬❪ ♦s t ♠r♥ ♣r♠trs r ♥♦t ② αk = (λk1, λk2, λk3) s♠t♦♥ s ♣r♦r♠ t t ♦♦♥ s ♦ t ♣r♠trs

π = (1/3, 2/3), λ1h = h ♥ λ2h = 3 + h, ♦r h = 1, 2, 3.

rr♦r rt ♦ ts ♠♦ ♦♠♣t t t ②s r s q t♦ 9.5% stss♦ tt t ①t② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t♦ ♥t②t ts s♠t t ♥ t r r♥ ♦♠s r② s♠♥ t s③ ♦ t s♠♣ ♥rss rtr♠♦r t rr♦r rt ♦ t ♠♦ s♠st♦ ♦♥r t♦ st tt t rr t♥ t t♦rt ♦♥ 9.5% ❲ s♦♥♦t tt t ♠r♥ ♣r♠trs ♦ ♦t ♦♠♣♦♥♥ts ♥ t ♦rrt♦♥ ♦♥tss♠ t♦ ♦♥r t♦ tr tr s

♥②ss ♦ tr r t sts

❲ ♥♦ str tr r t sts ② s♥ t ♠①tr ♠♦ ♦ ss♥♦♣s ♣r♠trs r st♠t t tr♦♣♦sts ♦rt♠♥t③ ♦♥ t ♠①♠♠ ♦♦ st♠t ♦ t ♦♥t♦♥ ♥♣♥♥


50 100 200 400 800

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

size of sample

Kul

lbac

k−Le

ible

r di

verg

ence

50 100 200 400 800

0.10

0.15

0.20

0.25

0.30

size of sample

Bay

es' e

rror

50 100 200 400 800

24

68

10

size of sample

β 11

50 100 200 400 800

0.0

0.2

0.4

0.6

0.8

size of sample

Γ 1[1

,2]

r sts ♦ ♠t♦♥ r r♥ ♦ t st♠t ♠♦ r♦♠ t tr ♦♥ rr♦r rt ♦ t st♠t ♠♦ ❱♦ t rst ♠r♥ ♣r♠tr ♦r t ss ❱ ♦ t ♦rrt♦♥ ♦♥tt♥ ♦t rs ♦r ss

♠♦ r♥♥ s ♣r♦r♠ r♥ trt♦♥s ♥ t ♣r♠tr ♥t③t♦♥ s r♥t ♥ t ♥trss ♣♥♥s r s♠ ♦rt♠ sst♦♣♣ tr trt♦♥s ♥ t st♠t s ♦t♥ ② t♥ t ♠♥ ♦ ts♠♣ ♣r♠trs ♠♦ st♦♥ s ♣r♦r♠ ② s♥ t♦ ♥♦r♠t♦♥rtr rtr♦♥ ❬❪ rtr♦♥ ❬❪ ♦♠♣t ♦♥ t ♠①♠♠ ♣♦str♦r st♠t

r s♦rr t st

t

s t st ❬♦r❪ srs ♥s ② ♦♦ tsts rt♦t t♦ s♥st t♦ r s♦rrs tt ♠t rs r♦♠ ①ss ♦♦♦♥s♠♣t♦♥ ♦♥t♥♦s rs ♥ ② t ♥♠r ♦ qrt♣♥t q♥ts♦ ♦♦ rs r♥ ♣r ② ♦♥ ♥tr r

♦ st♦♥

❲ st♠t t tr ♠①tr ♠♦s ♦♥t♦♥ ♥♣♥♥ ♦♥ tr♦sst ss♥ ♦♣ ♠①tr ♥ ♦♠♦sst ss♥ ♦♣ ♠①tr ♦r r♥t ♥♠rs ♦ sss ♣rs♥ts t s ♦ ♦t s ♥♦r♠t♦♥

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

rtr s ♦ ♦t rtr ♦t♥ t t ♦♠♣♦♥♥t ♦♠♦sst♠①tr ♠♦ ♦ ss♥ ♦♣s r t st ♦♥s ♦r ♥♦t tt t tr♠♦s st t♦ ♦♠♣♦♥♥ts

♦♥ ♥♣t

tr♦ ♦♠♦

♦♥ ♥♣t tr♦ ♦♠♦

❱s ♦ t ♥ rtr ♦r t tr ♠①tr ♠♦s st♠t♦♥ t r s♦rr t st

♥tr♣rtt♦♥ ♦ t st ♠♦

❲ ♥♦ sr t st ♠♦ ♦r♥ t♦ ♦t rtr t ♦♠♦sst♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ② s♥ t ♠r♥ ♣r♠trs♥ t ♥trss ♣♥♥s s♠♠r③ ② r ♠♦ ♦♥srst♦ sss ♦s t ♠♦rt② ♦♥ π1 = 0.60 r♦♣s t ♥s ♥ str♦♥ ♦♦ ♦♥s♠♣t♦♥ β1r♥s = 10.6 ♥ r s ♦ t ♦♦tsts s♣② ♦r t tsts ♦t ♥ ♠♠t ♠♥♦rt② ss r♦♣s t♥s ♥ s♠ ♦♦ ♦♥s♠♣t♦♥ β2r♥s = 1.36 ♥ s♠r s♦ t ♦♦ tsts ♦r ♦t sss t tr ♦♦♥ ♦♦ tsts r ♣♦st②♦rrt t ♣t ♦♣t ♥ ♠♠t t tst s ♣♦st② ♦rrtt t ♥♠r ♦ ♦♦ r♥s

Gammagt

0 50 100 150 200 250 3000.00

00.

005

0.01

00.

015

0 2 4 6 8 11 14 17 20 23 26 29 32 35 38

Drinks

0.00

0.04

0.08

0.12

πkp(xj |βkj , z = k)

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

inertia: 36 %

iner

tia: 2

1 %

Mcv

Alkphos

Sgpt

SgotGammagt

Drinks

❱rs ♥ t rst ♦rrt♦♥ r ♥ ② Γk

r ♠♠r② ♦ t ♦♠♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t r s♦rr t st ss s s♣② ♥ ♥ ss ♥ r


Prtt♦♥ st②

s t rs r ♥♠r r ♥ s♣② t ♥s ♥tr ss ♠♠rs♣s ♥ t rst ss ♣ ♠♣ ♦r s sss r ♥♦t s♣rt ♥ ts ♠♣ t strtr ♦ t t s ♥♦t r♦t ♦t sr s♣②s t ♥s ♥ t rst ♣ ♠♣ ♦ ss ♥ ts ♠♣sss r ttr s♣rt s♥ t rst ss rs s ♥tr ts♦♥ ss r tr♥s s ♦♥ t t♦♣ ♣rt ♦ t r♣ ♦ t s♦♥ ①ss sr♠♥♥t s s♠♠r② s ♥ r♠♥t t t ss ♥tr♣rtt♦♥ s♥ts ①s s t ② t rs ♥ r♥s r t♠ss sr♠♥♥t♦r♥ t♦ tr ♠r♥ ♣r♠trs

−2 0 2 4 6

−4

−3

−2

−1

01

23

First component (inertia 42%)

Sec

ond

com

pone

nt (

iner

tia 1

8%)

rst ♠♣ ♦ P

−15 −10 −5 0

−6

−4

−2

02

46

inertia 36 %

iner

tia 2

1 %

rst ♠♣ ♦ t P ♦ t ss

r ❱s③t♦♥ ♦ t ♣rtt♦♥ ② t ♦♠♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t r s♦rr t st ss s r♥ ② rs ♥ ss ② r tr♥s

♦t tt t ♣rtt♦♥s ♦t♥ ② t tr ♦♠♣♦♥♥t ♠♦s r s♠rt ♥♦t ♥t s s♦♥ ②

tr♦

♦♠♦ ♦♠♦

♦♥ ♥♣t

♦♠♦ ♦♠♦

♦♥s♦♥ ♠trs t♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♦sst ♦♠♣♦♥♥t ♠♦ ♥ t ♣rtt♦♥ ♦t♥ ② t tr♦sst ♦♠♣♦♥♥t ♠♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦

♦♥s♦♥

♥ ts t st t ♠①tr ♠♦ ♦ ss♥ ♦♣s ttr ts t t♦r♥ t♦ t ♥♦r♠t♦♥ rtr t♥ t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♥

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

♦t ♠♦s st t s♠ ♥♠r ♦ sss ♣ ♣r ss ♦s t♦s♠♠r③ t ♥trss ♣♥♥s ♥ t♦ r♥ ♦t t s♣rt♦♥ ♦ ♦tsss ♥ ② ss ♣

❲♥ t st

t

t st ❬+❪ ♦♥t♥s r♥ts ♦ t P♦rts ❱♥♦ ❱r♥ r ♥s ♥ t ♥s sr ② ♥ ♣②s♦♠♦♥t♥♦s rs ① t② ♦t t② tr t② rs sr♦rs r sr ♦① t♦t ♥st② ♦① ♥st② ♣ s♣ts ♦♦♥ ♦♥ ♥tr r qt② ♦ t ♥ t ② ①♣rts ♥s ♦t ♥s r ♦r t r ♥ ♥ str t t st t tr r♥t♠①tr ♠♦s ♦t tt ♦♥ t ♥ ♥♠r s ① ♦ t st②s♥ t s ♥ ♦tr

♦ st♦♥

❲ st♠t t tr ♠①tr ♠♦s ♦♥t♦♥ ♥♣♥♥ ♦♥ tr♦sst ss♥ ♦♣ ♠①tr ♥ ♦♠♦sst ss♥ ♦♣ ♠①tr ♦r r♥t ♥♠rs ♦ sss ♥ ♣rs♥t t s ♦ ♦t s ♥♦r♠t♦♥ rtr ♥ ♦t rtr st♥t② st t ♦♠♣♦♥♥t tr♦sst ♠①tr♠♦ ♦ ss♥ ♦♣s ❲ ♥♦ s♦ tt ts ♠♦ ♦s t♦ s♣rtt t ♥s r♦♠ t r ♦♥s t♥ t ♠♦ ♥tr♣rtt♦♥

♦♥ ♥♣t

tr♦ ♦♠♦

♦♥ ♥♣t tr♦ ♦♠♦

❱s ♦ t ♥ rtr ♦r t tr ♠①tr ♠♦s st♠t♦♥ t ♥ t st

Prtt♦♥ st②

♣rs♥ts t ♦♥s♦♥ ♠trs ♥ ♦rr t♦ ♦♠♣r t r♥ ♦ tst♠t ♣rtt♦♥s ♦r♥ t♦ t tr ♦♥ ♥ ♦♦r s rsts str♥t♥t tt t ♠♦ st tt♥ t t s t ♦♠♣♦♥♥t tr♦sstss♥ ♦♣ ♠①tr ♠♦s ♥ ts ♣rtt♦♥ s t ♦sst t♦ t tr ♦♥

r s♣②s t ♥s ♥ ♣ ♠♣ ♦ ♦t sss st♠t ②t ♦♠♣♦♥♥t tr♦sst ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r♥ t♦ts sttr♣♦ts sss r s♣rt ❲ ♥♦ t ts ♣r♠trs


t r ♥

t r ♥

t r ♥

❱s ♦ t st ♥ ♥① ♥ ♦♥s♦♥ ♠trs t♥ ttr ♣rtt♦♥ ♥ t st♠t ♣rtt♦♥ ② t ♦♠♣♦♥♥t tr♦sstss♥ ♦♣ ♠①tr t tr♦♠♣♦♥♥t ♦♠♦sst ss♥ ♦♣♠①tr t ♦r♦♠♣♦♥♥t ♦♥t♦♥ ♥♣♥♥ ♠①tr

−5 0 5

−15

−10

−5

05

inertia 9.7 %

iner

tia 7

.8 %

P ♦ t ss ♠♣

−5 0 5 10

−10

−5

05

inertia 22.5 %

iner

tia 1

8.8

%

P ♦ t ss ♠♣

r ❱s③t♦♥ ♦ t ♣rtt♦♥ ② t tr♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t ♥ t st ss s r♥ ② rs ♥ ss ② r tr♥s


♦♦♥ ♥tr♣rtt♦♥ s s ♦♥ t ♠r♥ ♣r♠trs ♦ t ♦♠♣♦♥♥ts ♥ ♦♥ t ♥trss ♦rrt♦♥ ♠trs s♠♠r③ ② r ♠♦rt② ss π1 = 0.59 s ♣r♥♣② ♦♠♣♦s t t ♥s s ss srtr③ ② ♦r rts ♦ t② ♣ ♦rs ♥ s♣ts t♥ t♠ ♦ t♠♥♦rt② ss π2 = 0.41 s ♣r♥♣② ♦♠♣♦s ② r ♥s ♠♦rt②ss s rr s ♦r ♦t sr ♦① ♠srs ♥ t ♦♦ rt ♦ttt t ♥ qt② ♦ ♦t sss s s♠r β1qt② = 5.96 ♥ β2qt② = 5.58 ♠♦rt② ss s rtr③ ② str♦♥ ♦rrt♦♥ t♥ ♦t sr ♠srs ♦♣♣♦st t♦ str♦♥ ♦rrt♦♥ t♥ t ♥st② ♥ t② ♠srs ♠♥♦rt② ss ♥r♥s tt t ♥ qt② s ♣♥♥t t rr ♦♦rt ♥ s♠ s ♦r t ♦rs ♥ t② ♠srs

♦♥s♦♥

♥ ts t st t ss♥ ♦♣ ♠①tr ♠♦s ♦s t♦ r t ♥♠r ♦ sss ♥ t♦ ttr t t t rtr♠♦r ts ♠♣t ♦♥ t st♠t

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

volatile.acidity

0.5 1.0 1.5

01

23

residual.sugar

0 10 20 30 40 50 60

0.00

0.04

0.08

0.12


−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

inertia: 31.9 %

iner

tia: 1

3 %

fxd.

vlt.

ctr.

rsd.

chlrfr..tt..

dnst

pH

slph

alchqlty

❱rs ♥ t rst ♦rrt♦♥r ♥ ② Γ2

r ♠♠r② ♦ t tr♦sst ♦♠♣♦♥♥t ss♥ ♦♣ ♠①tr♠♦ ♦r t ♥ t st ss s r♥ ♥ ♥ ss ♥ r

♣rtt♦♥ s s♥♥t s ♦♥ t ♥ sttr♣♦ts ♥ t ♠♦ ♣ tst♠t sss r r♥t s♥ t② r s♣rt ♥② t st♠t♦♥♦ t ♥trss ♣♥♥s ♣s t ♥tr♣rtt♦♥ s♥ ts ♥r♥s t ♥t♥ t ♥ qt② ♦ t ♠♥♦rt② ss ♥ ts ♣②s♦♠ ♣r♦♣rts

♦rst r t st

t

s t st srs ♦rst rs ❬❪ ♥ t ♥♦rtst r♦♥ ♦ P♦rt② s♥ ♠t♦r♦♦ rs s♥ ♦♥t♥♦s rs ♦r ♦t t ❲s②st♠ ♥ t♦ ♦t t ♠t♦r♦♦② t♠♣rtr ♥rt ♠t② t♦ ♥tr rs rt t♦ t s♣t ♦♦r♥ts ♥ tr♥r② ♦♥s ♥t♥ t ♣rs♥ ♦ r♥ t ss♦♥ s♠♠r ♦r ♥♦t s♠♠r ♥t ② ♥ ♦r ♥♦t ♥

♦ st♦♥

♣rs♥ts t s ♦ ♦t s ♥♦r♠t♦♥ rtr ♦r t tr♠①tr ♠♦s ♦r♥ t♦ ♦t rtr t ♠♦ ttr tt♥ t t s t♦♠♦sst ♠①tr ♠♦ ♦ ss♥ ♦♣s t tr ♦♠♣♦♥♥ts


♦♦♥ ♥tr♣rtt♦♥ s s ♦♥ t ♠r♥ ♣r♠trs ♦♥ t ♥trss ♦rrt♦♥ ♠trs s♠♠r③ ♥ r ♠♦rt② ss π1 = 0.57r♦♣s t rs ♦♣ t t♠♣rtr ♥ s♠ rt ♠t② ♠srs ♦ ♥ r s♦♥ ss π2 = 0.26 r♦♣st ♥tr rs s rs r ♦♣ t str♦♥ ♥ ♥ ♥♦ r♥


♦♥ ♥♣t

tr♦ ♦♠♦

♦♥ ♥♣t tr♦ ♦♠♦

❱s ♦ t ♥ rtr ♦r t tr ♠①tr ♠♦s st♠t♦♥ t ♦rst r t st

t ❲ ♠srs t s♠ s ♠♥♦rt② ss π3 = 0.17) r♦♣s ts♠♠r rs ♦♣ t s ♦ ❲ ♠srs ①♣t t ♦♥ t♠♣rtr s ♠♥ t t rt ♠t② s ♥trss ♦rrt♦♥♠tr① ♥r♥s t ♣♥♥s t♥ t s♠♠r ♥ t♠♣rtr ♥s ♦ ♥ ♥② ♥♦t tt t s♣ ♦♦r♥ts r♦② ♦♦t s♠ strt♦♥ ♥ t tr sss

temp

5 10 15 20 25 30

0.00

0.02

0.04

0.06

no yes0.0

0.2

0.4

summer

C 1 C 2 C 3 C 1 C 2 C 3


−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

inertia: 23 %

iner

tia: 1

8.4

%

Xaxs

Yaxs

smmr

wknd

FFMCDMCDC

ISI

temp

RH

wind

rain

❱rs ♥ t rst ♦rrt♦♥r ♥ ② Γk

r ♠♠r② ♦ t ♦♠♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t ♦rst r t st ss s s♣② ♥ r♥ ss ♥ r♥ ss ♥

Prtt♦♥ st②

♦t tt t ♣rtt♦♥s ♦t♥ ② t tr ♠♦s r s♠r t ♥♦t ♥ts s♦♥ ②

♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t

tr♦

♦♠♦ ♦♠♦ ♦♠♦

♦♥ ♥♣t

♦♠♦ ♦♠♦ ♦♠♦

♦♥s♦♥ ♠trs t♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♦ssttr♦♠♣♦♥♥t ♠♦ ♥ t ♣rtt♦♥ ♦t♥ ② t tr♦sst ♦♠♣♦♥♥t ♠♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦

♦♥s♦♥

♠♦ ♣♦♥ts ♦t tr sss ♦ ♦rst rs t s ♠♦r ♣rs t♥ t♦♥t♦♥ ♥♣♥♥ ♠♦ r♦② s♣rts t s♠♠r rs r♦♠ t♦tr ♦♥s ♥ t ♦♠♦sst ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦♥srst♦ ♥s ♦ s♠♠r rs rstrt♦♥s ♦♥ ♦♥ t ♣r♠trs s♣s ♦ t♦ttr t t t t♥ t tr♦sst ss♥ ♦♣ ♠①tr ♠♦ ♦r♥ t♦ ♦t rtr ts ♠♣t s s♥♥t s♥ t ♥♠rs ♦ sss st② ♦t ♠♦s r r♥t

♦♥s♦♥

♠①tr ♠♦ ♦ ss♥ ♦♣s ss t ♣r♦♣rts ♦ ♦♣s ♥♣♥♥t ♦ ♦ t ♠r♥ strt♦♥s ♥ ♦ t ♣♥♥② rt♦♥s s ts♠①tr ♦s t♦ ① ss strt♦♥s ♦♥♥ t♦ t ①♣♦♥♥t ♠② ♦rt ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t ♦r♦r t ts ♥t♦♦♥t t ♥trss ♣♥♥s ♥ ♣♣r♦ s ♦♥ ♣ ♣r ss ♦ tss♥ t♥t r ♦s s♦ t♦ s♠♠r③ t ♠♥ ♥trss ♣♥♥s♥ t♦ s③ t t ② s♥ t ♠♦ ♣r♠trs

r♥ ♦t ♥♠r ①♣r♠♥ts ♥ ♣♣t♦♥s ♣♦♥t ♦t tt ts♠♦ s s♥t② ① t♦ t t r♥ ② ♥ ♦tr ♦♥ rtr♠♦r t ♥r t ss ♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♦r ♥st♥ t rt♦♥♦ t ♥♠r ♦ sss

♥♠r ♦ ♣r♠trs ♥rss t t ♥♠rs ♦ sss ♥ rss♣② s ♦ t ♦rrt♦♥ ♠trs ♦ t ss♥ ♦♣s ♦ ♦ tsr ♣r♦♣♦s ♦♠♦sst rs♦♥ ♦ t ♠♦ ss♠♥ t qt②t♥ t ♦rrt♦♥ ♠trs s ♠♦ ♠② ttr t t t t♥ ttr♦sst ss♥ ♦♣ ♠①tr ♠♦ ♦r t ♥ r ♥ t♥♠r ♦ rs ♥rss ♦ ♠♦r ♣rs♠♦♥♦s ♦rrt♦♥ ♠trs ♦ ♣r♦♣♦s t♦ ♦ ts r ♥ tr ♦rs

♥② t ♠♦ ♥ ♥♦t str ♥♦♥♦r♥ t♦r rs ♥ ♠♦rt♥ t♦ ♠♦ts ♥ ♥ s s t ♠t strt♦♥ ♥t♦♥ s♥♦t ♥ ♥ rt ♦rr t♥ t ♠♦ts ♦ t♦ ♥ ♠t strt♦♥ ♥t♦♥ t ts ♠t♦ s tr ♣♦t♥t ts♦r tt♥t♦♥ s t♦ ♣ t ss♠s rr ♣♥♥s t♥ t

♦♥s♦♥

♠♦ts ♦ t♦ rs ts st♠t♦♥ ♦ s♦ ♦♥ t st♠t♦♥ ♦rt♠♥ ts stt② ♦ t♦ st

♦♥s♦♥ ♦ Prt

❲ s♥ tt t s ♠♣♦rt♥t t♦ ♣r♦r♠ t str ♥②ss ♥ t ♥t s♣ ♦ t rs ♥ ♦rr t♦ ♣r♦ ♠♥♥ rsts ♥ t♠t♦s ♦♥ ♠①tr ♠♦s s r♥t t srs r♦♠ ♦ ♠trtstrt♦♥s ♦r ♠① t

ss♠♣t♦♥ ♦ t ♦♥t♦♥ ♥♣♥♥ t♥ t rs s ♠♥♥ ♠♦ s♥ t ♣r♦s ss strt♦♥s ♦r t ♦♥♠♥s♦♥♠r♥s ♦ t ♦♠♣♦♥♥ts s ♠♦ s r♥t s♣② ♥ t s♠♣ s③ ss♠ ♦r♥ t♦ t ♥♠r ♦ rs ♥ ♥ s s t ♥♦r♠t♦♥♦♥ t ♥trss ♣♥♥② s ♥♦t ♣rs♥t ♥ t t st ♦r t ♥ ♥ssr② t♦ r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥

♠①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s s s♦ ♥ tr♥t t♦ t ♦♥t♦♥ ♥♣♥♥t ♠♦ ♦t tt ts ①t♥s♦♥ ♣r ♦s♣♣rs t♦ ♠♦r ♥t s♥ t ♥♠r ♦ ♣r♠trs st②s ♠t ♦rt ♠♦ ♥tr♣rtt♦♥ ♥ t t♦ ♣r♦r♠ ② t ♣rtt♦♥r s♥ t♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts r ♥♦t ss ♦r t♦♥t♥♦s rs

s♦♥ tr♥t ♦♥ssts ♥ t ♥r♥ ss♥ ♠①tr ♠♦ ♣♣rs s ♠♦r ♠♥♥ ♦r ts ♠t♦ s ♦r t ♣r♠tr st♠t♦♥r♠t② ♠ts t ♥♠r ♦ srt rs

♥ ts ♦♥t①t t♦ ♠♥ ♦ts ♣♣r t♦ s s r t ♠♦ ♠stt♦ ♣r♦ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ts ♦♠♣♦♥♥ts ♥ t♠st ♣r♦ ♠♥♥ ♦♥ts rt♥ t ♥trss ♣♥♥s s♦♥ ♦t ♦ts ♣r♦♣♦s t♦ ♠①tr ♠♦s

rst ♠♦ ♦s t♦ ♣r♦r♠ t str ♥②ss ♦ t sts t ♦♥t♥♦s♥ t♦r rs t rs r♦♠ t ♠t t♥t ss ♠♦ ♦♣ ♦r ♥trss ♣♥♥t t♦r rs ♥ t ♦♠♣♦♥♥ts ♦ ts♠♦ r ♦♠♣♦s t ss♥ ♥ ② ♦st strt♦♥s s♣t②♦ ♦r ♣♣r♦ s t♦ s♠t♥♦s② ♣r♦r♠ t ♠♦ st♦♥ ♥ t ♣r♠trst♠t♦♥ ♥ ♠ ♦rt♠

s♦♥ ♠♦ s ♠①tr ♦ ss♥ ♦♣s s ♠♦ s r② ♥rs♥ t ♣r♦r♠s t str ♥②ss ♦ ♠① t sts t rs ♠tt♥ ♠t strt♦♥ ♥t♦♥s ♦r♦r t ♣r♦s s♦♠ s③t♦♥ t♦♦s t♦s♠♠r③ t ♥trss ♣♥♥s ♥ t♦ s♣② t ♥s ♦r t♠♦ ♦♠♣①t② ♥rss t t ♥♠r ♦ rs ♥ ♦r t ♦♠♦sstrs♦♥ s ts ♠♦ ♣♣rs s ♥♣♣r♦♣rt ♦r t sts t r ♥♠r♦ rs ♦ ♠♦r ♣rs♠♦♥♦s rs♦♥s ♦ ♦♥sr t♦ str s tsts

♥r ♦♥s♦♥ ♥ ♣rs♣ts

♦♥s♦♥

♥ ts tss ♥ ♥trst ♥ t str ♥②ss ♦ ♦♠♣① t♦r ♣rs② ♦s ♦♥ t t♦r ♥ ♠① t sts ♦t s t♦ ♥tr♦ ♠♦s ♣♣r♦s ♥ ♦rr t♦ str s t ②♠♦③♥ t ♥trss ♣♥♥s ♦r♦r ts ♠♦s t♦ s♠♠r③t t strt♦♥ ② ♣r♠trs t♦ tt t ♥tr♣rtt♦♥

♦ ♠♦s ♥ ♣rs♥t t♦ ♣r♦r♠ t str ♥②ss ♦ t♦rt sts ♠♥ s t♦ r♦♣ t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t♦s ♥ t♦ ♣t ♣rs♠♦♥♦s strt♦♥ ♦r ♦ ♦♠♥t♦r♣r♦♠s r qt♦s ♥ t t♦r t sts t ♥trss ♣♥♥sr ♥②③ ♦ t ♣rs♥t ♠♦s sr r♦♠ ts ♣r♦♠ r♥ t ♠♦st♦♥ st♣ ♥ t♦ ②s♥ ♣♣r♦s r ts r t s ♥♦trst t♦ ♣r♦r♠ t ts ♠♦s t str ♥②ss ♦ t st t ♦t♦ rs ♦r ♥ t rs r ♦r♥ ♦r ♥r② ♣♦ss ♥srt♦ ts ♣r♦♠ ♥ ♥ ② t ♠♦s ♦♣s

♠①tr ♠♦ ♦ ss♥ ♦♣s s ♥ ♥tr♦ t♦ str ♠①t sts s ♠♦ ♣r♠ts t♦ ♦t♥ ss ♦♥♠♥s♦♥ ♠r♥s ♦r ♦♠♣♦♥♥ts ♥ t♦ ♠♦③ t ♥trss ♣♥♥s ♥r ♠①tr♠♦ ♦ ss♥ ♦♣s ♦s ♥♦t sr r♦♠ ♦♠♥t♦r ♣r♦♠s t♦ ♣r♦r♠t ♠♦ st♦♥ s ts ♠♦ ♥ ♥ ♥t tr♥t t♦ t ♠♦s♣ t♦ t t♦r t ♥ t rs r ♥r② ♦r ♦r♥ ♥ t♦s t ♦♠♥t♦r ♣r♦♠s ♦ t ♠♦ st♦♥

Prs♣ts

r♦♦t ts tss ♥ ss ② t st ♦ t ♥sr♥ ② t s♠ strt♦♥ ♦r tr♥t ♥t♦♥s ♦ ss ♦ s ❬+ ♥❪

♠♦s ♥ ♥tr♦ ♥ str♥ r♠♦r ♦s② t②♥ s ♥ s♠s♣rs ♦r ♥ sst♦♥ ♦♥t①t ♦r ♦♥ ♥①♣t tt ts ♠♦s ♦t♣r♦r♠ t sr♠♥♥t ♣♣r♦s t ♦strrss♦♥ ♦♥② ♥ ♥s r ♥ tr ♦t s ♠♦r

♣tr

♠t♦s t♥ t sr♠♥♥t ♣♣r♦s s♥ t② ♠♦③ t t strt♦♥ t sr♠♥♥t ♣♣r♦s ♦s ♦♥ t ♦♥rs t♥ sss

♠♦s ♥tr♦ ♥ ts tss ♦ ♠♥ t sts t ♠ss♥ s♥ tr st♠t♦♥ ♥ ♣r♦r♠ ② ♥ ♦rt♠ ♦r ② s s♠♣r r ♥♦♥ t♦ ♠♥ s t

♠♦s st♠t ♦r♥ ♠tr① ♥ rqr r ♥♠r ♦♣r♠trs ♦ t s ♠♣♦rt♥t t♦ ♥tr♦ s♦♠ ♣rs♠♦♥♦s rs♦♥s ♦ t♠①tr ♠♦ ♦ ss♥ ♦♣s ♥ ♦rr t♦ ♠♥ t sts t r ♥♠r♦ rs s ♦♥ t ♦♠tr ♣♣r♦s s ♦r t ss♥ ♠①tr♠♦s ❬❪ s♦♠ ♦♥str♥s ♦ ♦♥ t ♦rrt♦♥ ♠tr① ♦ tss♥ ♦♣s ♦r ts ♣♣r♦ ♦ ♠ t st♠t♦♥ rr tr♣rs♠♦♥♦s ♣♣r♦s ♥rt r♦♠ t ss♥ r♠♦r ♦ s♦ s♦r ♥st♥ t ♠♦s ♦r r t sts ❬❪ ♥② ♥♦tr rsr ①s♦ ♦♥sst ♥ ♥r③t♦♥ ♦ t ♠①tr ♠♦ ♦ ♣♥♥② trs ♥t ♦♣s ♥ t rt strt♦♥ ♦r ♥② ♦♣s ♦ ♠① rs t② ♠t ♦r♦r t ♠♦ st♦♥ st♣ ♦ ts ♠t♦ s ss♥② ts ♣♣r♦ ♦ ♦ ♥ ♥r♥ ② ♠①♠③t♦♥ ♦ t ♦♦② s♥ ♠t♦s ♥rt r♦♠ ❬❪ s ♥♦ ♣r♦r ♥♦r♠t♦♥ ♦ t ts ♠t♦ ♦ ♥r♥

♦rrt♦♥ ♦♥t ♦ t ss♥ ♦♣ s ♦♦ ♣r♦♣rts ♥ t♠r♥ strt♦♥s r st♠t ♦r st t ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts ♦r t ♠①tr ♠♦ ♦ ss♥ ♦♣s s s♠♣r♠tr ♣♣r♦ ♦r ♥st♥ s ♦♥ t ♦rs ♦ ❬♦ ❲❪s♥ t ♣r♦♣rts ♦ t ♦rrt♦♥ ♦♥t ♦ s②♠♣t♦t② r♥t

♥② t ♠①tr ♦ t t♦ ①tr♠ ♣♥♥② strt♦♥s ♦ ♥tr♥t t♦ t ss♥ ♦♣s s ♠♦ ♥ ♥ ② s♥ ♦♣s♥ t ♠①♠♠ ♣♥♥② strt♦♥ ♦ ♥ s t strt♦♥ tt♥s t rét♦♥ ♣♣r ♦♥ ② ♥ s♦♠ ♦♥str♥ts t strtr ♥ tr t ♠♦ st♦♥ ♦ s② ♣r♦r♠

t rts t♦ st ②♦ rt ②♦ ♥r ♦♦ ♠

♥ ♦ tr ♥ s♦t s ♥ ♦ ♥ts tr t♦

s s t ♥ ♦♦rs ♥

♣♣♥①

♣♣♥① ♦ Prt

♥r ♥tt② ♦ t ♠①tr ♦ t t♦

①tr♠ ♣♥♥② strt♦♥s

♦ strt♦♥ s ♥r② ♥t ♥ t ♦ ♦♥t♥s t sttr rs ♦r ♥ t ♦ ♦♥t♥s t st t♦ rs ♥ t st tr♠♦ts ♦ ♣r♦ ts ♣r♦♣♦♣rt② rst② s♦ t ♥r ♥tt② ♦t ♠♦ ♥ ♦t ♦ t ♦♦♥ s♠♣ ss t♦ rs t tr ♠♦ts♥ tr ♥r② rs ♥ ♦♥ t♦ t ♥r ♥tt② ♦ t♠♦

Pr♦♣♦st♦♥ ♦ rs t tr ♠♦ts ♠①tr ♠♦ ♦t t♦ ①tr♠ ♣♥♥② strt♦♥s s ♥r② ♥t ♥ dkb = 2m

kb1 = m

kb2 = 3

Pr♦♦ ♣♣♦s tt tr ①stsαkj = (ρkj, ξkb, τ kb, δkb) ♥ αkj = (ρkj, ξkb, τ kb, δkb)s s

∀xkbi p(x

kbi ;αkb) = p(x

kbi ; αkb).

❲ ♠♦♥strt tt ts qt② ♥♦s tt αkj = αkj ♠♦♥strt♦♥s s♣t ♥ tr ♣rts r tr♠♥ ② t tr ♣♦ssts ♦ (δkb, δkb)qt② ♦♥ rt♦♥ q ♦r ♦t ♣r♠trs ♥♦ rt♦♥ q ♦r ♦t ♣r♠trs ❲ s♦ tt ♥♦s t qt② t♥ t ♣♥♥② rt♦♥s δkb = δkb ♥ t♥ t ♦♥t♥♦s ♣r♠trs s ♥♦sαkb = αkb

• qt② ♦ t ♣♥♥② rt♦♥s δkb = δkb❲t♦t ♦ss ♦ ♥rt② ss♠ tt

∀h, h′ ∈ 1, . . . , 3, h 6= h′ : δh2hkb = 1 ♥ δh2h′

kb = 0.

♥ t rt♦♥ ♥ ② s t♦ t ♦♦♥ s②st♠ ♦ ♥♥ qt♦♥s♦r h ∈ 1, . . . , 3 ♥ h′ ∈ 1, . . . , 3 \ h

(1− ρkb)ξ1hkb ξ

2hkb + ρkbτ

hkb = (1− ρkb)ξ

1hkb ξ

2hkb + ρτhkb

(1− ρkb)ξ1hkb ξ

2h′

kb = (1− ρkb)ξ1hkb ξ

2h′

kb .

♣♣♥① ♣♣♥① ♦ Prt

❲ s t s♦♥ ♥ ♦ t ♣r♦s s②st♠ t t ♦♦♥ s ♦ t ♦♣(h, h′) ♥ s ♦t♥ tt

ξ11kbξ12kb

=ξ11kbξ12kb

♥ξ11kbξ13kb

=ξ11kbξ13kb

.

♦ ξ11kb = ξ11kbξ12kbξ12kb

= ξ11kbξ13kbξ13kb

r s ♥tr♣t ε ∈ R+ s tt ε = ξ12kb

ξ12kb=

ξ13kbξ13kb

❲

r♠♥ tt∑3

h=1 ξ1hkb =

∑3h=1 ξ

1hkb = 1 ♦r♦r

3∑

h=1

ξ1hkb = ξ11kbε+ ξ12kbε+ ξ13kbε = ε.

♦ ε = 1 ❲ ♦♥ tt ξ1hkb = ξ1hkb s♠ rs♦♥♥ s s t♦ ♦t♥tt ξ2hkb = ξ2hkb r♦♠ ts ♦t♥ t qt② t♥ ρkb = ρkb ♥ τhkb = τhkb♥② ♦t♥ tt αkb = αkb

• ♥② ♦♥ rt♦♥ s q t♥ ♦t ♣r♠tr③t♦♥s❲t♦t ♦ss ♦ ♥rt② ss♠ tt δ121kb = δ222kb = δ323kb = 1 ♥ δh2h

′

kb = 0♦trs δ122kb = δ221kb = δ323kb = 1 ♥ δh2h

′

kb = 0r♦♠ t s②st♠ ♦ ♥♥ qt♦♥s ♥ ② ①trt t ♦♦♥ s②st♠

(1− ρkb)ξ13kbξ

21kb = (1− ρkb)ξ

13kb ξ

21kb

(1− ρkb)ξ13kbξ

22kb = (1− ρkb)ξ

13kb ξ

22kb

(1− ρkb)ξ11kbξ

21kb + ρkbτ

1kb = (1− ρkb)ξ

11kb ξ

21kb

(1− ρkb)ξ11kbξ

22kb = (1− ρkb)ξ

11kb ξ

22kb + ρkbτ

1kb.

r♦♠ t rst t♦ ♥s ♦ t ♣r♦s qt♦♥ tt ξ22kb = ξ21kbξ22kbξ21kb

❲

♦♥sr t st t♦ ♥s r ξ22kb s r♣ ② ξ21kbξ22kbξ21kb

♥ r t st ♥ s

♠t♣ ② ξ21kbξ22kb

s

(1− ρkb)ξ11kbξ

21kb + ρkbτ

1kb = (1− ρkb)ξ

11kb ξ

21kb

(1− ρkb)ξ11kbξ

21kb = (1− ρkb)ξ

11kb ξ

21kb + ρkbτ

1kbξ21kbξ22kb.

s ρkbτ 1kb+ ρkbτ1kbξ21kbξ22kb

= 0 s rst s ♥ ♦♥trt♦♥ t t strt ♣♦stt②

♦ t tr♠s ♦ t s ♥♦t ♣♦ss t♦ rs♣t ♥ ♦♥② ♦♥ rt♦♥ sq t♥ ♦t ♣r♠tr③t♦♥s

• ♦ rt♦♥ q t♥ ♦t ♣r♠tr③t♦♥s

♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r

♠♦s

❲t♦t ♦ss ♦ ♥rt② ♦♥sr t ♦♦♥ s②st♠

(1− ρkb)ξ11kbξ

21kb + ρkbτ

1kb = (1− ρkb)ξ

11kb ξ

21kb

(1− ρkb)ξ12kbξ

22kb + ρkbτ

2kb = (1− ρkb)ξ

12kb ξ

22kb

(1− ρkb)ξ13kbξ

23kb + ρkbτ

3kb = (1− ρkb)ξ

13kb ξ

23kb

(1− ρkb)ξ12kbξ

21kb = (1− ρkb)ξ

12kb ξ

21kb + ρkbτ

2kb

(1− ρkb)ξ13kbξ

22kb = (1− ρkb)ξ

13kb ξ

22kb + ρkbτ

3kb

(1− ρkb)ξ11kbξ

23kb = (1− ρkb)ξ

11kb ξ

23kb + ρkbτ

1kb

(1− ρkb)ξ11kbξ

22kb = (1− ρkb)ξ

11kb ξ

22kb

(1− ρkb)ξ12kbξ

23kb = (1− ρkb)ξ

12kb ξ

23kb

(1− ρkb)ξ13kbξ

21kb = (1− ρkb)ξ

13kb ξ

21kb

r♦♠ t ♥s ♥ ♦t♥ tt ξ11kbξ12kb

<ξ11kbξ12kb

r♦♠ t ♥s ♥ ♦t♥

tt ξ11kbξ12kb

>ξ11kbξ12kb

♦ t s ♥♦t ♣♦ss t♦ rs♣t ♥ ♥♦ rt♦♥ s q

t♥ ♦t ♣r♠tr③t♦♥s

Pr♦♣♦st♦♥ r ♥r② rs ♠①tr ♠♦ ♦ t t♦ ①tr♠

♣♥♥② strt♦♥s s ♥r② ♥t ♥ dkb = 3 mkb1 = m

kb2 =

mkb3 = 2

Pr♦♦ ♣♣♦s tt tr ①stαkj = (ρkj, ξkb, τ kb, δkb) ♥ αkj = (ρkj, ξkb, τ kb, δkb)s s

∀xkbi p(x

kbi ;αkb) = p(x

kbi ; αkb).

② rt♥ t s②st♠ t qt♦♥s rt t♦ ♦t♥ tt ∀j =1, . . . , 3 : ξj1kb(1 − ξj1kb) = (1 − ξj1kb)ξ

j1kb s ∀j = 1, . . . , 3 : ξj1kb = ξj1kb ❲

strt♦rr② ♦t♥ t qt② t♥ t ♦trs ♣r♠trs s♦ αkb = αkb

♦♥s♦♥ ♠①tr ♠♦ s st ② s♦♥ ♦ ♠♦ts ♥♦r rs♦ ♦t♥ t ♥r ♥tt② ♦ t ♠♦s ♥ rtt♥ ②s♦♥ ♦ ♠♦ts ♥♦r rs s ♦♥ ♦ t ♦♦♥ ♠♦s t tr ♥r②♦♥ ♥ t t♦ tr♠♦ts ♦♥

♥r ♥tt② ♦ t ♠①tr ♠♦ ♦

♠t♥♦♠ strt♦♥s ♣r ♠♦s

♥r ♥tt② ♦ t ♠♠ ♠♦ t tr ♦s t k0 =r♠♥

kℓkb ♥ t ♠tr① Mb r

Mb(k, h) = ατk0b(h)

kb .

② ♥♦t♥ ② ξb = mink

ℓkb + 1 ♥r②

r♥K Mb = min(g, ξb).

♣♣♥① ♣♣♥① ♦ Prt

♦r♦r② ♣r♠trs ♦ t ♠♠ ♠♦ t tr ♦s r ♥r②♥t ♣ t♦ s♣♣♥ ♣r♦

min(g, ξ1) + min(g, ξ2) + min(g, ξ3) ≥ 2g + 2.

♥r ♥tt② ♦ t ♠♠ ♠♦ t ♠♦r t♥ tr ♦s ♥t s♠ ② tt ❬❪ ♥r③ t rst t ♦s ② ♦sr♥ tt ♦s ♦ t♦r rs ♥ ♦♠♥ ♥t♦ tr t♦r rss ♥ ♣♣② t rs t♦r♠♦r♦r② ❲ ♦♥sr ♠♠ ♠♦ t ♦s r ≥ 3 tr ①sts tr♣rtt♦♥ ♦ t st 1, . . . , ♥t♦ tr s♦♥t ♥♦♥ ♠♣t② ssts S1 S2 ♥S3 s tt γi =

∏

j∈Siξj t

min(g, γ1) + min(g, γ2) + min(g, γ3) ≥ 2g + 2,

t♥ t ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥

♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt

♦♦ ♦ t ♠①tr ♠♦ ♦ ♠t♥♦♠

strt♦♥s ♣r ♠♦s

♥ ts t♦♥ ♣r♦♦ ♦ Pr♦♣♦st♦♥ s ♥ ❲ rst② ♥ ♥♣r♠tr③t♦♥ ♦ t ♦ strt♦♥ tt♥ t ♥trt ♦♠♣tt♦♦ ♦♠♣tt♦♥ ❲ s♦♥② ♥ t ♣r♦r strt♦♥ ♦ t ♥ ♦♣r♠tr③t♦♥ ♦r♥ t♦ t ♦tr ♣r♠tr③t♦♥ r② ♥r♥ trt♦♥ t♥ t ♠ ♠♦s ❲ ♦♥ ② t ♥trt ♦♠♣tt♦♦ ♦♠♣tt♦♥ s t trt rst

♣r♠tr③t♦♥ ♦ t ♦ strt♦♥

❲t♦t ♦ss ♦ ♥rt② ss♠ tt t ♠♥ts ♦ δkb r ♦rr ② rs♥ s ♦ t ♣r♦t② ♠ss ss♦t t♦ t♠ ♥ ♥tr♦ t ♥

♣r♠tr③t♦♥ ♦ akb ♥♦t εkb r εkb ∈ Ekb =[

1mb ; 1

]

×, . . . ,×[

1♠b−ℓkb

; 1]

♥ r εkbh s ♥ ②

εhkb =

aδkbhkb h = 1aδkbhkb∏h−1

h′=1(1−εh

′kb)

♦trs.

♠♠ ♦♥t♦♥ ♣r♦t② ♦ ①b s

p(xb|z, ℓkb, δkb, εkb) =ℓkb∏

h=1

(εhkb)n(h)kb (1− εhkb)

nhkb ,

♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt ♦♦ ♦ t ♠①tr ♠♦

♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

Pr♦♦

p(xb|z, ℓkb, δkb, εkb) = p(xb|z, ℓkb,αkb)

=♠b∏

h=1

(αhkb)♥hkb

=

ℓkj∏

h=1

(α(h)kb )

♥(h)kb

(

α(ℓkb+1)kb

)♥ℓkbkb

= (ε1kb)n(1)kb

ℓkb∏

h=2

[

(εhkb)♥(h)kb

(

h−1∏

h′=1

(1− εhkb)n(h)kb

)]

ℓkb∏

h=1

(1− εhkb)nℓkbkb

=

ℓkb∏

h=1

(εhkb)n(h)kb (1− εhkb)

nhkb .

Pr♦r strt♦♥

♠♠ ♣r♦r strt♦♥ ♦ εkb s

p(εkb|ω, δkb) =♠b

♠b − ℓkb.

Pr♦♦ ❲ r♠♥ tt akb|ω ∼ Dtℓkb+1

(

1, . . . , 1;♠b)

♥ tt

p(akb, δkb|ω) = p(α|ω) = p(εkb, δkb|ω).

♦ t ♣ ♦ t ♣r♦r strt♦♥ ♦ εkb

p(εkb|δkb,ω) =

∏ℓkbh=1(ε

hkb)

γhkb−1(1− εhkb)∑ℓkb+1

h′=h+1(γh

′

kb−1)

∫

εkb∈Ekb

∏ℓkbh=1(ε

hkb)

γhkb−1(1− εhkb)∑ℓkb+1

h′=h+1(γh

′kb−1)dεkb

.

s εhkb ♦♦s tr♥t t strt♦♥ ♦♥ t ♣r♠trs s♣[

1♠b−h+1

, 1]

♥♦t ② Be(γhkb,∑ℓkb+1

h′=h+1(γh′

kb−1)+1) ♦ ssr t ♣♦stt② ♦ t ♣r♠trs♦ t tr♥t t strt♦♥s ♣t γhkb = 1 s♦

p(εkb|δkb,ω) =♠b

♠b − ℓkb.

t♦♥ t♥ ♠ ♠♦s

♠♠ t t ♠♦ t ℓ⊖kb ♠♦s ♥ t ♣r♠trs (δ⊖

kb, ε⊖kb) ♥ t t

♠♦ t ℓkb ♠♦s ♥ t ♣r♠trs (δkb, εkb) ♦t ♠♦s r ♥ s s

♣♣♥① ♣♣♥① ♦ Prt

tt ℓ⊖kb = ℓkb − 1 tt t ℓ⊖kb ♠♦s ♥ t rst ♣r♦ts t s♠♦t♦♥s ∀h ∈ δ⊖

kb, h ∈ δkb ♥ t s♠ ♣r♦t② ♠sss (ε⊖hkb = εhkb, h < ℓkb)s ♠ ♠♦s ♦♦ ts rt♦♥

p(xb|z, ℓkb, δkb, εkb)p(xb|z, ℓ⊖kb, δ

⊖

kb, ε⊖kb)

=(♠b − ℓkb + 1)♥

ℓkb−1

kb −1

(♠b − ℓkb)♥ℓkbkb

(εℓkb)♥(ℓkb)

kb (1− εℓkb)♥ℓkbkb .

Pr♦♦ ❲ strt ② t ♦♦♥ rt♦♥

p(xb|z, ℓkb,αkb)

p(xb|z, ℓ⊖kb,α⊖kb)

=(αℓkbkb )

♥(ℓkb)

kb (αℓkb+1kb )♥

ℓkbkb

(α⊖ℓkbkb )♥

ℓkb−1

kb

.

♦t tt εhkb = ε⊖hkb ♥ h = 1, . . . , ℓkb − 1 s♥ α(h)kb = α

⊖(h)kb ♥ τℓkb(h) =

τℓkb−1(h) ♥ h = 1, . . . , ℓkb − 1 ♥ ② s♥ t r♣r♠tr③t♦♥ ♥ εkbt ♣r♦♦ s ♦♠♣t

♥trt ♦♠♣tt ♦♦

♥trt ♦♠♣tt ♦♦ s ♥② ♣♣r♦①♠t ② ♥t♥t s♠ ♦r t srt ♣r♠trs ♦ t ♠♦s ♦t♦♥s ♥ ② ♣r♦r♠♥ t①t ♦♠♣tt♦♥ ♦♥ t ♦♥t♥♦s ♣r♠trs ②

p(xb|z, ℓkb) ≈(

1

mb − ℓkb

)♥ℓkbkb

ℓkb∏

h=1

Bi(

1♠b−h+1

; ♥(h)kb + 1; ♥hkb + 1

)

♠b − h,

r Bi(x; a, b) = B(1; a, b)−B(x; a, b) B(x; a, b) ♥ t ♥♦♠♣t t ♥t♦♥ ♥ ② B(x; a, b) =

∫ x

0wa(1 − w)bdw r♦♠ t ♣r♦s ①♣rss♦♥ ts s

strt♦rr t♦ ♦t♥ p(xb, z|ω)

Pr♦♦ ♦ Pr♦♣♦st♦♥ ♦r t ♠♦ t ℓkb − 1 ♠♦s t st ♠♦s♦t♦♥s r ♥♦♥ ♥ ♥ ② δ

⊖

kb t♥ t ♦♥t♦♥ ♣r♦t② ♦ xb ♦r ♠♦ t ℓkb ♠♦s s

p(xb|z, ℓkb, δ⊖

kb, εkb) =1

♠b − ℓkb + 1

∑

τ∈1,...,♠b\δ⊖kb


kb, τ,α⊖kb, εkb),

s ② ♣♣r♦①♠t♥ ts s♠ ② ts ♠①♠♠ ♠♥t ♦t♥ tt


kb, εkb) ≈1

♠b − ℓkb + 1p(xb|z, ℓkb, δkb,α⊖

kb, εkb).

② s♥ ♠♠ ♦t♥ tt


kb, εkb)

p(xb|z, ℓ⊖kb, δ⊖

kb, ε⊖kb)

≈ (♠b − ℓkb + 1)♥ℓkb−1

kb −1

(♠b − ℓkb)♥ℓkbkb

(εℓkbkb )♥(ℓkb)

kb (1− εℓkbkb )♥ℓkbkb .

♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt ♦♦ ♦ t ♠①tr ♠♦

♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s

s p(xb|z, ℓkb = 0) = (mb)−nk ② ♣♣②♥ rrs② t ♣r♦s ①♣rss♦♥ ♦t♥ tt

p(xb|z, ℓkb, εkb) ≈(

1

♠b − ℓkj

)♥ℓkbkb

ℓkb∏

h=1

(εhkb)♥(h)kb (1− εhkb)

♥hkb

♠b − h+ 1.

♣♣♥①

♣♣♥① ♦ Prt

♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥

♥ ♦st strt♦♥s

Pr♦♣♦st♦♥ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s s♥r② ♥t

Pr♦♦ ♣♣♦s tr r t♦ ♠①tr ♠♦s ♦ ss♥ ♥ ♦st strt♦♥s♥♦t ② p(xi;θ) ♥ p(xi; θ) s tt

∀xi,g∑

k=1

πkp(xi;αk) =

g∑

k=1

πkp(xi; αk), 0 < πk, πk ≤ 1,

g∑

k=1

πk =

g∑

k=1

πk = 1.

♠ s t♦ ♣r♦ tt θ = θ ♠♦♥strt♦♥ s s♣t ♥ t♦ ♣rts ♥ trst ♦♥ s♦ t qt② ♦ t ss♥ strt♦♥s ♣r♠trs ♥ ♦ t♣r♦♣♦rt♦♥s ♥ t s♦♥ ♦♥ s♦ t qt② ♦ t ♣r♠trs ♦ t ♦strrss♦♥s

♦♥t♥♦s ♣r♠trs ♥ ♣r♦♣♦rt♦♥s❲ s♠ qt♦♥ ♦r t ♣♦ss s ♦ x

i s♦ ♦t♥ tt

∀x

i ,

g∑

k=1

πkφ(x

i ;µk,Σk) =

g∑

k=1

πkφ(x

i ; µk, Σk), 0 < πk, πk ≤ 1,

g∑

k=1

πk =

g∑

k=1

πk = 1.

♥tt② ♦ t ♥t ss♥ ♠①trs ♠♦s s ❬❪ ♦r t♥rt s ♥ ❬❨❪ ♦r t ♠trt s ♥♦s tt g = gπk = πk µk = µk ♥ Σk = Σk

Pr♠trs ♦ t ♦st rrss♦♥st s r ❬❪ tt ∀j = 1 + c, . . . , e ∀(x

i ,xji )

g∑

k=1

fk(x

i )p(xji |x

i ;βkj) =

g∑

k=1

fk(x

i )p(xji |x

i ; βkj)

♣♣♥① ♣♣♥① ♦ Prt

♥♦s tt βkj = βkj r fk(x

i ) = πkφ(x

i ;µk,Σk) t♥ ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s s ♥tt t t♦r ♦ s③ c ♥♦t ② yi = (y1, . . . , yc) r t ♠♥tsr ③r♦ ①♣t t ♠♥t j′ s q t♦ a ❲t♦t ♦ss ♦ ♥rt② ♦♥sr tt t fk(yi ) r ♦rr s tt Σ−1

k (j′, j′) < Σ−1k+1(j

′, j′)r♦♠ qt♦♥ tt

g∑

k=1

fk(y

i )α1(βkj|yi ) =g∑

k=1

fk(y

i )α1(βkj|yi ),

t(

α1(βkj|yi ))−1

= 1 +∑mj

h=2 exp(β0hkj + βj

′hkj a) ❲ t ♦

qt♦♥ ② f1(yi )α1(β1j|yi ) ts

1 +

g∑

k=2

fk(y

i )α1(βkj|yi )f1(yi )α1(β1j|yi )

=α1(β1j|yi )α1(β1j|yi )

+

g∑

k=2

fk(y

i )α1(βkj|yi )f1(yi )α1(β1j|yi )

.

tt♥ a → ∞∑g

k=2

fk(y

i )α1(βkj |y

i )

f1(yi )α1(β1j |y

i )= 0 ♥

∑gk=2

fk(y

i )α1(βkj |y

i )

f1(yi )α1(β1j |y

i )= 0 s♥

t fk(yi ) r ♦rr ❲t♦t ♦ss ♦ ♥rt② mj > 2 ss♠ ttβj

′h1j > βj

′h+11j 1 < h < mj

lima→∞

α1(β1j|yi )α1(β1j|yi )

= lima→∞

exp(

(βj′2kj − βj

′2kj )a+ (β02

kj − β02kj ))

= 1.

♦ qt♦♥ ♥♦s tt βj′2kj = βj

′2kj ♥ β02

kj = β02kj ② r♣t♥

ts r♠♥t ♦r h = 3, . . . ,mj t♥ ♦r j = 1, . . . , ♦♥ ttβ1j = β1j ② r♣t♥ ts r♠♥t ♦r j = 1 + , . . . , + t♥ ♦rk = 2, . . . , g ♦♥ tt qt♦♥ s tr t♥ θ = θ

♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥

♦♣s

♠♦ ♥tt② s ♣r♦ ② t♦ ♣r♦♣♦st♦♥s rst ♣r♦♣♦st♦♥♣r♦s t ♠♦ ♥tt② ♥ t rs r ♦♥t♥♦s ♥♦r ♥trs ♣r♦♣♦st♦♥ ♣rs♥ts t rs♦♥♥ ♥ s♠♣ s s♥ t ♦s ♥♦t ♦♥srt ♦r♥ rs s♦♥ ♣r♦♣♦st♦♥ ♣r♦s tt t ♠♦ rqrs t st♦♥ ♦♥t♥♦s ♦r ♥tr r t♦ ♥t

Pr♦♣♦st♦♥ ♥tt② t ♦♥t♥♦s ♥ ♥tr rs ♠①tr ♠♦ ♦ ss♥ ♦♣s s ② ♥t ❬❪ t rs r♦♥t♥♦s ♥ ♥tr ♦♥s t ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts rss♥ ♦r P♦ss♦♥ strt♦♥s s

∀x ∈ Rc × N

d,

g∑

k=1

πkp(x;αk) =

g′∑

k=1

π′kp(x;α

′k)

⇒ g = g′, π = π′, α = α′.

♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s

Pr♦♦ ♥tt② ♦ t ♠trt ss♥ ♠①tr ♠♦s ♥ ♦ t♥rt P♦ss♦♥ ♠①tr ♠♦ ❬ ❨❪ ♥♦s tt ♠♣s

g = g′, π = π′, βkj = β′kj ♥ Γk = Γ

′k.

❲ ♥♦ s♦ tt Γk = Γ′k ♥ Γk = Γ

′k

t j ∈ 1, . . . , c ♥ h ∈ c + 1, . . . , e ❲ ♥♦t ② ρk = Γk(j, h)

ρ′k = Γ′k(j, h) vk = Φ−1

1 (P (xj;βkj)) εk(xj) = πk

φ1(vk)σkj

ak =b⊕k (xj)−ρkvk√

1−ρ2k♥

a′k =b⊕k (xj)−ρ′kvk√

1−ρ′2k ❲t♦t ♦ss ♦ ♥rt② ♦rr t ♦♠♣♦♥♥ts s s

σkj > σk+1j ♥ σkj = σk+1j t♥ µkj > µk+1j t♥ ♠♣s tt

1 +

g∑

k=2

(εk(xj)Φ(ak))/(ε1(x

j)Φ(a1)) =

g∑

k=1

εk(xj)Φ(a′k)/(ε1(x

j)Φ(a1)).

t γt = (xj, xh) ∈ R× N : a1 = t ♥ tt♥ xh → ∞ s s (xj, xh) ∈ γt

∀t,∫ a′1tφ(u)du

Φ(t)= 0.

s a′1 = a1 s♦ ρ′1 = ρ1 ♣t♥ ts r♠♥t ♦r k = 2, . . . , g ♥ ♦r t♦♣s (j, h) ♦♥ tt Γk = Γ

′k

❲♥ ♦t rs r ♥tr s t s♠ r♠♥t t γ(t,ξ) = (xj, xh) ∈N × N : a1 ∈ B(t, ξ) ♦t tt ρ1 6= ρ′1 t♥ ∃n0 s s ∀xj > n0 a

′1 > t + ξ

tt♥ xh → ∞ s s (xj, xh) ∈ γ(t,ξ) ♦t♥ t ♦♦♥ ♦♥trt♦♥

∫ a′1t+ξ

φ(u)du

Φ(t− ξ)= 0 ♥

∫ a′1t+ξ

φ(u)du

Φ(t− ξ)> 0.

♦ a′1 = a1 t♥ ρ1 = ρ′1 ♣t♥ ts r♠♥t ♦r k = 2, . . . , g ♥ ♦r t♦♣s (j, h) ♦♥ tt Γk = Γ

′k

Pr♦♣♦st♦♥ ♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♠①tr ♠♦ ♦ ss♥ ♦♣s s ② ♥t ❬❪ t st ♦♥ r s ♦♥t♥♦s ♦r ♥tr

Pr♦♦ ♥ ts ♣r♦♦ ♦♥sr ♦♥② ♦♥ ♦♥t♥♦s r ♥ t♦ ♥r② rs ♦s② t s♠ rs♦♥♥ ♥ ①t♥ t♦ t ♦tr ss ❲ ♥♦s♦ tt Γk = Γ

′k ♥ Γk = Γ

′k

t j = 1 ♥ t h ∈ 2, 3 ❲ ♥♦t ρk = Γk(j, h) ρ′k = Γ′k(j, h) vk =

Φ−11 (P (xj;βkj)) εk(x

j) = πkφ(vk;0,1)σkj

ak =b⊕k (xj)−ρkvk√

1−ρ2k♥ a′k =

b′⊕k (xj)−ρ′kvk√1−ρ′2k

❲t♦t

♦ss ♦ ♥rt② ♦rr t ♦♠♣♦♥♥ts s s σkj > σ[k+1]j ♥ σkj = σ[k+1]j

t♥ µkj > µ[k+1]j ♦t tt ♠♣s tt

1 +

g∑

k=2

(εk(xj)Φ(ak))/(ε1(x

j)Φ(a1)) =

g∑

k=1

εk(xj)Φ(a′k)/(ε1(x

j)Φ(a1)).

♣♣♥① ♣♣♥① ♦ Prt

tt♥ x1 → ∞ ♥ ss♠♥ tt ρk > 0 t♥ Φ(a′k)

Φ(ak)= 1 ♦ s♥(ρk) = s♥(ρ′k).

② ♥♦t♥ κ = lima→∞

φ(a)Φ(a)

♥ tt♥ x1 → ∞ κ 1κ

φ(a′k)

φ(ak)= 1. s a′1 = a1 s♦ ρ′1 = ρ1

♥ b⊕k (xj) = b′⊕k (xj) s♦ βkh = β′

kh♦t tt t s♠ rst ♥ ♦t♥ ② t♥♥ x1 t♦ −∞ s ρk < 0

♣t♥ ts r♠♥t ♦r k = 2, . . . , g ♥ ♦r t ♦♣s (j, h) ♦♥tt Γk = Γ

′k t♥ Γk = Γ

′k

♦r♣②

❬❪ ③③♥ ♥ ❲ ♦♠♥ ♦♦ t s♦♠ t ♦♥ t t②sr ♣♣ ttsts

❬❪ P ♥rs♥ ♦r♥ ♥ ♥ ttst ♠♦s s ♦♥ ♦♥t♥ ♣r♦sss ♣r♥r rs ♥ ttsts ♣r♥r❱r ❨♦r

❬❪ ♠ ♥ ② ♠♥ str♥ ♦rt♠ ♦r ♠① ♥♠r♥ t♦r t t ♥♦ ♥♥r♥

❬r❪ rst t♦r t ♥②ss ♦♠ ♦♥ ❲② ♦♥s

❬❪ ♥♦r♠t♦♥ t♦r② ♥ ♥ ①t♥s♦♥ ♦ t ♠①♠♠♦♦ ♣r♥♣ ♥ ♦♥ ♥tr♥t♦♥ ②♠♣♦s♠ ♦♥ ♥♦r♠t♦♥ ♦r② ss♦r ♣s é♠ ó ♣st

❬❪ ♠♥ ts ♥ ♦s ♥tt② ♦ ♣r♠trs♥ t♥t strtr ♠♦s t ♠♥② ♦sr rs ♥♥s♦ ttsts

❬❱❪ rtr ♥ ❱ssts ♠♥s ♥ts ♦ rs♥ ♥ Pr♦♥s ♦ t t♥t ♥♥ s②♠♣♦s♠ ♦♥ srt ♦rt♠s ♣s ♦t② ♦r ♥str♥ ♣♣ t♠ts

❬❪ r ♣r♦st str♥ ♠♦ ♦r rs ♦ ♠① t②♣t② ♥ ♥tt②

❬❪ ♦②r♦♥ ♥ r♥t ♦s str♥ ♦ ♠♥s♦♥ t r ♦♠♣tt♦♥ ttsts ♥ t ♥②ss

❬❪ r♥ ① ♥ ♦rt ssss♥ ♠①tr ♠♦ ♦rstr♥ t t ♥trt ♦♠♣t ♦♦ Pttr♥ ♥②ss♥ ♥ ♥t♥ r♥st♦♥s ♦♥

♦r♣②

❬❪ r♥ ① ♥ ♦rt ①t ♥ ♦♥t r♦ t♦♥s ♦ ♥trt ♦♦s ♦r t t♥t ss ♠♦ ♦r♥ ♦ttst P♥♥♥ ♥ ♥r♥

❬❪ ♥ ♥ ♥tr ♥ ♦rt♠ ♦rs♠ ♥ ♥♦♥♣r♠tr st♠t♦♥ ♥ ♠trt ♠①trs ♦r♥♦ ♦♠♣tt♦♥ ♥ r♣ ttsts

❬❨❪ ♥ ♥tr ♥ ❨♦♥ ♠①t♦♦s ♥ ♣ ♦r ♥②③♥ ♥t ♠①tr ♠♦s ♦r♥ ♦ ttst♦tr

❬r❪ P r♥ sr② ♦ str♥ t ♠♥♥ t♥qs ♥ r♦♣♥♠t♠♥s♦♥ t ♣s ♣r♥r

❬❪ ♦♥♥♥s rt ♠r ♥ st③ ♠r ♣t♠③t♦♥ t♦rt ♥ ♣rt s♣ts

❬❪ r♥ ♥r② ♥ t ♠①♠♠ ♦♦ st♠t♦♥ ♦ ♥rt ss♥ ♠①trs ♦r r♦♣ t ♥ ♦r ♦ t ♦rt♠ ♥♥♥ ♦r♥ ♦ ttsts

❬❪ r♥ ♥ ♦r♠ ss♥ Prs♠♦♥♦s str♥ ♦s ♥r♥t ♥ t ② Pr♦t♦♥ ttsts ♥ ♦♠♣t♥♣ ♥ ♣rss

❬❪ r♥r ♦ ♥ ❳ ♥ ♦♥ ♦r♥ ♠trs ♥ tr♠s ♦ st♥r t♦♥s ♥ ♦rrt♦♥s t ♣♣t♦♥t♦ sr♥ ttst ♥

❬♦③❪ ♦③♦♥ ♦ st♦♥ ♥ s ♥♦r♠t♦♥ rtr♦♥ t ♥r t♦r② ♥ ts ♥②t ①t♥s♦♥s Ps②♦♠tr

❬❪ ♥ ♥ tr② ♦s ss♥ ♥ ♥♦♥ss♥ str♥ ♦♠trs ♣s

❬❪ r♥t ♥ ♦♥ rt♦♥ ♦ t ♦rt♠ Prss ♣s♦♥ ♦rt♠ ♦♠♣tt♦♥ ttsts ♥ t ♥②ss

❬+❪ P r② tr② ① ♦ ♥ ♦ttr♦ ♦♠♥♥ ♠①tr ♦♠♣♦♥♥ts ♦r str♥ ♦r♥ ♦ ♦♠♣tt♦♥♥ r♣ ttsts

❬r❪ ❱ rt♥♦ Prs♦♥ ♦♠♠♥t♦♥ s♦r s♠

❬+❪ P ♦rt③ rr ♠ t♦s ♥ s ♦♥ ♥ ♣rr♥s ② t ♠♥♥ r♦♠ ♣②s♦♠ ♣r♦♣rtss♦♥ ♣♣♦rt ②st♠s

♦r♣②

❬+❪ ① ♦t t ♥ ♦rt♠s ♦r ♠①trsttst ♥ ♥♠r s♣ts ♣♣♦rt r ♥r

❬❪ ① ♥ ♦t st♦st ♣♣r♦①♠t♦♥ t②♣ ♦rt♠ ♦r t ♠①tr ♣r♦♠ t♦sts ♥ ♥tr♥t♦♥ ♦r♥♦ Pr♦t② ♥ t♦st Pr♦sss

❬❪ ① ♥ ♦rt str♥ rtr ♦r srt t ♥t♥t ss ♠♦s ♦r♥ ♦ sst♦♥

❬❪ ① ♥ ♦rt sst♦♥ ♦rt♠ ♦r str♥♥ t♦ st♦st rs♦♥s ♦♠♣tt♦♥ ttsts t ♥②ss

❬❪ ① ♥ ♦rt ss♥ ♣rs♠♦♥♦s str♥ ♠♦sPttr♥ ♦♥t♦♥

❬❪ ♥tr ♥ ♥ st♠t♦♥ ♦r ♦♥t♦♥♥♣♥♥ ♠trt ♥t ♠①tr ♠♦s

❬❪ ♥t ❱ ♥t③♠♦♥t ♥ r♦ rt♦♦♥ r♦tt♦♥ ♥P❳ ♥s ♥ t ♥②ss ♥ sst♦♥

❬❪ ♦ ♥ ♣♣r♦①♠t♥ srt ♣r♦t② strt♦♥st ♣♥♥ trs ♥♦r♠t♦♥ ♦r② r♥st♦♥s ♦♥

❬❪ P ♦rt③ ♥ ♦rs t ♠♥♥ ♣♣r♦ t♦ ♣rt ♦rst rss♥ ♠t♦r♦♦ t

❬❪ ♦rt ♥ r st♠t♦♥ ♥ srt ♣r♠tr ♠♦s ttst ♥

❬❩+❪ ♣ ö♦♣ ❩♥ t ♠s♣rs r♥♥♦♠ ♣rss ♠r

❬❩❪ ③r♥ ♥ ❩r③② ♣♣t♦♥ ♦ r♦ sts ♥ t ♣rs♠♣t ♥♦ss ♦ r♥r② s②st♠ sss rt ♥t♥ ♥ rt② ♥ ♦♠♣t♥ ②st♠s t ♥tr♥t♦♥ ♦♥r♥Pr♦♥s ♣s

❬t❪ tr♥♦ ♥ rt ss t ♦♥ ♥ ♥♥ ♦♥t♦♥

❬❪ P ♠♣str r ♥ ♥ ①♠♠ ♦♦ r♦♠♥♦♠♣t t t ♦rt♠ ♦r♥ ♦ t ♦② ttst♦t② rs t♦♦♦ ♣s

♦r♣②

❬❪ ♥ r♣② ♥ ♦♥② ❯s♥ ♥ t t♦ ♣t sst♦♥ rs t ♣♣t♦♥s ♥ ♦♦ t♥tt② sts♦r♥ ♦ t ♦② ttst ♦t② rs ♣♣ ttsts

❬❪ s♣♥ ♥ ♥♠♥ ❯s♥ t♥t ss ♦s t♦rtr③ ♥ ssss t rr♦r ♥ srt sr♠♥ts ♦♠trs ♣♣

❬❪ rtt ♥ s ♥ t str ♥②ss♦♥♦♥ ② t♦♥

❬❪ rtt ♥t ♠①tr ♠♦ ♦r t str♥ ♦ ♠①♠♦t ttsts Pr♦t② ttrs

❬♦r❪ ♦rs②t P srs tt♣rs♠❯P sr t

❬♦r❪ ♦r♠♥♥ ♥r ♦st t♥t ss ♥②ss ♦r ♣♦②t♦♠♦s t♦r♥ ♦ t ♠r♥ ttst ss♦t♦♥

❬❪ r② ♥ tr② ❯ rs♦♥ ♥ ♣ ♦r♥♦r♠ ♠①tr ♠♦♥ ♥ ♠♦s str♥ ♥ r♣♦rt ♦♠♥t

❬❲❪ r ②♥ ♥ ❲②s ②s♥ ♠♦ st♦♥ ♦r tt♥t ♣♦st♦♥ str ♠♦ ♦r s♦ ♥t♦rs r❳ ♣r♣r♥tr❳

❬❪ rürt♥ttr ♥t ♠①tr ♥ r♦ st♥ ♠♦s♣r♥r

❬❪ rürt♥ttr ♥t ①tr ♥ r♦ t♥ ♦s

❬❲❪ r ♥ ❲②s st♠t♥ t ♥ r ttstr♥

❬❪ ♥st ♥ r r②t♥ ②♦ ②s ♥t t♦ ♥♦♦t ♦♣ ♠♦♥ t r r t♦ s ♦r♥ ♦ ②r♦♦♥♥r♥

❬❪ ♦♥ ♥ r♣② ①tr ♦ t♥t trt ♥②③rs ♦r ♠♦s str♥ ♦ t♦r t ttsts ♥ ♦♠♣t♥ ♣s

❬❪ ♦rt ♥ ♦ str♥ t r♥♦ ♠①tr ♠♦s ♦♠♣rs♦♥ ♦ r♥t ♣♣r♦s ♦♠♣tt♦♥ ttsts t ♥②ss

♦r♣②

❬❪ ♦rt ♥ t♥t ♦ ♠♦ ♦r ♦♥t♥♥② t ♦♠♠♥t♦♥s ♥ ttsts♦r② ♥ t♦s

❬♦♦❪ ♦♦♠♥ ①♣♦rt♦r② t♥t strtr ♥②ss s♥ ♦t ♥t ♥ ♥♥t ♠♦s ♦♠tr

❬♦❪ ♦t ❯tst♦♥ s ♠♦ès ♠é♥ ♣♦r sst♦♥ t♦♠tq ♦♥♥és ♦r♥s P tss ❯♥rsté ♥♦♦ ♦♠♣è♥

❬♦❪ ♦rt t ♥②ss ♦♠ ❲②

❬r❪ P r♥ ♥ s ♦ t ♦r ♣♥③ ♦♦ st♠t♦♥ ♦r♥ ♦ t ♦② ttst ♦t② rs t♦♦♦ ♣s

❬r❪ P r♥ rs ♠♣ r♦ ♥ ♦♥t r♦ ♦♠♣tt♦♥♥ ②s♥ ♠♦ tr♠♥t♦♥ ♦♠tr

❬❪ rt ♥ P r ♦♠♣r♥ ♣rtt♦♥s ♦r♥ ♦ sst♦♥

❬❪ ♥rs t♥t strtr ♠♦s t rt ts t♥♥t♦rs ♦ ♣♥♥ ♠♦s ♦♦♦ t♦s sr

❬r❪ r♣r ♦ ♣♥♥ t♥t strtr ♠♦s Ps②♦♠tr

❬❪ ♥ ❲ ♦♥ ♥ ❨ ♥ ♥ ①t♥s♦♥ ♦ ♠t♣ ♦rrs♣♦♥♥ ♥②ss ♦r ♥t②♥ tr♦♥♦s sr♦♣s ♦ rs♣♦♥♥ts Ps②♦♠tr

❬♥❪ ♥♥ t♦s ♦r ♠r♥ ss♥ ♠①tr ♦♠♣♦♥♥ts ♥s ♥ t ♥②ss ♥ sst♦♥

❬❪ ♥t ♥ ♦r♥s♥ ♦r② t♦s ①tr ♠♦ str♥ s♥ t ❯❳ ♣r♦r♠ str♥ ❩♥ ♦r♥♦ ttsts

❬❪ ♥t ♥ ♦r♥s♥ str♥ ♠① t ❲② ♥trs♣♥r② s t ♥♥ ♥ ♥♦ s♦r②

❬❲❪ P ♦ ❳ ♥ ❲♥r ♥♦r♠t♦♥ ♦♥s ♦r ss♥♦♣s r❳ ♣r♣r♥t r❳

❬♦❪ P ♦ ①t♥♥ t r♥ ♦♦ ♦r s♠♣r♠tr ♦♣st♠t♦♥ ♥♥s ♦ ♣♣ ttsts ♣s

♦r♣②

❬❪ ❩ ♥ ①t♥s♦♥s t♦ t ♠♥s ♦rt♠ ♦r str♥ rt ts t t♦r ❱s t ♥♥ ♥ ♥♦ s♦r②

❬❲❪ ♥tr ❲♥ ♥ P tt♠♥s♣rr ♥r♥ ♦r ♠①trs♦ s②♠♠tr strt♦♥s ♥♥s ♦ ttsts ♣s

❬❨❪ ♥ ♥ ❨ ♦ts ②s ♦t ♦ t♣ tr ♥tr♥t♦♥ ttst

❬❪ qs ♥ r♥ ♦s str♥ ♦r ♠trt♣rt r♥♥ t ♣♣♦rt r ♥r

❬❪ ♦r♥s♥ ♥ ♥t ①tr ♠♦ str♥ ♦ t sts tt♦r ♥ ♦♥t♥♦s rs ♥ Pr♦♥s ♦ t ♦♥r♥ ♦♠ ♣s

❬♦❪ ♦ trt ♠♦s ♥ ♠trt ♣♥♥ ♦♥♣ts ♦♠ Prss

❬P❪ qs ♥ Pr ♦s str♥ ♦ ♠trt ♥t♦♥ t ♦♠♣tt♦♥ ttsts ♥ t ♥②ss

❬r❪ r♥ ♦♥sst♥t st♠t♦♥ ♦ t ♦rr ♦ ♠①tr ♠♦s♥② r

❬❪ ♦s♠s ♥ rs ♦s str♥ s♥ ♦♣s t♣♣t♦♥s r❳ ♣r♥ts

❬❪ ♥ r ♥ ♥♦r♠t♦♥ ♥ s♥② ♥♥s ♦ t♠t ttsts

❬❪ rs ♥ ♦ts♦ ♥t ♠①trs ♦ ♠trt P♦ss♦♥ strt♦♥s t ♣♣t♦♥ ♦r♥ ♦ sttst P♥♥♥ ♥♥r♥

❬r❪ ♦s♣ rs ♥ t s♦rtst s♣♥♥♥ str ♦ r♣ ♥ ttr♥ ss♠♥ ♣r♦♠ Pr♦♥s ♦ t ♠r♥ t♠ts♦t②

❬r③❪ ❲ r③♥♦s ♦t♦♥ ♠♦ ♦r ♠①trs ♦ t♦r ♥♦♥t♥♦s rs ♦r♥ ♦ sst♦♥

❬❪ rs ♥ P s♠②rt③s ①t ②s♥ ♠♦♥ ♦r rtP♦ss♦♥ t ♥ ①t♥s♦♥s ttsts ♥ ♦♠♣t♥

❬❲❪ ss♥ ♥ ❲♥r ♥t st♠t♦♥ ♥ t rt♥♦r♠ ♦♣ ♠♦ ♥♦r♠ ♠r♥s r st ♦r r♥♦

♦r♣②

❬❪ P t♦ r♠é ♥ ♠r♦s ②s♥ ♠t♦s ♦r r♣str♥ ♥ ♥s ♥ t ♥②ss t ♥♥ ♥ s♥ss♥t♥ ♣s ♣r♥r

❬❪ ♠♥♥ ♥ s ♦r② ♦ ♣♦♥t st♠t♦♥ ♦♠ ♣r♥r

❬+❪ rt ♦ ♥r♦♥t r♥ ① ♥ ♦rt ♠①♠♦ P ♦ t ♦s ❯♥s♣rs ♣rs ♥ ♠♣rs sst♦♥ ①♠♦ rr② Pr♣r♥t

❬❪ r♥ ♥ ❲ r③♥♦s ①tr s♣rt♦♥ ♦r ♠①♠♦ t ttsts ♥ ♦♠♣t♥

❬♦❪ ♦② st sqrs q♥t③t♦♥ ♥ P ♥♦r♠t♦♥ ♦r② r♥st♦♥s ♦♥

❬❪ ♠ ❲❨ ♦ ♥ ❨ ♦♠♣rs♦♥ ♦ ♣rt♦♥ r② ♦♠♣①t② ♥ tr♥♥ t♠ ♦ trt②tr ♦ ♥ ♥ sst♦♥ ♦rt♠s ♥ r♥♥

❬❱❪ r r♥ ♥ ❱ ❱♥ ♦s str♥♦r ♦♥t♦♥② ♦rrt t♦r t ♦r♥ ♦ sst♦♥

❬❱❪ r r♥ ♥ ❱ ❱♥ ♦s str♥♦r ♦♥t♦♥② ♦rrt t♦r t ♣♣♦rt rr

❬❱❪ r r♥ ♥ ❱ ❱♥ ♥t ♠①tr ♠♦ ♦♦♥t♦♥ ♣♥♥s ♠♦s t♦ str t♦r t ♠tt

❬❱❪ r r♥ ♥ ❱ ❱♥ ♦s str♥♦ ss♥ ♦♣s ♦r ♠① t ♠tt

❬❪ rr② ♥s♦♥ r♥ ♥ s ②s♥ ss♥♦♣ t♦r ♠♦s ♦r ♠① t ♦r♥ ♦ t ♠r♥ ttstss♦t♦♥

❬❪ ♥ ♦r♥ r♥♥ t ♠①trs ♦ trs ♦r♥♦ ♥ r♥♥ sr

❬❪ ♥ ♥ rs♥♥ ♦rt♠ ❲② rs ♥Pr♦t② ♥ ttsts ♣♣ Pr♦t② ♥ ttsts ❲②♥trs♥ ❨♦r

❬❪ r♥ ♥rs♥ ♥ P ♦rt ②s♥ ♠♦♥ ♥♥r♥ ♦♥ ♠①trs ♦ strt♦♥s ♥♦♦ ♦ sttsts

♦r♣②

❬P❪ ♥ ♥ P ♥t ♠①tr ♠♦s ❲② rs ♥Pr♦t② ♥ ttsts ♣♣ Pr♦t② ♥ ttsts ❲②♥trs♥ ❨♦r

❬P❪ ♦st ♥ P♣♦r♦ t♥t ss ♠♦s ♦r ♠① rst ♣♣t♦♥s ♥ r♦♠tr② ♦♠♣tt♦♥ sttsts t♥②ss

❬t❪ té♥ t♥t r ②rs r ♦ ♦ ♥ ♥ ♠♦s♥s ♥ t♥t r ♠①tr ♠♦s

❬❪ s♥ ♥ ♥tr♦t♦♥ t♦ ♦♣s ♣r♥r

❬s❪ ❯ ss♦♥ ①♠♠ ♦♦ st♠t♦♥ ♦ t ♣♦②♦r ♦rrt♦♥♦♥t Ps②♦♠tr

❬❪ ♥ ♥ t trt ♦rrt♦♥ ♠♦s t ♠① srt ♥ ♦♥t♥♦s rs ♥♥s ♦ t♠t ttsts♣s

❬Pr❪ Pr③♥ ♥ st♠t♦♥ ♦ ♣r♦t② ♥st② ♥t♦♥ ♥ ♠♦♥♥s ♦ ♠t♠t sttsts

❬P❪ Ptt ♥ ♥ ♦♥ ♥t ②s♥ ♥r♥ ♦r ss♥ ♦♣ rrss♦♥ ♠♦s ♦♠tr

❬P❪ Prs♦♥ ♦♥trt♦♥s t♦ t ♠t♠t t♦r② ♦ ♦t♦♥P♦s♦♣ r♥st♦♥s ♦ t ♦② ♦t② ♦ ♦♥♦♥

❬P❪ P ♥ ♥ ♦st ♠①tr ♠♦♥ s♥ t tstrt♦♥ ttsts ♥ ♦♠♣t♥

❬❪ ❨ ♥ ♥ t♥r ♥♦♠ ts ♦s ♥ t♥tss ♥②ss ♦r t♥ r② ♦ ♥♦st sts ♦♠trs♣♣

❬❪ tr② ②♣♦tss tst♥ ♥ ♠♦ st♦♥ ♥ r♦ ♥♦♥t r♦ ♥ ♣rt ♣s ♣r♥r

❬❪ ♠♦♥r♥② ♦♥♣t rs♦♥ ♦ t ♠♥s ♦rt♠Pttr♥ ♦♥t♦♥ ttrs

❬❪ P ♦rt ♥ s ♦♥t r♦ sttst ♠t♦s ♦♠ tsr

❬❪ rs♦♥ ♥ P r♥ ♥ ②s♥ ♥②ss ♦ ♠①trs t♥ ♥♥♦♥ ♥♠r ♦ ♦♠♣♦♥♥ts t sss♦♥ ♦r♥ ♦ t♦② ttst ♦t② rs ttst t♦♦♦②

♦r♣②

❬❲❪ ♦ss♥ ♣ ♥ ❲♦s♦♥ ♦② ♣♥♥t t♥tss ♠♦s t ♦rts ♥ ♣♣t♦♥ t♦ ♥r r♥♥ ♥t ❯ ♦r♥ ♦ t ♦② ttst ♦t② rs ttsts♥ ♦t②

❬♦❪ P ♦rt ②s♥ ♦ r♦♠ s♦♥t♦rt ♦♥t♦♥st♦ ♦♠♣tt♦♥ ♠♣♠♥tt♦♥ ♣r♥r

❬+❪ ♦ss♥ ❨ ♦♥ rst ♦♠♥ ♥ ❲♦s♦♥ t♥t ss ♥②ss ♦ ♥r ♣r♦♠ r♥♥ ♥r♦♠ ♦♠♠♥t② s♠♣ ♦ ②r ♦s r ♥ ♦♦ ♣♥♥

❬❪ r③ st♠t♥ t ♠♥s♦♥ ♦ ♠♦ ♥♥s ♦ ttsts

❬❪ P ❳ ♦♥ ❨ ♥ ♥ s ①♠③t♦♥ ② ♣rts♥ ♦♦ ♥r♥ ♦r♥ ♦ t ♠r♥ ttst ss♦t♦♥

❬❪ ♠t ♥ st♠t♦♥ ♦ ♦♣ ♠♦s t srt ♠r♥s ②s♥ t ♠♥tt♦♥ ♦r♥ ♦ t ♠r♥ttst ss♦t♦♥

❬+❪ trss ♥s♦♣ st♦♥r s rs♥ ♥ ❯s♥ t♥t ss ♥②ss t♦ ♥t② ♣ttr♥s ♦ ♣tts sr ♣r♦s♦♥ ♥ rr trt♠♥t ♣r♦r♠s ♥ t s r♥ ♦♦ ♣♥♥

❬t❪ t♣♥s ♥ t st♥ ♥ ♠①tr ♠♦s ♦r♥ ♦ t ♦② ttst ♦t② rs ttst t♦♦♦②

❬t❪ t♣♥s ♥ t st♥ ♥ ♠①tr ♠♦s ♦r♥ ♦ t ♦② ttst ♦t② rs ttst t♦♦♦②

❬❪ r ♥tt② ♦ ♥t ♠①trs ♥♥s ♦ t♠tttsts

❬❪ r ♥tt② ♦ ♠①trs ♦ ♣r♦t ♠srs ♥♥s ♦t♠t ttsts

❬❱r❪ ❱r♠♥t t t♥t ss ♠♦s ♦♦♦ ♠t♦♦♦②

❬❱r❪ ❱r♠♥t t ♠①tr t♠ rs♣♦♥s t♦r② ♠♦s ♥♣♣t♦♥ ♥ t♦♥ tst♥ Pr♦♥s ♦ t t sss♦♥ ♦ t♥tr♥t♦♥ ttst ♥sttt s♦♥ P♦rt ♣s

♦r♣②

❬❱❪ P ❱♥ tt♠ ♥ ♦t♥ rt ♠♥tt♦♥ ❯s♥ r♥trt② sr ②s♥ ♥r♥ t s♣t t♦ ①trs ♦ ♦♥r ♦s ♦r♥ ♦ sst♦♥

❬❱❪ ❱r♥ ♥ ♦♥ ♠♣ ♥ ♦② ♦♥r♥t ♠t♦s♦r rt♥ t ♦♥r♥ ♦ ♥② ♦rt♠ ♥♥♥♦r♥ ♦ ttsts

❬❲❪ ❲s ♥ ♦ ♥t ♥t ♠①trs ♦ ♦t♦♥ ♠♦s♦r str♥ ♠①♠♦ t ttsts ♥ ♦♠♣t♥

❬❲❪ ❲ ♥ t ♦♥r♥ ♣r♦♣rts ♦ t ♦rt♠ ♥♥s ♦ ttsts

❬❨❪ ❨♦t③ ♥ ♣r♥s ♥ t ♥tt② ♦ ♥t ♠①trs♥♥s ♦ t♠t ttsts

❬❩❪ ❩ ❩♠♦♠ ♥ s ♦ r♥ ♥st② st♠t♦♥ t♣♣t♦♥s t♦ ♦♥♦♠trs r❳ ♣r♣r♥t r❳

model-based clustering for categorical and mixed data sets

Documents