Á¦ÂnÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁnÅfÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐgÃ]ÎtÔsÔ]Ì...

37

Upload: others

Post on 24-Aug-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

� � ��� ����

� � �� ���� ��� � ����� ��� ��� � � ���� � �� � � � ���

�! !"�#%$'&' )(+*-,/.0$213&4 657*8 8 :9;#+.<&!"=.<&>9+$2?4@#+.<&A"=.<&>9+$2?B&8#3(DCECF -1G9DH7IJ >KLNMDOPOPQSRTMDUPQ<UPMVOXWYU[Z]\_^:Ma`cbd^ceTfYW>UgeJh3i6jlk4mnZVmpo

qsr=t]u_v4wNtx \zy{MDiY|P\_hPi}MDUPMD~�j+^:Q�^�\Dy�fdMDhP^:MD~�WY�XW>f-eJ^}�%MD^�y�\+f>hP^6W>�a\_Ul�Pi:Q���Q�eJQ��_W�Q�UgeJW>ic�_WYU7�eJQ�\_UP^!\Dy�e:�PW�y�\_i6�����3`E�Xo-|��}�3Q<f8��^ceJ\+\7��y�\_i)^6WYe6eJQ<U3��eJ�3W���MD~<h3W�\Dy��DM�i:Q�M�OP~<W��eJ\VM4�3�7W>�lf>\_U3^6e8M�U�e>|3�S|%MDU3�lMD^:��Q<U3�Vy�\_i�e:�PW WY�XW>f-e}\Dy�eJ�PQ�^�MDf-eJQ<\DU¡\_U¡eJ�3W�Pi6\_O%MDOPQ�~<Q�eJQ<WY^S\Dy%^:\D��W�i6W>^6�X\_U3^:W���MDi6Q�MDOP~�W>^�¢£m�¤¥U)eJ�PQ�^�f8�%MD�7eJW>i¦��W�Q�U�e:i:\+�PhPfYW^:W-�_WYiJMD~SW-�+e:W>UP^6Q<\_UP^N\DyFeJ�PQ�^}MDU%MD~�j+^:Q<^Ym§�Q<i6^6e4` x W>fYe:Q<\_Ua¨Pm�b©o-|%��W)�3Q<^:fYhP^:^NeJ�PW!^6eJM�eJh3^T\Dy¦MDfYe:Q<\_U3^}�+Q�^c�>ªM©�«�+Q<^}\_OP^6W>i¬���M�eJQ�\_UP^)Q<U��Pi6\_O%MDO3Q<~<Q�e¥j­eJ�3W>\_icj_|=�PWYf>Q<^6Q<\_U®M�U%MD~�j+^6Q<^>|¦MDUP��f>MDhP^JM�~��¯\7�PWY~<Q�UP�P|MDUP����W[M��3��MDUPf>W¡e:�PW°eJ�PWY^:Q�^;e:�%M�eze:�PW°�£M�Q<U�i:\D~<W¡\DyTfdM�hP^JMD~��¯\+�PW>~�^VQ<^4e:\y{MDfYQ<~<Q�e8M�e:W�eJ�3W4WY��MD~�h%M�e:Q<\_Ul\Dy�eJ�PW)WY�XW>fYe�\Dy�1±$2²D -C3MDfYe:Q<\_U3^�MDU3�[�G\_~<Q�f>Q<WY^�eJ�%M2e��W>i:W!hPU%M�U�e:Q<fYQ<�%M�e:W>�l�PhPi6Q<UP�ze:�PWAf>\_UP^ceJi6hPfYe:Q<\_Ua\DyFeJ�PWA�¯\7�PWY~0m¤¥U x W>f-eJQ<\DUa¨3m´³)��WAW-�+eJWYUP�ae:�PWAQ<�PWYUgeJQ��PfdM�e:Q<\_U°M�U%MD~�j+^6Q<^�\Dy�µ��%M��3eJWYiT¶4e:\f>\DUP�PQ�e:Q<\_UPMD~_MDf-eJQ�\_UP^�\Dy+eJ�PW=y�\_i6��·6�3\}�4Q�y�j_\Dh)^6W>W�¸D¹�M�UP�A^6eJ\+f8�%MD^ceJQ�fº�G\_~�Q<f>Q�W>^\DyGeJ�PW�y�\Di:��·c�P\ ���}Q�eJ�;�Pi:\DO%MDOPQ�~<Q�e¥j}»¯Q�y%jD\_h¯^:W>WN¸3m¼¹¾½¿W}^6�%MD~<~3^:W>W�e:�%M�e=eJ�3WWY��MD~�h%M�e:Q<\_U°M�UP��Q��PW>Uge:Q��%f>M�eJQ�\_U¡\DyFeJ�PWY^:WA�¯\_i:W WY~�MDOG\_iJM2eJW]Q�U�e:W>ic�_W>Uge:Q<\_UP^}f>MDU

b'À_¶

Page 2: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b'ÀD¨ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏOGW�\_O7e8MDQ�UPW>�®y�i:\_�Öe:�PW�MDU%MD~�j7^6Q<^;\Dy��Pi:Q���Q�eJQ��DW°Q<UgeJWYi6�DW>UgeJQ�\_UP^>mפ¥U x W>f-eJQ�\_U¨Pmn¶3|È��WlhP^6Wle:�PWlQ<UgeJWYi6�DW>UgeJQ�\_UËf>MD~<fYhP~<h3^z�PWY�DW>~<\D�XWY�ËQ�Uص��PMD�3eJWYi¯¶¾e:\Ù�_Q��DWM¾�_iJM��P�PQ<f>MD~�f8�%M�iJMDf-eJW>i6Q<Ú>M�eJQ�\_U�\DyNeJ�PWl^:W-e¯\Dy�^6W>�¯Q��¬Û[MDi:��\'�7Q<MDU��¯\7�PWY~<^4y�\Di�}�PQ�f8�ae:�PWAfdMDhP^:MD~�WY�XW>f-e�\�y�\_U3WA��MDi6Q�MDO3~<W�\DU[MDU3\DeJ�PWYi}fdMDUlOGW!Q��PW>Uge:Q��%WY�Ñm½¿WTMD�P�3i:W>^6^�Q�U x W>fYe:Q<\_U¯¨Pm¼¨!e:�PW}�Pi:\DOP~<WY�Ü\DyÑW-��MD~<h%M2eJQ<U3�]eJ�PWNWY�XW>f-e�\Dy±^:WÝ�Þ hPW>Uge:Q�MD~G�P~�M�UP^¬ß×U%MD�¯W>~�jD|g^:W Þ h3W>UPfYW>^}\DyÑe:Q<�¯W-�«��MDicj+Q<UP�!MDf-eJQ<\DUP^]`�^:\_�¯W}e8MD��W>Uf>\DUPf>hPi6i:WYU�e:~�j3oz�PW>^6Q<�_UPWY�àe:\��3i:\+�PhPf>W�M¾f>WYi6e8M�Q<Uà\_h3eJfY\_�¯WDm�½¿Wl�Pi6\'�7Q��PWaM�_i:MD�P�PQ�fdMD~��¯WYe:�P\+�Ë\Dy�W>^ceJQ<��M�e:Q<UP�­e:�PW�W-�±WYfYe�\Dy]^6hPf8�Ø�3~�MDUP^zy�i6\_�ÖUP\_U3W-����GW>i:Q���WYUge8MD~N\_OP^6W>i6��M�e:Q<\_U3^£Q<UË�}�PQ<f8�×^6\_��W[\Dy eJ�PW­MDfYe:Q<\_UP^£MDi:W[Q<U3á%h3W>UPfYW>�Ogj¡eJ�3W>Q<i��3i:W>�3W>f>WY^:^6\_i:^Y|X^6\_��W \DOP^:WYi6��M�eJQ�\_UP^}MDi6W Q<U3áPhPW>UPfYW>��O�j£eJ�PWAMDf-eJQ�\_UP^>|MDUP�[^:\_�¯WVf>\DU3y�\_hPUP�3Q<UP�£��MDi6Q�MDO3~<W>^TMDi6W4hPUP�¯WdM�^:hPi6W>�Ñm]½�WV^:�3\©�âe:�%M�e�e:�PW>i6WQ<^=^6hPOP^ce8MDUgeJQ<MD~ÓMD�3��MDUgeJMD�_W�eJ\!MDU%M�~�j+Ú>Q�UP�!MA�3~�MDU;Q<Uge:\!Q�eJ^=fY\_UP^6e:Q�e:hPW>Uge�MDfYe:Q<\_U3^iJM2eJ�PWYi}e:�%MDU°eJi6WdM�e:Q<UP�ze:�PWA^:WYeN\Dy�MDf-eJQ<\DUP^}MD^NMz^:Q<U3�_~<W WYU�e:Q�e¥jDm§�Q<UPMD~<~�j_|}Q<U x WYfYeJQ�\_Uã¨Pm´ä¿��W¿M��P�Pi:WY^:^leJ�PW Þ hPWY^6eJQ�\_Uã\Dy4�3Q<^6e:Q<UP�DhPQ<^6�PQ<UP��PQ�i:W>f-eSy�i:\_�åQ�UP�PQ�i:W>f-eFWY�XW>fYe:^>mF½¿Wº^:�P\'��e:�%M�eF�PQ�i:W>f-eFWY�XW>fYe:^�f>MDU4OGWÈQ<�PWYU�e:Q��%WY�Ogj­eJ�3Wz�_iJMD�3�PQ<f>MD~��¯WYe:�P\7�¾�3WY�_WY~<\_�GW>��Q<U x W>fYe:Q<\_UÙ¨Pm¼¨Pm)k�UÙW-�3MD�¯�P~<W4hP^:Q�UP�MD~�~<W>�DW>�A^:WÝ�!�PQ<^6f>i:Q���Q�U%M�e:Q<\_U]Q�UAf>\_~�~<W>�DWÈMD�P�¯Q<^:^6Q<\_U �}Q�~<~_^6W>i6�DW�e:\T�PWY��\DUP^6e:iJM�e:WeJ�3W)M�^:^:h3���3e:Q<\_U3^�UPW>WY�PW>�[y�\Di��Pi:\_�GW>i}M�U%MD~�j+^6Q<^�\Dy��PQ<i6W>f-eTW-�±WYfYe:^>m

æ­ç'è éSê¡ë rÈìlí¾î¾w ë=ï ì êðTñ6òºñ6ò óãô±õPö¬÷�ø]ù±ú)ó�ô±õPù±ú!ûÈø]ü�ý�þ%÷�ÿ]ûÈÿ ö��cö¥õPö��Fùk�f-eJQ�\_UP^AMD�P�¯Q�eTe¥��\aQ<Uge:W>i:�3i:WYeJM�eJQ�\_UP^���i:W>MDfYe:Q��DW�M�UP�Ù�PW>~�Q<OGW>iJM2eJQ��DWDm����PW;i:WÝ�MDf-eJQ��DW Q<UgeJWYi:�Pi6WYeJM�eJQ�\_U¡^:WYW>^NMDf-eJQ<\DUlMD^�Mzf>\_U3^:W Þ hPWYUPf>W!\Dy�MDUlMD�DW>Uge�¼^}OXWY~<Q<W-y�^>|�PQ�^:�G\_^:Q�eJQ<\DUÑ|±M�UP�­W>Ug�+Q<i6\_UP�¯W>Uge8M�~�Q<U3�Ph3eJ^Y|�MD^]Q�U�·6kT�%MD� M�eJW4e:�PWzMD�3�P~<W)OXWÝ�fdM�hP^:W�¦�_Wº�%MDUP�3W>�!Q�eÑe:\N�3Q<�amn¹����PWÈ�PW>~�Q<OGW>i:M�eJQ��_W¦Q�U�e:W>i6�Pi:W-e8M�e:Q<\_U ^6W>W>^�MDf-eJQ<\DUMD^�MDU¯\_�3eJQ�\_U¯\DyÑf8�P\DQ<f>WNQ<U¯f>\DU�e:W>�¯�P~�M2eJW>�¯�PWYf>Q<^6Q<\_U¯��MD��Q<UP�P|gh3^:h%MD~�~�jzQ�Ug�_\_~����Q<U3�]f>\_�¯�%MDi6Q<^6\_Uz\DyGf>\_U3^:W Þ hPWYUPf>WY^>|3MD^=Q<UÙ·ck��%MD���M�^¦��\_U3�PW>i6Q<UP� �}�%M�e�  \+���\_hP~<�Ù�P\aQ�yº�PW¯M�e:WzeJ�PW¯MD�P�3~<WDm¼¹ ½¿W;^6�%MD~<~��PQ<^ceJQ<U3�_hPQ<^6�¾eJ�PWze¥��\a�+Q<W-�}^AO�jfdM�~<~<Q�UP�zeJ�PWA�%i6^6el·6MDfYe:¹¡MDUP�aeJ�PW)^:WYf>\_UP�×·:MDfYe:Q<\_U±mn¹®k�U­MDfYeTQ<^}�+Q<WY��W>��y�i:\_�eJ�3W�\_h7eJ^:Q��PWD|FMDU�MDf-eJQ<\DUÙy�i6\_��e:�PW�Q�UP^:Q��PWDm����PWYi:W-y�\_i:W�|¦MDU�MDfYe!f>MDU¿OGW;�Pi:WÝ��PQ�fYeJWY�¯MDUP�¯fdMDU;^:WYi6�DW�MD^=W-�+Q<�PWYUPf>W}y�\_i¦eJ�3W}MDfYe:\_i��¼^È^6eJQ��VhP~�Q3MDU3���¯\DeJQ���M�eJQ�\_UP^`{�3i:\'�+Q<�PWY�àe:�PWaMDfYe:\_i;Q<^z�%MDice;\Dy]\DhPiz��\+�PWY~Eo-m¿k�fYe:Q<\_U3^>|�Q�U®f>\_UgeJi:MD^6e>|�f>MDUUPWYQ�eJ�3W>iºOXWN�Pi:WY�PQ<f-eJWY�£UP\_iº�Pi6\©�+Q��PWTW-�+Q<�PWYUPf>WT^:Q<U3f>W4`{Ogj¯�PWY�%U3Q�eJQ�\_UÓo�e:�PWYj�MDi6W�GW>UP�PQ�UP�;�PW>~�Q<OGW>i:M�eJQ�\_U°MDUP�leJh3i:UaQ<UgeJ\�(+*-,E&�\_U3f>WAW-�7W>fYh3eJWY�Ñm

Page 3: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��_ÊË̬ÎzÅ�Ç Í�Ô���ÁNÅFÌ-Í�Î b'À_ä���PW}f>\_U7y�hP^:Q�\_U;OXW-e¥��WYW>U¡M�fYeJQ�\_UP^=M�UP�¯MDfYe:^º�PMD^=~<WY�;e:\��TW-��fY\_�VO��¼^��PMDiJM2��P\d�¡`��N\©�+Q�f8��b��DÀ��go�MDUP�V\�eJ�PWYi�\+�P�PQ�e:Q<WY^FQ�U)e:�PW�^6\��¥f>MD~<~�W>�!WY�+Q<�PWYUgeJQ�M�~+�3W>f>Q�^:Q�\_UeJ�3W>\_icj_|_�}�PQ�f8�zW>UPfY\_hPiJM��_W>^=�PWYf>Q�^:Q<\DUz�£MD��W>i6^�eJ\]eJMD�DW�Q�UgeJ\ f>\DUP^:Q��PW>i:M�eJQ�\_U4eJ�3WWY�+Q��PW>UPfYW!e:�%M�eTMDUaMDf-eJQ<\DUl��\_hP~����3i:\'�+Q<�PW�|%Q�y�W>U%MDf-eJWY�Ñm����3Q<^}OPQ<Ú>MDi:i6W]eJ�PWY\_i6j^:WYW>�¯^Ne:\��PM'�DW)~�\7\D��WY�ly�i6\_���DW-�±i6WYj��n^TQ<U3áPhPW>UgeJQ<MD~ÑOG\7\D� �Ñ#% "!S$$#D.0*z$�%'&V )(*-.<&-.0$21®`��DWY�Xi:WYjËb��_À_ä_o-|FQ<U��}�PQ<f8��M�fYeJQ�\_UP^4MDi:W;eJi6WdM�e:W>�®MD^)\_i:�PQ�U%MDicjÙWY�_WYUgeJ^`{i:M�eJ�3W>i�e:�%MDU¾Q<Uge:W>i6�DW>UgeJQ�\_UP^Jo]MDUP�Ñ|SMDfYf>\_i6�PQ<U3�_~�jD|XeJ�3WVWY�XW>f-eJ^A\DyºMDf-eJQ<\DUP^ MDi6W\_O3eJMDQ<U3W>�zeJ�Pi6\_hP�_�¯f>\DUP�PQ�e:Q<\_UPMD~<Q�ÚdM�e:Q<\_U)iJM�e:�PW>i=eJ�PMDU¯e:�Pi:\DhP�_��MA�¯W>f8�PMDUPQ<^6�;��¯\7�PQ�yEj+Q<UP�à\_�GW>iJM2eJQ<\DU ~<Q��DW����3`E�Xo-m ` x W>W x e8MD~�U%MD��W>i�b���*_³,+-  Q<OPO%M�i:�åM�UP�R�M�i:�GW>i]b���*DÀ.+ x �gj7i6��^ b��/�0.+3Û�W>WY�°MDUP��  ~�j+�¯\_hPiNb����D¨.+3RNQ�eJf8�3f>\+f8�[b����DÀ3m´o��i:MD�PQ�eJQ<\DU%MD~±�3W>f>Q�^:Q�\_U°eJ�PWY\_i6j21�Q�UP^6e:i:hPf-eJ^NiJM�e:Q<\_UPMD~ÑMD�DW>UgeJ^�eJ\¯f8�P\+\_^:WAeJ�3W\_�3e:Q<\_U¡��eJ�%M2eT��M2�7Q<�¯Q<ÚYW>^�WÝ�7�XWYfYeJWY�[h3e:Q<~�Q�e¥j_|43

5z`E�Xo�687:9<;�`>=@? ���7`��Xo6oBA�`>=3o)C�}�PWYi:WDA�`>=3oFQ<^�eJ�PW�h7eJQ<~�Q�e¥j!\DyG\_h3e:f>\_�¯WE=�+gQ<UVfY\_UgeJi:MD^6e>|�·cWY�+Q<�PWYUgeJQ�M�~3�PWYf>Q<^6Q<\_U3¹eJ�3W>\_icj°fdMD~�~<^�y�\_i���M2�7Q<�¯Q<ÚYQ<UP�!eJ�PWAfY\_UP�PQ�eJQ<\DU%MD~±WÝ�7�XWYfYe8M2eJQ<\DU

5GFIH�`��XoJ6 7 9 ;£`K=L? �XoBA�`>=3o)CQ<U°�}�PQ�f8�a�[Q<^!`�Q<�¯�Pi:\_�GW>i6~�j3oÈe:i:W>M�eJWY�[MD^NMDUa\_OP^6W>ic�_W>���Pi:\D�X\_^6Q�e:Q<\_UÑm���PWz�%MDi:MD�P\d�7W>^]e:�%M�e WY��WYi:�_W4y�i6\_� eJ�PQ�^Ty{MD~�~�MDf-jaMDi:WV\DO��+Q�\_hP^��T�%M�e:Q<W>Uge:^^:�3\_hP~<��M'�D\_Q<���_\_Q�UP�ae:\[eJ�3W£�P\+fYe:\_i­·¬eJ\[i6W>�Ph3f>W£e:�PW£�Pi6\_O%MDO3Q<~<Q�e¥j­eJ�PM�eV\_U3WQ<^A^6W>i:Q�\_hP^6~�j¾Q�~<~<¹¾` x �gj+i:�¯^;b���/�03|S�Ñm=bd¶�0go)+F��\Di:�DWYi:^)^:�P\_h3~<��UPWY�DW>i!�+h3i:i6j¾e:\��\_i:�Ó|_eJ\ i6W>�Ph3f>W�eJ�PW��Pi:\_OPMDOPQ<~�Q�e¥jA\DyG�%Md�+Q<U3� \©�DW>i6^:~<WY�3e+�^ceJhP�PWYUgeJ^È^:�P\DhP~<�zUP\�e�Pi6W>�%MDi6W)y�\_iNW-�3MD�¯^>|G~<WY^6eNeJ�PQ�^N��\_hP~��[�Pi6\'�_W4e:�PW>� OGW>�PQ�UP�[Q�Uae:�PW>Q�iT^ceJhP�PQ�W>^�+MDUP�¾^6\¡\_UÑm�¤¥U¾^:�P\_iced|±MD~<~Fi:WY��WY�PQ�MD~FMDf-eJQ�\_UP^�^6�P\_hP~��­OGWVO%MDUPQ�^:�PWY�¾~<WY^6e�e:�PWYjQ<U3f>i:W>MD^:W e:�PWA�Pi:\_OPMDOPQ<~�Q�e¥j¯eJ�%M2eNM;i6W>�¯W>�3j¡Q<^}Q<UP�3W>W>�lUPWYW>�PWY�Ñm���PWA\7�3�PQ�e¥j£Q<U¡eJ�3Q<^���Q<UP�l\Dy�~<\D�_Q<fT^6eJWY��^�y�i6\_� e:i:W>M�eJQ�UP�;MDfYe:Q<\_U3^}MD^}MDfYe:^eJ�PM�e�MDi:W¾�_\'�DW>i:U3W>�åOgj×�%M�^6e[MD^:^6\7fYQ�M�e:Q<\_UP^£Q<UP^ceJWdM�� \Dy4MD^l\_ONM6WYfYe:^�\Dy)y�i:WYWOQPSRUTUVWRYX�Z�[B\^]`_:a�X)bdcfe�gU[�h�X�iEijX�kle�bme�\^[jn�hB_)ToZW_p\�cU[�h�bdZ�bmX�kle�gU[�X)VW]�q@b^kJX)Vrc�[BV,e�XfZ>ToRURUVW[�ZWZ[Ba�[Bkse�gU[JZ�\^bmt�g�eW[�Z>eugUbmk�eueWgo_veL_)k�]s_p\me�[BVWko_pe�b^a)[)wkUX�kohB_)ToZW_p\Ye�gU[�X)VW]xhB_)ksyY[lToZ�[�cse�XEt�TUb^cU[c�[$h�bdZ>b^X)koZ�z{�| X�\m\^Xv}ubmkot�_�Z�TUt)t�[�Z>e�b^X)k~Xp�L��er_p\^ko_p��[BVx�>�����)�)��w��fb^yUyo_)VWc�_)koc��L_)V�RY[BV��K����v���fToZ>[$c� ���,��������bmk��E���Y��wvVW_pe�gU[�V�e�go_)k � ���S� �������Y�>��wv}ugU[BVW[2�N�2���`ZKer_pkocoZ���X)VNe�gU[�Z�TUy��KTokohQeWbma�[h�X�koc�bme�b^X)ko_)\un>��bm�.bme@}�[BVW[G�YqUz���gU[lZ�[Bi�_pk�e�bdhBZ�X)�NeWgU[feK} XEX)RY[BVr_veWX)VrZ _)V�[`h�\^X�Z�[B\^]DVW[B\d_veW[�c��Z�[B[E�[�h�e�b^X)k¡��z ¢�Qw�yUT�efe�go[�[$£�T¤_ve�b^X)kU¥¦VW[BiEXvav_p\Sb^k�e�[�V�RUVW[�er_veWbmX�k"Xp�2e�go[j���������LX�R¤[�VW_pe�X�V@b^Z\^[�ZWZu_pi�yobmt�TUX)ToZ�_)koc'h�\^[�_pVW\^]DZ>TURoRUV�[$Z�Z�[�Z�b^k���[�V�[�koh�[f��VWX)i§[B¨Y[$hQeue�XDh�_pToZ�[)z

Page 4: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b'À_À Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ïf8�P\_Q�f>W�|�MD^]�PQ�fYeJM�eJWY�­Ogj[eJ�3WV^:WY�£MDUge:Q<f>^T\DyÈeJ�PWV�+�7`��Xo}\_�GW>iJM2eJ\_iYm'���PQ<^°·6WY�+Q���PWYU�e:Q�MD~�¹V�PWYf>Q<^6Q<\_U�eJ�PWY\_i6j£�Pi:W>MDf8�PW>^�e:�%M�e�\DUPW]^:�3\_hP~<�°U3WY�_WYi}Q<�_UP\Di:W��DW>U�hPQ<U3W^6eJM�eJQ�^6e:Q<fdM�~PWY�+Q<�3W>UPfYWV`�Q<U;\_hPiÈfdM�^:WD|�eJ�3WNW-�+Q<�PWYUPf>WNeJ�%M�eºM�U£MDf-eºUP\_i:��MD~�~�j)�Pi:\���+Q<�PWY^�i:WY�gMDi:�3Q<UP�V�}�PW-eJ�PWYiNe:�PWAMDf-e}Q<^�UPWYW>�PWY�Óo-|ÓOPh7e}�PW>fYQ<^:Q�\_U¡eJ�3W>\_icj°�Pi:\D�XWYii:WY��Q�UP�P^FhP^�e:�%M�e�MDfYe:Q<\_U3^¬ß�Ogj e:�PW>Q�iF�DW>i6j!�PW-�%UPQ�eJQ<\DU+ß�i6W>UP�PWYi�^6hPf8�zWY�+Q��PW>UPfYWQ<i6i:WY~<WY��MDUgeNeJ\¯eJ�PWV�3W>f>Q�^:Q�\_U­M�e��%MDUP�±|Xy�\_iTMDfYe:Q<\_U3^¯*8#3(D1N#+ �e:�PWV�Pi6\_O%MDO3Q<~<Q�eJQ�W>^eJ�PM�eNMDfYe:^}UP\_i:��MD~�~�j�\_OXW-j_m4©���PWs�¯\_i:MD~¾\Dy¿e:�PQ<^ã^6e:\_i6j f>MDU�OGW^:hP�¯�£M�i:Q<ÚYW>� Q<U eJ�PWy�\_~<~�\'�}Q<UP��¯UPW>�¯\_UPQ�f]i:�gj+��WY^��

½��%M2eJWY�DW>iNWY�+Q<�3W>UPfYW4MDUaMDf-eN��Q��_�ge��Pi:\'�+Q<�3Wª U°y{MDfYe:^�eJ�%M�eN�Pi6W>fYW>�PWY��eJ�3W)M�fYed|x �3\_hP~<�lUPW-�_WYiTOGWAhP^:WY��eJ\;�PWY~<�l\_UPWA�PWYf>Q��PWª U°�}�PWYe:�PW>i}eJ\¯f8�P\+\_^:W e:�%M�e}^JM���W!MDf-edm¦�7Q��PW>Uge:Q�MD~��PW>fYQ<^:Q�\_U­eJ�3W>\_icj[��MD^AM°�%MD^:^6Q<UP�°WY�PQ<^6\7�3WVQ<U­e:�PWV�P�PQ�~<\_^6\_�P�7�Q<f>MD~=~<Q�eJWYiJM�e:hPi:W�|�MDU3�¿UP\��P�PQ<~�\_^:\D�P�PW>i e:\7�%Mdj¾eJMD�DWY^4eJ�PW¯\_i6Q<�_Q�U%MD~��DW>i:^6Q<\_U�\DyeJ�3Q<^)e:�PW>\Di6j�^:W>i6Q<\_h3^:~�jDm x eJQ<~�~0|F^:\_�¯W£i6W>fYW>Uge;M�e6eJW>�¯�3e:^4�%Md�_W°OGW>WYU��£M��PW¯e:\i:W-�+Q��_W[Q<UgeJWYi:W>^ce¡Q�U«�DWY�Xi:W-j��n^¡WÝ�7�XWYfYeJWY��h3eJQ�~<Q�e¥j�O�jËi:WY�P~�M�f>Q<U3�¬;�`>=@? �Xo4�}Q�e:�;�`>=@? �@Cv­ÙoÝ|��}�PW>i6W®­ ^6e8M�UP�P^£y�\_i¯��MDi:Q�\_hP^£OPMDf8���_i:\_h3UP��f>\_UgeJWÝ�+eJ^>|Nf8�P\_^6W>UeJ\Ø^:hP�P�3i:W>^6^¾^:�Ph3i:Q<\DhP^¾MD^6^:\+f>Q�M2eJQ<\DUP^�`{MD^[Q�U `/¶7m�b'¶_o:o�`�¯=i:Q�f>WØb���3b�+;RNQ�eJf8�+�f>\+f8��b����_À_o-m x hPf8�×M�e6eJW>�¯�3e:^¯WYf8�P\�MDUË\'�_W>i6~�j®i6W>^6e:i:Q�fYeJQ��_WaW>�¯�PQ<i6Q<fYQ<^6eze:iJM2��PQ�eJQ<\DUÑ|�M�f>f>\Di:�PQ�UP�[eJ\a�}�PQ<f8��i:M�eJQ�\_U%MD~ÈM��_W>UgeJ^4~<Q��DW£M�UP���PQ�W£Ogj¿\DUPW£^6\_hPi:fYW\DyAQ�U3y�\_i:��M�e:Q<\_U+ß×^6eJM�eJQ�^6eJQ�fdMD~�MD^6^:\+f>Q�M2eJQ<\DUP^¬ß�MDUP�à�PW>UPfYW¾W-�7�GW>fYe:W>��h3e:Q<~<Q�eJQ�W>^^:�3\_hP~<�ÙM��P��Q�e]UP\l\De:�PW>i \D�XWYiJM�e:Q<\_U­O3h3e�°�Mdj_WY^��¼^)fY\_UP�PQ�eJQ<\DU%MD~<Q�ÚdM�e:Q<\_U±m����PQ�^eJi:MD�PQ�eJQ<\DU¾Q<^ iJM��PQ<�P~�j[�DQ��+Q<UP�¡��Mdj¾eJ\aMl��\_i6W;MDf>fY\_�¯��\+�%M�e:Q<UP�¡fY\_UPfYW>�3e:Q<\_U��iJM2eJQ<\DU%MD~ÈMD�_WYUgeJ^4^6�P\_hP~���MDf-eVMDfYf>\_i6�PQ<U3��eJ\aeJ�PWY\_i:Q�W>^)\Dy}MDf-eJQ<\DUP^�+=U%M2eJhPi:MD~<~�j_|^:h3f8�e:�PW>\_i6Q<WY^¾�PW>��MDUP�âMDf-eJQ�\_U7�¥^6�XWYf>Q��%f�f>\DUP�PQ�e:Q<\_UPMD~<Q�ÚdM�e:Q<\_U `{W�mn�3m ���3`E�Xo:o-|�}�PQ�~<WVi6W>^:WYi6�+Q<U3�±°�M'jDW>^��n^!f>\_U3�PQ�e:Q<\_U%M�~<Q<Ú>M�eJQ�\_U°y�\_i]i6W>�Pi6W>^:WYUgeJQ<U3���%M�^:^:Q��_W;\_O7�^:WYi6��M�e:Q<\_UP^�`�^:W>W²  \_~<�P^6Ú>�¯Q<�3e]MDUP�³¯�W>MDi:~}b����_³,+�Û�W>WY��M�UP�´  ~�j+�¯\_hPiVb���D¨�+½�\7\+�3��MDi:�Ùb����_ä_o-m¤¥U¾�Pi:Q�UPf>Q��P~<W�|GMDfYe:Q<\_UP^TMDi6W4UP\De��%MDice�\DyÈ�3i:\_O%M�OPQ<~�Q�e¥j¡e:�PW>\_icj_|±MDUP�­hPU3�PW>i¬�^6eJMDUP�%M�OP~�j[^:\��]�3i:\_O%M�OPQ<~�Q�eJQ�W>^NfdMD�7eJhPi6W¯U3\_i:��MD~Fi:W>~<M�eJQ�\_UP^6�PQ<�P^�Q<U­eJ�3WV��\_i6~<�Ñ|�}�PWYi:WdM�^¾MDfYe:Q<\_U3^�i6W>�Pi6W>^:WYUgeÙQ<UgeJWYi6�_WYUgeJQ<\DUP^�e:�%M�e­�GW>iceJhPi6O eJ�3\_^:W�i:W>~<M�eJQ�\_U7�µ �Tohrgj[Babdc�[BkohB[�b^Z�VW[BkocU[BVW[�cJb^VWV�[�\m[�a:_)k�e.}ubme�gUb^kje�gU[u_�hQe�X�V�¶ Z,Xv}uk�RUVWX)y¤_pyUb^\mbmeK]JZ�Ro_)hB[)·pb^ki�To\4eWb^_)t)[Bk�elc�[�hBb^Z�b^X)k�Z�b4eWTo_veWbmX�koZ�w�gUXv} [�a)[�V�wY[�_�hrg¸_pt�[Bk�efZ�gUX�TU\^c¸c�[�¹¤kUb4eW[B\^]�yY[jhBX)t)kobmº$_pk�eXp��gUXv}±Xpe�go[BV@_pt�[Bk�eWZ�iEb^t)g�e@bmk�eW[BVWRUV�[Beu[�_)hrgsXp��gUbdZuR¤[�koc�b^kUt~n>}�X)TU\dc¥»y¤[$qE_)hQerZBz

Page 5: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��_ÊË̬ÎzÅ�Ç Í�Ô���ÁNÅFÌ-Í�Î b'À�*^:�3Q<�P^Ymˤ«e¯Q<^zU3\���\_UP�PWYi>|�e:�PW>U±|�eJ�PM�e¯MDfYe:Q<\_U3^�M�i:W°eJi6WdM�e:W>�ËMD^zy�\_i:WYQ<�_U®W>U+�eJQ�eJQ�W>^¦eJ�3i:\_hP�D�P\_h3eÈeJ�3WN~�Q�e:W>iJM2eJhPi6W�\_U¯�Pi:\_OPMDOPQ<~�Q�e¥j4MDUP�¯^6eJM�eJQ�^6eJQ�f>^�+�e:�PWYj;^6W>ic�_WUPWYQ�eJ�3W>iAMD^ M�i:�_hP�¯W>Uge:^ \Dyº�Pi:\_OPMDOPQ<~�Q�e¥jaW-�7�Pi6W>^6^:Q<\DUP^AUP\_i MD^ W-�_WYU�e:^!y�\Di]f>\_U+��PQ�eJQ<\DUPQ<UP�4^:hPf8��W-�7�Pi:WY^:^6Q<\_UP^Ym¦�_W>UBQ<U eJ�3Wã^ce8M�e:Q<^ceJQ<f>MD~£�3W>f>Q�^:Q�\_U7�«e:�PW>\Di:WYe:Q<f ~�Q�e:W>iJM2eJhPi6Wâ`{W�mn�Pm x M'��MD�DWb��gä2¨�o-|g�}�PWYi:W MDf-eJQ�\_UP^ºMDi:WNeJ�PWN��MDQ<U;e8MDi6�_WYe�\DySM�U%MD~�j+^6Q<^>|ge:�PW�^cj7�4OX\_~�^��DQ��_WYUeJ\àMDf-eJQ�\_UP^¡^6W>i6�DW��¯W>i6W>~�j×MD^°Q�UP�PQ<fYW>^£y�\Di°�PQ<^ceJQ<U3�_hPQ<^6�PQ<U3��\_UPW¾�Pi6\_O%MDO3Q<~<Q�e¥jy�hPUPf-eJQ�\_UËy�i6\_� MDUP\�eJ�PWYi>|�UP\De£MD^¯W>UgeJQ�eJQ�W>^;eJ�%M�e¯^ce8MDUP�ØQ<Uà~<\_�_Q�fdMD~�i:W>~<M�eJQ�\_U7�^:�3Q<�P^ºeJ\4e:�PWT��MDi6Q�MDOP~�W>^È\_U£�}�3Q<f8�¡�Pi6\_O%MDO3Q<~<Q�eJQ�W>^ÈMDi6W��PW-�%UPW>�±m x M'��MD�DWz`cb�gä�¨P|�Ñm}bd¨�oV�PW-�%UPWY� ·:MDf-eJ¹�MD^£M ·cy�hPU3fYeJQ�\_UËM�e:eJMDf8�PQ<U3�¿MÙf>\_U3^:W Þ hPWYUPf>W­e:\�W>MDf8�^6eJM�eJW°\�yTe:�PW¡��\Di:~<�±|n¹�MDUP�®�PW°eJi6WdM�e:W>�àM­f8�%MDQ�U®\DyN�PW>fYQ<^:Q�\_UP^Y|º\DUPWl~<WdM��PQ<UP�eJ\a\De:�PW>iY|�MD^)Ml^:Q<U3�_~<Wz�PWYf>Q�^:Q<\DUÑm£RN\'��W-�_W>iY|¦eJ�PW;~�\_�_Q<fVe:�%M�e!~<W>MD�P^AhP^AeJ\aQ<U7�y�W>ize:�PWlf>\_UP^6W Þ hPWYUPf>WY^°\Dy]M�fYeJQ�\_UP^;MDU3�Ë^ceJiJM2eJW>�DQ<W>^zy�i6\_�Ö�¯\_i:W°WY~<W>�¯W>UgeJMDi6jf>\DUP^:Q��PW>i:M�eJQ�\_UP^ Q<^�~<WYyEe]\_h3eA\DyÈeJ�3W;y�\Di:��MD~<Q�^:�amA§Ó\Di]W-�3MD�¯�P~<W�|Ñf>\DUP^:Q��PW>i eJ�3WMDf-eJQ<\DUP^��­·6iJMDQ�^:W;e8M2�7WY^>|¼¹ ·c~<\'��WYi!eJM2�7W>^Y|n¹�MDUP�å·6iJM�Q<^:W¯Q<Uge:W>i:WY^6e)iJM�e:W>^Ymn¹¼���PWf>\DUP^:W Þ h3W>UPfYW>^ \Dy¦MD~<~±e:�Pi:WYW4M�fYeJQ�\_UP^}�VhP^ceNOXW!^:�GW>fYQ��%WY�[^:WY�%MDi:M�eJWY~�j_|Ó�Pi6Q<\_i�e:\MDU%M�~�j+^:Q�^�+FUP\_UPW;fdM�U¿OGW¯Q�U3y�W>i6i:WY��y�i6\_� e:�PW�\�eJ�PWYi:^>m¯k�^!Mli:WY^:hP~�ed|SQ�yº��W�MDi6W�_Q��_W>Ule¥��\£�Pi6\_O%MDOPQ�~<Q�eJQ<WY^>|.;G½ÙMDUP�±;f¾¦|Ó�PWYUP\De:Q<UP�zeJ�3W)�Pi6\_O%MDO3Q<~<Q�eJQ�W>^��3i:WY��MDQ�~��Q<U3��hPU3�PW>i�MDfYe:Q<\_UP^x¿\_ixÀ£|Xi6W>^6�XWYfYeJQ��_WY~�j_|ÓeJ�3W>i:W)Q<^NUP\���Mdjl��W)fdMDU��PW>�Ph3f>Wy�i:\D�BeJ�3Q<^�Q<U3�Ph3e�e:�PWA�Pi:\_OPMDOPQ<~�Q�e¥jÁ;G½.¤¾Ùf>\Di:i:WY^:�G\_UP�PQ�UP�zeJ\zeJ�3WEM6\DQ<Uge}MDf-eJQ<\DU¿±Ã¡Àâ\_i=Q<UP�3W>W>�£MDUgj�°�\+\_~<W>MDU;f>\_�4OPQ<U%M2eJQ<\DUz\DyXe:�PW}�Pi:\D�X\_^6Q�e:Q<\_UP^l¿ãMDU3�²À£m���PQ�^¦��W>MDUP^FeJ�%M�e>|_Q<U4�Pi:Q�UPf>Q��P~<W�|2e:�PW�Q<�¯�%MDf-e¦\DyGM�~<~+MDUgeJQ�f>Q<�PM�eJWY�xM6\_Q<Uge�MDfYe:Q<\_U3^��\_hP~<�aUPWYW>�leJ\;OXWA^6�XWYf>Q��PW>�[Q�UlMD�3��MDUPfYWÝß�MDU�Q�UP^:h3i:�¯\_hPUge8MDO3~<W]e8M�^:�Óm���PWÙ�XWYf>hP~�Q�MDi�^6eJM�eJhP^l\Dy)MDf-eJQ�\_UP^°Q<UØ�Pi:\_OPMDOPQ<~�Q�e¥j®eJ�3W>\_icj×fdMDU�OGW¾^:WYW>U�¯\_^6e�f>~�WdMDi6~�j®Q<Uàf>\D���%M�i:Q<^6\_UËe:\�e:�PW[^ce8M�e:hP^£\DyA\DOP^:WYi6��M�eJQ�\_UP^YmÄ°ºjË^6�XWYf>Q �yEj+Q<UP�[M��Pi6\_O%MDO3Q<~<Q�e¥j[y�hPU3fYeJQ�\_U´;�`WÅ©o]\_U�eJ�PW��X\_^6^:Q�OP~<W¯^6eJM�eJWY^4\Dy�eJ�PW¯��\_i6~<�Ñ|��W�M�h3eJ\_��M�e:Q<fdM�~<~�jÙ^6�XWYf>Q�yEj��P\'� �Pi6\_O%MDO3Q<~<Q�eJQ�W>^!^:�P\_h3~<�®f8�%MDUP�DWl�}Q�eJ�®WY�DW>icjf>\DUPf>WYQ���MDOP~�WT\_O3^:W>ic��M�eJQ�\_U²Æ_|+^:Q�UPf>W�;£`�Å©o=�GW>i6��Q�eJ^ÈhP^ºeJ\4fY\_�¯�Ph3eJW4`{Ogj¯f>\DUP�PQ��eJQ�\_UPQ�UP�A\_UÇÆ©o¦eJ�PWT�X\_^ceJWYi:Q<\Diº�3i:\_O%M�OPQ<~�Q�eJQ�W>^J;�`�È�?ÉÆ©o�y�\_iºWY�_WYi6j��%MDQ�i�\DyÑWY�_WYUgeJ^È M�UP��Æ_m¦RN\'��W-�_W>iY|�^:�GW>f>Q�yEj+Q<UP�";�`WÅ©oÑeJWY~<~<^FhP^¦U3\DeJ�PQ�UP�]M�OX\_h7e¦�P\'�®�Pi6\_O%MDOPQ�~��Q�e:Q<WY^º^6�P\_hP~���f8�PMDUP�_WTQ<U¯i:W>^6�X\DUP^:WNeJ\)MDU£WÝ�7e:W>i6U%MD~XMDfYe:Q<\_U����3`K¿ o-m�¤¥U£�_WYUPW>i:MD~0|Q�y�MDUlM�fYeJQ�\_Ul���3`K¿ o�Q�^�eJ\;OGW �PW>^6f>i:Q�OXWY�aMD^}MVy�hPUPfYe:Q<\_U¡e:�%M�e�eJMD�DWY^s;�`WÅ©o�M�UP�eJi:MDUP^cy�\_i:�¯^�Q�e;eJ\´;L½�`WÅ©o-|�e:�PW>UÊ;£`�Å2o4e:W>~<~�^;hP^£UP\�eJ�PQ�UP�¿M�OX\_h7e�e:�PW[U%M2eJhPi6W\Dy�;L½¦`�Å2oÝ|XW-�_WYUÙ�}�3W>UË¿sQ�^]MDU[W>~<WY��WYUge8MDicj�W-�_WYU�e y�\DiN�}�PQ�f8�®;£`K¿ o}Q<^}��WY~<~���PW-�%UPW>��`�WDm¼�Pm�|F·ciJMDQ�^:W�e:�PW�e:W>�¯�XWYiJM�e:hPi:W�O�j£b��PW>�Di:W>WY¹ \_i ·¬eJhPi6UVeJ�PW�^6�Pi:Q�UP��~<WYi\_UP¹go-m¦½�Q�e:�£eJ�3W]W-�7f>WY�3eJQ�\_U°\Dy�e:�PW]eJi6Q��+Q�MD~Ói6W Þ hPQ<i6W>�¯W>Uge�e:�%M�e�;L½¦`�Å©o�OXW�Ú>W>i6\Q�ysÅ°Q<�¯�P~�Q<W>^¸Ì`¿4|ºM­i6W Þ hPQ<i6W>�¯W>UgeVe:�%M�e;MD�3�P~<Q�W>^4h3UPQ�y�\_i6��~�j¾eJ\¾W-�_W>icj§;�`WÅ©o-|

Page 6: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b'À�/ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï�Pi6\_O%MDOPQ�~<Q�e¥jVeJ�PWY\_i6j¯�P\+W>^�UP\De�e:W>~�~Gh3^��P\'�Ä;G½�`WÅ©o=^:�P\_h3~<�¡�PQ��±WYi�y�i6\_�Í;�ν `WÅ©o-|�}�PWYi:WÏ; Î `�Å©oAQ�^!^6\_�¯W�\De:�PW>i)�Pi:W>MDfYe:Q<\_U��Pi6\_O%MDOPQ�~<Q�e¥j[y�hPUPf-eJQ�\_UÑmaµ�\_UP�3Q�eJQ�\_U7�Q<U3�z\_UÇ¿ Q<^�fY~<W>MDi:~�j£Q�U%MD�PW Þ hPM�eJW]y�\_i�fdMD�3e:hPi:Q�UP�VeJ�3Q<^�e:iJMDUP^cy�\_i:��M�e:Q<\_UÑ|7MD^���W�%Md�_W�^:W>WYUzQ<U)�£MDUgj!W-�3MD�¯�P~�W>^FQ<Uzµ��PMD�3eJWYi:^�b�MDUP�z¶4`{^:WYW�WDm¼�Pm x W>f-eJQ�\_U£b_mn¶3m�b©o-|OGW>fdM�hP^:W!f>\_U3�PQ�e:Q<\_UPQ�UP�Vi6W>�Pi6W>^:WYUgeJ^��PMD^:^6Q��_WA\_O3^:W>ic��M�eJQ�\_UP^}Q<UaMDUlhPUPf8�PMDUP�_Q�UP���\_i:~��l�}�3W>i:W>MD^TMDfYe:Q<\_UP^�f8�PMDUP�_WAeJ�PW ��\_i:~��ÑmZ]iJMd�}Q<U3�[MDUPMD~<\_��j­eJ\l�+Q<^:hPMD~È�XWYi:f>WY�3eJQ�\_UÑ|S��W��£Mdj¾^:Mdj�e:�%M�e!eJ�PW¯Q<U7y�\_ic���M�eJQ�\_U f>\DU�eJMDQ<U3W>�×Q<UÐ;�`WÅ©o�Q�^°MDU%M�~<\_�_\DhP^£eJ\®M��3i:W>fYQ<^6W��3W>^:fYi:Q��3eJQ�\_U×\Dy)MeJ�3i:W>WÝ�¥�PQ���WYUP^:Q�\_U%MD~X\_ONM6WYfYe+ÓQ�e�Q<^�^6h.ѯf>Q<WYUgeNy�\Di��Pi:WY�PQ<f-eJQ<U3�¯�3\©� eJ�%M2e}\_ONM6W>f-e�}Q<~�~_OGWº�+Q<W-��WY�4y�i:\D� MDUgj!MDUP�_~�WÈ\_h3eJ^6Q<�PWÈeJ�3W�\_OSM6W>fYe>|DOPh7e�Q�eSQ<^SQ<UP^6h.ѯf>Q�W>Uge�y�\Di�Pi6W>�PQ�fYeJQ�UP�£�3\©� e:�PW4\DONM6W>f-e]�}Q<~�~�OXW!�7Q�WY��W>�¾Q�y=�£M�UPQ<�Ph3~�M�e:W>�aMDUP�­^ Þ hPW>WYÚ>W>�OgjàW-�+eJWYi:U%MD~}y�\_i6f>W>^Ym×k��P�3Q�eJQ�\_U%MD~�Q<U3y�\_i6�£M2eJQ<\DU®MDOX\Dh3e;eJ�PW��P�gj7^6Q<f>MD~}�Pi:\_�+�W>iceJQ�W>^�\�y=eJ�PWV\DONM6W>f-e �VhP^ce]OGW4^:h3�P�P~<Q�W>�[y�\Di���MD��Q<UP��^:hPf8�¾�Pi6W>�PQ�fYe:Q<\_UP^Ym'°�jMDU%M�~<\_�DjD|2e:�PW�M��P�PQ�e:Q<\_UPMD~_Q<U7y�\_i:��M�eJQ�\_UAi:W Þ h3Q<i:WY�4y�\_iF�PW>^6f>i6Q<OPQ�UP�Te:�PW�e:iJMDUP^cy�\_ic���M�eJQ�\_U�y�i6\_�Ò;�`WÅ©o=eJ\�;L½¦`�Å2oº^:�P\_h3~<�¡Q��PW>UgeJQ�yEj�e:�P\_^6W]W>~�W>�¯W>UgeJ^�\Dy�e:�PW���\_i:~��eJ�PM�e¦i6W>��MDQ<U)Q<Ug��MDi:Q<MDUge�hPU3�PW>i�e:�PW�M�fYeJQ�\_Uz���7`>¿ oÝmG���3Q<^¦WÝ�7e:iJM�Q�U3y�\_i6�£M�e:Q<\_UAQ�^�Pi6\©�+Q��PW>�zOgj)fdMDhP^:MD~3��UP\'�}~�W>�P�_W�|�MDUP�4e:�PW����7`BÓ oS\_�GW>iJM2eJ\_iFW>U%M�OP~<WY^¦hP^�e:\]fdMD�+�eJh3i:W)e:�PW!Q<Ug��MDi:Q<MDUgeTWY~<WY��WYU�e:^z`EeJ��hP^��3WY�%UPQ�UP�Ï;L½¦`�Å©o:o�O�ja~�\7f>MD~<~�j¡�¯\7�PQ�yEj+Q<UP�eJ�3W¯�DiJMD�P�Ù\_i]e:�PWz^6e:i:hPf-eJhPi:MD~=W Þ h%M2eJQ<\DUP^>m����PW;UPWÝ�+e)^6W>f-eJQ<\DUÙ�}Q�~<~�f>\_�¯�%MDi6WeJ�3Q<^��PW-�+Q<f>W]e:\VeJ�PWT��MdjlMDf-eJQ�\_UP^�M�i:W �%MDU3�P~<WY�°Q<U¡^ce8MDUP�PMDi:�l�PWYf>Q<^6Q<\_U�eJ�PWY\_i6jDmðTñ6òºñQÔ óãô±õPö¬÷�ø]ùØö6øÖÕÄ�FôSö¬ùPö¬÷�ø óãø]ûJ�Q×Tù3öcù¤¥UP^ceJWdM��°\Dy�Q<Uge:i:\+�PhPf>Q�UP�!UPWY�ã\_�GW>i:M�eJ\_i6^�Q�UgeJ\4�3i:\_O%M�OPQ<~�Q�e¥jzf>MD~<fYhP~<hP^Y|+e:�PWTe:iJM2��PQ�eJQ<\DU%MD~�MD�3�Pi:\gM�f8�[�%MD^TOXWYW>UaeJ\¡M2e:eJi6Q<OPh7eJW eJ�PW)�PQ��XW>i6W>UPfYW>^TOXW-e¥��WYW>U¾^:WYW>Q�UP�MDUP���P\_Q�UP�¡eJ\a�PQ��±WYi:W>U3f>W>^!Q<U¾e:�PWzeJ\DeJMD~¦WY�+Q��PW>UPfYW£Md��MDQ�~�MDOP~�WDmVµ�\_UP^:Q��PW>iAeJ�3W^6eJM�eJWY��WYUgeJ^���·ceJ�3W¡O%MDi6\_��W-eJWYi4i:W>MD�PQ�UP�[��MD^V\_OP^6W>ic�_W>�®eJ\[OXW£�Ó¹ÙM�UP�·ceJ�3WO%MDi6\_�¯WYeJWYi;i:WdM��PQ<UP�Ù��MD^£^6WYe£e:\�~�WY�DW>~}�Smn¹ ���PWay�\_i:�¯W>iz�PWY~<�P^�hP^¯�Pi:WY�PQ<f-eeJ�3WV��W>M�eJ�PWYi>|Ñe:�PWz~�M�e6eJWYi��P\+W>^ UP\�edm ½��PQ<~�W)e:�PWzWY�+Q<�3W>UPfYWz�PW>^6f>i:Q�OXWY�ÙQ<U­eJ�3W�%i6^6e!^6eJM�eJWY��WYU�e)Q<^ ~�Q<�¯Q�e:W>�[e:\ae:�PW;i:WdM��PQ<UP�a\DyºeJ�PW;O%M�i:\_�¯WYe:W>i>|�e:�PW�^6W>fY\_UP�^6eJM�eJWY��WYUge;MD~<^6\�e:W>~<~�^)h3^4eJ�%M2e4eJ�3W£O%MDi6\_�¯WYeJWYi)��MD^V��MDUPQ��PhP~�M2eJW>��Ogj¿^6\_�¯WMD�_WYUged|SMDUP�¾fY\_UP�PQ�eJQ�\_UPQ<U3�£\_U­e:�PQ<^ M��P�PQ�e:Q<\_UPMD~SWY�+Q<�PWYUPf>W;^6�P\_hP~��¾i:W>U3�PW>i eJ�3WO%MDi6\_�¯WYeJWYi�i:W>MD�PQ<U3�¯Q�i:i6W>~<W-��MDUge�eJ\¯�Pi:WY�PQ<f-eJQ�UP�VeJ�PWAi:MDQ<UÑm���PW­�Pi:MDfYe:Q<f>MD~�M�^:�GW>fYe:^¡\Dy e:�PQ<^£MD�P�Pi6\gMDf8�×MD�¯\_hPUge¯eJ\�WY�VOPi:MDf>Q�UP�¿eJ�3WMDf-eJQ<U3��M��_W>UgeJ^!MD^ ��MDi6Q�MDOP~�W>^ Q�UÙe:�PW¯MDU%MD~�j+^:Q<^Y|Sf>\_UP^ceJi6hPfYe:Q<UP��MDU�MDhP�_�¯W>UgeJWY��PQ�^6eJi6Q<OPh7eJQ<\DU¿y�hPUPf-eJQ<\DUàQ�UPf>~�hP�PQ�UP�­eJ�PW°�3W>f>Q�^:Q�\_UP^z\DyNeJ�P\D^:WaMD�_WYU�e:^>|�MDUP�®Q<U7�y�W>i6i:Q�UP��eJ�PWÙW-�±WYfYea\Dy)MDfYe:Q<\_U3^°O�j f>\_UP�3Q�eJQ�\_UPQ�UP�¿eJ�3\_^:WÙ�PWYf>Q<^6Q<\_U ��MDi6Q�MDO3~<W>^

Page 7: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��_ÊË̬ÎzÅ�Ç Í�Ô���ÁNÅFÌ-Í�Î b'ÀY�eJ\Ù�%M�i6eJQ�f>hP~<MDi)��MD~�hPW>^Ym¬����hP^Y|�y�\_izWÝ�PM����P~�WD|¦e:�PWaMD�_W>Uge;��MDUPQ��PhP~�M2eJQ<U3��eJ�3WO%MDi6\_�¯WYeJWYi¡�¯Q<�_�ge°WYU�e:W>i¡e:�PWÙ^6j+^6e:W>� MD^lM��PWYf>Q�^:Q<\DU �DM�i:Q�M�OP~<W ·6^ Þ hPWYW>Ú>Q�UP�eJ�3W¡O%MDi6\_��W-eJWYi:¹�+ºM�yEe:W>i4Q�UPf>\Di:�G\_iJM�e:Q<UP�leJ�3Q<^)��MDi6Q�MDO3~<W¯Q<UgeJ\[e:�PW¡�Pi6\_O%MDO3Q<~<Q�e¥j�PQ�^6eJi6Q<OPh7eJQ<\DUÑ|3��W4f>\DhP~<�aQ<U3y�WYi�eJ�PW!Q<�¯�%MDf-eT\�y¦��MDUPQ<�3hP~�M�e:Q<UP�)eJ�PW!O%MDi6\_��W-eJWYi^:Q����3~�j­Ogj�f>\DUP�PQ�e:Q<\_U3Q<UP�¡e:�PW£M�hP�_�¯W>UgeJWY���PQ�^6e:i:Q<O3h3eJQ�\_UÙ\_U�e:�PW¯WY�_WYUge�·ceJ�3WO%MDi6\_�¯WYeJWYi���M�^T^ Þ h3W>W>ÚYW>�­Ogj¡y�\Di:f>W�=°M�UP�l�%MD^}i:W>MDf8�PW>��~<W-�_W>~��Sm¼¹§%\_i¯e:�PQ<^�f>\_U3�PQ�e:Q<\_UPQ�UP�Ù��W-eJ�P\+�Ëe:\¿��\_i:�®�Pi6\_�XWYi:~�jËQ�UØW-�DM�~<h%M�e:Q<UP�ÙeJ�3WWY�XW>f-e[\�y)y�h3e:hPi:W�MDf-eJQ<\DUP^>|NeJ�PW��£M�UPQ<�Ph3~�M�e:Q<UP��M��_W>Ugea�VhP^ce�OGW¾eJi6WdM�e:W>�åM�^MDUàQ<�PW>MD~�WÝ�3�GW>i6Q<�¯W>UgeJWYi¯MDfYe:Q<UP�Ù\_h3e¯\DyNy�i:WYWa�}Q�~<~/|ÈMDUP�®eJ�PWaMD^6^:\+f>Q�M2eJW>�à�PWÝ�f>Q�^:Q�\_U���MDi6Q�MDOP~�W>^N�VhP^ce OXW!eJi6WdM�e:W>�ÙMD^TW-�7\_�_WYUP\_hP^¥ß�fdM�hP^JMD~�~�j�hPU%M��XW>f-eJWY�¾O�j\De:�PW>iº�DM�i:Q�M�OP~<WY^�Q�U�eJ�3W�^6j+^ceJW>�amº§%\_i�WÝ�3MD���3~<WD|+Q�y±eJ�PW�MDhP�_�¯W>UgeJWY�¡�Pi6\_O%MDOPQ�~��Q�e¥jay�hPUPf-eJQ�\_U­W>U3f>\+�PW>^]e:�PW4y{MDf-e]eJ�%M2e�eJ�3WVf>hPi6i:WYU�eA\'�}UPWYi \Dy=eJ�3WVO%MDi6\_��W-eJWYieJWYUP�P^!eJ\[^ Þ hPW>WYÚ>W¡eJ�3W£O%MDi6\_�¯WYeJWYi)W>MDf8�¿e:Q<�¯W¯^6�PW�y�WYW>~�^VMDiceJ�Pi6Q�eJQ�^A�%MDQ<U±|F��W�}Q<~�~�OGWzhPU%MDO3~<WVeJ\lhP^6W¯e:�%M�e y�hPU3fYeJQ�\_U­y�\_i W-�DM�~<h%M�e:Q<UP�¡e:�PWzWY�XW>f-eJ^!\Dy��PW>~�Q<O7�W>i:M�eJW;^ Þ h3W>W>ÚYQ<UP��\DyºeJ�PW;O%MDi6\_�¯WYeJWYi>|SWY�DW>U�O�j¾e:�PW;^JMD�¯W;\'�}UPW>iYm�L}W>f>MD~<~�Q<UP�eJ�3W°�PQ��XW>i6W>UPfYW°OXW-e¥��WYW>U M�fYeJ^;M�UP�àMDfYe:Q<\_UP^Y|¦�}�PW>U3WY�_WYi���W°^:W-e�\Dh3eVeJ\¾f>MD~��f>h3~�M�e:W�eJ�3W£WY�XW>f-ez\Dy}M��XWYUP�PQ<U3�­MDf-eJQ<\DUÑ|���W¡�VhP^ceVQ��_UP\_i6W£MD~�~È��WYf8�%MDUPQ�^:�¯^eJ�PM�e]fY\_UP^6e:iJMDQ�UPW>�[\_iTe:i:Q<�D�_W>i6W>�[e:�PW4WÝ�3WYf>h3e:Q<\_U¾\�y=eJ�%M�e]MDfYe:Q<\_U�Q<U[e:�PWV�%M�^6edmk�fYf>\_i6�PQ<U3�_~�jD|XeJ�3WzWY�_WYUge­·B���PWzO%MDi6\_��W-eJWYi���M�^ ^ Þ hPW>WYÚ>WY�P¹[�Vh3^6e WYU�e:W>i eJ�3WMDhP�D��WYU�e:W>���Pi6\_O%MDO3Q<~<Q�e¥j­y�hPUPf-eJQ�\_U�MD^)Q<UP�PWY�XWYUP�PWYU�e;\�yNMD~<~ÈWY�DW>UgeJ^ze:�%M�eV\+f-�f>h3i:i:WY���Pi:Q�\_i¯e:\�eJ�PW�eJQ<�¯W[\�yA�£MDU3Q<�PhP~<M�eJQ�\_UÑ|�^:Q<�¯Q<~<MDiVeJ\�e:�PW[��Mdj×MDf-eJQ<\DU��MDi:Q<MDOP~�W"ØsW>Uge:W>i:WY��e:�PW!MDhP�_�¯W>UgeJWY��U3WYe¥��\_i:�°Q�U�§�Q<�_hPi6WA¶3mp³7m���PQ<^A^6\_~<h3e:Q<\_U¾fY\_i:i6W>^6�X\_U3�P^)�3i:W>fYQ<^6W>~�j[eJ\leJ�PWz��M'j�M�fYeJQ�\_UP^AMDi6WzeJi:W>M�eJWY�Q<U��PW>fYQ<^6Q<\_U°M�U%MD~�j+^6Q<^>|7MD^��PW>�PQ�fYe:W>�°Q<U¯eJ�PW�~<Q�e:W>i:M�eJhPi6WT\DU¡Q<U3áPhPW>UPfYW]�PQ<MD�_iJM���^`{¤¬Z]^Jo)`{RT\'��MDi:�[MDUP�¾Û�M�eJ�PWY^:\_U�b���/3bY+ x �%MDf8�geJWYi4b��Y/_À.+�¯�W>MDi:~Èb���/Y/DOÑ|%f8�%MD�±mÀgoÝmjÈMDf8���PW>fYQ<^:Q�\_Ul�DM�i:Q�M�OP~<W]Q�^}i:W>�3i:W>^6W>UgeJWY�¾MD^NW-�7\_�_WYUP\_hP^}��MDi:Q<MDOP~�W¯`/M;�%MDic�W>Uge:~<W>^6^)U3\7�PW;Q�U­eJ�PW¯�PQ�M��_iJMD�£o-|±MDUP��Q�e:^ Q<�¯�%MDfYe]\_U�\�eJ�PWYiA��MDi6Q�MDO3~<W>^�Q<^AMD^¬�^:WY^:^6W>��MDUP�°W>U3f>\+�PW>�lQ<U�eJW>i6��^�\DyFf>\_UP�3Q�eJQ�\_U%MD~Ó�Pi6\_O%MDO3Q<~<Q�eJQ�W>^>|g^6Q<�¯Q<~<MDi=eJ\VeJ�3WQ<�¯�%MDf-e�\Dy�MDUgjl\DeJ�3W>i��%MDi6W>UgeNUP\+�PWAQ<U°eJ�PWA�PQ<MD�_i:MD��mÚÙ���PW¡�PQ�ѯf>hP~�e¥j��}Q�eJ��eJ�PQ�^VMD�P�3i:\gMDf8�¿Q<^)e:�%M�e!��W£UPW>WY��eJ\¾MDUge:Q<f>Q��%M�e:W�Q�UMD�3��MDU3f>WD|}M�UP�Øi6W>�Pi6W>^:WYUge°W-�7�P~�Q<f>Q�eJ~�j_|�MD~<~}MDf-eJQ�\_UP^;�}�P\_^:W�WY�XW>f-eJ^���W[�¯Q<�D��e�}Q<^6�[eJ\£WY��MD~<hPM�eJW)Q<U�e:�PW)y�h7eJhPi6WDm'���PQ�^�i6W>UP�PWYi:^]e:�PW4�¯\7�3W>~<Q�UP�¯�Pi:\+f>WY^:^]hPU+��PhP~�j�f>h3�VOXWYi:^6\_��W�|�Q�y�U3\DezeJ\DeJMD~<~�j¿h3UP�£M�U%MD�_W>MDOP~<W�m�¤¥UËfYQ<i6f>hPQ�e;�PQ�MD�DUP\_^:Q�^>|Û$��gU[JPKÜÝ\mbme�[�VW_pe�TUVW[)¶ Zub^koZ>bdZ>e�[Bk¤h�[lX)k�cUbma�X)Vrh�b^kUtje�gU[J\^b^kU��Zubmk'eWgU[JPKÜ®��V�X�iÞ_pk]�hB_pT¤Z�_)\b^k�eW[BVWRUVW[�eW_pe�b^X)kD�ß�@Xv}�_pVrcf_)kocJà�_pe�gU[$Z>X�k�����áU�)·:�@Xv}�_pVrcE�����)â��Yb^Z,_ve,X�cUcoZN}ubme�gJRoV�[�a:_)bm\^b^kUtRUVr_)h�e�bdh�[)zu��gU[�hB_pT¤Z�_)\Nb^k�eW[BVWRUVW[�eW_pe�b^X)k'bdZ@}ugo_peG_p\^\mXv}@Z@ToZueWX�e�VW[�_peGc�[�hBb^Z�b^X)k"av_pVWb^_)yU\^[�Z@_�ZVWX�X)eukUX�c�[�Z�w�TUko_�Z�Z�X�h�bd_ve�[$cx}ubme�g"_)\m\�X)e�gU[�VukUX�c�[�ZJ��[�ãUh�[�R�e�e�gU[�bmVGc�[�ZWh�[�kocU_pk�erZW��z

Page 8: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

bU*Y0 Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ïy�\_i)W-�3MD�¯�P~<W�|�Q�e)��\DhP~<��OGW¡Md�}�g��MDi:��eJ\[i:W>�3i:W>^6W>Uge;WY�DW>icj�f>\DUPf>WYQ���MDOP~�W¡MDf-e\Dy±f>\_�¯�X\DUPW>Ugeºi:WY�P~�MDfYW>�¯W>Uge�`{^6Q<�¯Q<~�M�i:~�jD|�W-�_WYi6j¯f>\_U3f>W>Q���MDOP~<W�fY\_UPUPWYfYe:Q<\_U¯eJ\!M�_\D~�e8M��_W}^:\_hPi6f>W�|3f>h3i:i:WYUge�^:\DhPi:fYWD|3W-eJf�m´ozMD^�M!U3\7�PWNQ<U;eJ�3W��PQ<MD�_iJM���mF¤¥UP^6e:WdMD�±|eJ�3W�WY�XW>fYe:^È\DyG^:hPf8�¯i:WY�P~�MDfYW>�¯W>UgeJ^ÈM�i:W�Q����P~�Q<fYQ�eFQ<UVe:�PW�f>Q�i:f>h3Q�e¦�3Q�MD�_i:MD�âQ�e:^:W>~�yMDUP��fdMDU[OXW!�PW>�3hPf>WY�­y�i:\_� e:�PW)�3Q�MD�_i:MD�a|P�_Q��DW>U­Q�eJ^}fdM�hP^JMD~FQ<Uge:W>i:�3i:WYeJM�eJQ�\_UÑm¤¥UØWYf>\_U3\_��W-eJi6Q<f°�¯\7�PWY~<Q�UP�¾~<Q��DWY�}Q�^:W�|�Q�ez��\_hP~��ØOGWaM'�}�g��MDi:�ËeJ\�i:W>�3i:W>^6W>UgeWY�DW>icj;f>\_UPfYW>Q��DM�OP~<W���MDi6Q�MDUgeÈ\DyG�G\_~<Q�fYjVQ�UgeJW>ic�_WYU�e:Q<\_U¯MD^ºM UPW-�Ø��MDi:Q<MDOP~�W�Q<UzeJ�3WW>fY\_UP\_�¯Q<f;W Þ hPM�eJQ�\_UP^>ma¤¥UP^ceJWdM��Ñ|�eJ�3W£WY�XW>f-eJ^)\Dy�^:h3f8��Q<UgeJWYi6�_WYUgeJQ<\DUP^4f>MDU�OGW�PWY�PhPf>WY�4y�i:\D� eJ�PWÈ^ceJi:h3fYeJh3iJMD~�Q<Uge:W>i:�3i:WYeJM�eJQ�\_UA\Dy+eJ�P\_^6W�W Þ h%M�eJQ�\_UP^Y|�Q�y3\_U3~�j]��WfdM�U�eJQ�W!e:�PW!Q<�¯��WY�PQ�M�e:W]WY�XW>f-eJ^]\�y¦WdM�f8�­�X\D~<Q<f-j¡eJ\;eJ�3W4f>\Di:i:WY^:�G\_UP�PQ�UP����MDi6Q��MDOP~�W>^zM�UP���%MDi:MD��W-eJWYi:^VQ�U¿eJ�PW°W Þ h%M�eJQ�\_UP^YmÝ���3W°f>\_�¯�X\DhPUP��MDf-eJQ�\_U·6iJM�Q<^:We8M©�3WY^�MDUP�£~<\'��WYi�Q�U�e:W>i6W>^6e�iJM�e:W>^Y|n¹Vy�\_iºW-�3MD�¯�P~�WD|+UPW>WY�°UP\De�OGW�Q�UgeJi:\+�PhPfYW>�°M�^M UPWY�à��MDi:Q<MDOP~�W�Q<UVe:�PW}W Þ h%M�e:Q<\_U3^>|gOGW>fdM�hP^:W�eJ�3W�WY�XW>fYeº\DyÓe:�%M�eºMDf-eJQ<\DUzfdMDU;OGW�PWY�PhPf>WY�[Q�y���WA�%Md�_WAeJ�3W Þ hPMDUgeJQ�e:Q<WY^¯·ce8M2�3M�e:Q<\_U£~<WY�DW>~�¹�M�UP�Ë·6Q<UgeJWYi:WY^6e}iJM2eJW>^6¹MD~�i:WdM��3j�i6W>�Pi6W>^:WYUgeJW>�ØMD^�`{WYQ�eJ�3W>iVWÝ�3\D�_W>UP\DhP^�\DizW>UP�P\D�_W>UP\DhP^8o4��MDi:Q<MDOP~�W>^VQ�UeJ�3W!W Þ h%M�eJQ�\_UP^Ym���PWlMDOPQ�~<Q�e¥j¾eJ\¾�Pi6W>�PQ�fYeVe:�PW¡WY�XW>f-e;\DyNQ<UgeJWYi6�DW>UgeJQ�\_UP^4�}Q�eJ�P\_h7ezW>U�hP�¯W>i¬�M�e:Q<UP�;eJ�P\D^:W4Q�UgeJW>ic�_WYU�e:Q<\_U3^�Q<U[MD�3��MDUPfYW4Q�^T\_U3W4\Dy�eJ�PW)�£MDQ�U[M��3��MDUge8MD�_WY^T��W�Pi:M'�Ëy�i:\_�fdM�hP^JMD~%�¯\+�PW>~�Q<UP� M�UP�;\_UPW�\DyGeJ�PW���MDQ�U4y�hPUPf-eJQ<\DUP^=^:WYi6�_WY�¡OgjVeJ�3WUP\De:Q<\_U!\Dy3fdMDhP^:M�eJQ�\_UÑm x Q<UPfYWºe:�PWºU+h3�VOXWYi�\DyPMDfYe:Q<\_U3^�\_iFMDf-eJQ�\_U4fY\_�VOPQ�U%M�eJQ�\_UP^Q<^zWYUP\_i6��\_h3^>|�e:�PWYj�fdMDUPU3\De¯OXWli:WY�Pi:WY^:WYU�e:W>� WÝ�7�P~<Q�f>Q�e:~�j�Q<U�eJ�3W��¯\7�3W>~�OPh7eiJM2eJ�PWYi!�4hP^6e!OGW�Q�UP�PW-�7WY�¿Ogj¾eJ�PW;�Pi6\_�G\_^:Q�eJQ<\DUP^]eJ�%M2e)W>MDf8��M�fYeJQ�\_UÙW>U3y�\Di:f>WY^�PQ�i:W>f-eJ~�j_m�¤¥U3�PQ<i6W>fYe�f>\DUP^:W Þ h3W>UPfYW>^�\Dy±W>U3y�\_i6f>Q�UP�!e:�P\_^6W��Pi6\_�X\D^:Q�e:Q<\_U3^ÈMDi:W�eJ�3W>UQ<U7y�W>i:i6W>�ly�i:\_�BeJ�PW!fdMDh3^JMD~Si:WY~�M�e:Q<\_U3^:�PQ��P^}MD�¯\_UP�zeJ�PW ��MDi6Q�MDOP~�W>^�i6W>�Pi6W>^:WYUgeJW>�Q<U)eJ�PW��¯\7�3W>~0m�½�W��}Q<~�~+i6WYe:hPi:UzeJ\TeJ�PQ�^�eJ�PWY��W�Q<U;µ��%M��3eJWYi�*z` x WYfYeJQ�\_U�*7mp³7m¼¨�o-|�}�PWYi:W���W]y�hPi6e:�PW>i�W-�7�P~�\_i:W�e:�PW�Q�Ug�DM�i:Q�M�UPf>W�MD^:^6hP���7eJQ<\DUP^�e:�%M�e��4hP^6e�OXWT��W-ey�\_i�e:�PQ<^�WYUPf>\+�PQ<U3�¯^6f8�PW>�¯WAeJ\z��\Di:�ÓmðTñ6òºñrä óãô±õPö¬÷�ø]ùËûÈø]üæå�÷�ç ø�õ��Sþ2èÝû=ô±õ�ç]ûJ�cùk�^]MDUÙMD~�e:W>i6U%M�eJQ��_W!eJ\�°�Mdj_WY^:Q<MDUÙf>\_UP�3Q�eJQ�\_UPQ�UP�P|Ó�P�PQ�~<\_^6\_�P�PWYi:^;`>éSW-�}Q<^Vb��*DÀ.+ AMDi:�PWYU3y�\_i6^ b���/�/goà�%Md�_Wâ^6e:hP�PQ<WY��M�UP\DeJ�3W>i×�Pi6\_O%MDOPQ�~<Q�e¥j e:iJMDU3^6y�\_i6�£M�e:Q<\_UfdM�~<~<WY�Ë·6Q<��MD�_Q�UP�P|¼¹4�}�PQ<f8�l��MD^��PWYW>�¯W>�ahP^:W-y�hP~FQ<U°eJ�PW)MDU%M�~�j+^:Q�^�\Dy¦^6hPONM6hPUPfÝ�eJQ��_W¡fY\_UP�PQ�eJQ�\_U%MD~�^4MDU3�®�}�3Q<f8���¯\_i:W°M��PW Þ h%M�e:W>~�j¿i6W>�Pi6W>^:WYUgeJ^;eJ�PW¡e:iJMDUP^cy�\_ic���M�eJQ�\_UP^¦M�^:^:\+f>Q<M�eJWY�z�}Q�eJ�;M�fYeJQ�\_UP^Ym�½��PW>i6WdMD^�°�MdjDW>^Èf>\_U3�PQ�e:Q<\_UPQ�UP�]\Dy ;�`WÅ.?ÉÆ©oeJi:MDUP^cy�W>i:^]e:�PWzW>Uge:Q<i:W4�Pi:\_OPMDOPQ<~�Q�e¥ja��MD^:^�y�i6\_� ^6e8M2eJW>^ WÝ�7f>~<h3�PW>��OgjËÆzeJ\¡eJ�3Wi:WY�£M�Q<UPQ�UP�¾^6eJM�eJWY^¾`{Q�U®�Pi:\_�G\_iceJQ<\DU®e:\�e:�PW>Q�i¯fYhPi:i6W>Uge��Pi:\_OPMDOPQ<~�Q�e:Q<W>^Y|�;£`�Å2o6o-|

Page 9: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��_ÊË̬ÎzÅ�Ç Í�Ô���ÁNÅFÌ-Í�Î bU*+bQ<��MD�_Q�UP����\Di:��^[�PQ��±WYi:WYU�e:~�j��×WdM�f8�åW-�7f>~�hP�PWY� ^6e8M2eJWêÅ¿e:iJMDUP^cy�W>i6^[Q�e:^���MD^6^Q<U3�PQ��+Q<�3h%MD~<~�jÙeJ\�MÙ^6W>~�W>fYe�^:W-e£\�y�^ce8M�e:W>^²ëJìd`WÅ©o4eJ�%M�e�MDi6Wlf>\_UP^6Q<�PWYi:WY�ËeJ\�OGW·6fY~<\_^6W>^6e:¹4eJ\�ů`�^:WYW x W>f-eJQ�\_U�*7m¼¨Pmn¶go-m¦kT~�e:�P\_hP�_�£�Pi:\'�+Q<�PQ�UP�VM!��\Di:W]M��PW Þ h%M�e:WMDUP�Ù�_WYUPW>i:MD~�y�i:MD��W-��\Di:��y�\DiAMDfYe:Q<\_UP^z`W  Q<OPO%M�i:�ÙMDUP��RTMDi:�GW>izb���*DÀgo-|±Q<��MD���Q<U3�T~�WdMd�_WY^�eJ�3W��Pi6W>fYQ<^:W�^:�GW>fYQ��%f>M�eJQ�\_U4\Dy7eJ�PW�^:WY~<W>f-eJQ�\_U!y�h3UPfYe:Q<\_U~ë ì `WÅ©oFM�~<�¯\_^6ehPUPfY\_UP^ceJiJM�Q<UPWY�Ñm�µ�\_UP^6W Þ hPW>Uge:~�j_|ÈeJ�3W°�Pi:\_O3~<W>��\Dy}WYU�hP��WYiJM�e:Q<UP��y�h3eJhPi6W°MDfÝ�eJQ�\_UP^�Q�^�i6W>�P~<MDf>WY�°O�j�eJ�PW �3i:\_OP~�W>� \DySW>U3f>\+�PQ<UP�z�3Q<^6eJMDUPfYW>^}MD�¯\_UP�V^ce8M�e:W>^�Q�UM���Mdj¿eJ�PM�e4��\_hP~���OXW£OX\De:��W>fY\_UP\_�¯Q<f>MD~�MDUP��i6W>^:�GW>f-e:y�hP~�\Dy}f>\_�¯�¯\_U¿hPU+��PWYi:^6eJMDUP�PQ�UP�V\Dy±eJ�PWTfdMDhP^:MD~±~<Md�}^�eJ�PM�e�\D�XWYiJM�e:W�Q<U¯eJ�PWT�P\_��MDQ<U±mf���PW]^6W>fY\_UP�i:W Þ hPQ<i6W>�¯W>Uge�Q�^�UP\�e�eJi6Q��+Q�M�~0|_fY\_UP^:Q��PW>i6Q<UP�!eJ�%M2e�Q<U3�PQ<i6W>fYe�iJM���Q��%fdM�e:Q<\_U3^=\DySMDfÝ�eJQ�\_UP^)\DyEeJWYU�i:W>^6hP~�eVQ<U¿��\_i:~��P^)e:�%M�e;M�i:W Þ hPQ�eJW£�3Q<^:^6Q<�¯Q<~<MDi eJ\[e:�PW¡\_UPW�y�i:\_��}�PQ�f8�a��W)^ce8MDice)`/§�Q�UPWzb���*Dä_o-m

���PWz�PQ�Ñ�fYhP~�e:Q<WY^]MD^6^:\+f>Q�M2eJW>���}Q�eJ�¾��MD��Q<U3��eJ�3WVf>~�\_^:WY^6e6�0��\Di:~<�ÙM��P�Pi:\_MDf8�f>\DU3y�\_i:� eJ\!fdMDhP^:MD~Ó~�Md�}^=�}Q<~�~3OGWNy�hPiceJ�PWYiÈW>~�M�OX\_i:M�eJWY�;Q<U£µ��%MD�3e:W>i�*�` x W>f-eJQ�\_U*7m¼¨�o-m����PWz^ceJi6hPfYe:hPiJMD~¦MD�P�Pi6\gMDf8�¾�PhPi6^:hPWY��Q�U­eJ�PQ�^�OG\7\D��WY^:fdM��XWY^Ae:�PW>^6Wz�PQ�y���%fYhP~�e:Q<W>^}Ogj°O%M�^:Q<U3�¯e:�PWAUP\DeJQ�\_Ul\Dy�Q<UgeJWYi6�_WYUgeJQ<\DUP^T�3Q<i:WYfYe:~�j°\_Ulf>MDhP^JM�~F�¯W>f8�+�MDUPQ�^:�¯^]MDU3�¾O�j[fdMD�3Q�e8M�~<Q<ÚYQ<UP�¯\_U­e:�PWV�Pi6\_�XWYi6e:Q<W>^�\DyÈQ<Ug��MDi:Q<MDUPfYWzMDUP�ÙMDh7eJ\_U7�\_�4j¯eJ�PM�e}MDfYf>\_�¯�%MDUgj¯eJ�PWY^:WA��WYf8�%MDUPQ�^:�¯^>m`���3Q<^��¯W>f8�%MDU3Q<^:�z�¥�¯\+�PQ��%f>M�eJQ�\_UMD�P�3i:\gMDf8��fdM�U�OGWz�7Q�WY��W>�¿MD^!Ma^6�XWYf>Q�M�~¦Q<UP^ce8MDU3f>W¯\Dy�e:�PW;f>~�\_^:WY^6e6�0��\Di:~<��MD�7��Pi6\gMDf8�Ñ|N�}�PW>i6W�e:�PW¾f>~�\_^:WYUPW>^6^��¯WdMD^6hPi:W¾Q�^°f>i:M�yEeJWY� ^6\ËM�^¡eJ\®i:WY^:�GW>fYeaeJ�3WfdM�hP^JMD~T��WYf8�%MDUPQ�^:�¯^¡Q<UËeJ�PW­�3\_�£M�Q<U�+NeJ�PW¾^6W>~<WYfYe:Q<\_UØy�h3UPfYe:Q<\_UÐë ì `�Å2ozeJ�%M2eW>U3^:hPWY^�Q<^�i6W>�Pi6W>^6W>UgeJWY�­Q<U�`/¶7m�b_b'o]`{^6W>W!�PQ<^6f>hP^6^:Q<\DUae:�%M�e�y�\_~�~<\'�}^8oÝm

���PW[\D�XWYiJM�e:Q<\_U%M�~<Q�e¥j�\DyTeJ�PQ�^¯�¯W>f8�PMDUPQ<^6�;�«��\+�PQ��%fdM�e:Q<\_U�^:WY�£MDUge:Q<f>^;��MD^�PWY��\_U3^6eJi:M�eJWY��Q�U�µ��%MD�3e:W>i)¶�MDU3��~�W>�¾eJ\leJ�PW Þ h%MDUge:Q�e8M2eJQ��DW;�Pi:WY�PQ<f-eJQ<\DUP^ \DyeJ�3W¾WY�XW>fYe:^l\DyVM�fYeJQ�\_UP^Y|TQ�UPf>~�hP�PQ<U3��MDfYe:Q<\_U3^£eJ�%M2e¡��WYi:W�UP\De°fY\_UgeJWY���P~<M�eJWY��PhPi6Q<UP��eJ�PWz�¯\7�3W>~W�¼^Nf>\_UP^ceJi6hPfYe:Q<\_UÑm����PW;���;fdMD~�f>hP~�hP^�e:�%M�e]WY��WYi:�_WY�Ë`>���PWÝ�\_i6W>� ¶3m¼¨Pm�b'o�WÝ�+eJW>U3�P^]eJ�PQ�^T�3i:W>�3Q<fYe:Q<\_Uly{MDfYQ<~<Q�e¥j£e:\£fdM�^:W>^T�}�PW>i6W4^:\D��W)\Dy¦eJ�3W��MDi:Q<MDOP~�W>^�MDi6W]hPUP\DOP^:WYi6�_WY�Ñm�¤¥U�µ��%M��3eJWYi�*)��W y�hPi6e:�PW>i�h3^:W eJ�PW �¯W>f8�PMDUPQ<^6�;��¯\7�PQ��%fdM2eJQ<\DU Q<UgeJWYi:�Pi6WYe8M2eJQ<\DU e:\ �3i:\'�+Q<�PW�^:WY�£M�U�e:Q<fY^®y�\DiàfY\_hPUgeJWYi6y{MDf-eJh%M�~^6eJM�eJWY��WYUgeJ^>|7MD^È\_h3e:~<Q�UPW>�;Q<U x WYfYe:Q<\_Uab_m¼¨Pmn¨3m¦¤¥U¯e:�PQ<^Èf8�%M��3eJWYi>|���W}�}Q<~�~3W-�+e:W>UP�eJ�3WVMD�P�3~<Q<f>M�eJQ�\_UP^}\Dy¦e:�PWV���;fdMD~�f>hP~�hP^NQ<U[^6WY�DW>iJM�~��PQ�i:W>f-eJQ�\_UP^>|XMD^T\_h3eJ~�Q<UPWY�[Q�UeJ�3W¸¯=i6WYy{MDfYWDm

Page 10: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

bU*_³ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ïæ­çpí î ì ê í ïYë=ï ì ê v¡ï ð w ë=ï ì êËñ v ê í ò ë ìlwjó­v ñFë=ï w

q ì-ï ï w ï t ñ���PW!Q<UgeJWYi6�DW>UgeJQ�\_UP^Nf>\_UP^6Q<�PWYi:WY�[Q<Ul\_h3iTM�U%MD~�j+^6Q<^�\Dy�Q<�PWYUgeJQ��PfdM�e:Q<\_U�` x W>fYe:Q<\_U3^¶3mn¶Uô7¶3m¼¨�o¦��WYi:W]~�Q<�¯Q�e:W>�;eJ\VM�fYeJQ�\_UP^ºeJ�%M�e���WYi:WY~�jzy�\_i6f>W MA�DM�i:Q�M�OP~<WN\_i�M4�_i:\DhP�\Dy��DM�i:Q�M�OP~<WY^A� eJ\ae8MD��W�\DU¿^:\D��W;^:�GW>fYQ��%WY�¿��MD~<h3W;�Sm£¤¥U��_W>U3W>iJM�~�`�^:WYW�eJ�3W�Pi6\7fYW>^:^°fY\_UgeJi6\_~�WÝ�PM����P~�W­Q<U x W>f-eJQ<\DU�¶3m´³+mp¶goÝ|�Q<UgeJWYi6�DW>UgeJQ�\_UP^°��M'jËQ<Ug�D\_~��DWf>\D���P~�W-�­�X\_~�Q<fYQ<W>^TQ<U­�}�PQ�f8�¿M£��MDi6Q�MDO3~<W4� Q<^��£MD�3WVeJ\°i:WY^:�G\_UP��Q<U�M¡^:�GW>fÝ�Q��PW>����MdjÙeJ\a^:\_�¯W�^6WYe~õ \Dy�\DeJ�PWYi!��MDi6Q�MDO3~<W>^¥ß�^:M'jD|�eJ�Pi6\_hP�_�¿May�hPUPf-eJQ�\_U%MD~i:WY~�M�e:Q<\_U3^:�PQ��4��6÷ö±`{¸go�\_i�e:�Pi:\_h3�_�¯MA^ceJ\+f8�%MD^ceJQ<f�i6W>~<M�eJQ�\_UP^:�3Q<�4�}�PWYi:WYO�j;� Q�^^:W-e�e:\��4�}Q�e:�)�3i:\_O%M�OPQ<~�Q�e¥js; ì `��l? ¸_o-m�½�W=�}Q<~�~_^:�P\'�A|�O%MD^:WY�)\DU¡¯FWdMDi6~±`¬b����D¨DOÓo-|eJ�PM�e4Q��PW>Uge:Q�yEj+Q<UP�leJ�3W£WY�XW>f-eV\Dy}^6hPf8���G\_~<Q�f>Q�W>^AQ<^!W Þ hPQ��DM�~<W>Uge!eJ\�Q<�PWYU�e:Q�yEj+Q<U3�eJ�3W!WÝ�7�Pi:WY^:^:Q�\_Uø;�`>=@?Wù��C8¸goÝmé�WYe";�`>=@? ���7`��Ö68öÑ`{¸go:o6o}^ce8MDU3�[y�\_i�eJ�PW)�PQ<^ceJi6Q<OPh3e:Q<\_U�`�\DyÈ¢Vo��Pi6WY��MDQ<~�Q<UP�hPUP�3W>i4e:�PW£�G\_~<Q�fYj����7`�� 6Òö±`/¸_o:o-m±��\[f>\_�¯�Ph3e:WÏ;�`>=@? ���3`E� 6úö±`{¸go:o6o-|���Wf>\DUP�PQ�e:Q<\_U°\DUøõ MDUP�l�}i:Q�eJW

;�`>=@? ���7`��û6÷ö±`{¸go:o6o6 7pü ;�`>=@? ���7`��û6«ö±`/¸go6o)C8¸goB;£`{¸ ? ���7`��ý6÷ö±`/¸_o:o:o6 7 ü ;�`>=@?Wù��C8¸go�? þ)ÿ���� ü � ;£`{¸go6 È ü�� ;�`>=@?Wù�uC8¸_oU? þ)ÿ���� ü ���

���PW!W Þ h%MD~�Q�e¥j ;�`/¸ ? ���3`E�Ö6«ö±`/¸go6o:oJ6Ä;�`/¸go^6e:W>�¯^>|¦\�y�f>\_h3i:^:W�|�y�i:\D� eJ�PW¯y{MDfYe!eJ�%M2e�õ f>MDUPUP\�eVOGW£Ma�PW>^6f>W>U3�%MDUge;\Dy��ê+�PWYUPf>W�|ÓMDUgj°f>\_Uge:i:\_~ÑWÝ�7W>i6e:W>�a\_U°��fdMDUl�%Md�DW!U3\¯W-�±WYfYe}\_U°e:�PW �PQ<^ceJi6Q<OPh3e:Q<\_U\Dy"õ)m ����hP^Y|���W�^:WYWae:�%M�e;eJ�PWafdM�hP^JMD~}W-�±WYfYe�\Dy MÙ�X\D~<Q<f-j����7`�� 6Öö±`/¸go6ofdM�UàOGW°WY��MD~�h%M�e:W>���PQ�i:W>f-eJ~�j�y�i6\_� e:�PW°W-�7�Pi6W>^6^:Q<\DUË\Dys;�`>=@?Wù�uCJ¸go4^:Q����P~�j�O�j^:h3OP^6e:Q�eJh7eJQ<U3� ö±`/¸go y�\_i!�ËMDU3�¿e8MD��Q�UP�[eJ�3W£W-�7�GW>f-e8M�e:Q<\_U�\'�DW>i-õ `�hP^:Q�UP�[eJ�3W\_OP^6W>ic�_W>�[�PQ<^ceJi:Q�OPh3e:Q<\_UÇ;�`/¸_o:o-m���PQ<^àQ<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥jfYi:Q�e:W>i6Q<\_U y�\_i®f>\DUP�PQ�e:Q<\_UPMD~¡�G\_~<Q�fYjÜQ�^®^:\_�¯WY�}�%M2e^6e:i:Q�fYeJWYi4eJ�PMDU�eJ�%M�e4y�\_iVhPUPfY\_UP�PQ�eJQ�\_U%MD~ºQ<UgeJWYi6�DW>UgeJQ�\_UÑm�µ�~�WdMDi6~�jD|=Q�yNM­�G\_~<Q�fYj���3`E� 6�ö±`/¸_o:o!Q�^4Q��PW>UgeJQ��ÓMDOP~�W�e:�PW>U�e:�PW°^:Q����P~�W�Q�UgeJW>ic�_WYU�e:Q<\_U®���7`�� 6��XoQ<^NQ<�3W>UgeJQ��ÓMDOP~�W)M�^T��W>~<~/|Ó^:Q<U3f>W)��W4f>MDU¾MD~���Mdj+^�\DO3e8MDQ�Uae:�PW4~<M�e:e:W>i}Ogja^:WYe6eJQ�UP�

Page 11: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��+Ê� ÙÂ�ÆÈÎØÌcÏlÅ�ÂTÆ�ÆÈÕ¦Õ�ÆTÁNÅ�Í�Õ�Ã�ÎËÃ!ÁNÅFÌ-Í�ÎØ̬ԠÆÈÎ;ÅF̬Õ�Ì¥Ã���ÒSÆ��8bU*D¶ö±`/¸go²6��SmÊ���3W�fY\_Ug�_W>i6^:W­�3\7WY^�UP\�e��3\_~<�Ñ|º�P\'��WY�_WYi>|}OGW>fdM�hP^:W�f>\_UP�3Q�eJQ�\_U7�Q<U3��\_UÊõ �¯Q<�_�ge;f>i6WdM�e:W[�PWY�XWYUP�PWYUPf>Q�W>^£e:�%M�e¯�}Q<~<~��3i:WY�DW>Uge°eJ�3W�^6hPf>fYW>^:^cy�hP~i:WY�PhPf-eJQ<\DU�\Dyf;�`>=@?Wù��CJ¸go�eJ\¯M¯�PM�e6�0y�i:W>W WÝ�7�Pi:WY^:^:Q�\_UÑmk�^ceJ\+f8�%MD^6e:Q<f��X\_~�Q<f-j_|��}�3Q<f8��Q����G\_^:WY^4M�UPWY� f>\DUP�PQ�e:Q<\_UPMD~=�PQ<^ceJi6Q<OPh3e:Q<\_U; ì `��l? ¸go¯y�\_i¯�S|Tf>MDU×OXW��%MDUP�3~<W>�×Q�U×M¿^6Q<�¯Q<~�M�i¯��MDUPU3W>i>m ½¿W­i6W>�gM�i:� eJ�3W^6e:\7f8�PMD^6e:Q<fNQ<UgeJWYi6�_WYUgeJQ<\DU°MD^�M)iJM�UP�P\_� �Pi6\7fYW>^6^�Q<U;�}�PQ�f8�£eJ�PWNhPUPfY\_UP�PQ�eJQ�\_U%MD~Q<Uge:W>i6�DW>UgeJQ�\_U ���7`�� 6 �XozQ<^zW>U7y�\_i:fYW>�Ø�}Q�eJ�à�Pi:\_OPMDOPQ<~�Q�e¥j¬; ì `��l? ¸_o-m ����hP^Y|�_Q��_W>UêõÒ6 ¸3|�e:�PW£Q�UgeJW>ic�_WYU�e:Q<\_U��+�7`�� 6B�XoA�}Q�~<~=\+f>f>h3i)�}Q�e:�¿�Pi6\_O%MDO3Q<~<Q�e¥j; ì `��l? ¸go�MDUP�°�}Q<~�~X�Pi6\7�3hPf>W!MzfdMDh3^JMD~ÑW-�±WYfYeN�_Q��_W>UlOgj�;�`>=@?Wù�uCJ¸go-m=kN�_W>i:MD�_Q�UP�\'�_WYiN�­MDUP�a¸��_Q��_WY^

;�`>=3oU? �����mþ�� ü � 6�7 þ 7 ü ;�`>=@?Wù��C8¸go�; ì `E�l? ¸goB;£`{¸go °�WYfdMDhP^6W¬; ì `E�l? ¸go¡Q<^°^6�XWYf>Q��%W>�ãW-�+eJWYi:U%M�~<~�jD|T��WÙ^:W>W�MD�gM�Q<U eJ�PM�e°eJ�PWÙQ��PW>U7�eJQ��ÓMDOPQ�~<Q�e¥j�\Dy�;�`>=@?Wù��CJ¸go�Q<^¡M¿UPW>fYW>^6^JMDicj�MDUP�×^6h.ѯf>Q<WYUge°f>\_U3�PQ�e:Q<\_Uày�\_i£eJ�3WQ<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥j[\Dy}MDUgj�^6e:\7f8�%M�^6eJQ�f£�G\_~<Q�fYj¾e:�%M�e)^:�%MD�GW>^)eJ�PW¡�3Q<^6e:i:Q�OPh3eJQ�\_UÙ\Dy� Ogj°eJ�PWA\_h3e:f>\_�¯W \Dy`õ)mª y±^6�XWYf>Q�M�~3Q<�¯�G\_i6eJMDUPf>W�Q<Uz�P~<MDUPUPQ�UP�]Q�^=M x �}LN¤�¯��«~<Q<��W�MDf-eJQ<\DUa`/§�Q��DW>^ÈM�UP��TQ�~<^6^:\_U�b��*7b©o��}�P\_^:W4Q<�¯��WY�PQ�M�e:WAWY�XW>fYe:^ � 6â���3W>�GW>UP�Ù\_U­e:�PWV^JM2eJQ<^cy{MDf-�eJQ�\_U¾\Dyº^:\D��WzWYU%MDOP~�Q<UP�°�Pi6W>fY\_UP�PQ�eJQ<\DU��£`�� oT\_UÙM°^6WYe�� \�y���MDi6Q�MDOP~�W>^Ym¸�F\i:WY�Pi:WY^:WYU�e�^:hPf8�[MDfYe:Q<\_U3^>|3��WA~<WYesõ÷6�� ;�¿"!ãMDU3��^6WYe

; ì `��l? ¸_o�6 #$% $&;£`E�l? »(' ! o�Q�y)��`*�]ol6ãy{MD~<^6W¤Cb Q�y)��`*�]ol6ãeJi6hPW!MDUP���ý6ã�@C0 Q�y)��`*�]ol6ãeJi6hPW!MDUP��� +6ã�

æ­ç�, - ó­t ê é@ñë ó­t/.10�t]w ë ì¯u�v ê ð w ë=ï ì ê é í­t ê32ë¦ï54 v76ËïYt78

¤¥Uãµ��%MD�7eJW>i¡¶���W¾�3WY�_WY~<\_�GW>�×^6WY�DW>iJM�~ �_iJMD�3�PQ<f>MD~}f>i:Q�eJWYi:Q�MÙy�\_i£i6W>fY\_�_UPQ�Ú>Q<U3��}�PWYU�eJ�3W�W-�±WYfYe)\Dy�\DUPWz�DM�i:Q�M�OP~<Wz\_U�MDUP\�eJ�PWYi>|L;£`K=L? ���3`E�Xo:o-|�Q�^ Q<�PWYUgeJQ��%MDOP~<WQ<UleJ�3W)�Pi6W>^6W>UPfYW4\Dy�hPUP�¯WdMD^6hPi:WY�[��MDi:Q<MDOP~<WY^>m����PW>^6W4fYi:Q�e:W>i6Q�M3|%~�Q<�DW]e:�PW!O%MDf8�_��P\+\_iN`>���3W>\_i6W>� ¶3mn¶3m´³Do¦MDUP��y�i6\_Uge6�«�P\7\Di}`>���PWY\_i:WY� ¶3mp¶7mn¨�oÝ|�MDi:W}^6�XWYf>Q�M�~Gf>MD^:WY^\Dy�Ma�¯\_i:W;�_WYUPW>i:MD~=f>~<MD^:^!\Dy�^:W>�¯Q��¬Û[MDi6�D\'�+Q�MDUÙ�¯\7�PWY~<^]y�\_i �}�PQ�f8�¿i6W>�GWdM�e:W>�MD�P�3~<Q<f>M�eJQ�\_UV\DyGeJ�3WNQ�U3y�W>i6W>UPfYW}i:hP~�W>^È\Dy±�+�TfdMD~�f>hP~�hP^N`>���PWY\_i:WY� ¶3m¼¨Pm�b'o��}Q<~<~7i:WÝ��PhPfYWD;£`K=L?�ù�GoFe:\AM]�%M�e6�0y�i:WYW�W-�7�Pi6W>^:^6Q<\_U±|�e:��hP^=i:WYUP�PW>i6Q<UP�AQ�e¦Q<�3W>UgeJQ��ÓMDOP~�WDmL���PW

Page 12: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

bU*�¨ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï^:WY��Q �cÛ[M�i:�D\'�+Q�M�U¿�¯\7�PWY~È\Dy}§�Q��_hPi:W�¶3m�b¾`�\_i4§�Q<�_hPi6W£¶7m^/%`Ey8o6oAQ�^4M�U¿W-�3MD�¯�P~�W�}�PWYi:WA�PQ<i6W>f-e}MD�P�P~�Q<f>M�eJQ�\_U£\DyFWYQ�eJ�3W>i�e:�PW O%MDf8�_�¥�3\7\_i�\_i�y�i:\DU�ec�¥�P\+\_i�f>i:Q�eJWYi:Q<\DU��\_hP~<�lUP\�e�OXW ^6h.ѯf>Q�W>Uge}y�\_i�Q<�3W>UgeJQ�yEj7Q�UP��;�`>=@?Wù�Go�M�UP�°j_WYe�e:�PW W-�7�Pi6W>^6^:Q<\DU�Q�^i:WY�PhPfYQ<OP~�Wa`{�PWYUPf>W¯Q<�PWYUgeJQ��%MDOP~<WdoTOgjÙM°^:W Þ h3W>UPfYW£\Dy�Q<U3y�WYi:WYUPf>W;i:h3~<W>^A\Dy����PWÝ�\_i6W>� ¶3m¼¨Pm�b_m�¤¥U�e:�PQ<^N^:WYfYeJQ�\_U���W4WY^6eJMDOP~<Q�^:��M�fY\_���3~<WYe:WAf8�%MDiJM�fYeJWYi:Q�ÚdM�e:Q<\_Ua\DyeJ�3Wzf>~�M�^:^ \Dy���\+�PWY~<^]Q�U­�}�PQ<f8�¾eJ�3W¯f>MDhP^:MD~¦WY�XW>f-e¸;£`K=L?Wù�Go�Q<^]Q��PW>Uge:Q��ÓMDO3~<WVQ�U���4fdM�~<f>h3~<hP^YmðTñrä�ñ6ò 9 þ%û;:=< öcô�ûJ�Ïå�÷�ø]üAö«õ%ö¬÷�ø]ù è-÷ºþ?>gü"�Fø�õ%öA@Aô�û�õ%ö¬÷�ø���PWY\_i:WY� ¨3mp¶3m�bNf8�%MDiJM�fYeJWYi:Q�Ú>W>^ÈeJ�3WTfY~�MD^6^�\Dy�·:���'�«Q<�PWYUgeJQ��%MDOP~<WY¹ ��\+�PWY~<^=Q<U;eJ�3Wy�\_i6� \�y¦y�\_hPiT�_iJMD�3�PQ<f>MD~�fY\_UP�PQ�eJQ�\_UP^>|GMDUgjD\_UPWz\Dy¦�}�3Q<f8�¾Q<^T^:h.ѯfYQ<W>Uge]y�\DiTeJ�3WQ<�3W>UgeJQ��%fdM�e:Q<\_U!\Dy�;�`>=@?Wù�Óo��}�PWYUV��MDU3�z¢ M�i:W�^6Q<UP�D~<WYe:\_U!UP\7�3W>^�Q�U)e:�PW��_i:MD�P�Ñm���PWY\_i:WY�s¨Pmn¶3mp³NeJ�PWYU;MD^:^6W>iceJ^�e:�PW�f>\D���P~�WYe:W>UPWY^:^}`�\_i�UPWYf>WY^:^:Q�e¥j3o=\�y%eJ�PWY^:W�y�\_h3if>\DUP�PQ�e:Q<\_U3^�++\_UPW]\DyS�}�PQ�f8�°�VhP^ce��P\_~��¡Q<U�eJ�PW]�¯\+�PW>~Óy�\_iE;�`>=@?Wù�ÓoÈeJ\zOGW]Q��PW>U7�eJQ��ÓMDOP~�W�Q�U ���ÙfdMD~�f>hP~�hP^>mã½��PWYe:�PW>i£e:�PW>^6W[y�\_hPi�f>\_U3�PQ�e:Q<\_UP^�MDi6W­UPW>fYW>^6^JMDicjQ<U®�_WYUPW>i:MD~!`{Q�UàMDf>fY\_i:�%M�UPf>Wl�}Q�e:�®e:�PWl^:WY�£MDUge:Q<f>^;\Dy�Z]WY�PUPQ�e:Q<\_U®¶3mp³7m¼¨�o4�PWÝ��GW>UP�P^�\DUl�}�3WYeJ�3W>i�eJ�3W Q<U3y�WYi:W>U3f>W i:h3~<W>^�\Dy��+�!fdMD~�f>hP~�hP^�MDi6W]f>\D���P~�WYe:WDml���PQ�^Þ hPW>^ceJQ�\_UÑ|PeJ\ze:�PWAOXWY^6eT\Dy��)j¡��UP\'�}~<WY�P�_WD|ÓQ�^�^6eJQ�~<~X\_�XWYUÑmB�CEDGF(H�DJI KELNMOL*P `� !MD~�~<WY^�MDUP�±¯�W>MDi:~=b��Y�gä_o!S -,%� (D1G9T¢B9+ -1±$2,¥ �,{"�$]&-.E1N#DC< -,¬$21¯²2(DIÝ.{(RQ>C< 8&�.E1¡(A&' -?z.I(TSÙ(DIVUg$2²'.{(D1¯?£$>9+ -C*8#3(DI6(+*-,¬ -IÝ.XW2 :93QZY²#DI6(\[%#^]£K`_B&-H\a×*-.0 -1%,]*8$21G9D.E,{.0$21-%d$2Iz,E#% ;.{9+ -1%,{. b�(RQ>.EC .E,Y$�%x;�`>=@?Wù�ÓoA.<&!,�#3(D,c] &>(D,/.<&K%dY°$21± V$�% ,E#% J%d$2CEC<$2"=.E1N#x%d$2H+Iz*8$21G9D.E,{.0$213&>Kb_mÇ�Ñ#% -IJ �.<&�1±$�Q8(+*�Uo(c9+$'$2IE[3(D,E#E%ÝIJ$2?�� ,¥$}¢ .E1e]z@S,E#3(D,�.<&�f�`���gNg!¢Voih�j K³7mÇ�Ñ#% -IJ A.<&!1±$�9D.EIJ 8*-,¥ :9k[3(D,�#'%ÝI8$2?Ü� ,¬$)¢ .E1^]£K¶3mÇ�Ñ#% -IJ ! 65_.<&-,�&A(V&' -,�$�%�1±$>9+ 8&�À ,�#3(D,lQ>C<$'*�U'& (DCECmQ8(+*�Uo(c9+$'$2Il[3(D,E#g&f%ÝIJ$2?� ,¬$�¢ &'$®,E#3(D,�;£`*n�?�ù�XoÙ.<&­.{9+ -1%,/. b�(RQ>C< >K `_ &A[% 8*-.{(DCA*:(2&' ¿$�%¾,E#+.<&*8$21G9D.E,{.0$21�$'*8*-H+IJ&l"�#% -1³À *8$213&-.<&-,E&[ -1%,{.EI8 -CoY�$�%£1±$21G9+ 8&'*8 -1G9g(D1%,�&l$�%�¿@=.E1¾"�#+.0*8#­*:(2&' x;�`n�?Wù�XoAI8 :9DH%*8 8&A.E?z?£ :9D.{(D,¥ -CoY;,¬$¸;�`n-o>K o¨PmÇ�Ñ#% -IJ V 65_.<&-,=&' -,�&z$�%]1±$>9+ 8&sõ 1 (D1G9~õ 3 &-H%*8#l,E#3(D,qp

`{QEoÇõ 1 Q>C<$'*�U'&V -²D -IdY�9D.EIJ 8*-,¬ :9k[3(D,E#"%ÝIJ$2?Ü� ,¥$4¢`¬.{K{ >K�@�`/¢�gNg �Ê?Úõ 1 oih rts j o)+

Page 13: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��+Ê� ÙÂ�ÆÈÎØÌcÏlÅ�ÂTÆ�ÆÈÕ¦Õ�ÆTÁNÅ�Í�Õ�Ã�ÎËÃ!ÁNÅFÌ-Í�ÎØ̬ԠÆÈÎ;ÅF̬Õ�Ì¥Ã���ÒSÆ��8bU*_ä`{Q�QEoÏõ 3 Q>C<$'*�U'&)(DCECEQ8(+*�Uo(c9+$'$2Iu[3(D,E#g&eQ- -,{"� 8 -1�õ 1 (D1G9V¢`¬.{K{ >K�@�`/¢�gNg¸õ 1 ?Éõ 3 oih j rts op+`�Q<Q�QEo²õ 3 Q>C<$'*�U'&s(DCECvQ8(+*�Uo(c9+$'$2Iw[3(D,E#g&xQ- -,/"� 8 -1 � (D1G9Íõ 1 `¬.{K{ >K�@`E�ygXg¡õ 1 ?Úõ 3 oih�j o)+�(D1G9`{Q��3o�õ 3 9+$' 8&]1±$2,�(+*-,{.E²2(D,¥ )(D1zY{Q8(+*�Uo(c9+$'$2I|[3(D,�#g&`%ÝI8$2?� ,¥$!¢ `¬.{K{ >K�@`E�ygXg)¢±?Éõ 1 Cvõ 3 oih rts jm} r\~q� o `U�Ñ#+.<&N*8$21G9D.E,/.0$21V#%$2C�92&�. %�`{Q�orô±`�Q<Q<Q�o�(DIJ ?£ -,�(D1G9£1±$;?£ -?�Q- -I4$�%xõ 3 .<&)(°9+ 8&'*8 -1G9g(D1%,�$�%�� o`�kã^6�XWYf>Q<MD~ÓfdMD^6W�\Dy±f>\_UP�3Q�eJQ�\_U;¨)\+f>fYhPi:^È�}�PWYU�õ 3 6��)MDUP��e:�PW>i6WTQ�^ÈUP\O%M�f8�g�«�P\+\_i��%M�e:�ay�i6\_�B� eJ\Áõ 1 \_i�y�i6\_� õ 1 eJ\¯¢�mpo

� H�FmF��� $21G9D.E,/.0$21y�Dmj���PQ<^Nf>\_U3�PQ�e:Q<\_Uly�\_~<~�\'�}^��PQ<i6W>fYe:~�j¡y�i6\_� L}hP~<W)³[`�^:WYW¸���PWÝ�\_i6W>� ¶3m¼¨Pm�b'o-m}¤«yN`/¢�gNgA��oih j eJ�3W>U[��WVfdM�U[Q<�¯�¯W>�PQ<M�eJWY~�j¡f8�PMDUP�_W¡;£`K=L?Wù�Go�e:\;�`>=@? �Xo-|7^:\;eJ�PW Þ hPWYi6jlQ<^�Q��PW>Uge:Q��ÓMDO3~<WDm� $21G9D.E,/.0$21v�gmV¤«yºeJ�PWYi:W;Q<^ UP\l�PQ�i:WYfYeJWY���PM�eJ�¾y�i:\D� � eJ\a¢�Q<U�];|ÑeJ�3W>U`/¢�gNg ��oih j mâRTWYUPf>W�|TOgjØLNh3~<W­¶7|s;�`>=@?Wù�Óo±6 ;�`>=3o¯MDU3�×^:\¿e:�PW Þ hPW>icj Q�^Q<�3W>UgeJQ��ÓMDOP~�WDm� $21G9D.E,/.0$21��_mB¤«y)eJ�PWYi:W�Q<^aM®^:WYea\Dy4UP\+�PW>^±À e:�%M�eaOP~�\7f8��^aMD~<~�O%MDf8�_��P\+\_iA�%M�e:�P^)y�i6\_��� e:\­¢ `{Q/mnW�mº`/¢�gXgA�Ê? Àzoih�j oÝ|�eJ�3W>U���W£WÝ�7�%MDUP�ê;�`>=@?Wù�ÓoMD^����N;£`K=L?�ù��C�n-oB;£`*n�?�ù�Xo�MDU3�Ñ|�Ogj L}hP~<W­³7|�i6WY�}i:Q�eJW ;£`K=L?Wù�uC�n-o�MD^²;£`K=L? �@C�n-o-m¤«yºeJ�PW Þ hPWYi6jà`*n�?�ù�±o�Q<^]Q��PW>UgeJQ��ÓMDOP~�WD|±e:�PW>UÙeJ�3Wz\_i:Q��_Q<U%M�~ Þ hPWYi6jÙ�Vh3^6e!MD~�^:\°OGWQ<�3W>UgeJQ��ÓMDOP~�WDm x WYW)WÝ�3MD���3~<W>^�Q�Ua§�Q<�_h3i:W ¨Pm�b_m� $21G9D.E,/.0$21��%m ¤«y�e:�PW>i6W×Q<^�Må^6WYe®\DyaUP\+�PWY^<õ 1 eJ�PM�e�OP~�\7f8�sMD~<~¯�PQ��i:WYfYe:W>�¿�PM�eJ�P^ y�i6\_��� eJ\a¢ M�UP�¿M°^6WYe)\Dy�U3\7�PWY^~õ 3 eJ�%M�e!OP~�\7f8�­MD~<~�O%MDf8�_��P\+\_i��%M�eJ�3^�OGWYe¥��W>WYUB¢ MDUP� õ 1 Q<U�] ! |;e:�PW>U ��W WÝ�7�%MDUP� ;�`>=@?Wù�ÓoÄ6� ü sq� ü ~ ;�`>=@?Wù��C8¸ 1 C8¸ 3 oB;£`{¸ 1 C8¸ 3 ?Wù�XoSM�UP�)i6WY�}i6Q�eJW�;�`>=@?Wù��CJ¸ 1 C8¸ 3 o�MD^G;£`K=L?Wù��C.ù¸ 1 C8¸ 3 ohP^6Q<UP�£LNhP~�W)³7|Ó^:Q<U3f>W)MD~<~�O%M�f8�g�«�P\+\_iT�PM�eJ�P^NOGWYe¥��W>W>UÝõ 1 M�UP�­¢ MDi:W!OP~�\7f8��W>�Ogj�õ 3 Q<U�] ! m ½¿W f>MDU i:WY�PhPf>W ;�`>=@?Wù��C.ù¸ 1 C8¸ 3 oàe:\ ;£`K=L?Wù¸ 1 C8¸ 3 oàhP^:Q�UP�L}hP~<W[¶3|�^:Q�UPf>W�`/¢�gXgA�Ê?Éõ 1 Cpõ 3 oih rts j�} r5~T� m ½�W­f>MDU i:W-�}i:Q�eJWø;�`>=@?Wù¸ 1 C8¸ 3 o�M�^;�`>=@? ¸ 1 CJ¸ 3 o£Q�y�`{¢7gXg¡õ 1 ?Úõ 3 oih rts m ���PW¾\_U3~�jà��Mdj e:�%M�e¡e:�PQ<^¡Q�UP�PWY�XWYUP�PW>U3f>WfdM�UPUP\De��P\_~��Q�^ÙQ�y¯eJ�PWYi:WØQ�^�Mã�%M2eJ�ây�i:\_� ¢ eJ\8õ 1 eJ�3i:\_hP�D���|;^6Q<UPfYW`/¢�gNg¸õ 1 ?Éõ 3 oih j rts m RN\©��WY�DW>iY|!��W�f>MDUãOP~<\+f8�ØeJ�3Q<^°�%M�e:� Ogj�fY\_UP�PQ�eJQ<\DUPQ<UP�MDUP�l^6hP���¯Q<U3�z\©�DW>i�� MDUP�l^:\;�PWYi:Q��_We� þ�� ;£`K=L?�ù¸ 1 C8¸ 3 C6�2Î o�;�`��2Î�?Wù¸ 1 C8¸ 3 o-mJ�T\'���W×fdMDU i6WY�}i:Q�eJW«;�`>=@?Wù¸ 1 C8¸ 3 C:� Î o¿MD^ê;�`>=@? ¸ 1 C8¸ 3 C:� Î o�hP^:Q�UP� L}hP~<W�³+m ���PW;�`��2ÎW?�ù¸ 1 C8¸ 3 o¾eJW>i6� f>MDU OGWØi:W-�}i:Q�e:eJWYU MD^´;�`��2Î�? ¸ 3 oÙhP^:Q�UP� L}hP~<Wض3|£^6Q<UPfYW

Page 14: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

bU*DÀ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï

B2

B1X

B

X

Z

YY

(a) (b)

§�Q��_hPi:Wl¨Pm�b��Ùµ�\DUP�PQ�e:Q<\_Uض�\Dy'���PW>\_i6W>� ¨Pmn¶3m�bDmפ¥Uâ`/MgoÝ|�eJ�3W[^:W-e��oÀ 1 C:À 3Z�OP~�\7f8��^�MD~<~POPMDf8�_�¥�P\+\_iÈ�%M�e:�P^ºy�i6\_�s� eJ\)¢�|+MDUP�²;£`*n 1 C�n 3 ?Wù�±ol68;�`n 1 C�n 3 oÝm¦¤¥U`{O%o-|7e:�PW]UP\+�PW"À OP~<\+f8��^�MD~<~ÓO%M�f8�g�«�P\+\_i��%M�e:�P^�y�i6\_� � eJ\z¢�|PMDUP�Ï;£`*n�?�ù�XoºQ�^Q<�3W>UgeJQ��ÓMDOP~�W]hP^:Q�UP�¯f>\_UP�3Q�eJQ�\_U°¨Pmõ 1 Q�^°M�f8�PQ<~��×\DyA� MDUP�×eJ�3W¾�_iJMD�3�ãQ�^°MDf-j7fY~<Q�fDm ���3W Þ hPWYi6j×f>MDU×eJ�PWYi:WÝ�y�\_i6WTOGWTi6WY�}i6Q�e:e:W>U£MD^ � ü s � ü ~ � þ�� ;£`K=L? ¸ 1 C8¸ 3 C6� Î o�;�`�� Î ? ¸ 3 oB;£`{¸ 1 C8¸ 3 ?Wù�XoÝ|3M�UP����W�%Md�_W~;£`{¸ 1 C8¸ 3 ?Wù�Xoj6 ;�`/¸ 3 ?Wù�Xo�;�`/¸ 1 ?Wù�@C8¸ 3 o-m x Q<U3f>W�õ 3 fY\_UP^6Q<^6e:^]\Dy=U3\_UP�PWY^:f>WYU7��%MDUge:^V\Dy���|���W¡f>MDU�i:W-�}i:Q�e:WÏ;�`/¸ 3 ?�ù�Xo4MD^¡;£`{¸ 3 o!hP^6Q<UP�­LNhP~�W¡¶3m x Q<U3f>W�õ 3OP~�\7f8��^¯MD~<~�O%MDf8�_�¥�3\7\_i;�%M2eJ�P^zy�i:\D� � eJ\³õ 1 |º��WlfdMDUài:W-�}i:Q�e:W�;£`{¸ 1 ?Wù�@C8¸ 3 oMD^s;£`{¸ 1 ? �@CJ¸ 3 o}h3^:Q<U3�£L}hP~<W)³7m����PW4WYUgeJQ<i6W Þ hPWYi6jafdM�U�eJ��hP^NOGW)i6WY�}i:Q�e:e:W>U­M�^� ü s � ü ~ � þ�� ;�`>=@? ¸ 1 C8¸ 3 C:� Î o�;�`�� Î ? ¸ 3 o�;�`/¸ 1 ? �@C8¸ 3 oB;£`{¸ 3 o-m x WYWzW-�3MD�¯�P~<WY^]Q<U¾§�Q<���hPi6W!¨3m´³7m �

B�CEDGF(H�DJI KELNMOLN� �Ñ#% x%d$2H+Ia*8$21G9D.E,{.0$213&°$�%Ý�Ñ#% 8$2IJ -?��GK���K��¾(DIJ ¡1± 8*8 8&8&>(DIdY%d$2IA.{9+ -1%,/. b�(RQ>.EC .E,*Y¯.E1l�+�­*:(DC<*-H+C H�&>KÁ�Ñ#3(D,º.<&8@º. %)(DCEC�%d$2H7Iz*8$21G9D.E,{.0$213&V$�%Ç�Ñ#% )($2IJ -?��GK���K��Á%Y(D.EC�.E1ã(Ë#DI6(\[%#1��@4,E#% -1�,�#% -IJ ¾ 65_.<&-,E&a1±$`bÈ1%.E,¬ l&' ��>H% -1±*8 ­$�%.E1%d -IJ -1±*8 AIÝH7C< 8&!,E#3(D,ºI8 :9DH%*8 8&x;�`>=@?Wù�GoA,¬$£(z#3(D,�(d%ÝIJ 8 z 655[GIJ 8&8&-.0$21GKkâ�Pi:\+\DyF\DyG���PW>\Di:W>��¨3mp¶3mp³VQ<^��_Q��_WYU�Q�U AMD~�~<W>^�MDUP�±¯�WdM�i:~º`cb���gä_oÝmðTñrä�ñQÔ �«� � û=þz¡}ù ÷�ø£¢3¤BôSöB�Fø]ô�פ¥UVQ����P~�W>�¯W>UgeJQ�UP�D���PW>\Di:W>�¨Pmn¶3m�b�MD^�M�^6j+^ceJW>��M�e:Q<fº��W-eJ�P\+�4y�\_iF�PWYe:W>i6��Q�UPQ<UP�Q<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥jD|}f>\_UP�3Q�eJQ�\_UP^a¶àMDUP�ã¨à��\_hP~�� ^6W>W>� e:\Ëi6W Þ hPQ<i6WÙW-�7�%MDhP^ceJQ��_W^:W>MDi:f8�±mT¤¥U­\Di:�PWYiTeJ\��Pi:\'�DW)eJ�PM�e�fY\_UP�PQ�eJQ�\_U[¶¯�P\+W>^�U3\De]�P\D~<�Ñ|%y�\_iNQ<UP^ce8MDU3f>WD|��W¯U3W>W>�ÙeJ\l�Pi6\'�_W;eJ�%M�eAU3\a^6hPf8��OP~<\+f8��Q<UP�l^:W-e�À f>MDUÙW-�7Q<^cedm;§%\_iceJhPU%M2eJW>~�j_|

Page 15: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��+Ê� ÙÂ�ÆÈÎØÌcÏlÅ�ÂTÆ�ÆÈÕ¦Õ�ÆTÁNÅ�Í�Õ�Ã�ÎËÃ!ÁNÅFÌ-Í�ÎØ̬ԠÆÈÎ;ÅF̬Õ�Ì¥Ã���ÒSÆ��8bU*�*

Z1

Y

X

(a) (b) (c)

Z1 Z1

Z2 Z2

Y

X

Y

X

§�Q��_hPi:W�¨3m´³,��µ�\_UP�3Q�eJQ�\_U;¨A\Dy����PWY\_i:WY� ¨Pmn¶3m�bDm�¤¥U[`/MgoÝ|.õ 1 OP~<\+f8��^�M�~<~P�PQ�i:WYfYeJWY��%M�e:�P^�y�i6\_� � eJ\4¢£|3M�UP�¡eJ�PWTW>�¯�3e¥j�^:WYe�OP~<\+f8��^�MD~�~GOPMDf8�_�¥�P\+\_i��PM�eJ�P^�y�i:\_�õ 1 e:\�¢�Q�U�] ! M�UP�àMD~<~�O%MDf8�_�¥�P\+\_iz�%M2eJ�P^zy�i:\D� � eJ\³õ 1 Q�U¥]¯m�¤¥U `{O%oMDUP� `{fdo-|fõ 1 OP~<\+f8��^4M�~<~=�PQ�i:W>f-eJWY���%M2eJ�P^Ay�i:\_� � eJ\�¢�|�MDUP�´õ 3 OP~�\7f8��^)MD~<~O%MDf8�_�«�P\7\Di��%M�e:�P^Ty�i6\_� õ 1 eJ\¡¢ Q<U¦] ! MDUP�¾M�~<~SO%MDf8�_�¥�3\7\_iN�%M�e:�P^�y�i6\_� �eJ\Áõ 1 Q�U�]¯meJ�3W]y�\_~<~�\©�}Q�UP�4e:�PW>\_i6W>�¯^}MD~<~�\'� h3^�eJ\;^:Q��_UPQ��PfdMDUgeJ~�j£�3i:hPUPW e:�PW ^:W>MDi:f8��^:�%MDfYW^:\�MD^�eJ\;i6W>UP�PWYi�eJ�PW eJWY^6e}eJi:MDfYeJMDOP~�WDmB�CEDGF(H�DJI KELNMOLNM¨§ %E;�`nV©$?�ù�Xo].<&].{9+ -1%,{. b�(RQ>C< L%d$2I!$21± �?z.E1%.E?¯(DCÓ&' -,uÀ"©/@=,E#% -1;�`n�ª�?Wù�Xo!.<&!.{9+ -1%,{. b�(RQ>C< l%d$2I!(D1zY°$2,�#% -IA?z.E1%.E?¯(DCÓ&' -,fÀ«ª�K���PWY\_i:WY� ¨Pmn¶3mn¶]MD~�~<\'�}^�h3^�eJ\TeJW>^ce¦fY\_UP�PQ�eJQ<\DUV¶T�}Q�eJ�VMT^:Q<U3�_~<Wº��Q�UPQ<��MD~_O3~<\+f8�g�Q<U3�)^6WYe�À�m¦¤«y@Às�¯W>W-eJ^ÈeJ�PWTi:W Þ hPQ�i:WY��WYU�e:^�\Dy±f>\_U3�PQ�e:Q<\_U�¶!e:�PW>U¯eJ�PW Þ h3W>i6j¯Q�^Q<�3W>UgeJQ��ÓMDOP~�WY+P\De:�PW>ic�}Q<^6WD|%f>\DUP�PQ�e:Q<\_Ua¶zfdM�UPUP\DeTOXWA^:M�eJQ�^6�%WY�Ñm�¤¥Ua�Pi6\©�+Q�UP�zeJ�PQ�^eJ�3W>\_i6W>�a|3��W!hP^6W!e:�PW y�\_~<~�\©�}Q�UP�4~�W>�¯�£M7m¬ D­I®I®¯�K LNM°L±K²§ %A,E#% ��>HÓ -IdY¡;£`K=L?Wù�Óo!.<&!.{9+ -1%,{. b�(RQ>C< )(D1G9£. %)(¯&' -,�$�%A1±$>9+ 8&õ C .0 8&;$21¿(l9D.EIJ 8*-,¥ :9�[3(D,E#¸%ÝI8$2? � ,¥$V¢;@�,E#% -1Ù,�#% ��>HÓ -IdY�;�`/¸�?Wù�Xo).<&4.{9+ -12(,{. b�(RQ>C< >KB�CEDGF(H�DJI KELNMOLN³ !S -,Ñ¢ 1 (D1G9]¢ 3 Q- N,{"�$ &-H´Q-&' -,E& $�%}1±$>9+ 8&�&-HÓ*8#¯,E#3(D,= -.E,�#% -I`{Q�o�1±$�1±$>9+ 8&4¢ 1 (DIJ ¡9+ 8&'*8 -1G9g(D1%,�&°$�%!� $2I°`�Q<QEo�(DCECÈ1±$>9+ 8&V¢ 1 (D1G9l¢ 3 (DIJ 9+ 8&'*8 -1G9g(D1%,E&�$�%�� (D1G9 (DCECg1±$>9+ 8&F¢ 1 (DIJ º1±$21G9+ 8&'*8 -1G9g(D1%,�&�$�%F¢ 3 Kl_ãIJ :9DH%*-.E1N#&' ��>H% -1±*8 f%d$2Ij;£`K= 1 C$= 3 ?Wù�Xo! 65_.<&-,�&7µN[% -I � $2IJ$2CEC�(DIdY���K¶�GK·�Z¸4. % (D1G9£$21%CoYV. %T,�#% -IJ (DIJ !IJ :9DHÓ*-.E1N#;&' ��>H% -1±*8 8&l%d$2IkQ-$2,E#Á;£`K= 1 ?�ù�Xo)(D1G9~;�`>= 3 ?Wù�@C:= 1 oYK

Page 16: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

bU*Y/ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï

Y2

Y1

Z

X

§�Q��_hPi:W!¨Pmn¶.�D���PW>\Di:W>� ¨Pmn¶3m�bVW>UP^6hPi:WY^AM£i:WY�PhPfYQ<UP�¡^6W Þ hPWYUPf>Wzy�\_i�;£`K= 3 ?�ù�@C:= 1 oMDUP�±;�`>= 1 ?Wù�XoÝ|ÓMD~�e:�P\_hP�_�°U3\_UPWAW-�7Q<^ceJ^}y�\_iD;�`>= 1 ?Wù�@C:= 3 o-m���PW��Pi:\DO%MDOPQ�~<Q�e¥jÞ;�`>= 1 C:= 3 ?Wù�Xo¡�¯Q<�_�ge°�%M�^:^°eJ�PW¾e:W>^6eaQ�U÷���3W>\_i6W>� ¨Pmn¶3m�b�Q�y��W�MD�P�3~�j e:�PW��3i:\+f>W>�3hPi:W�eJ\àOX\De:��;�`>= 3 ?Wù�@C$= 1 o°MDUP�8;£`K= 1 ?�ù�Xo-|]O3h3e[Q�y4��WeJicj�eJ\�MD�3�P~�j�eJ�PW¡e:W>^6e;e:\³;�`>= 1 ?Wù�@C$= 3 o4e:�PW>U®��W°�}Q�~<~�U3\Dez�%UP�ËM¾i:W>�3hPf>Q�UP�^:W Þ hPW>UPfYW�\�yTi6hP~<WY^>m�§�Q��_hPi:W�¨Pmn¶­^:�P\'�}^�M6hP^6eV^:hPf8�ËMDU�W-�3MD�¯�P~�WDm®���PW>\Di:W>�¨Pmn¶3mpä[�_h%MDi:MDUgeJW>WY^zeJ�%M�e>|ÈQ�y}eJ�PWYi:WlQ<^VMÙi6W>�Ph3f>Q<U3��^6W Þ hPWYUPf>Wly�\_i�;�`>= 1 C:= 3 ?Wù�Xo-|eJ�3W>U£��WT^6�P\_hP~��¡MD~���Mdj+^�OGWTMDOP~<W�e:\)�%UP��^:hPf8�¡M)^:W Þ hPW>UPfYW]y�\_iÈOX\�eJ�Ï;�`>= 1 ?Wù�XoMDUP�±;�`>= 3 ?Wù�@C:= 1 o�Ogjl�Pi:\D�XWYi�f8�P\_Q<fYWA\Dy¦¢ 1 mB�CEDGF(H�DJI KELNMOLN¹¨§ %z,�#% -IJ ° 65_.<&-,E&�(l&' -,jõ 1 ,�#3(D,N?£ 8 -,E&�(DCEC�$�%;,E#% ;IJ ��>H7.EIJ )(?£ -1%,E&u%d$2I�õ 1 .E1�*8$21G9D.E,/.0$21��3@F,�#% -1¡,E#% �&' -,¦*8$213&-.<&-,/.E1N#z$�%�,�#%  *8#+.EC�9DIJ -1l$�%¦�.E1%,¥ -I8&' 8*-,¥ :9!"=.E,E#z,E#% N(D1±*8 8&-,¥$2IJ&�$�%¦¢ "=.ECECP(DCp&'$]?£ 8 -,F(DCECÓ$�%�,E#% }IJ ��>H7.EIJ -?£ -1%,E&%d$2Isõ 1 .E1�*8$21G9D.E,/.0$21º�GK���PWY\_i:WY� ¨Pmp¶7mpÀ�i:WY��\'�DW>^leJ�PW�UPW>WY� e:\à^6WdMDi6f8� y�\_i�õ 1 Q<U�f>\_U3�PQ�e:Q<\_U�¨®\Dy���PWY\_i:WY��¨Pmn¶3m�b_mJ¯=i:\+\Dy�^�\Dy@���PWY\_i:WY��^�¨3mp¶3mn¶Uô+¨3mp¶3mnÀVMDi6W]�_Q��_W>U°Q�U� AMD~<~�W>^�M�UP�¯�W>MDi:~º`¬b����gäDo-mðTñrä�ñrä Õ8�SþÓöA»�öcø�¼âû å´�c÷ºù.�Sü¾½À¿]÷ºþ(� ¢�ÁÂ:]þ2�Sù3ù3öc÷�øÒèÝ÷�þ§å�÷�ø"½

õPþ%÷��Äà ç'�SþÓö��Fù���PW M�~<�_\_i6Q�e:�P� �PWY�PUPW>�°OgjÏ���3W>\_i6W>� ¨Pmp¶7m�b]U3\De�\_U3~�j¯�PWYe:W>i6��Q�UPW>^�e:�PW]Q��PW>U7�eJQ��ÓMDOPQ�~<Q�e¥j;\Dy�MVf>\_UgeJi6\_~ Þ hPWYi6j°OPh7e}MD~<^6\z�Pi:\'�+Q<�3W>^}Mzf>~�\_^:WY�7�«y�\_i6�BWÝ�7�Pi:WY^:^:Q�\_Uy�\_i�;�`>=@?Wù�Óo¦Q�U¯e:W>i:�¯^È\DyGeJ�PWN\_OP^6W>i6�DW>�¡�3i:\_O%M�OPQ<~�Q�e¥j)�PQ<^ceJi:Q�OPh3e:Q<\_Ua`E�}�PW>U�^:hPf8�Mzf>~�\_^:WY�ay�\_i6��WÝ�3Q�^6e:^8o}MD^�y�\_~<~�\'�}^>m

ÅGÆRÇ�È5ÉVÊÌË­ÇJÍ µ�~�\_^:WY�%§%\_i:��`�;�`>=@?Wù�%o:o\Î

Page 17: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê��+Ê� ÙÂ�ÆÈÎØÌcÏlÅ�ÂTÆ�ÆÈÕ¦Õ�ÆTÁNÅ�Í�Õ�Ã�ÎËÃ!ÁNÅFÌ-Í�ÎØ̬ԠÆÈÎ;ÅF̬Õ�Ì¥Ã���ÒSÆ��8bU*Y�ϱÇRÐRÆÑÉ\ÍÂÒmË�ÇÑÉ�Ó�Ë­Ô­Õ´Æ�Ö�Ó·×eËÙØ É�Ú�ÖkØ�ËRÓTÛ ;£`K=L?Wù�Go5Î

Ü ÆÑÉ�ÐRÆÑÉ\ÍkÝ(Ê É�Ú�ÖÀÓ�ÞßÈÀÔ ËáàiÖ5âÙãqØ�ËRÓTÛäÖ�åJÐÀÓ·ÖZàæàdÊ Ë�ÇßØ�ËRÓ ;�`>=@?Wù�Godç Ê�Ç®É�ÖÀÓTÛ¾à�ËÙØ7Ë�è�àæÖÀÓNé�ÖZâé�ÞÀÓ*ÊêÞRèRÔ Ö5à)Ë�ÇRÔ × ç ËRÓlÅ5ë«ÏNì�í"Ú�ÖÀÇeÉ�Ú�ÖkÕ´Æ�Ö�Ó·×�ÊÌà�Ç�ËÙÉîÊÌâRÖ�ÇÑÉVÊ ï�ÞGèRÔ Ö Îð Î ÏÌØ `E�ygXg)¢Voæh j É�Ú�ÖÀǺÓ�Ö5É�ÆRÓÇ ;�`>=3o5Îñ Î Ü É�Ú�ÖÀÓXí"Ê àiÖ ç Ê Ø `��ygXg)¢zoæh�j É�Ú�ÖÀÇ{Ó�Ö\É�ÆRÓTÇ ;�`>=@? �Xo\Îò Î Ü É�Ú�ÖÀÓXí"Ê àiÖ ç Ô Ö\É À 6 °�~<\+f8��Q<UP� x WYe'`��øCJ¢Vo ÞRÇ�â ;kn 6µ�~�\_^:WY�%§%\_i:��`n�?Wù�Xo\ó Ê Ø ;kn7+6 §+k�¤�é É�Ú�Ö�ǺÓ�Ö\É�ÆRÓTÇ��^� ;£`K=L? n�C:�XoEôE;en�Îõ Î Ü É�Ú�ÖÀÓXí"Ê àiÖ ç Ô Ö5É õ 1 6 µ��PQ�~<�Pi6W>US`E��o®ö `/¢ �k�U3f>W>^ceJ\_i6^2`/¢4o:o\çõ © 6 °�~�\7f8��Q�UP� x W-e©`�� Cpõ 1 o\ç õ Ù 6 °�~<\+f8��Q<UP� x WYed`Wõ 1 C8¢Vodç ÞRÇ�âõ 3 6 õ © �õ Ù ó Ê Ø ¢ +÷ õ 1 ÞRÇ�â � +÷ õ 3 É�Ú�Ö�ÇøÓ�Ö\É�ÆRÓTÇ� ü sq� ü ~ � þ�� ;£`K=L? ¸ 1 C8¸ 3 C:�2Î o�;�`��2Î�? ¸ 3 oB;£`{¸ 1 ? �@C8¸ 3 oB;£`{¸ 3 o5Îù Î Ü É�Ú�ÖÀÓXí"Ê àiÖ ç Ó�Ö5É�ÆRÓÇ^Å\ë)ÏNì Î

x e:W>�P^)¶�M�UP��¨aQ<Ug�D\_�DW¯eJ�PWzy�hPUPf-eJQ�\_U¬°�~<\+f8�+Q�UP� x W-e'`E�øC8¢Vo-|��}�PQ<f8��^:WY~<W>f-eJ^)M^:W-e!\Dy�UP\+�PWY^¡õÜeJ�PM�e!�D�¥^:WY�%MDi:M�eJWz� y�i:\_� ¢�m x hPf8��^:WYe:^)f>MDU�OXWzy�\_hPU3��Q�U�G\_~�j+UP\_�¯Q�M�~XeJQ���W¯`K��Q�MDUlW-e]MD~/mÑb����Y/go-m x eJWY�­¶;f>\_UgeJMDQ<UP^NM;i:WYf>hPi6^:Q��DW4f>MD~<~±e:\eJ�3WVMD~��_\_i:Q�eJ�P��µ�~<\_^6W>�%§%\_i6�[`*n�?Wù�XoNQ�eJ^6W>~�y¬|ÓQ�U­\_i:�3W>iTe:\£\_O7e8MDQ�U­MDU­WÝ�7�Pi:WY^:^:Q�\_Uy�\_i�f>MDhP^JM�~�WY�XW>fYe�;�`n�?Wù�XoÝmðTñrä�ñ{ð úsç=�û� û=þ�×���PW°fY\_UP�PQ�eJQ<\DUP^V\Dyx���PWY\_i:WY�Ö¨Pmn¶3m�b°^6�%MDi6�P~�j¿�PW>~�Q<UPW>M�eJW¯eJ�PW°OG\_hPUP�PMDi6j¿OXWÝ�e¥��W>W>UaeJ�PW fY~�MD^6^�\Dy�Q<�PWYUgeJQ�yEj+Q�UP�V�¯\7�PWY~<^]`{^:h3f8�[MD^�e:�P\_^6W!�3W>�PQ�fYeJWY�lQ<Ul§�Q��_hPi:W¶3mm/go£MDU3� UP\DUPQ<�PWYUgeJQ�yEj+Q�UP���¯\7�PWY~<^Ù`{§�Q<�DhPi:W¾¶7m^�goÝm ���PWY^:W�f>\_U3�PQ�e:Q<\_UP^£~<WdM��eJ\[MDU¿W-�±WYfYe:Q��_W£MD~<�D\_i:Q�eJ�P� y�\_i)�3WYeJWYi:�¯Q<UPQ�UP�leJ�PW¯Q<�PWYUgeJQ��%MDOPQ<~�Q�e¥j[\Dy�f>\DU�e:i:\_~Þ hPW>i6Q<WY^]\Dy=eJ�3W)e¥j+�XW�;�`>=@?Wù�%o-|G�}�PW>i6W4� Q<^�M£^:Q�UP�_~�W!��MDi6Q�MDOP~�WDm x hPf8� Þ hPWYi:Q<WY^MDi6W�Q<�3W>UgeJQ��ÓMDOP~�WTQ�U°���Af>MD~<fYhP~<h3^�Q�y�MDUP�¡\_U3~�j¯Q�yÑe:�PWYj��¯W>WYe�eJ�PWTf>\_UP�3Q�eJQ�\_UP^º\Dy���PWY\_i:WY� ¨Pmn¶3m�b_mÇ���PW°M�~<�_\_i6Q�e:�P� y�hPiceJ�PWYi4�_Q��_WY^VM�fY~<\_^6W>�7�0y�\_i:��WÝ�7�Pi:WY^:^:Q�\_Uy�\_i�e:�PWAfdMDhP^:MD~�WY�XW>f-e';�`>=@?Wù�%o�Q<U°eJWYi:�¯^�\Dy�WY^6e:Q<��MDOP~<W��Pi:\_OPMDOPQ<~�Q�e:Q<W>^Ymk��P�3~<Q<f>M�eJQ�\_UP^Ñe:\Tf>MDhP^JM�~7M�U%MD~�j+^6Q<^�\Dy3UP\_UPWÝ�3�GW>i6Q<�¯W>Uge8M�~+�PM�e8M�Q<UAeJ�3W�^6\7fYQ�MD~MDUP����WY�PQ<f>MD~%^:fYQ<WYUPf>WY^�MDi6WT�PQ�^:fYhP^:^6W>�°Q<U£µ��%MD�7eJW>i�¶4M�UP��y�hPiceJ�PWYi�WY~�MDOG\_i:M�eJWY�Q<U˵��%M��3eJWYi:^£ä�MDU3�ØÀ3m®¤¥Uص��%MD�3e:W>i-�Ë`0µ�\_i6\_~<~<MDi6j¬�3mp³7m�b�*_o)��Wl�}Q<~�~�MD�3�P~�jeJ�3W>^:W)i:WY^:hP~�eJ^�eJ\��Pi:\_O3~<W>�¯^�\DyN*:(DH�&>(DCF(D,{,/IÝ.*Q>H+,/.0$21%|%e:�%M�eTQ<^>|Pe:\�WY^6e:Q<��M�eJW eJ�3W

Page 18: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b�/�0 Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï�Pi6\_O%MDOPQ�~<Q�e¥j£eJ�PM�e M¯^:�GW>f>Q��%f4\DOP^:WYi6��M�eJQ�\_U¿`{WDm¼�Pm�|ÑM¯�PQ<^6WdMD^6W4fdM�^:W'o}Q<^NfdMDh3^JMD~�~�jM�e6eJi:Q�OPh3eJMDOP~<WTeJ\¯Mz�_Q��DW>UaWY�_WYUge;`{W�mn�3m�|PWÝ�7�X\_^6hPi:Wdo-mæ­çcæ ü ó­t é í­t ê¡ë=ï54 w}v ë¦ï ì ê ì;uâq ï>v êËñ���PQ�^F^6W>f-eJQ<\DUÑ|©O%MD^:WY�4\_U�¯FWdMDi6~�MDUP�)LN\_O3Q<UP^È`¬b����gäDo-|©f>\_U3f>W>i6UP^SeJ�PWÈ�Pi6\_O%MDOPQ�~<Q�^c�eJQ�f)W-��MD~<h%M2eJQ<\DUa\�y=�P~�M�UP^TQ�Uae:�PW4�3i:W>^6W>UPfYWV\Dy=h3UP��W>MD^:h3i:W>���DM�i:Q�M�OP~<WY^>|%�}�PW>i6WWdM�f8�[�P~<MDUlf>\_UP^6Q<^ceJ^N\Dy¦^6WY�_WYiJMD~Ff>\DUPf>hPi6i:WYU�eT\_i}^:W Þ hPW>UgeJQ<MD~FMDfYe:Q<\_U3^TM�UP��W>MDf8�MDf-eJQ<\DUa��Mdj¡OGWAQ<U3á%h3W>UPfYW>�[OgjlQ�eJ^��Pi6W>�PWYf>W>^6^:\_i6^]Q<U°e:�PWA�P~�MDU±m¦½¿WAW>^ce8MDO3~<Q<^6�M­�_i:MD�P�PQ�fdMD~�f>i:Q�eJWYi:Q<\DU¿y�\_iVi6W>f>\D�_UPQ<ÚYQ<UP���}�PW>U�eJ�3W°WY�XW>fYe:^;\DyTM­�_Q��DW>U®�P~�M�UfdM�U£OGWT�Pi6W>�PQ�fYe:W>�£y�i6\_� �%MD^6^:Q��DW�\_O3^:W>ic��M�eJQ�\_UP^º\_U��¯WdM�^:hPi6W>�£��MDi6Q�MDOP~�W>^=\_U3~�j_m½��PWYU¡eJ�PWTf>i:Q�eJWYi:Q<\DU�Q�^�^:M�eJQ�^6�%WY�Ñ|PM4fY~<\_^6W>�7�0y�\_i:� WÝ�7�Pi:WY^:^:Q�\_U°Q<^º�Pi6\©�+Q��PW>�£y�\DieJ�3W!�3i:\_O%M�OPQ<~�Q�e¥j¯eJ�%M�e�e:�PWA�P~�M�Ul�}Q�~<~±MDf8�PQ<W-�_W)Mz^:�GW>f>Q��%W>�a�_\gM�~0mðTñ{ðTñ6ò ý ÷=õ%öA»=û�õPöc÷�ø�F\­��\�eJQ���M�e:W¯e:�PW£�3Q<^:fYhP^:^6Q<\_U±|=f>\_UP^6Q<�PWYiVMDU¿W-�3MD�¯�P~<W��PQ<^6f>hP^6^:W>�®Q�U�L}\_OPQ<U3^`cb���_¶3|2MD�_�Xm�³_o-|�MD^F�PW>�PQ�fYe:W>�4Q�U!§�Q<�_h3i:W=¨Pm¼¨PmL���PW���MDi6Q�MDOP~�W>^Ñ� 1 MDUP�)� 3 ^6eJMDUP�

X2

U1

U2

X1

Z

Y

§�Q��_hPi:W�¨3mn¨�� ���PW �Pi:\DOP~<WY� \Dy W-��MD~<h%M2eJQ<U3��e:�PW WY�XW>fYeB\Dy eJ�PW �P~�M�U`/���7`�� 1 opC=���3`E� 3 o:oS\_Uz¢£|Dy�i6\_�âUP\_UPWÝ�7�XWYi:Q<�¯W>UgeJMD~7�PM�e8M�eJMD�DWYUz\_U4� 1 CJõsC�� 3 |MDUP��¢�my�\_i!eJi:W>M�eJ�¯W>Uge:^)eJ�PM�eV�P�gj+^:Q�f>Q<MDUP^V�Pi6W>^6f>i:Q�OXW�eJ\¾M[�PM�eJQ�W>UgezM�e)e¥��\¾�3Q��XW>i:WYUgeeJQ���WY^>|'õ�i6W>�Pi6W>^6W>UgeJ^�\_OP^:WYi6��M�e:Q<\_UP^°e:�%M�eleJ�PWÙ^6W>f>\DUP� �P�gj+^:Q�f>Q<MDU�f>\_UP^6hP~�e:^

Page 19: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê´É+ÊâÅ�ÂTÆØ̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í�ÎBÍ�ÕàÄÈÒ�Ã]Î�Ï b�/7beJ\£�PWYe:W>i:�¯Q<U3W!� 3 |XMDUP�¾¢�i6W>�Pi6W>^6W>UgeJ^]e:�PWV�%M�e:Q<WYU�e��n^T^:hPic�+Q���MD~0mx���PWV�3Q<�P�PWYU��MDi:Q<MDOP~�W>^�5 1 MDUP�Ï5 3 i6W>�Pi6W>^6W>Uged|+i:WY^:�GW>fYe:Q��DW>~�jD|��%MDi6eÈ\�yGe:�PW}�%M�e:Q<W>Uge��n^=�PQ�^6e:\_i6jMDUP��eJ�PW¯�%M�e:Q<WYU�e��n^)�PQ<^6�X\D^:Q�e:Q<\_U¾e:\�i6W>f>\'�DW>i>mlk ^:Q����P~�W;i:WdM�~<Q<Ú>M�eJQ�\_U¾\Dy�^:hPf8�^6e:i:hPf-eJhPi6W¾f>\_h3~<� OGW�y�\_h3UP�×MD�¯\_UP��kT¤¬Z x �%M�e:Q<WYU�e:^>|��}�PWYi:W®õ i6W>�Pi6W>^6W>UgeJ^W>�3Q<^:\+�PWY^�\Dy"¯�µE¯Fm����3Q<^¯Q<^¯M�fY\_���¯\_U®\_�P�G\_iceJhPUPQ�^6e:Q<flQ<U3y�WYfYe:Q<\_Uà\Dy]kT¤¬Z x�%M�e:Q<WYU�e:^ eJ�%M�e�`{MD^]eJ�3W¯�3Q�MD�_i:MD� ^:�P\'�}^8o �P\+W>^AUP\DeA�PM'�DW£M°�PQ�i:WYfYeAWY�XW>f-e!\_U^:h3i6�+Q���MD~=¢�OGW>f>MDhP^:W;Q�e fdMDU�OGWVeJi:W>M�eJWY��W-�±WYfYe:Q��_WY~�jD|�OPh3e!Q�e Q�^AMDUÙQ<UP�3Q<fdM2eJ\_i\DyGeJ�PW}�%M2eJQ<WYUge�¼^ºh3UP�PW>i6~�j+Q<U3�!Q����4hPUPW�^6eJM�eJh3^�`r5 3 o-|_�}�PQ�f8�£fdM�U�f>MDhP^:W}�PW>M�eJ�±m���PW�eJWYi:�¯^=� 1 MDU3�¯� 3 ^6eJMDUP�;y�\_i=O%MDf-eJi:Q���|_MA�3i:hP�AeJ�%M2eÈ�Pi:W-�_W>Uge:^E¯ºµE¯×`Wõ�oMDUP�¿�£Mdj�MD~<^6\[�Pi6WY�DW>Uge¯�PWdM�e:��Ogj¿\�eJ�PWYiV��WYf8�%MDUPQ�^:�¯^>m[Z]\7f-eJ\_i6^)hP^6W>��eJ�3W�%M�e:Q<WYU�e��n^4WdMDi6~<Q<WYi¡¯ºµE¯â�PQ<^ceJ\_icj×`r5 1 o!eJ\[�Pi:WY^:f>i6Q<OGW£� 1 |¦OPh7ezQ�e:^)��MD~<h3W���MD^UP\De}i6W>fY\_i:�PWY��y�\_i��PM�e8M;MDU%M�~�j+^:Q�^>m���PW¾�Pi6\_OP~<WY� ��W¾y{MDf>W[Q<^°MD^�y�\_~<~�\©�}^Ym k�^6^:hP�¯W­��W¾�PM'�DWÙf>\_~�~<W>f-eJWY�×M~�M�i:�_WTMD�¯\_hPUge�\�yS�%M�e8M!\_U£e:�PW]OGW>�%Md�+Q<\_iº\DyS��MDUgj¯�%M�eJQ�W>UgeJ^�MDUP�¡�3��j+^6Q<f>Q<MDUP^Y|�}�PQ�f8�¡Q<^º^:hP�¯��MDi:Q�Ú>W>�¯Q<U¯eJ�PWNy�\_i:� \�yº`/MDU�W>^ceJQ<��M�e:W>�Óo M6\_Q<Ugeº�PQ�^6eJi6Q<OPh7eJQ<\DU²;\DyÈeJ�3Wz\_OP^:WYi6�DW>�Ùy�\_hPi���MDi6Q�MDOP~�W>^;`E� 1 C�õsC]� 3 CA¢zoÝm)kÜU3WY�Ü�PM�eJQ�W>Uge f>\D��WY^Q<U±|¦MDUP����W£�}Q�^:��e:\[�PW-eJW>i6��Q�UPW�e:�PW£Q����PMDfYe)\Dy�eJ�3W¾`{h3UPf>\_U3�PQ�e:Q<\_U%M�~Eo]�P~�M�U`/���7`�� 1 opCV���7`�� 3 o6oA\_U�^:hPic�+Q���MD~0|S�}�PWYi:W£� 1 M�UP��� 3 M�i:W;e¥��\­�3i:W>�3WYeJWYi:�¯Q<UPWY��P\_^:MD�_WY^N\�y�O%M�fYeJi6Q<��e:\;OXW!MD�P�¯Q<U3Q<^6e:W>i6W>�lM�e�e¥��\��3i:W>^6�XWYf>Q��%W>�aeJQ<�¯W>^Ym¤¥UÙ�_W>UPWYiJMD~/|�\_hPi]�3i:\_OP~�W>� MD�¯\_hPUgeJ^]e:\¡eJ�%M�eA\�y�W-��MD~<h%M2eJQ<U3�°M¡UPWY�s�P~�M�UOgj[��M�eJf8�3Q<UP�°eJ�3Wz�XWYi6y�\_i6�£MDU3f>Wz\Dyº\De:�PW>i]�3~�MDUPU3W>i:^]�}�3\_^:W;�PWYf>Q<^6Q<\_U¾^ceJi:M�eJWÝ��_Q�W>^;MDi6W°Q<UP�PQ�^:fYW>i:U3Q<OP~�WDm´¯=�gj+^:Q�f>Q�M�UP^;�P\ÙUP\De;�Pi6\©�+Q��PWaMÙ�3W>^:fYi:Q��3eJQ�\_U�\Dy�MD~<~Q<U3�Ph3eJ^;eJ�PM�e¡�Pi6\_�¯�3eJWY�ËeJ�3W>� eJ\��Pi:WY^:f>i6Q<OGW­M��_Q��DW>UËe:i:W>M�eJ�¯W>Uge+�MD~<~�e:�PWYjf>\D���Vh3UPQ<f>M�eJW°eJ\�hP^¯Q<^;eJ�%M�e�5 1 ��MD^£fY\_UP^:h3~�eJWY� Q<Uà�PW-eJW>i6��Q�UPQ<U3�Ù� 1 M�UP�eJ�PM�eÁõ�MDUP��� 1 ��W>i6W�fY\_UP^6hP~�e:W>�®Q<U��PW-eJWYi:�¯Q<UPQ�UP�­� 3 mê°�h3eÏ5 1 |ÈhPU3y�\Di6eJh+�U%M�e:W>~�j_|_��MD^�U3\Deºi:W>fY\_i:�3W>�Ñm=¤¥U¯W>�PQ��PW>�¯Q<\D~<\_�DjD|�e:�PW}�P~�MDU;W-�DM�~<h%M�e:Q<\_Uz�3i:\_OP~�W>�Q<^)��UP\'�}UËM�^Ù·ceJQ���WÝ�«��MDicj7Q�UP�°eJi6WdM�e:��WYU�e4�}Q�eJ��eJQ<�¯W-�0��MDi6j+Q<U3��fY\_U3y�\_hPU3�PW>i6^:¹`/L}\_OPQ�UP^4b���_¶goÝm]¤¥U�MDi6e:Q��%fYQ�MD~SQ<Uge:W>~<~�Q<�_WYUPf>W)MD�P�3~<Q<f>M�eJQ�\_UP^Y|Óe:�PWVW-�DM�~<h%M�e:Q<\_U�\Dy^:h3f8���P~�M�UP^AW>U%MDO3~<W>^A\_U3W£MD�DW>Uge)e:\l~<WdM�i:UÙeJ\�MDfYe!Ogj�\_OP^:WYi6�+Q<U3�ae:�PW��GW>icy�\_ic���MDUPf>W¯\Dy�MDUP\De:�PW>i)MD�_WYU�e>|�W-�_WYU�Q<U�fdMD^6W>^!�}�PW>i6W�eJ�3W£MDf-eJQ�\_UP^!\Dy�e:�PW�\�eJ�PWYiMD�_WYUgeÈMDi:W��Pi:WY�PQ<f>M�eJWY�z\_U4y{MDf-eJ\_i6^�eJ�PM�e=MDi6W�UP\De��+Q<^:Q�OP~<WÈeJ\TeJ�PW�~�WdMDi6UPW>iYm�¤«yÓeJ�3W~<W>MDi:U3W>iAQ<^!�XWYi:�¯Q�e6eJW>�¾e:\­MDf-e4M�^!��W>~�~�M�^)\_O3^:W>ic�_W�|¦eJ�PWYU�eJ�3W¯eJMD^:�¾OGW>fY\_��WY^�VhPf8�lW>MD^:Q�W>i��=e:�PW]eJ\_�G\_~�\_�Dj¯\DyFeJ�PW f>MDhP^JM�~��PQ�MD�DiJMD� f>\DhP~<�aMD~�^:\zOGW Q<U3y�WYi:i:WY�`/M2e�~�WdMD^ce£�PMDi6e:Q�MD~�~�j3o-|ºMDUP�àeJ�PWaWY�XW>f-eJ^�\Dy]^6\_��Wa�Pi6WY�+Q<\_h3^:~�j®h3UPQ<�PWYUgeJQ��%MDOP~<WMDf-eJQ<\DUP^�f>\_h3~<�lOXWA�PW-eJWYi:�¯Q<UPWY�Ñmk�^ Q�U­eJ�PWVQ��PW>Uge:Q��%f>M�eJQ�\_U­\Dy�MDf-eJQ<\DUP^z` x W>f-eJQ<\DU¾¨Pmp¶_o-|±e:�PWz�£M�Q<U[�3i:\_OP~�W>�Q<UË�P~�M�U Q<�PWYUgeJQ��PfdM�e:Q<\_UàQ<^¯eJ�PW�f>\_UgeJi6\_~�\�ya·6f>\DU3y�\_hPUP�3W>i:^Y|n¹àeJ�%M2e¡Q<^Y|�hPUP\_O+�

Page 20: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b�/g³ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï^:WYi6�DW>� y{M�fYeJ\Di:^¡eJ�PM�e°eJi6Q<�_�_WYi°MDf-eJQ<\DUP^°MDUP�ã^:Q��VhP~�e8MDUPWY\_hP^6~�jËM��XW>fYele:�PW¾i:WÝ�^:�G\_UP^6WDmºRT\'��WY�DW>i>|ÓhPU3~<Q<��W]eJ�PW �3i:\_OP~�W>� eJi6WdM�e:W>�lQ<U x WYfYe:Q<\_U°¨Pmn¶3|P�3~�MDU°Q��PW>U7�eJQ��%fdM2eJQ<\DU]Q<^Ñy�hPiceJ�PWYi�f>\_�¯�P~�Q<fdM2eJW>��O�j�eJ�PW¦y{MDf-e�eJ�%M2e�^:\_�¯W=\Dy�eJ�3WºfY\_U3y�\_h3UP�PW>i6^`{W�mn�3mxõ]oNM�i:W)M��±WYfYe:W>�­Ogjaf>\_Uge:i:\_~S��MDi:Q<MDOP~<WY^>m�k�^T��W4i6W>��MDi:��W>�­Q�U­µ��PMD�3eJWYi¶3|�\_UPW}\DyXeJ�PWN�PW>MD�P~<Q�W>^ceÈ^:Q<U3^ºQ�U¯e:�PW}�PW>^6Q<�_U�\DyX^6e8M2eJQ<^ceJQ�fdMD~PWÝ�7�XWYi:Q<�¯W>Uge:^]`/µ�\'�b��gä¤/3|��Ñm�¨�/gozQ<^�e:\�MD�UM6hP^6e¡y�\Di�^6hPf8�×�DM�i:Q�M�OP~<WY^>|�OGW>f>MDhP^:W�eJ�PW¾M��oM6hP^6e:��WYUge��\_hP~<�¯^:Q��VhP~<M�eJW��3\_~<�PQ�UP�AM]��MDi:Q<MDOP~<W�f>\_UP^ce8MDUge�+3�P\_~��PQ<U3� f>\_UP^ce8MDUge�M �DM�i:Q�M�OP~<WeJ�PM�e]^ce8MDUP�3^�OGWYe¥��W>W>U�MDU­MDfYe:Q<\_U[MDUP�­Q�eJ^Nf>\_UP^6W Þ hPWYUPf>W;Q<Uge:W>i6y�WYi:WY^T�}Q�e:�[eJ�3W�_WYi6j Þ h%MDUge:Q�e¥j¡��W!�}Q�^:�leJ\;W>^ceJQ��£M�e:WÝß eJ�PW]e:\De8MD~±WY�XW>fYeN\DyFeJ�PM�eTMDfYe:Q<\_UÑm����\;\De:�PW>i�y�WdM�e:hPi:WY^�\DyF§�Q��_hPi:WT¨Pm¼¨zMDi:WT��\_iceJ�°UP\�eJQ<U3�Pm=§�Q�i:^6e>|+e:�PW Þ h%MDU+�eJQ�e¥j¸;£`K=L?Wù� 1 C.ù� 3 o�f>MDUPUP\�e¦OGW�f>\_�¯�Ph3e:W>�zQ�y%��W�e:i:WdM2e�eJ�3W�f>\_Uge:i:\_~+�DM�i:Q�M�OP~<WY^�� 1MDUP�°� 3 MD^}Mz^:Q<U3�_~<W]fY\_�¯�X\_h3UP�°�DM�i:Q�M�OP~<WN��ml���PWA�_iJM��P�lf>\_i6i:W>^6�X\DUP�PQ<U3�;e:\^:h3f8��fY\_���G\_hPU3�PQ<UP�]��\_hP~��z�PW>�3Q<fYe=� M�^=f>\_UPU3W>fYe:W>�¯eJ\A¢åOgjzOG\DeJ�;MDU¯MDi6i:\'�MDUP�lMVfYhPi6�DW>��MDi:fz`�eJ�3i:\_hP�D� 5 o�M�UP�°eJ��hP^���\_hP~<�¡y�\Di:�BMVOX\'�ã�%M�e6eJW>i6U�`{^:WYW§�Q��_hPi:W}¶7m^�goÝ|g�}�PQ<f8��Q<^=Q<U3�PQ<f>M�eJQ��_W�\Dy±UP\_UPQ��PW>UgeJQ��ÓMDOPQ�~<Q�e¥j_m x W>fY\_UP�Ñ|+eJ�PW}f>MDhP^JM�~WY�XW>f-e�;�`>=@?Wù� 1 o}Q<U�Q<^:\D~�M�e:Q<\_UlQ<^TUP\De�Q<�PWYU�e:Q��ÓM�OP~<W!OXWYfdMDh3^:WÁ5 1 f>i6WdM�e:W>^ M�OX\'��%M�e6eJWYi:U�MDi:\_h3UP�°eJ�PWA~<Q�UP���ÿþ õ)|3�}�PQ�f8��~�Q<W>^�\DU�Mz�PQ�i:WYfYeJWY���PM�eJ�ly�i:\_� �eJ\¯¢ `{^6W>WAeJ�PWA�PQ�^:fYhP^:^6Q<\_UaQ<U x WYfYe:Q<\_U�¶3mpä_o-m���PW£y�W>M�eJhPi6W�e:�%M�e!y{MDf>Q�~<Q�eJM�eJWY^]eJ�PW�Q<�PWYUgeJQ��%MDOPQ<~�Q�e¥j[\DyD;�`>=@?Wù� 1 C.ù� 3 oAQ�^)eJ�3WQ<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥jV\DyL;£`K=L? � 1 C8¸,C.ù� 3 o«ß eJ�PW]f>MDhP^:MD~±W-�±WYfYe}\Dy�e:�PW MDf-eJQ<\DU°���3`E� 3 6� 3 o�MD~<\DUPWD|%fY\_UP�PQ�eJQ<\DU%MD~Ñ\_UleJ�3W)\_O3^:W>ic��M�eJQ�\_UP^NMd�DM�Q<~�M�OP~<W M2eNe:�PWAeJQ<�¯W \Dy�eJ�PQ�^MDf-eJQ<\DUÑmË���PQ<^zf>MDU�OGW£�_WYi:Q��PW>�®hP^:Q�UP�­eJ�3W°O%MDf8�_�¥�P\+\_iVfYi:Q�eJW>i6Q<\_U±|¦\_OP^6W>i6�+Q�UP�eJ�PM�e��'� 1 Cpõ � OP~�\7f8��^}MD~�~XO%M�f8�g�«�P\+\_i��%M�e:�P^�OXW-e¥��WYW>Ua� 3 MDU3��¢¯ml����hP^>|%eJ�3WQ<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥j¯\Dyf;�`>=@?Wù� 1 C.ù� 3 o�fdMDUaOXWAi6WdMD�PQ�~�j��Pi:\'�DW>U[Ogj°�}i6Q�eJQ�UP�;�`>=@?Wù� 1 C,ù� 3 o 6 ;�`>=@? � 1 C.ù� 3 o `�¨Pm�b'o

6 7 ü ;£`K=L? ¸.C:� 1 C.ù� 3 o�;�`/¸ ? � 1 o `�¨Pm´³Do6 7 ü ;£`K=L? ¸.C:� 1 C6� 3 o�;�`/¸ ? � 1 o)C `�¨Pmp¶_o

�}�PWYi:Wa`{¨3m�b©oTMDUP�à`{¨3mp¶go}y�\_~<~�\©� y�i:\_� LNh3~<Wz³+|�MDUP�à`�¨Pm´³DoNy�\_~<~�\'�}^Ty�i:\D��L}hP~<W¶3ml���PWA^6hPOP�_i:MD�P�P^}eJ�%M2e}�XWYi:�¯Q�e�e:�PW!MD�P�P~�Q<fdM2eJQ<\DU£\DyFeJ�PWY^:W!i:hP~�W>^}MDi6WA^:�P\'�}UQ<Ua§�Q��_hPi6W]¨Pmpä�`�Q<U x WYfYeJQ�\_Ua¨Pmn¨3mp¶goÝm���PQ<^!�PWYi:Q���M�e:Q<\_U�MD~�^:\a�PQ<�_�3~<Q<�D��e:^ �P\'� f>\_U3�PQ�e:Q<\_U%M�~¦�P~�M�UP^)f>MDU¿OGW¯WY��MD~��h%M�e:W>�Ñm�k�^6^:hP�¯W¡��W°�}Q�^:��eJ\ÙW-�DM�~<h%M�e:W£eJ�3W¡WY�XW>f-e�\�yNe:�PW°�P~<MDU �2���3`E� 1 6� 1 o)Cº���7`�� 3 6«ö±`�� 1 C8¸go6o � m=§%\_~�~<\'�}Q<U3�TeJ�3W�MDUPMD~�j+^:Q�^=\Dy x W>f-eJQ<\DU�¨Pmp³7|_��WN�}i6Q�e:W;�`>=@? ���3`E� 1 6�� 1 o)CJ�+�7`�� 3 6÷ö±`�� 1 C8¸_o:o:o 6 ;�`>=@? � 1 C8���3`E� 3 6ÐöÑ`E� 1 C8¸go6o:o

Page 21: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê´É+ÊâÅ�ÂTÆØ̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í�ÎBÍ�ÕàÄÈÒ�Ã]Î�Ï b�/D¶6 7 ü ;�`>=@? ¸.C:� 1 C8���3`E� 3 6ÐöÑ`E� 1 C8¸go6o:oB;£`{¸ ? � 1 o6 7 ü ;�`>=@? ¸.C:� 1 C:� 3 o�;�`/¸�? � 1 o�? þ ~ ÿ����mþ s � ü �

k��_MDQ<UÑ|GeJ�PWzQ��PW>Uge:Q��ÓMDO3Q<~<Q�e¥jl\DyÈeJ�PQ�^]f>\DUP�PQ�e:Q<\_UPMD~��3~�MDU­i6W>^ceJ^A\_U­eJ�3WzQ<�PWYU�e:Q��3�MDOPQ�~<Q�e¥jl\Dy=eJ�PW4W-�7�Pi:WY^:^6Q<\_UÝ;�`>=@? ¸.C:� 1 C,ù� 3 o-|X�}�PQ<f8�¾i6W>�PhPfYW>^ eJ\Ï;£`K=L? ¸.C:� 1 C6� 3 oOGW>fdM�hP^:W`�'� 1 Cpõ � OP~<\+f8��^NMD~<~XO%MDf8�_�¥�P\+\_i}�%M�e:�P^}OXW-e¥��WYW>U[� 3 MDUP�a¢£m���PW!f>i:Q�eJWYi:Q<\DU¡�PWY�DW>~�\_�XWY�[Q�U¡eJ�PWAUPWÝ�7eN^6W>fYe:Q<\_U°�}Q�~<~±WYU%MDOP~�W hP^�eJ\;i:WYf>\_�2�UPQ�Ú>W¯Q<U��_W>UPWYiJMD~/|�Ogj��_i:MD�P�PQ�fdMD~=�¯WdMDU3^>|S�}�PWYe:�PW>izMl�Pi6\_�X\D^:W>�¿�P~�MDU�fdM�U¿OGWWY��MD~�h%M�e:W>�¾y�i:\_� eJ�3W'M6\DQ<Uge]�PQ�^6e:i:Q<O3h3eJQ�\_U­\_U¾eJ�3W¯\DOP^:WYi6��MDOP~�W>^AMDUP�±|�Q�yº^:\P|Ñe:\Q<�3W>UgeJQ�yEj°�}�PQ<f8�­fY\'�DM�i:Q�M2eJW>^N^:�3\_hP~<��OXW!�¯WdMD^6hPi:WY�¾MDUP���P\'� eJ�PW-j[^6�P\_hP~��[OGWMD�oM6h3^6eJWY�ÑmðTñ{ðTñQÔ ý¬�¬ûÈø >gü"��ø�õPö�@Aô�û�õPö¬÷�ø ��� ÷Èõ3û�õPöc÷�ø ûÈø]ü óãù7ù�ç=� :"½

õ%ö¬÷�ø]ùª hPia^6e8M�i6eJQ�UP�®�X\DQ<UgeaQ<^aMË��UP\'�}~<WY�P�_W�^:�GW>fYQ��%f>M�eJQ�\_U ^6f8�PW>�¯W¿Q�Uãe:�PWÙy�\_i:�\DyTM¾fdMDh3^JMD~��PQ<MD�_i:MD��|�~<Q<��W£eJ�3W°\_UPW°^:�3\©�}UËQ<U®§�Q��_hPi6W¡¨Pm¼¨P|=eJ�%M�eV�Pi:\'�+Q<�PWY^M Þ h%MD~�Q�eJM�eJQ��_W¯^:hP�¯�£M�i6jÙ\Dy�e:�PW°MDU%M�~�j+^6e��n^)hPUP�PWYi:^6eJMDUP�PQ�UP�¾\Dy�e:�PW¡i6W>~<W-��MDUge�%M�eJM2�¥�DW>UPWYiJM�e:Q<UP�z�Pi6\7fYW>^6^:W>^Ym��� F��ѯ��*F �k *8$21%,/IJ$2Cm[GIJ$ÙQ>C< -?�fY\_UP^6Q<^6e:^)\�y�Ml�PQ<i6W>fYe:W>�¿MDfYj+f>~�Q<f;�_i:MD�P�Ø`/Z}kx )o=]��}Q�e:��_WYi6e:W-��^6WYe��;|¦�%MDiceJQ�eJQ<\DUPW>��Q<UgeJ\ay�\_hPi4�3Q<^KM6\DQ<Uge4^6WYe:^��Ö6ÿ�'�øC�õxC-5�Cz¢ � |�}�PWYi:W� � eJ�PWN^:W-e�\DyXf>\DU�e:i:\_~%��MDi6Q�MDOP~�W>^}`�W-�7�G\_^:hPi6W>^Y|3Q<Uge:W>i6�DW>UgeJQ�\_UP^Y|+e:i:WdM2eJ�¯W>UgeJ^Y|W-eJfDmpo)+õ�� eJ�PWA^:W-e�\�y�\_O3^:W>ic�_WY����MDi:Q<MDOP~�W>^>|3\DyEeJWYUaf>MD~<~�W>��*8$2²2(DIÝ.{(D,¬ 8&)+5�� eJ�3W!^6WYeN\Dy�hPUP\_O3^:W>ic�_WY�®`{~<M�eJWYU�e8o���MDi:Q<MDOP~<WY^�+%M�UP�¢�� MDUl\_h3eJfY\_�¯W]�DM�i:Q�M�OP~<W�m

½¿Wâ~�WYe×eJ�3W f>\_UgeJi6\_~­��MDi:Q<MDOP~�W>^ OGW \_i:�3W>i:WY� � 6 � 1 C:� 3 C Zt C6���^:h3f8� e:�%M�e×W-�_WYi6j ���åQ�^ MÜUP\_U3�PW>^6f>W>U3�%MDUge \�y¾�����Rªs`�� � 0goËQ�U ]¯|��� k'_p\me�[�V�ko_pe�b^a)[fZ>RY[�hBb4¹¤h�_veWbmX�ksZWhrgU[BiE[fToZ>b^kUtEh�X�TUk�e�[BV���_)h�e�To_)\YZ>eW_pe�[�ij[�k�eWZ�}�_�Z�c�[Ba�[B\m¥X)RY[�c�y]! uX�yUb^koZl�K�$�)á)�ow����)á�p��w_�Zuc�[�ZWh�VWb^y¤[$csb^k��[�h�e�b^X)k#"Uz �Uz ¢oz

Page 22: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b�/D¨ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏMDUP�å��W�~�WYe[e:�PW�\_h7eJf>\D��W¿¢ OXW�MØ�PW>^6f>WYUP�%MDUge�\Dyz���3mÖéSW-e%$&��^6eJMDUP�y�\_i¾eJ�3WË^6WYe�\Dy¡\_O3^:W>ic�_WY� UP\7�3W>^ÙeJ�%M�e�MDi6WËU3\_UP�PWY^:f>WYUP�%MDUge:^�\Dy°M�U�jWY~<W-��¯W>Uge¡Q�UËeJ�3W[^:W-e��'���¤C6����� 1 C tZ C:��� � m k�[GC�(D1�Q<^�MDU \_i6�PW>i6W>� ^6W Þ hPWYUPf>W`Uù� 1 C-ù� 3 C tt C.ù���_oº\Dy���MD~�hPW�M�^:^:Q��_UP�¯W>UgeJ^ºeJ\4e:�PW�fY\_UgeJi6\_~G��MDi6Q�MDO3~<W>^Y|+�}�3W>i:W ù�'��¯WdMDUP^�·¬���°Q�^V^:W-e;eJ\¾�'�Dm¼¹�k *8$21G9D.E,{.0$21G(DCO[GC�(D1�Q�^zMDU®\_i:�3W>i:WY�Ë^6W Þ hPWYUPf>W`�ùö 1 `{¸ 1 o)C�ùö 3 `/¸ 3 o)C tZ C�ùö(�3`/¸)��o6o-|��}�PW>i6W]WdM�f8�Ïö*�NQ<^�M!y�hPU3fYeJQ�\_U£y�i6\_� M4^:W-eDõ+�Ne:\���AMDUP�°�}�PWYi:W ùö*�_`/¸,�do�^6e8M�UP�P^�y�\_i�eJ�3W ^6e8M2eJW>�¯W>Uge£·6^6WYe����]eJ\�ö-�_`{¸,�'o��}�PWYU7�WY�DW>i�õ+�)M�e:eJMDQ<UP^}eJ�3W)��MD~<h3W4¸,��mn¹Ê���3WV^:hP�3�X\_ice�õ+�)\�y=WdMDf8�øö-�D`/¸,�'o�y�hPUPf-eJQ�\_U�VhP^ceNUP\De}f>\DU�eJMDQ<UaMDUgj°��MDi6Q�MDOP~�W>^�e:�%M�eNMDi6WA�PW>^6f>W>U3�%MDUgeJ^T\Dy���� Q�U3];mª hPis�Pi6\_OP~�W>� Q<^âeJ\ -²2(DC HP(D,¥ MDU h3UPf>\_U3�PQ�e:Q<\_U%M�~à�3~�MDU/. Ogj fY\_�;��Ph3e:Q<UP�æ;�`>=@?Wù� 1 C ù� 3 C Zt C.ù���_o-| �}�PQ�f8� i6W>�Pi6W>^6W>UgeJ^�e:�PW Q<�¯�%MDfYe \�y eJ�3W�P~<MDU `�ù� 1 C tt C,ù���_o \_U eJ�3W�\_h3e:f>\_�¯W���MDi:Q<MDOP~<W ¢¯m ���PW�WÝ�7�Pi:WY^:^:Q�\_U;�`>=@?Wù� 1 C®ù� 3 C tZ C,ù���_o]Q<^A^JMDQ���e:\�OGWl.{9+ -1%,/. b�(RQ>C< �Q<Uv] Q�y¬|�y�\_iAW-�_W>icj�MD^:^6Q<�_U7��¯W>UgeA`Uù� 1 C²ù� 3 C tt C.ù���go-|�eJ�PW]WÝ�7�Pi:WY^:^:Q�\_U°fdM�U°OXWT�PWYe:W>i:�¯Q<U3W>�¡hPU3Q Þ hPW>~�j;y�i:\_�eJ�3WDM6\_Q�U�eT�PQ�^6eJi6Q<OPh7eJQ<\DU�\Dy�eJ�PW)\_OP^:WYi6��MDOP~�W>^k�'�øC8¢JCpõ � m}ksfY\_UgeJi6\_~��3i:\_OP~�W>�Q<^�Q��PW>Uge:Q��ÓMDO3~<W��}�PWYUPWY�DW>i";�`>=@?Wù� 1 C�ù� 3 C tt C.ù����o�Q�^�Q<�PWYUgeJQ��%MDOP~<W�mª hPi°�£M�Q<UØQ��PW>Uge:Q��ÓMDO3Q<~<Q�e¥j®f>i6Q�eJWYi:Q<M�MDi:W[�Pi:WY^:W>Uge:W>� Q�UÊ���PWY\_i:WY��^°¨Pm¼¨Pm�bMDUP��¨Pm¼¨PmnÀ3m ���PWY^:W�Q<Ug�_\_��W��_�«^:WY�%MDiJM2eJQ<\DU�e:W>^ceJ^l\_U×��MDi:Q�\_hP^°^:h3OP�_iJM��P�P^l\Dy];|¦�PWY�PUPW>�®Q<U�eJ�PW£^JMD�¯W£��MDUPUPWYiVMD^)Q<U x W>fYe:Q<\_U¿¨Pmp¶7m�½�W¡�PW>U3\DeJW¡Ogj®] !`/M�UP� ]=! | i6W>^6�XWYfYeJQ��_WY~�j3ole:�PW¿�_i:MD�P�P^a\_O3eJMDQ<U3W>� Ogj��PWY~<W-eJQ<U3�ày�i6\_� ] MD~<~MDi6i:\'�}^­�G\_Q<UgeJQ�UP�Ëe:\â`{WY��WYi:�_Q�UP�Ëy�i6\_�£olUP\+�PWY^­Q<Uã��mÖ�F\ i:WY�Pi:WY^:W>UgeÙeJ�3W�PWY~<WYe:Q<\_U®\DyTOG\DeJ�àQ<UPfY\_�¯Q<UP�¾M�UP�Ë\Dh3eJ�_\DQ<UP�ÙMDi6i:\'�}^>|º��WahP^6Wae:�PWlUP\De8M2eJQ<\DU

] !+0 m£§�Q�U%MD~�~�j_|XeJ�PW¯W-�7�Pi6W>^:^6Q<\_U¬;£`K=L?�ù�uC8¸go�16 ;£`K= CJ¸ ?Wù�Go�2Y;�`/¸ ?Wù�Xo ^ce8MDU3�P^!y�\DieJ�3W¯�3i:\_O%M�OPQ<~�Q�e¥ja\Dy�¢ 6 =­�_Q��DW>U¾eJ�PM�e¸õ 6 ¸aQ�^ \_OP^:WYi6�DW>��M�UP�¾� Q�^ �PW>~��f>\DUP^6eJMDUge�M2eN�Sm

ðTñ{ðTñrä ý¬�¬ûÈø£>gü"�Fø�õ%öA@Aô�û�õ%ö¬÷�ø � ó 98�Fø'�Sþ%ûJ��å¿þÓö«õ��SþÓö¬÷�øB�CEDGF(H�DJI KEL±K L*P `>¯�WdM�i:~FMDUP�aL}\_OPQ<U3^ b����gä_o�Ñ#% Â[GIJ$ÙQ8(RQ>.EC .E,*Y�;£`K=L?�ù� 1 C Zt C,ù���_oV.<&z.{9+ -1%,{. b�(RQ>C< V. %J@�%d$2I� -²D -IdYab�3546387F@,E#% -I8 z 65_.<&-,E&)(¯&' -,fõ9�°$�%V*8$2²2(DIÝ.{(D,¬ 8&�&>(D,/.<&K%dY2.E1N#

õ+�;:<$&�YC `�¨Pmn¨go= PKc�[Bk�eWb4¹¤h�_veWbmX�k�Xp�@h�X�koc�bme�b^X)ko_)\�RU\d_pkoZJh�_pk~yY[DX)y�er_pb^kU[�c¡��VWX)i8��gU[BX�V�[�i÷¢¤z ¢¤zm�EToZ�bmkote�go[lij[Be�gUX�c'c�[$Z�hBV�b^y¤[$cxb^k��[�h�e�b^X)k'¢ozÉ�j_pkocs[�ã�[�ijRo\mbm¹o[�c�b^k��[�h�e�b^X)k�¢oz ¢oz^�)z

Page 23: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê´É+ÊâÅ�ÂTÆØ̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í�ÎBÍ�ÕàÄÈÒ�Ã]Î�Ï b�/_ä`c.{K{ >K�@Lõ9�°*8$213&-.<&-,�&z$�%]1±$21G9+ 8&'*8 -1G9g(D1%,�&)$�%=�'���oC:���>� 1 C tt C:��� � o)(D1G9

`{¢7gXgA���,? � 1 C tt C:���@? 1 Cpõ 1 Cpõ 3 C Zt Cpõ+�>oih j A>B j ADC s B E E E B j�F `�¨Pm´äDoG£#% -1¾,�#% 8&' V*8$21G9D.E,{.0$213&)(DIJ  &>(D,{.<&�b� :92@È,�#% 4 IH] 8*-,}$�%A,E#% «[GC�(D1¾.<&s#D.E²D -1�QZY

;£`K=L?Wù� 1 C tZ C.ù���_o 6 7ü s �KJKJKJ � ü F ;�`>=@? ¸ 1 C tt C8¸)�,C:� 1 C tZ C:���_o�L�vÿ 1;£`{¸,�N? ¸ 1 C tt C8¸,�@? 1 C6� 1 C tt C:�'�@? 1 o `�¨PmpÀ_o

°�W-y�\_i:W �Pi6W>^6W>UgeJQ�UP�;Q�eJ^��Pi:\+\Dy¬|3~�WYe�hP^��PW>�¯\_UP^ceJi:M�eJW �P\'�Ä���PW>\Di:W>��¨Pmn¨3m�b f>MDUOGW�hP^6W>�4e:\Te:W>^6e�eJ�PWºQ<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥jT\DyPe:�PWºf>\_UgeJi6\_~+�Pi:\_O3~<W>� ^:�P\'�}UzQ<U)§�Q��_hPi:W¨Pm¼¨PmV§�Q<i:^ced|±��Wz�}Q<~�~�^:�3\©� eJ�%M2e�;£`K=L?Wù� 1 C.ù� 3 o�f>MDUPUP\�eAOXW;Q<�3W>UgeJQ��%W>�­�}Q�eJ�P\_h7e�¯WdMD^6hPi:Q�UP�Áõ¸+%Q<Ul\De:�PW>i���\_i:�P^Y|%eJ�%M�e}e:�PWA^:W Þ hPWYUPf>W�õ 1 6���C�õ 3 6��¯��\_hP~��UP\De�^JM�e:Q<^cyEjlf>\_UP�3Q�eJQ�\_UP^4`{¨Pm¼¨�oWô±`{¨Pmpä_oÝmx���3W4e¥��\¡�_�«^:WY�%MDiJM2eJQ<\DU�eJWY^6e:^]W>UPfY\7�3W>�Q<U�`�¨Pm´äDo�MDi:W `{¢7gXgA� 1 oih j s B j ~MDUP� `/¢�gNg � 3 ? � 1 oæh�j ~ ���PWNe¥��\)^:hPOP�DiJMD�P�3^�MD^6^:\+f>Q�M2eJW>�;�}Q�e:�¯e:�PW>^6WTe:W>^6e:^�MDi6WT^6�P\'�}U¡Q<U�§�Q��_hPi:W�¨3m´ä7m½�Wa^6W>WleJ�%M2e­`{¢7gXgA� 1 oV�P\_~��P^zQ<U¥] ! s � ! ~ O3h3e;eJ�%M�e[`{¢7gXgA� 3 ? � 1 o)y{MDQ�~<^4e:\�P\_~�� Q<U ]�! ~ m�����hP^>|TQ<U \_i6�PW>i�eJ\��%MD^6^¡eJ�PW�eJW>^ced|}��W¾�4hP^6e°�PM'�DW¾W>Q�eJ�PWYiõ 1 6 �¤õ � \_i±õ 3 6 �¤õ � +V^:Q�UPf>W¬õ�Q<^�Mà�PWY^:fYW>UP�%M�U�e­\Dy4� 1 |]\_U3~�j×eJ�3W^:WYf>\_U3�ÙMD~�e:W>i:UPM�eJQ��_WA^JM�e:Q<^c�%W>^V`{¨Pm¼¨�o-m����3W)eJWY^6e:^]MD�P�3~<Q<f>MDOP~�WAe:\�e:�PW)^6W Þ hPWYUPf>Wõ 1 6²��C'õ 3 6 �¤õ � M�i:W°`{¢7gXgA� 1 oæh j s B j ~ MDUP��`/¢7gXgA� 3 ? � 1 Cpõ]oih j ~ mN§�Q��_hPi:W¨PmpäN^6�P\'�}^�eJ�%M2e�OG\DeJ�!eJWY^6eJ^�MDi:W�UP\'�®^JM�e:Q<^c�%W>�Ñ|�OXWYfdMDh3^:WÂ�'� 1 Cpõ � �D�¥^:WY�%MDi:M�eJWY^¢ y�i6\_� � 3 Q�Uß]�! ~ mÙR�Md�+Q<U3�­^JM�e:Q<^c�%W>�®f>\DUP�PQ�e:Q<\_U3^¡`�¨Pmn¨gorô±`�¨Pm´äDo-|¦W Þ hPM�eJQ�\_U`{¨3mpÀgo��3i:\'�+Q<�PWY^TMzy�\_i6�VhP~�M!y�\_i�eJ�PWAW-�±WYfYeT\Dy��3~�MDU�`Uù� 1 C.ù� 3 o�\_U�¢Á�

;�`>=@?Wù� 1 C.ù� 3 ol6 7 ü ;£`K=L? ¸.C:� 1 C6� 3 o�;�`/¸ ? � 1 o)C `�¨Pmd*Do�}�PQ�f8��fY\_Q<UPfYQ<�PWY^��}Q�eJ��`�¨Pmp¶_o-m���PW Þ hPW>^ceJQ<\DU¯UPM�eJhPi:MD~<~�jVM�i:Q<^6W>^¦\�yX�}�PW-eJ�PWYi=eJ�PW�^6W Þ hPW>U3f>W"õ 1 6���C�õ 3 6�¤õ � f>MDUlOXW]Q��PW>Uge:Q��%WY�£�}Q�e:�P\_h3e�WÝ�7�%MDhP^ceJQ��DW ^:W>MDi:f8�Ñm����PQ�^ Þ hPW>^ceJQ<\DU¡�}Q<~<~GOGWMDUP^c��WYi:WY�­Q<Uaµ�\_i6\_~<~<MDi6j�¨Pm¼¨Pmpä;M�UP�±���3W>\_i6W>��¨Pm¼¨PmnÀ3m� H�FmF��îF(�¾B�CEDJF(H�DJI K L±KEL*P�M���PW��Pi6\7\�yÓ�_Q��DW>Uz�PWYi:W�Q<^�O%MD^6W>�;\_UVeJ�3W�Q<U3y�WYi:W>U3f>W�i6hP~<WY^¦\DyX���Nf>MD~<fYhP~<hP^�`>���PWÝ�\_i6W>� ¶3mn¨3m�b©oÝ|G�}�3Q<f8�­y{MDfYQ<~<Q�e8M�e:W eJ�PWVi6W>�Ph3fYeJQ�\_U­\DyÈf>MDhP^:MD~�W-�±WYfYe]y�\_i6�VhP~<MD^Ne:\

Page 24: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b�/_À Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï

X2

GX2

U1

U2

X1

U1

X2

U2

X1

(a)

Z

YX1 X2

G ,

(b)

Y

Z

§�Q��_hPi:W)¨Pmpä,�D���PWVe¥��\l^6hPOP�_i:MD�P�P^ \Dyl] hP^:WY�ÙQ<U[e:W>^6e:Q<UP��eJ�PWzQ��PW>UgeJQ��ÓMDOPQ�~<Q�e¥j\DyFeJ�PWA�P~<MDU�`Uù� 1 C,ù� 3 o�Q�Ua§�Q<�_h3i:W]¨Pm¼¨Pm�%M�ec�«y�i6W>W4WÝ�7�Pi:WY^:^:Q�\_UP^Ym k�U¾MD~�eJWYi:U%M�e:Q��DW)�Pi6\7\�y¬|XhP^6Q<UP�£~�M�e:W>UgeT��MDi6Q�MDO3~<W!W>~<Q���Q �U%M�e:Q<\_U±|PQ<^��_Q��_WYUaQ�U�¯�WdM�i:~FMDUP�aL}\_OPQ<U3^!`cb����_ä_o-mN �ÑDPO�P'M ���3W f>\_UP�3Q�eJQ�\_U�õ+�;:Q$&�]Q����P~�Q<WY^Eõ+�;:<$ªTy�\Di�MD~<~�SRQ4Xml���3W>i:WÝ�y�\_i6WD|3��W!�%Md�_W

;�`/¸,�S? ¸ 1 C tt CJ¸,�@? 1 C:� 1 C Zt C6�'�@? 1 C.ù�'�¤C.ù�'�>� 1 C Zt C.ù���_o6Ä;�`/¸,�N? ¸ 1 C tt CJ¸,�@? 1 C:� 1 C tt C6�'�@? 1 o ���PQ�^]Q<^T^:\°OGW>f>MDhP^:WzU3\¡UP\+�PWVQ<Uv�¤õ 1 C tt Cvõ9�oC:� 1 C tZ C:���@? 1�� f>MDU¾OGWMz�PWY^:f>WYUP�%MDUge�\Dy=MDUgj°U3\7�PWAQ�U��'���¤C tt C:��� � mºRNW>UPfYWD|GLNh3~<W!¶;MD~<~�\©�}^hP^�e:\¯�3W>~<W-eJW eJ�3W!�PM�e}�DM�i:Q�M�OP~<WY^�y�i:\_� eJ�PWAWÝ�3�3i:W>^6^:Q�\_UÑm

N �ÑDPO �TM ���3W f>\_UP�3Q�eJQ�\_U¡Q�U�`{¨Pmpä_oº�GW>i6��Q�eJ^�h3^�eJ\zQ<Ug�D\_�DW!L}hP~<W]³;M�UP�¡�}i:Q�eJWY�;£`K=L? ¸ 1 C tZ C8¸,�¤C:� 1 C tZ C:�'�@? 1 C.ù�'�YC.ù�'��� 1 C tt C,ù���_o6Ä;£`K=L? ¸ 1 C tZ C8¸,�oC:� 1 C tZ C:�'�@? 1 C:�'�YC,ù�'��� 1 C tt C.ù���_o

����hP^Y|Ó��W!�%Md�_W;�`>=@?Wù� 1 C tt C,ù���go

Page 25: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê´É+ÊâÅ�ÂTÆØ̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í�ÎBÍ�ÕàÄÈÒ�Ã]Î�Ï b�/�*6 7 ü s ;£`K=L? ¸ 1 C.ù� 1 C.ù� 3 C tt C.ù���goB;£`{¸ 1 ?Wù� 1 C tt CNù���go6 7dü s ;£`K=L? ¸ 1 C:� 1 C.ù� 3 C tt C.ù���goB;£`{¸ 1 o6 7 ü ~ 7 ü s ;�`>=@? ¸ 1 C8¸ 3 C:� 1 C.ù� 3 C tt C.ù���goB;£`{¸ 1 o�;�`/¸ 3 ? ¸ 1 C:� 1 C.ù� 3 C tt C,ù���go6 7 ü ~ 7 ü s ;�`>=@? ¸ 1 C8¸ 3 C:� 1 C:� 3 C.ù� © C tt C,ù���_o;£`{¸ 1 o�;�`/¸ 3 ? ¸ 1 C:� 1 ommm

6 7 ü F Ó�Ó�Ó 7 ü ~ 7 ü s ;�`>=@? ¸ 1 C tt C8¸)�,C:� 1 C tZ C:���_oU ;�`/¸ 1 o�;�`/¸ 3 ? ¸ 1 C:� 1 o tt ;£`{¸)� ? ¸ 1 C:� 1 C8¸ 3 C:� 3 C tt C8¸)�-? 1 C:���-? 1 o

6 7ü s �KJKJKJ � ü F ;�`>=@? ¸ 1 C tt C8¸)�,C:� 1 C tZ C:���_o�L�vÿ 1;£`{¸,�,? ¸ 1 C tt C8¸,�@? 1 C6� 1 C tt C:�'�@? 1 o

�V DXWY�ZI�[*F � KEL±KELN� _�1zY4&' ��>HÓ -1±*8 Dõ 1 C tt Cvõ+�[$�%!*8$2²2(DIÝ.{(D,¬ 8&N&>(D,/.<&K%dY2.E1N#¯,E#% *8$21G9D.E,{.0$213&z.E1®`�¨Pmn¨gorô±`�¨Pm´äDoz"=.ECEC Q- ¡*:(DCEC< :9£MD�P�¯Q<^6^:Q�OP~<W©@N(D1G9­(D1zY­ 655[GIJ 8&8&-.0$21;�`>=@?Wù� 1 C�ù� 3 C Zt C,ù���_o�,E#3(D,�.<&�.{9+ -1%,{. b�(RQ>C< «QZY],�#% T*-IÝ.E,¬ -IÝ.0$21°$�%¸�Ñ#% 8$2IJ -?w�GK¶�GK��"=.ECECEQ- z*:(DCEC< :9�]]�¥Q��PW>Uge:Q��ÓMDO3~<WgK]\���PWAy�\_~<~�\'�}Q<UP�)f>\_i6\_~<~<MDi6j�Q<^�Q����¯W>�PQ<M�eJW�m^ F(H�F _`_*¯�H�a�KEL±KELNM _�*8$21%,/IJ$2CO[GIJ$ÙQ>C< -? .<&7]�(¥.{9+ -1%,/. b�(RQ>C< ¯. %¡(D1G9�$21%CoY­. %£.E,#3(2&)(D1�(g9D?z.<&8&-.*Q>C< T&' ��>HÓ -1±*8 >K]]�¥Q<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥j£Q<^T^:h.ѯf>Q�W>Uge OPh7e UP\De�UPW>fYW>^:^:MDi6j�y�\_i��DW>UPWYiJMD~��P~�MDU�Q<�PWYUgeJQ���ÓMDO3Q<~<Q�e¥j;MD^��PWY�%U3W>�°Q<U x W>f-eJQ�\_U£¨Pm¼¨Pmp³7m`���PW�i:WdM�^:\_UP^�MDi:WNe¥��\�y�\_~<�Ñm¦§�Q<i6^6ed|�eJ�3Wf>\D���P~�WYe:W>UPWY^:^�\�y¦eJ�PW!eJ�Pi6W>W)Q<U3y�WYi:W>U3f>W4i6hP~<WY^T\�yÈ���;f>MD~<fYhP~<h3^TQ�^T^ceJQ<~�~SM��GW>UP�+�Q<U3�¡f>\_UUM6W>fYe:hPi:W�m x W>fY\_UP�Ñ|Ñe:�PW�4+e:�¾^6eJWY�ÙQ<U[e:�PWVi:WY�PhPf-eJQ<\DU¾\DyT`{¨PmnÀgo}i:WYy�i:MDQ<U3^y�i:\D�Üf>\_U3�PQ�e:Q<\_UPQ�UP�]\_U;��MDi:Q<MDOP~�W>^�õ+��eJ�%M�e�MDi6W}�PW>^6f>W>U3�%MDUgeJ^�\Dy±���Jß�U%MD�¯W>~�j_|b���gU[`e�[�V�i ndcf¥I_)c�iEbdZ�Z�b^yUbm\^bmeK]qj}�_)Z�ToZ�[�csb^kfe�[�_pVW\�_pkoc# uX�yUb^koZl�K�$�)�@g���e�X�[Ba�X)��[LeK}�X_)ZWZ�XhBb^_pe�b^X)koZ�hL�>�$�T uX�yUb^koZB¶ Zickj`lnmporqtsvu>owq�x>yx��X)VWi�TU\d_"��[$£�T¤_ve�b^X)k��r"Uz �z"����Qw¤}ugUb^hrg�hBX)b^koh�bdc�[$Z}ubme�g���¢oz ���`}ugU[�k�c b^Z�h�X)iERU\^[�eW[s_)koc�h�X�k�er_pb^koZlkUX�TUkUX)y¤Z>[�V�a�[�c¡h�X)kU��X)TUkocU[BVrZB·�_)kocÇ�ß���e�go[|{~}duD���[q���u~�Uko_pe�TUVW[`Xp�.e�gU[JhBX)koc�bme�b^X)k¤Z�bmk���¢¤z ¢�w����¢ozKg)��z

Page 26: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b�/�/ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï��MDi:Q<MDOP~�W>^�e:�%M�eA��M'j[OXW¯M��XW>f-eJWY��Ogj­e:�PW¯MDfYe:Q<\_UÙ���7`�����6s�'�'oÝmz¤¥UÙf>W>ice8MDQ�UfdM�hP^JMD~º^6e:i:hPf-eJhPi6W>^Y|Èe:�PW£Q��PW>Uge:Q��ÓMDO3Q<~<Q�e¥j­\Dy}f>MDhP^:MD~�W-�±WYfYe:^zi:W Þ hPQ�i:WY^)eJ�PM�e4��Wf>\DUP�PQ�e:Q<\_U!\_U)^:hPf8�V��MDi6Q�MDOP~�W>^Y|DMD^F�PWY��\_U3^6eJi:M�eJWY�VOgjAeJ�PWºy�i:\_Uge6�«�P\+\_iSf>i:Q�eJWYi:Q<\DU`>���3W>\_i6W>� ¶3mn¶3m¼¨�o-mðTñ{ðTñ{ð ý¬�¬ûÈø£>gü"�Fø�õ%öA@Aô�û�õ%ö¬÷�ø � ó ý�þ%÷}ôu�Sü�ç]þ2����PWY\_i:WY� ¨Pm¼¨Pm�b4�Pi6\'�7Q��PW>^�M£�PWYf>~�M�iJM�e:Q��_W!f>\DUP�PQ�e:Q<\_Uly�\_iN�P~�M�U[Q<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥jDm¤«ezf>MDU®OXW°hP^6W>��eJ\¾i:M�eJQ�yEj�e:�%M�e;M­�3i:\_�G\_^:WY��y�\_i:�Vh3~�M�Q�^4��MD~<Q��¿y�\_izM­�DQ��_WYU�P~<MDUÑ|SOPh3e!Q�eA�3\7WY^)UP\�eA�Pi:\'�+Q<�PW�MDU�WY�XW>f-eJQ��DW��3i:\+f>W>�3hPi:W;y�\_i �PWYi:Q��+Q�UP�l^:hPf8�y�\_i6�VhP~�M�^�OXWYfdMDh3^:W eJ�PWAf8�P\DQ<f>W \�y�W>MDf8�øõ+�]Q�^�UP\De}^:�GW>~�~<WY�°\_h3e}�Pi6\7fYW>�PhPi:MD~<~�j_m���PWÙ�G\_^:^6Q<OPQ�~<Q�e¥jàWÝ�3Q�^6e:^le:�%M�el^6\_��W¾h3U3y�\_i6e:hPU%M�e:W¾f8�P\_Q�f>WÙ\Dy¡õ9�¾^:M�eJQ�^6yEj+Q<U3�`{¨3mn¨�oÈMDUP��`�¨Pmpä_oÈ�¯Q<�D��e��Pi6WY�_WYUge}hP^�y�i6\_� f>\_UgeJQ�U�hPQ<UP�!eJ�PW�i:W>�3hPfYe:Q<\_U¡�3i:\+f>W>^6^eJ�3i:\_hP�D�[MDU3\DeJ�PWYi�i:WY�PhPfYe:Q<\_Ua^:W Þ hPW>UPfYW4Q�^�y�WdMD^6Q<OP~�WDm���PQ<^zQ�^VQ<~�~<hP^ceJi:M�eJWY��Q<U®§�Q��_hPi6W£¨PmnÀ3m�RNW>i6W�� Q�^zMDUàMD�P�¯Q<^6^:Q<O3~<W£f8�3\_Q<fYWy�\_i-õ 1 |ºOPh3e¯Q�y}��W���MD�DW¡e:�PQ<^zf8�3\_Q<fYWle:�PW>U®��Wa�}Q�~<~�U3\De;OXWaMDO3~<W¡eJ\ÙfY\_�;��P~�WYeJWVe:�PWzi:WY�PhPf-eJQ<\DUÑ|�^:Q�UPf>WzUP\l^6WYe¸õ 3 f>MDUÙOXWVy�\DhPUP�¾eJ�%M2e ^JM�e:Q<^c�%W>^Af>\DUP�PQ��eJQ�\_UË`{¨Pmpä_op�¡`{¢7gXgA� 3 ? � 1 Cd�ÝCpõ 3 oih�j ~ m£¤¥U�eJ�PQ�^!WÝ�3MD���3~<W;Q�e ��\_hP~���OGW¯�}Q�^:WYieJ\�f8�P\+\_^:W®õ 1 6 õ 3 6ø�P|��}�3Q<f8� ^:M�eJQ�^6�%WY^¡OX\�eJ� `{¢7gXgA� 1 ? ��oih j s B j ~ M�UP�`/¢�gNg � 3 ? � 1 C���oæh j ~ m

U1

U2

X1

X2

W

Y

§�Q��_hPi:W�¨3mpÀ.��k�U�MD�P�¯Q<^6^:Q<O3~<W�f8�P\DQ<f>W�õ 1 6�� eJ�%M2eÈi:hP~�W>^È\_h3e�MDUgj¯MD�P�¯Q<^6^:Q�OP~<Wf8�P\_Q�f>W y�\_ixõ 3 m���PW[\DO��+Q�\_hP^;��Mdj®eJ\¿M'�D\_Q<�àO%MD�Ëf8�P\_Q�f>W>^�\Dy]fY\'�DM�i:Q�M2eJW>^Y|�~�Q<�DW°e:�PW[\_U3WQ<~�~<hP^ceJi:M�eJWY��Q�U�§�Q<�DhPi:W¯¨PmnÀ3|�Q�^)eJ\[Q<UP^6Q<^6e)\_U�M�~���M'j+^Vf8�3\7\_^6Q<UP�­M×·6�¯Q<UPQ��£MD~�¹

Page 27: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Ê´É+ÊâÅ�ÂTÆØ̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í�ÎBÍ�ÕàÄÈÒ�Ã]Î�Ï b�/Y�

X1

X2

Z 1

Y

U

Z 1

§�Q��_hPi:W)¨Pm^*,�'�N\_U�hPUPQ Þ hPWYUPW>^6^A\Dyº��Q�UPQ<��MD~SMD�P�¯Q<^6^:Q�OP~<W!^:W-eJ^��"õ 1 MDU3�¬õDÎ1 MDi6WWdM�f8���¯Q<UPQ��£M�~±MDU3�[MD�3��Q�^:^:Q�OP~<W�mõ+��|_UPMD��WY~�jD|_MT^6WYe=\�y%f>\'��MDi:Q<M�eJWY^�^:M�eJQ�^6yEj+Q<UP�V`{¨Pmpä_oÑeJ�%M2e¦�%MD^�UP\��Pi6\_�GW>i�^6hPOP^6WYe^JM2eJQ<^cyEj+Q<UP��`{¨3m´ä_oÝm;RT\'��WY�_WYi>|�^:Q<U3f>WzeJ�PWYi:W�MDi:Wzh3^:h%MD~�~�j[�£MDUgj¾^6hPf8�¿�¯Q<U3Q<��MD~^:W-eJ^�`{^6W>W�§�Q<�DhPi:WV¨3md*_oÝ|ÑeJ�PW Þ hPWY^6eJQ�\_U�i:W>��MDQ�UP^]\Dyº�}�PWYe:�PW>i!WY�DW>icjÙf8�P\_Q<fYWz\DyM��¯Q<U3Q<��MD~Jõ+��Q�^­·6^:M�y�W>¹.�¯µ�MDU¿��W£OGW£^:h3i:W£e:�%M�e)UP\­f8�P\_Q�f>W�\Dy}M��¯Q<U3Q<��MD~^:h3OP^:W Þ h3W>UPfYW�õ 1 C tt Cpõ+�T�}Q<~�~ÓWY�_WYi}�Pi:W-�_W>UgeNhP^�y�i6\_� �%UP�PQ�UP�;MDUlMD�P�¯Q<^6^:Q�OP~<Wõ+�>� 1 |3�}�PWYU[^6\_��WAM��P��Q�^:^6Q<OP~�W]^:W Þ h3W>UPfYW�õ ì1 C Zt Cvõ ì� WÝ�7Q<^6e:^>����PW�U3W-�+e�i6W>^6hP~�eº�_h%M�iJMDUgeJWYW>^ºeJ�PWN^JM�y�W-e¥j;\DyÑW-�_WYi6j¯��Q�UPQ<��MD~+^:hPO3^:W Þ hPWYUPf>Wõ 1 C tt Cpõ+�AMDUP�l�PW>U3f>W!�Pi:\'�+Q<�PWY^�M�UaW-�±WYfYe:Q��_WAeJWY^6e}y�\_iî]]�¥Q��PW>UgeJQ��ÓMDOPQ�~<Q�e¥j_m

B�CEDGF(H�DJI KEL±K L±K²§ %�,E#% -IJ } 65_.<&-,�&�(D1�(g9D?z.<&8&-.*Q>C< ¦&' ��>H% -1±*8 `õ ì1 C tZ Cpõ ì� ,E#% -13@%d$2I4 -²D -IdY�?z.E1%.E?¯(DCECoY�(g9D?z.<&8&-.*Q>C< T&-H´Q-&' ��>H% -1±*8 sõ 1 C tt Cpõ+�@? 1 $�%4*8$2²2(DIÝ.{(D,¥ 8&8@,E#% -I8 !.<&!(D1�(g9D?z.<&8&-.*Q>C< �&' -,fõ9�_Kkâ�Pi:\+\DyFQ<^��DQ��_WYU�Q�U�¯�WdM�i:~SMDUP��L}\_OPQ<U3^!`cb����_ä_o-m���PW>\Di:W>� ¨Pm¼¨Pm¼¨)U3\©�Ëj+Q<W>~��P^ºMDU¯WY�XW>fYe:Q��DW��PWYf>Q<^6Q<\_U;�Pi6\7fYW>�Ph3i:WTy�\Di¦eJW>^ceJQ�UP�]]�¥Q<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥j¯M�^}y�\D~<~<\'�}^Ym^ F(H�F _`_*¯�H�a�KEL±KELN³ _ *8$21%,{IJ$2C�[GI8$ÙQ>C< -?B.<&¾]-(¥.{9+ -1%,{. b�(RQ>C<  . %)(D1G9°$21%CoY¯. %A,E#% %d$2CEC<$2"=.E1N#£(DC4#+$2IÝ.E,E#+?� 65_.E,�& "=.E,�#°&-H%*8*8 8&8&>K�gK��Ñ -,�4Á6âb�K�+K � #%$'$'&' )(D1zY;?z.E1%.E?¯(DC�õ+�;:<$&�;&>(D,/.<&K%dY2.E1N#¡`�¨Pm´äDo>K

Page 28: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b���0 Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï��K § % 1±$z&-HÓ*8#-õ9�l 65_.<&-,�&A,E#% -1¿ 65_.E,º"=.E,�#"%Y(D.EC H+IJ �f� -Cp&' ]&' -,�4Á6�4&��b�K�GK § %!4Á6<7��ãb£,E#% -1¿ 65_.E,º"=.E,�#°&-H%*8*8 8&8&�f� -Cp&' AIJ -,{H7IÝ1Ù,¬$z&-,¥ A[��+K

k y�hPiceJ�PWYiÈ��MDi6Q�MDUge=\�y����PWY\_i:WY� ¨Pm¼¨Pmn¨ f>MDU¯OXW�^ce8M�e:W>�;eJ�%M�eºMd�D\_Q<�P^=e:�PW}^:WdM�i:f8�y�\_iA�¯Q<UPQ��£MD~F^:W-eJ^~õ9�2m²���PQ�^!y�\D~<~<\'�}^]y�i6\_��eJ�3W�i6WdMD~�Q<ÚdM2eJQ<\DU­eJ�%M�e>|�Q�y�M�U�MD�7��¯Q<^:^6Q<OP~�WA^:W Þ hPWYUPf>W¯W-�7Q�^6eJ^Y|±��WVfdMDU¾i6WY�}i6Q�eJW¸���PW>\Di:W>� ¨Pm¼¨Pm�b4Q<U[e:W>i:�¯^�\�yºMDUW-�7�P~�Q<fYQ�e�^6W Þ hPWYUPf>W!\DyFf>\'��MDi:Q<M�eJWY^î� 1 Cd� 3 C tt Cd���)eJ�%M2e}fdMDU°W>MD^:Q�~�j�OXW Q��PW>U7�eJQ��%W>�lQ�U3];mB�CEDGF(H�DJI KEL±K LN¹ �Ñ#% [GIJ$ÙQ8(RQ>.EC .E,*Y÷;£`K=L?Wù� 1 C tZ C.ù���_oã.<&¥]�(¥.{9+ -1%,{. b�(RQ>C< ×. %(D1G9l$21%CoY¯. %A,�#% J%d$2CEC<$2"=.E1N#¡*8$21G9D.E,/.0$21a#%$2C�92&l%d$2IV -²D -I\Y;b�3Q4�3�7 p

`/¢�gNg ���N? � 1 C Zt C:���@? 1 Cd� 1 Cd� 3 C tt Cd�%�>oih j A~B j A�C s B E E E B j�F C"�#% -IJ ��%��.<&�,E#% �&' -,�$�%[(DCEC]*8$2²2(DIÝ.{(D,¬ 8&�.E1 ] ,E#3(D,;(DIJ Q-$2,E#Ø1±$21G9+ 8&'*8 -12(9g(D1%,E&[$�%��'���oC:���>� 1 C tZ C:��� � (D1G9¾#3(D²D � -.E,�#% -IV¢ $2IV����(2&a9+ 8&'*8 -1G9g(D1%,.E1�] ! A � ! ADC s �KJKJKJ � ! F KeS�$2IJ 8$2²D -IJ@�. %4,E#+.<&;*8$21G9D.E,/.0$21Ù.<&!&>(D,/.<&�b� :9£,E#% -1�,�#% "[GC�(D1 -²2(DC H3(D,¬ 8&)(2&

;�`>=@?Wù� 1 C tt C.ù���go 6 7� s �KJKJKJ � � F ;£`K=L? � 1 C tZ CV�k�,C:� 1 C tt C6���go�L�vÿ 1;�`*�v��? � 1 C tt C��v�@? 1 C:� 1 C tt C6�'�@? 1 o `�¨Pm^/_o

kå�Pi6\7\�y�\�yG���3W>\_i6W>� ¨Pm¼¨PmnÀ3|7e:\_�_W-eJ�PWYi��}Q�e:�°^:WY�DW>i:MD~S�_W>UPWYiJMD~�Q<Ú>M�eJQ�\_UP^ºfdMDUlOGWy�\_hPU3��Q�Uݯ�WdM�i:~ÈMDUP��L}\_OPQ�UP^¯`cb��Y�gä_o-m¡��7e:W>UP^6Q<\_U3^Ae:\�] �«Q<�PWYU�e:Q��ÓM�OPQ<~�Q�e¥jaMDi6Wi:WY�X\Di6eJWY��Q�U��Ah3i:\_��QSMDUP��Û�Q�jgMD�2Md��M[`¬b������goÝm���PW}i:W>MD�PW>iÈ^6�P\_hP~��¯U3\DeJW�e:�%M�ed|�M�~�eJ�3\_hP�_�¯µ�\_i6\_~<~<MDi6j!¨Pm¼¨Pmpä MDUP�����PW>\Di:W>�¨Pm¼¨PmnÀ­MDi:W��Pi:\+f>WY�PhPi:MD~�Q�U�eJ�3W¡^:WYUP^:W°\Dy�\D�±WYi:Q�UP�[^cj7^ceJWY�£M�e:Q<f£e:W>^ceJ^4y�\_i)�P~�M�UQ<�3W>UgeJQ��ÓMDOPQ�~<Q�e¥jD|±eJ�3WYj�MDi6W�^ceJQ<~�~T$2I69+ -Ip(c9+ A[% -1G9+ -1%,/m¯¤«e!Q<^ Þ hPQ�e:W¯�G\_^6^:Q<O3~<WzeJ�%M2eMDU¿MD�P�¯Q<^:^6Q<OP~�WV^:W Þ hPW>UPfYW°W-�7Q<^ceJ^)Q<U�\_UPW¯\_i:�PWYi:Q�UP��\�y�eJ�3W�fY\_UgeJi:\D~È��MDi6Q�MDO3~<W>^MDUP�£UP\De�Q�U¡MDU3\DeJ�PWYi��}�PWYU°OX\�eJ�¡\_i6�PW>i6Q<UP�D^�MDi6W�fY\_UP^:Q�^6e:W>Uge��}Q�e:�£eJ�3W]MDi6i:\'�}^Q<U�];mJ���PW)�DiJMD�P�3]ÜQ<Ua§�Q��_hPi6W]¨Pmm/¯Q�~<~<h3^6eJi:M�eJWY^�^:h3f8�­M;fdMD^6WDmº¤«eTQ�^�\_O3eJMDQ<UPWY�y�i:\D� §�Q<�_h3i:Wa¨Pmn¨�OgjØ�PWY~<W-eJQ<U3��eJ�3W­MDi6i:\'�}^�� 1 þ � 3 MDUP�à� 1 þ õ)|�^6\eJ�PM�eTe:�PWAe¥��\�f>\_UgeJi6\_~���MDi:Q<MDOP~<WY^)`�� 1 MDU3��� 3 o�fdMDU�OXW!\_i:�3W>i:WY�¾MDi6OPQ�e:iJMDi6Q<~�jDm

Page 29: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Êt��ÊØÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏlÃ]Î�Ô Å�Â�Æ=̬Çå̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í}Î b��7b

X2

U1

U2

X1

U1

X2

U2

X1

G

Z

(b)

Y

(a)

Z

YGX1

§�Q��_hPi:W!¨Pmm/.�}µ�MDh3^JMD~F�PQ�M��_iJMD� ] Q�U��}�PQ�f8�­�Pi:\D�XWYi�\_i6�PW>i6Q<UP�¯\Dy=eJ�3W4f>\DU�e:i:\_~��MDi:Q<MDOP~�W>^�� 1 MDUP�l� 3 Q<^�Q����G\_ice8MDUgedm���PW]\Di:�PWYi:Q<U3�°`�� 1 C:� 3 o=��\DhP~<�¡^ceJQ�~<~ÓMD�P�¯Q�eÈe:�PW MD�P�¯Q<^6^:Q�OP~<W}^:W Þ hPW>UPfYW�`��Cpõ]oMD^;OGWYy�\_i6WD|ºOPh3e¯UP\�MD�P�¯Q<^:^6Q<OP~�W¡^:W Þ hPW>UPfYW[f>MDUËOGW°y�\_hPUP�®y�\_izeJ�3Wa\Di:�PWYi:Q<U3�`�� 3 C6� 1 o-mJ���PQ<^�f>MDUlOXW ^6W>WYU�Q����¯W>�PQ<M�eJWY~�jzy�i6\_�Be:�PW �_iJM��P��]�! s |PQ<U¡�}�3Q<f8�`/M�f>f>\Di:�PQ�UP�¿eJ\ã`�¨Pmpä_o;�}Q�eJ�Q4�6 b©o;��W¾UPWYW>�×eJ\��%U3� M�^:WYe±õ�^6hPf8�×eJ�%M2e�'� 3 Cpõ � �D�¥^6W>�%MDi:M�eJWY^N¢ y�i6\_�B� 1 mJ�N\�^6hPf8�[^6WYeNW-�7Q<^ceJ^Ym���PW�Q����P~�Q<f>M�eJQ�\_U \Dy%eJ�3Q<^F\_i:�PWYi¦^:WYUP^:Q�eJQ��7Q�e¥j!Q<^SeJ�%M�e>|��}�PWYUPWY�DW>il]��XWYi:�¯Q�eJ^^:W-�_WYiJMD~�\_i6�PW>i6Q<UP�_^±\Dy7e:�PW=f>\DU�e:i:\_~��DM�i:Q�M�OP~<WY^>|©MD~�~D\_i6�PW>i6Q<UP�_^±UPW>WY�)OGW=W-�3MD�¯Q<UPWY�OGWYy�\_i6W���W°fdMDU�OGW£^6hPi:W£e:�%M�e;M��P~�MDU�Q<^)UP\De�]]�¥Q��PW>Uge:Q��ÓMDO3~<WDma½��PWYe:�PW>iVMDUWY�XW>f-eJQ��_W;^:W>MDi:f8��W-�7Q<^ceJ^]e:�Pi:\_h3�_�­eJ�PWz^6�%MDfYW¯\�y�^6hPf8��\_i:�PWYi:Q�UP�_^]i6W>��MDQ<UP^]MDU\_�GW>U Þ hPW>^ceJQ<\DUÑmæ­ç~� � ï r=t w ë .10�t w ëlñ v ê í ü ó­t ï r é í­t ê¡ë=ï\4 wNv 2

ë¦ï ì êðTñp��ñ6ò Õåö¬þ2�Sô±õ »��Fþ%ù.ç]ù��¯÷Èõ3ûJ��¢��~�Sô±õ3ù ����PW¯fdMDh3^JMD~=W-�±WYfYeA��W¯�%Md�_W�MDU%M�~�j+Ú>WY��^6\ay{MDiY|L;£`K=L?Wù�Óo-|S�¯WdMD^6hPi:WY^!e:�PWl,¬$2,0(DCWY�XW>f-e \DyºM���MDi6Q�MDO3~<W�`{\_i�M£^:W-e \Dy=��MDi6Q�MDOP~�W>^Jo�� \_UÙM£i:WY^:�G\_UP^6W4��MDi:Q<MDOP~<W)¢�m¤¥U¾��MDUgj�f>MD^:WY^>|Ñe:�PQ<^ Þ hPMDUgeJQ�e¥j��P\+W>^]U3\DeAMD�PW Þ h%M�eJWY~�j�i:WY�Pi:WY^:W>Uge e:�PW4e8M�i:�_W-e

Page 30: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b��g³ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï\Dy�Q<Ug�_WY^6eJQ��gM�e:Q<\_UaMDUP�[M�e:e:W>UgeJQ�\_UlQ<^�y�\+f>hP^6W>�[Q�UP^6e:WdMD��\_UleJ�PW!�PQ<i6W>f-eTW-�±WYfYeT\Dy� \DU°¢�ml���PWTeJW>i6� ·6�PQ<i6W>f-e�W-�±WYfYeJ¹zQ�^��¯WdM�U�eºeJ\ Þ hPMDUgeJQ�yEj�MDU�WY�XW>f-e�eJ�%M2e�Q�^UP\De���WY�PQ�M2eJW>�aOgj[\�eJ�PWYiT��MDi:Q<MDOP~�W>^}Q<UaeJ�PW)��\+�PW>~S\_iY|X�¯\_i6W4MDfYf>hPi:M�eJWY~�jD|XeJ�3W^:WYUP^:Q�eJQ��7Q�e¥jl\DyÈ¢ eJ\�f8�%MDUP�_WY^�Q�Ua� �}�PQ<~�W)M�~<~�\De:�PW>iNy{MDfYe:\_i:^NQ<UleJ�PW4MDU%MD~�j+^:Q<^MDi6W¡�PW>~����3�7W>�Ñm¬��M�e:hPiJM�~<~�jD|¦�P\_~��PQ<U3�[eJ�P\D^:W¡y{MDf-eJ\_i6^4�3�7W>�®��\DhP~<��^6WY�_WYi¯MD~<~fdM�hP^JMD~%�PM�eJ�P^=y�i6\_�s� eJ\!¢å�}Q�e:�;e:�PW}W-�7f>WY�3eJQ�\_U£\�yXe:�PW}�PQ<i6W>fYeÈ~�Q<UP�!�ÿþ ¢�|�}�PQ�f8��Q�^�UP\DeNQ<UgeJWYi:fYW>�3e:W>�aO�jaM�U�j¡Q<UgeJWYi:�¯W>�PQ<MDi:Q�W>^>mk fY~�MD^6^:Q�fdMD~�WÝ�PM����P~�Wl\DyTe:�PWahPOPQ Þ hPQ�e¥j®\Dy]�3Q<i:WYfYe¯WY�XW>f-eJ^[`{^6W>W­RNW>^6^:~<\'�b���*�À.+7µ�MDi6e¥�}i6Q<�_�geTb���/��go�e:W>~�~<^=eJ�PWN^6e:\_i6j;\�y�M)O3Q<i6e:�7�¥fY\_UgeJi6\_~%�PQ<~�~3eJ�PM�eºQ<^È^:hP^¬��GW>fYeA\�y��Pi6\7�3hPf>Q�UP�¡eJ�Pi6\_�VOG\_^:Q�^]Q<U¾��\_�¯W>U�MDUP�Ñ|SM2e eJ�PWz^:MD��WVe:Q<�¯WD|Ñ�PMD^AMUPWY�gM�eJQ��_W]Q�UP�PQ�i:W>f-e�W-�±WYfYe�\_U�eJ�Pi6\_�VOG\_^:Q�^�Ogj¡i6W>�PhPfYQ<UP�)eJ�PW]i:M�eJW�\DyS�Pi:WY�_U%MDU+�f>Q�W>^£`{�Pi6W>�_UPMDUPfYjÙQ�^A�+U3\©�}U�eJ\aW>UPfY\_hPi:MD�_W;eJ�Pi6\_�VOG\_^:Q�^8o-m¯¤¥UÙeJ�PQ�^!WÝ�3MD���3~<WD|Q<Uge:W>i:WY^6e�Q<^=y�\7fYhP^:WY�¡\_U¯eJ�PWN�PQ<i6W>f-e�W-�±WYfYe�\�yÑeJ�PWN�PQ�~<~POGW>f>MDhP^:W�Q�eºi:WY�Pi:WY^:W>Uge:^�M^6eJMDOP~�W�OPQ<\_~�\_�_Q�fdMD~�i:WY~�M�e:Q<\_U3^:�PQ��4eJ�%M2ed|ghPUP~�Q<��W�eJ�PW�e:\De8MD~3WY�XW>fYe>|�Q<^=Q<Ug��MDi6Q�MDUge�e:\��MDi:Q�e8MD~�^6eJM�eJh3^ÈMDUP�;\DeJ�3W>i¦^6\7fYQ�MD~+y{MDfYe:\_i:^�e:�%M�e=��MdjVM2�±WYfYe=��\_��WYU��¼^=f8�%MDUPfYW>^\Dy��_WYe6eJQ�UP�;�Pi:WY�_U%MDUgeN\_i�\Dy�^:hP^ce8MDQ�UPQ<U3�¯�3i:W>�DU%MDUPf-j_mk�UP\�eJ�PWYi4f>~<MD^:^)\Dy�WÝ�PM����P~�W>^)Q<Ug�_\_~��_WY^4~<WY�gMD~È�PQ�^:�Ph3e:W>^)\©�DW>iVi:MDf>W¯\_i)^:W-��PQ�^:f>i6Q<�¯Q<U%M2eJQ<\DU Q<U��PQ�i:Q<U3�Pm RTW>i6WD| UPWYQ�e:�PW>i°eJ�3W�W-�±WYfYe�\DyV^6W-�ã\_iliJM�f>W�\_UMD�P�3~<Q<f>MDUgeJ^�� Þ hPMD~<Q��%fdM�e:Q<\_U×U3\_i°eJ�PWÙWY�XW>f-e­\Dy Þ h%MD~�Q��%f>M�eJQ�\_U×\_Uã�PQ<i6Q<UP�àMDi6We8M�i:�_W-eJ^N\Dy=~<Q�eJQ��gM�eJQ�\_UÑm�LTM2eJ�PWYi>|Ó�PWYy�WYUP�%MDUge:^]�VhP^ce]�Pi6\©�DW4eJ�PM�e�^6W-�[MDUP�[iJMDfYW�P\)UP\DeA9D.EIJ 8*-,{CoY}Q<U3á%h3W>UPfYW]�PQ<i6Q<UP�A�PWYf>Q�^:Q<\DUP^>|��}�%M�e:WY�_WYi�Q<UP�3Q<i:WYfYeºWY�XW>f-eJ^�e:�PWYj�¯Q<�_�ge��%Md�DW)\_Ul�PQ�i:Q�UP�zOgj°��Mdjl\Dy�MD�P�P~�Q<f>MDUge Þ h%MD~<Q��%fdM2eJQ<\DUÑm¤¥UzMD~�~DeJ�PWY^:W�WÝ�3MD���3~<W>^Y|2e:�PW�i6W Þ hPQ�i:W>�¯W>Uge�\DyP�3\_~<�PQ�UP�}eJ�PWº��WY�PQ�M2eJQ<U3�}��MDi6Q��MDOP~�W>^ �3�7W>���VhP^ce!OXW;Q<Uge:W>i:�3i:WYe:W>�¿MD^�`{�gj+�X\De:�PWYe:Q<f>MD~<~�jPoN^:WYe6eJQ�UP�ae:�PW>^6W¯��MDi6Q��MDOP~�W>^;eJ\�f>\_U3^6e8M�U�e:^£Ogjà�P�gj+^:Q<f>MD~}Q<UgeJWYi6�DW>UgeJQ�\_UÑ|�UP\�e£OgjËMDU%MD~�j�eJQ<f>MD~��¯WdMDU3^^:h3f8�lMD^�^:WY~<WYfYeJQ�\_UÑ|+f>\_U3�PQ�e:Q<\_UPQ�UP�P|g\_i�MD�oM6hP^ceJ�¯W>Ugedmȧ%\_iºW-�3MD�¯�P~�WD|+Q�eº�}Q<~<~ÓUP\�eOGWz^:h.ѯf>Q�W>Uge!eJ\l��W>MD^:h3i:WVeJ�3W�M�^:^:\+f>Q<M�eJQ�\_U­OGWYe¥��W>WYU¿eJ�PWzO3Q<i6e:�7�¥fY\_UgeJi6\_~¦�PQ�~<~MDUP�®eJ�3i:\_�VOG\_^6Q<^z^:WY�%MDi:M�eJWY~�j®MD�¯\_UP�Ù�Pi6W>�_UPMDUge£M�UP�ËU3\_UP�Pi6W>�_U%M�U�e¯��\_��WYUMDUP��e:�PW>UàMD�D�_i:WY�gM�eJW¯eJ�PW°i6W>^6hP~�e:^>mÙ¤¥UP^6e:WdMD�±|È��W°�VhP^cez�GW>i6y�\Di:��eJ�PW°^ceJhP�3jMD�¯\_UP����\_�¯W>U×�}�P\àOXWYfdMD�¯W¾�Pi:WY�_U%MDUgelOGWYy�\_i6W¾eJ�PW¾hP^6WÙ\Dy)e:�PW¾�PQ�~<~]M�UP�MD�¯\_UP�N��\_�¯W>UV�}�3\ �Pi:W-�_W>Uge:W>�¯�Pi:WY�_U%MDUPf-jVOgj)��W>MDUP^¦\De:�PW>iFeJ�%M�UVeJ�PW��3i:hP�Pm���PW!i:W>MD^:\_U�Q<^�eJ�PM�ed|%Ogjlf>\DUP�PQ�e:Q<\_U3Q<UP�z\_U�MDU�Q<UgeJWYi:�¯W>�PQ<M�eJW]��MDi6Q�MDOP~�W;`{�3i:W>�2�U%MDU3fYj;Q<U;eJ�3WNWÝ�3MD���3~<W'oÝ|g��WN�£MdjzfYi:W>M�eJWN^:�Ph3i:Q<\DhP^�M�^:^:\+f>Q<M�eJQ�\_UP^=OGWYe¥��W>WYU£�MDUP�Ù¢BWY�_WYU¾�}�PW>U­e:�PW>i6WzQ<^�U3\¡�PQ<i6W>f-e WY�XW>f-e \Dy=� \_U¾¢�m'���PQ<^�fdMDU¾W>MD^:Q�~�jOGW)Q�~<~<h3^6eJi:M�eJWY�lQ<U�e:�PW!��\+�PW>~��äþ õ�� 5�þ ¢¯|G�}�3W>i:W!�Ö�%M�^TUP\��PQ<i6W>f-eWY�XW>f-eº\_U;¢�ml¯=�gj+^:Q�fdMD~�~�j)�P\_~<�3Q<UP��õ fY\_UP^6eJMDUge=��\DhP~<�;�GW>i:�¯Q�e�UP\!MD^:^6\7fYQ�M�e:Q<\_UOGWYe¥��W>W>Uz� MDUP�z¢¯|gMD^�f>MDUVOGW�^6W>W>UzOgj!�PWY~<WYe:Q<UP��MD~<~+MDi6i:\'�}^�WYUgeJW>i6Q<UP�"õ)m`°�h7e

Page 31: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Êt��ÊØÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏlÃ]Î�Ô Å�Â�Æ=̬Çå̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í}Î b��D¶Q�y���WV��W>i6WVeJ\¡fY\_UP�PQ�eJQ<\DU[\_U®õ!|ÑM£^6�PhPi:Q�\_hP^ M�^:^:\+f>Q<M�eJQ�\_U���\_hP~��­OGWVf>i6WdM�e:W>�eJ�3i:\_hP�D�§5 `�hPUP\_O3^:W>ic�_WY�Óo e:�%M�e!�¯Q<�_�ge OGW;f>\_UP^ceJi6hPW>�¿MD^AMa�3Q<i:WYfYeAW-�±WYfYe!\Dy� \_U�¢�mðTñp��ñQÔ Õåö¬þ2�Sô±õ ¢��~�Sô±õPù±ú�ÕÄ�O@!ø ö¥õPöc÷�øzú)ûÈø]üÿ>gü"��ø�õPö�@Aô�û�õPö¬÷�ø

µ�\_UgeJi6\_~<~�Q<UP��MD~<~���MDi:Q<MDOP~�W>^¦Q�UzM �Pi:\_O3~<W>�Q<^�\_Og�7Q�\_hP^6~�j4M]�£MUM6\Di¦hPUP�PWYi6eJMD��Q<UP�P|Q�y3UP\De�MDU4Q����G\_^6^:Q<O3Q<~<Q�e¥j_m����3W�MDU%M�~�j+^:Q�^�\Dy3Q<�PWYUgeJQ��PfdM�e:Q<\_UAeJWY~<~<^ShP^�hPUP�PWYi��}�%M2ef>\DUP�PQ�e:Q<\_U3^ �PQ<i6W>fYe!W-�±WYfYeJ^)fdMDU�OGW�WY^6eJQ��£M2eJW>�¾y�i6\_� UP\_U3W-�7�XWYi:Q���WYU�eJMD~=�%M�eJMWY�DW>UÙ�}Q�e:�P\_h3e ^6hPf8�¿fY\_UgeJi6\_~0m&�T^6Q<UP�°\_h3i ���3`E�Xo�UP\De8M2eJQ<\DU®`{\_iøù�¿y�\_i]^6�P\_iceÝo-|��W)f>MDUlW-�7�Pi:WY^:^NeJ�PWA�PQ�i:W>f-eNWY�XW>fYeTMD^�y�\_~�~<\'�}^>mV DXWY�ZI�[*F � KELN³°L*P�� V *H�D��*���¡ ;D��*�*¢�Ñ#% !9D.EIJ 8*-,� IH] 8*-,�$�%�� $21£¢�.<&x#D.E²D -1 QZY�;�`>=@?Wù��CÏùÅZ!�£�oJ@È"�#% -IJ Dë�!�£ .<& ,E#% &' -,�$�%)(DCEC� -1G9+$$#+ -1±$2H�& ²2(DI-.{(RQ>C< 8&) 657*8 A[G,F� (D1G9V¢ .E1­,E#%  &\Y'&-,¬ -?¯K½�W£^:WYW�e:�%M�e!eJ�PW���W>MD^:hPi6W>�¯W>UgeV\�y��PQ<i6W>f-e4W-�±WYfYeJ^4Q<^)MD^:fYi:Q<OGW>��eJ\[MDU¿Q��PWdMD~~�M�OX\_i:M�eJ\Di6j�+�e:�PW°^:fYQ<W>Uge:Q<^6e¯fY\_UgeJi:\D~<^Vy�\_i;MD~�~��G\_^:^6Q<OP~�W¡f>\DUP�PQ�e:Q<\_U3^-ë�!Y£åM�UP�UPWYW>�¡UP\�e�OGW�Md��MDi:WT\Dy±e:�PW�^ceJi6hPfYe:hPi:WT\Dy±e:�PW��PQ<MD�_i:MD�Ü\_iÈ\DyÑ�}�3Q<f8����MDi6Q�MDO3~<W>^MDi6W)eJi6hP~�jlQ�UgeJW>i6��WY�PQ�M�i:Q<WY^TOGWYe¥��W>WYU¾� MDUP�­¢£m]ÛahPf8�¾\Dy=eJ�3W4W-�7�GW>i6Q<�¯W>Uge8MD~f>\DU�e:i:\_~NfdM�UØOGW[W>~�Q<�¯Q<U%M2eJW>�±|º�3\©��WY�DW>iY|TQ�y���W­��UP\'��eJ�PW�^6eJi6hPfYe:hPi:W[\Dy]eJ�3W�PQ<MD�_iJM���mק%\_i¯\_UPWaeJ�PQ�UP�P|ºeJ�PWYi:W­Q�^�UP\�UPW>WY� eJ\¿MDfYe:h%MD~<~�j��P\_~�� (DCEC¦\�eJ�PWYi��MDi:Q<MDOP~�W>^ºf>\_UP^ce8MDUge�+P�P\_~��PQ<UP�)f>\_U3^6e8M�U�e�e:�PW]�PQ�i:WYfYe��PMDi:WYU�e:^�\DyF¢ `�W-�7f>~�hP�PQ�UP���o}^:�P\_h3~<�¾^:h,Ñ�fYWDm�����hP^>|Ñ��Wz\_O3eJMDQ<U�eJ�PW4y�\D~<~<\'�}Q�UP��W Þ hPQ���MD~�W>Uge]�PW-�%UPQ�eJQ<\DU\Dy�Mz�PQ<i6W>fYeNW-�±WYfYedm^ F(H�F _`_*¯�H�a�KELN³°LN� �Ñ#% 9D.EIJ 8*-, IH] 8*-, $�% � $21 ¢ .<& #D.E²D -1 QZY;�`>=@?Wù�uCS¤»(' £¦¥T! o8@N"�#% -IJ �»(' £¦¥T! &-,0(D1G92&�%d$2I¯(D1zYlIJ :(DC .XW'(D,/.0$21à$�%z,�#% Â[3(DI8 -1%,E&$�%N¢;@� 657*-C H39D.E1N#!�àKµ�~<W>MDi:~�j_|�Q�yP���3\7WY^¦UP\De¦MD�P�GWdMDi�Q<U)e:�PW�W Þ hPM�eJQ�\_U4y�\_i�¢Ü`{W Þ hPQ���MD~�W>UgeJ~�j_|DQ�yP��Q�^UP\De�M;�%MDi:WYUge]\Dy¦¢zo-|%eJ�3W>Uø;�`>=@?Wù��C ¤»(' £ ¥T! o��PWY�%U3W>^]M¯f>\_U3^6e8M�U�e��PQ<^ceJi6Q<OPh3e:Q<\_U\_UØ¢ eJ�PM�e£Q�^�Q�UP�PWY�XWYUP�PW>Uge°\�y]�S|�eJ��hP^��£M�e:f8�PQ<U3��\_h3i£hPUP�3W>i:^ce8MDU3�PQ<UP��\Dy·6�%Md�+Q<U3�!U3\!�3Q<i:WYfYeÈW-�±WYfYedm¼¹�¤¥U��_WYUPW>i:MD~0|�M�^:^:h3��Q�UP�]eJ�%M2e=��Q�^ÈMA�PMDi:WYU�eº\Dy±¢�|µ�\_i6\_~<~<MDi6ja¨Pmpä7mp³£Q<�¯�P~<Q�W>^}eJ�%M�e�eJ�PWz�PQ�i:WYfYe W-�±WYfYeA\DyÈ� \_U�¢�Q�^]Q<�PWYUgeJQ��%MDOP~<W�}�PWYUPWY�DW>i�;£`K=L?t¤»(' £ o�Q�^£Q<�3W>UgeJQ��ÓMDOP~�WDmâÛ�\Di:W>\'�DW>i>|T^:Q�UPf>W[e:�PW­fY\_UP�PQ�eJQ<\DUPQ<UP�

Page 32: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b��D¨ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï�%MDice�\Dy�e:�PQ<^�W-�7�Pi6W>^:^6Q<\_U°fY\_i:i6W>^6�X\_U3�P^�eJ\zM)�P~�MDU�Q<U£�}�3Q<f8�¡eJ�3W]�%MDi6W>UgeJ^�\�yF¢MDi6WTeJ�3W�f>\DU�e:i:\_~G��MDi:Q<MDOP~<WY^>|g��W]f>\_U3f>~<h3�PW�eJ�PM�e�M)�PQ<i6W>fYe�WY�XW>f-e�Q<^ºQ<�PWYUgeJQ��%MDOP~<W�}�PWYUPWY�DW>iVeJ�3W�W-�±WYfYeV\�y�eJ�PW¯f>\Di:i:WY^:�G\_UP�PQ�UP�[�PMDi:WYU�e:^��F�P~�M�U¿Q<^AQ��PW>UgeJQ��ÓMDOP~�WDm½�Waf>MDUàUP\'� hP^6Wae:�PWaMDU%MD~�j7^6Q<^z\Dy x WYfYeJQ�\_U®¨Pm¼¨¿MDU3�ËM��P�P~�j¿eJ�PWl�_i:MD�P�PQ�fdMD~f>i6Q�e:W>i:Q<M)\DyL���PWY\_i:WY��^�¨3mn¨Pm�b MDU3�°¨Pmn¨3mpÀ4e:\4eJ�PWAM�U%MD~�j+^6Q<^�\�y��3Q<i:WYfYe�WY�XW>fYe:^>mº¤¥U�%MDiceJQ�f>hP~<MDi>|+��W!fdMDUl^ce8M�e:W!\DhPi}UPW-�+e}eJ�PWY\_i:WY��mB�CEDGF(H�DJI KELN³OLNM !S -,�;�¿§£ 6 �'� 1 C tt C:���¤C tt C6��¨ � Kê�Ñ#% �9D.EIJ 8*-,  IH] 8*-,$�%[(D1zY����à$21ࢠ.<&�.{9+ -1%,{. b�(RQ>C< l"�#% -1± -²D -Il,�#% Ù*8$21G9D.E,{.0$213&¾$�% � $2IJ$2CEC�(DI\Y�GK¶�GKª©£#%$2C�9"%d$2I),�#% ¾[GC�(D1�`Uù� 1 C,ù� 3 C tZ C.ù��¨�o4.E1�&'$2?£ V(g9D?z.<&8&-.*Q>C< ;$2I69+ -I-.E1N#[$�%,E#% !²2(DI-.{(RQ>C< 8&>K��Ñ#% 49D.EIJ 8*-,� IH] 8*-,º.<&!,E#% -1�#D.E²D -1�QZY£`{¨Pmm/go>K���PWY\_i:WY�s¨Pmpä7mn¶]Q<�¯�P~<Q�W>^�eJ�%M2e=Q�yPe:�PW�WY�XW>f-e=\DyG\_UPW��%MDi:WYUge=\DyX¢ Q<^�Q<�PWYUgeJQ��%MDOP~<WeJ�3W>UÙeJ�PWzW-�±WYfYe)\DyºWY�_WYi6jÙ�%M�i:W>Uge!\Dy�¢ Q<^ Q��PW>UgeJQ��ÓMDOP~�WzMD^]��W>~�~0m ª y�f>\_hPi6^:W�|eJ�3W �£MD�DUPQ�e:hP�PW�\�yFe:�PWAWY�XW>f-e���\_h3~<�°�PQ��±WYi�y�i:\_� �%MDi6W>Uge�eJ\z�%M�i:W>Uge>|ÓMD^�^:WYW>UQ<U�`�¨Pm^/_o-m���PWAy�\_~<~�\©�}Q�UP�VfY\_i:\_~�~�MDicj¯Q<^}Q<�¯��WY�PQ�M2eJWDm^ F(H�F _`_*¯�H�a�KELN³°L±K !F -,¦�eª^Q- z(�[3(DIJ -1%,T$�%�¢°Kø�Ñ#% ;9D.EIJ 8*-,� IH] 8*-,T$�%N�eªa$21¢�.<&[email protected]®#+ -1± -I6(DCp@}1±$21%.{9+ -1%,/. b�(RQ>C< 4. %z,E#% -IJ ¡ 65_.<&-,�&¯(Ù*8$21%d$2H71G9D.E1N#�(DIJ*;,�#3(D, -?�Q>I6(+*8 8&)(D1zY¯C .E1�U ���"þ ¢°KðTñp��ñrä ¢�ÁTû;� :"�B� � úx�°Á Õ öcù7ô�þÓöi� ö6ø]û�õPö¬÷�ø öcøúå�÷������E¼��ãó�ü¾½

�Bö¬ù3ù3öc÷�ø�F\�Q�~<~<h3^6eJi:M�eJWTeJ�PW!hP^:W!\Dy�eJ�PQ�^Ni6W>^6hP~�e>|GfY\_UP^6Q<�PWYi}e:�PW!^6eJh3�3ja\Dy`°�W>i:��W>~�WYj��n^]MD~��~<WY�_W>��^:WÝ�¿O3Q�MD^!Q<U��_iJM��Ph%M�e:W£MD�3��Q�^:^:Q�\_UË`�°�Q�f8�DWY~�W-eVMD~/m=b���*_äDo-|S�}�PW>i6W£�%M�eJM^:�3\©��W>�£M]�PQ<�D�PW>i¦i:M�eJW�\�y±M��P��Q�^:^6Q<\_U4y�\Di¦�£M�~<W�M��P�P~<Q�fdMDUge:^¦\'�_W>i:MD~<~3OPh3ed|_�}�3W>UOPi6\_�DWYU��P\'�}U�Ogj��3W>�%MDiceJ�¯W>UgeJ^Y|=M�^6~<Q��_�ge4OPQ<MD^ eJ\'��MDi:�¿y�WY�£M�~<W�MD�P�P~�Q<fdM�U�e:^>m���PWAW-�7�P~<MDU%M�e:Q<\_U¡��MD^�eJ�PM�e�y�W>��MD~<W�MD�P�P~�Q<f>MDUgeJ^�eJWYUP�°eJ\¯MD�P�3~�j£e:\VeJ�PWA�¯\_i:Wf>\D���GWYe:Q�eJQ��_W]�3W>�%MDiceJ�¯W>UgeJ^Y|3�}�PWYi:WAi:WQM6W>fYe:Q<\_Uli:M�eJWY^}MDi:W �PQ��_��+7O%MD^:WY��\DU¡eJ�PQ�^�%UP�3Q<UP�P|�°�W>i6�DW>~�WYj���MD^ WÝ�7\_UPW>i:M�eJWY�¾y�i:\_� f8�%MDi6�_W>^ \Dyº�PQ�^:fYi:Q<�¯Q<UPM�eJQ�\_UÑmx���PW�P�PQ�~<\_^6\_�P�PQ�fdMD~ºMD^6�XWYfYe:^z\Dy}^:hPf8�®i6WY�_WYi:^:MD~<^Y|º��UP\'�}U®MD^ x Q<�¯�P^:\DU��n^)�%MDi:MD�P\d�X|�}Q<~�~�OXW)�PQ<^6f>hP^6^:WY���¯\_i6W!y�hP~�~�j°Q�U¾µ��%M��3eJWYi À3mTRTW>i6W)��W4y�\+f>hP^T\_U[eJ�3W Þ hPW>^¬�eJQ�\_U¡\Dy��}�3WYeJ�3W>i}MD�oM6h3^6eJ�¯W>Uge�y�\Di��PWY�%MDi6e:��WYUge}Q<^�M��P�Pi:\D�Pi:Q<M�eJWNy�\_i�MD^6^:WY^:^:Q�UP�^:WÝ�®�PQ�^:f>i6Q<�¯Q<U%M2eJQ<\DU¿Q<U�fY\_~<~�W>�_W¡MD�P�¯Q<^:^6Q<\_U±m�µ�\_Ug�DW>UgeJQ�\_U%MD~��}Q�^:�P\_� �%MD^zQ�eeJ�PM�e]^6hPf8��MD�oM6hP^ceJ�¯W>Uge Q<^�MD�P�Pi6\_�Pi6Q�M�e:W)OGW>f>MDhP^:W�·¬½¿W)�+U3\©�eJ�PM�e MD�3�P~�j+Q<U3�

Page 33: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Êt��ÊØÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏlÃ]Î�Ô Å�Â�Æ=̬Çå̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í}Î b��_äeJ\£M¯�G\_�Ph3~�MDi}�PWY�%MDi6e:��WYUge;`{\DUPW!�}Q�eJ�af>\_U3^:Q<�3W>iJM�OP~�jl�¯\_i:W)MD�P�3~<Q<f>MDUgeJ^�eJ�PMDU�G\_^:Q�eJQ<\DUP^8oTQ�^�M6hP^ce!e:�PW;��Q<UP�Ù\DyÈeJ�3Q<UP�°eJ�PM�eAfdMDh3^:W>^Ai6W�M6W>f-eJQ�\_UP¹�`0µ�MDice¥�}i:Q��_�geb���/D¶3|��Ñm�¶�/goÝ|�OPh3e¯��W[�}Q<~�~�^:\+\_UØ^6W>W[e:�%M�e£MD�P�PQ�eJQ<\DU%MD~�y{MDf-eJ\_i6^£^6�P\_hP~��ØOGWf>\DUP^:Q��PW>i6W>�Ñmé�WYe¡h3^°MD^:^6hP�¯W�eJ�PM�e£e:�PW[i6W>~<W-��MDUge£y{MDf-eJ\_i6^£Q<UàeJ�PW °�W>i6�DWY~<WYjËW-�3MD�¯�P~�WMDi6W�f>\DU3�%�_hPi6W>�lMD^�Q<U£§�Q<�_h3i:WN¨Pm^�7|+�}Q�eJ�£e:�PWTy�\_~�~<\'�}Q<U3�!Q�UgeJW>i6�Pi:W-e8M�e:Q<\_U�\Dy�eJ�3W

X2

X1

Y

U

Z

§�Q��_hPi:W ¨Pmm�.� µ�MDhP^JM�~ i6W>~<M�eJQ�\_UP^:�3Q<�P^ i:WY~<W-�DM�U�e eJ\ °�WYi:�DWY~<W-j �¼^ ^6W-����PQ�^:f>i6Q<�¯Q<U%M2eJQ<\DU�^ceJhP�3j�+�M��oM6hP^6e:Q<UP��y�\_iV�PWY�%MDi6e:��WYUgezf8�P\_Q�f>W�`�� 3 o!\_iVfdM�i:W>WYi\_ONM6WYfYe:Q��_W4`Wõ�o�`{\_i=OG\DeJ�%o���\_h3~<�¯OXW}Q<UPMD�P�Pi6\_�Pi:Q<M�eJW�Q<U;W>^ceJQ<��M�e:Q<UP��e:�PW}�PQ<i6W>f-eWY�XW>f-e]\Dy¦�_W>UP�3W>i�\DU¾MD�P�¯Q<^6^:Q<\DUÑmE���PWzM��P�Pi:\D�Pi:Q<M�eJW!MD�oM6h3^6eJ�¯W>Uge�Q<^N�_Q��DW>U­Q�U`{¨3m�b�0_o-m��MDi:Q<MDOP~�W>^��� 1 � MD�P�P~�Q<f>MDUge�¼^��_W>UP�3W>i�+� 3 � MD�P�P~�Q<f>MDUge�¼^�f8�P\_Q<fYWA\Dy��3W>�%MDiceJ�¯W>Uge+õ�� MD�P�3~<Q<f>MDUge�¼^�fdMDi6W>WYiN\DONM6W>f-eJQ��DW>^�+¢�� MD�3��Q�^:^:Q�\_U°\_h3e:f>\_�¯W¯`/M�f>f>WY�3e~«2i:W�M6WYfYe8o)+5�� MD�P�P~�Q<fdM�U�e��n^}M��3eJQ�eJhP�PW�`�hPUPi:WYf>\_i6�PW>�%o-m�T\�eJW�e:�%M�es5åM��XW>f-eJ^�MD�3�P~<Q�fdMDUge�¼^�f>MDi:WYW>i�\DONM6W>f-eJQ��DW MDUP�¡MD~<^6\4eJ�PW]MD�P�¯Q<^:^6Q<\_U\_h3e:f>\_�¯WA¢ `{^:M'jD|PeJ�Pi6\_hP�_�l�_WYi:O%M�~F^6��Q<~<~�^A`{h3UPi:WYf>\_i6�PW>�Óo6o-m

Page 34: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b��_À Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ïk��oM6h3^6eJQ�UP�zy�\_i��PWY�%MDi6e:��WYUge}f8�P\_Q<fYW!MD��\DhPUgeJ^�eJ\;fY\_���3h3eJQ�UP�VeJ�PW]y�\D~<~<\'���Q<U3�¯WÝ�7�Pi:WY^:^:Q�\_U��

È�þ ~ ;�`>=@?Wù� 1 C6� 3 oJ6 7 þ ~ ;£`K=L? � 1 C:� 3 oB;£`E� 3 o `�¨Pm^�_o¤¥Uaf>\_UgeJi:MD^6e>|PeJ�PWA�PQ�i:WYfYeNWY�XW>f-eT\�y�� 1 \_U[¢¯|%MD^}�_Q��_W>UaOgj�`{¨3md*_oÝ|Pi:W>MD�P^

;�`>=@?Wù� 1 C.ù� 3 ol6 7 ü ;£`K=L? ¸.C:� 1 C6� 3 o�;�`/¸ ? � 1 o `{¨Pm�b�0go¤«eAQ<^�f>~<W>MDi�e:�%M�e eJ�3WVe¥��\lW-�7�Pi6W>^6^:Q<\DUP^)��Mdj[�3Q��XW>i]^6hPOP^6eJMDUgeJQ<MD~<~�j_m����PWV�%i:^ce�¯WdMD^6hPi:WY^Ne:�PW�`/Md�_WYiJMD�_Wdo}W-�±WYfYeN\Dy�^:W-�a\DU�M��P��Q�^:^6Q<\_UaMD�¯\_UP�;MD�3�P~<Q�fdMDUgeJ^�e:\M;�_Q��DW>Ua�PW>�%M�i6eJ�¯W>Uge>|XM Þ h%MDUgeJQ�eJQ�j¡eJ�%M2eTQ�^}^:W>U3^:Q�e:Q��DW!e:\¯e:�PW!y{M�fYe}eJ�%M2eT^6\_�¯W�_WYUP�PW>i¬�¥�PWY�%MDiceJ�¯W>Uge4fY\_�VOPQ�U%M�e:Q<\_UP^��£Mdj¾OGW£M�^:^:\+f>Q<M�eJWY���}Q�eJ�Ù�PQ��_�¿M��P��Q�^c�^:Q�\_U�iJM�e:W>^4�¯W>i6W>~�j¾OXWYfdMDh3^:W£^6hPf8��f>\D�VOPQ<UPM�eJQ�\_UP^AMDi6W�Q�UP�PQ<f>M�eJQ��_W;\Dy�f>W>ice8MDQ�UMD�3e:Q�e:hP�PW[`Q5]oAe:�%M�e4��MD^)hPUPi:WYf>\_i6�PW>�±m ���PW¡^6W>fY\_UP��WÝ�3�3i:W>^6^:Q�\_U�W>~�Q<�¯Q<U%M�e:W>^^:h3f8�°^:�PhPi6Q<\_h3^�MD^6^:\+f>Q�M2eJQ<\DUP^�Ogj¯^:WY�%MDi:M�eJWY~�j�MD�oM6hP^ceJQ�UP�)y�\_iºfdM�i:W>WYi�\_OSM6W>fYe:Q��DW>^`Wõ�o�Q<UlWdMDf8��\DyFeJ�PW e¥��\��_WYUP�PWYi:^>m��\l�_WYi:Q�yEj[eJ�PM�e�`{¨Pmm�go��P\+W>^ U3\DeA�Pi:\D�XWYi:~�j[��W>MD^:h3i:WzeJ�PWz�3Q<i:WYfYeAW-�±WYfYe!\Dy� 1 \_Ul¢�|7��W UP\�eJW�e:�%M�e�e:�PW]W-�7�Pi6W>^6^:Q<\DUa�3W>�GW>UP�P^}\_U¡e:�PW���MD~�hPW]\Dy�� 1 WY�DW>UQ<U;f>MD^:WY^º�}�PWYi:WNeJ�PWNMDi6i:\'� OGWYe¥��W>WYU£� 1 MDU3�£¢ Q<^ºMDO3^:W>Uge>mJ Þ h%M2eJQ<\DU­`�¨Pm�b0go-|\_U;eJ�PWN\De:�PW>iº�%MDUP�±|7OGW>fY\_��WY^�Q�UP^:WYUP^:Q�eJQ��_W�eJ\A� 1 Q<U;^6hPf8�¡fdM�^:W>^¥ßãMDU£W-�7W>i6f>Q<^6WeJ�PM�e���W!~<W>Md�_W y�\_i�eJ�3W!i6WdMD�3W>i�eJ\z�_WYi:Q�yEj_m­¬��\zf>MD^6e�eJ�PQ�^�MDU%MD~�j+^:Q<^�Q<U°MVfY\_UPfYi:WYe:W U�hP��WYi:Q�fdMD~G^:W-e:eJQ�UP�P|7~<WYe�hP^�Q��£MD�DQ<UPWMzf>\_~�~<WY�_W�fY\_UP^:Q�^6e:Q<UP�z\DyFe¥��\¯�3W>�%MDiceJ�¯W>UgeJ^Y|2¿âMDUP��À£|POG\De:��M��P��Q�e:e:Q<UP�)^6eJh+��PWYU�e:^�\_UleJ�3W!OPMD^:Q�^N\�y Þ h%MD~�Q��%f>M�eJQ�\_UÑ|/®;|GMD~<\DUPWDm�é�WYeNhP^}y�hPi6e:�PW>iNMD^6^:hP�¯W£`{QEoeJ�PM�eTe:�PWVM��P�P~<Q�fdMDUgeT�X\+\_~Sf>\DUP^:Q�^6eJ^T\Dy�b�0Y0���MD~<WY^�MDU3��b�0�0zy�W>��MD~<WY^�M�UP�®`{Q�QEoeJ�PM�e;äY0�MD�P�P~�Q<f>MDUgeJ^!Q<U¿W>MDf8���_WYUP�PW>i)�%Md�_W°�PQ��_� Þ h%M�~<Q��PfdM�e:Q<\_UP^�`��PW>UPfYW°MDi6WMD�P�¯Q�e6eJWY�Óo�MDU3�[ä¤0V�%Md�_WA~<\'� Þ h%MD~<Q��%fdM2eJQ<\DUP^ `��PW>UPfYW!MDi:W]i6W�M6WYfYeJWY�Óo-m�µ�~<W>MDi:~�j_|eJ�3Q<^�f>\D~<~<WY�_W fdM�UPUP\De}OGW!MDf>fYhP^:WY�[\Dy�^:WÝ�a�PQ<^6f>i:Q���Q�U%M�e:Q<\_UÑmkB�3Q��XW>i:WYUge4i6W>^:h3~�eA��\DhP~<��^:hPicy{MDf>W�|��P\'��WY�_WYi>|=Q�y���W¡MD�UM6hP^6e!y�\_iA�PW>�PMDi6ec��¯W>UgeJ^z�}�PQ�~<W°Q<�DUP\_i:Q�UP� Þ h%MD~�Q��%f>M�eJQ�\_UP^Y|È�}�3Q<f8�ËMD��\DhPUgeJ^zeJ\ÙhP^6Q<UP�Ë`{¨3m^�go)e:\W>^ceJQ��£M�e:W£eJ�3WaW-�±WYfYe¯\DyT�_W>U3�PW>i;\_UËMD�P�¯Q<^6^:Q<\DUÑm¿kT^:^:h3��W°eJ�PM�ezeJ�PWlU%M2eJhPi6W\Dy=e:�PWV�PWY�%MDi6e:��WYUgeJ^ Q<^T^:hPf8�ÙeJ�PM�e;(DCEC¦(D1G9­$21%CoY Þ h%MD~<Q��%W>�[�£MD~�WVMD�3�P~<Q�fdMDUgeJ^MD�P�3~�jaeJ\¡�PWY�%MDiceJ�¯W>Uge"¿4|X�}�PQ<~�WVMD~�~Fy�WY�£MD~�W>^�MD�P�P~�jae:\°�PW>�PMDi6e:��WYU�e"À�`{^:WYW��MDOP~<W ¨3m�b©oÝm¯ �@b^k�e�h | _)hQeWX)VWbmº�[ � ���*°d±°�²Y�)³� O °´³� { �,ToZ�bmkUtfeWgU[�bmkocU[BRY[Bkoc�[�koh�b^[�Z�b^k�e�gU[�t)Vr_pRog�_pk¤cJ[B\^b^i�¥b^ko_veW[+±�_)Z�b^kseWgU[Jc�[�V�b^av_ve�b^X)k�Xp����"oz ���p�Qz

Page 35: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Êt��ÊØÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏlÃ]Î�Ô Å�Â�Æ=̬Çå̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í}Î b���*

Û[MD~�W>^ §%W>��MD~<WY^ �F\De8MD~kT�P�¯Q�e:e:W>� k��P�3~<Q<WY� kT�P��Q�e:e:W>� kT�P�P~�Q<W>� k��P�¯Q�e6eJWY� k��3�P~<Q�W>�Z]W>�7edm�¿ ä¤0 äY0 0 0 äY0 ä¤0Z]W>�7edm2À 0 äY0 äY0 b�0�0 äY0 b'äY0�TUPMD�oM6hP^ceJW>� äY0¶µ äY0Xµ ä¤0Xµk��UM6hP^6e:W>� ³_ä*µ ¶�*7m´ä*µ��MDOP~<W£¨Pm�bY�£k��3��Q�^:^:Q�\_U�i:M�eJWlMD�¯\_UP�[�£MD~�W>^zM�UP��y�W>��MD~<WY^VQ<U�W>MDf8�®�PW>�PMDi6ec��¯W>Ugedm

½¿W4^:W>W)y�i:\_��eJ�3W4e8MDO3~<WAeJ�%M�e]MD�oM6hP^ceJQ�UP�£y�\_iN�PW>�PMDi6e:��WYU�e���\_h3~<�[y{MD~�^:WY~�jQ<U3�PQ<f>M�eJW!M�O3Q�MD^N\Dyȶ�*+m´ä��±³Dä[`W6ܶ��X³_o}Q�U�y{Md�_\Di�\Dy¦y�WY�£M�~<W!MD�P�P~�Q<f>MDUgeJ^>m�kTUhPU%M��oM6hP^6e:W>�¡`{^:\D��W-eJQ<�¯W>^±fdMD~�~<W>��·6fYi:hP�3W>¹�o�MDU%M�~�j+^:Q�^��%MD�P�GW>UP^SeJ\}�_Q��_W¦e:�PWÈf>\_i¬�i:WYfYeNi:WY^:hP~�e}Q<U°eJ�3Q<^�WÝ�PM����P~�WÝßãäY0XµBM��P��Q�^:^6Q<\_U¡i:M�eJW]y�\Di��£MD~�W>^�M�UP�ay�WY�£M�~<W>^MD~�Q<�DW8ß×e:�+h3^�W-�7\_UPWYiJM�e:Q<UP�;eJ�PWA^6f8�P\7\D~Ñy�i:\_�Bf8�%MDi:�DW>^N\Dy�^6W-�a�PQ�^:fYi:Q<�¯Q<UPM�eJQ�\_UÑmª hPi=MDU%M�~�j+^:Q�^�Q�^�UP\De¦��W>MDUge�eJ\�Q<�¯�P~�j]e:�%M�e¦e:�PW�°�WYi:��W>~<W-j4^ceJhP�7jV\Dy�°�Q�f8�DWY~WYeVMD~0mº`¬b���*_äDo Q�^)�3WYy�W>f-eJQ��_WD|=\Di!e:�%M�e4MD�oM6hP^ceJ�¯W>Uge4y�\_i!�PWY�%MDi6e:��WYUge4��MD^4UP\�eM6hP^ceJQ��PW>�zQ<U)eJ�%M�e�^6e:hP�3j_ml���3W��PhPi6�X\_^6W�Q<^FeJ\ W>�¯�P�%M�^:Q<ÚYW�e:�%M�e=UP\AM��oM6hP^6e:��WYUgeQ<^ �Dh%MDiJM�U�e:W>WY��eJ\a�_Q��_W;MDU�hPU�OPQ�M�^:W>��W>^ceJQ<��M�e:Wz\Dy�f>MDhP^:MD~ÈWY�XW>fYe:^>|��PQ<i6W>fYeA\DiQ<U3�PQ<i6W>fYe>|SMDOP^6W>UgeVMlfdM�i:WYy�h3~=W-�3MD�¯Q<U%M�e:Q<\_U¾\�y�eJ�3W¯f>MDhP^:MD~ÈMD^:^6hP�¯�3eJQ�\_UP^ eJ�%M2eW>U3^:hPi6W4Q<�3W>UgeJQ��%fdM�e:Q<\_U±mE���PW>\Di:W>� ¨Pmpä7mn¶��Pi6\'�7Q��PW>^NhP^N�}Q�eJ�aeJ�PW!hPUP�3W>i:^ce8MDU3�7�Q<U3�)\DyXeJ�P\D^:W]M�^:^:h3���3e:Q<\_U3^�MDU3���}Q�e:�¡MA�£M2eJ�PWY�£M�e:Q<f>MD~P�¯WdMDUP^º\Dy�WÝ�7�Pi:WY^:^:Q�UP�eJ�3W>�am¯½�W�U3\DeJWzeJ�PM�e!Q�y�MD�P�P~�Q<fdM�U�e:^�� Þ h%M�~<Q��PfdM�e:Q<\_UP^]��W>i6W�U3\De4i6W>fY\_i:�PWY�¿Q�UeJ�3WT�%M2e8M3|geJ�3W>U;eJ�PWN�PQ<i6W>f-e�W-�±WYfYe�\DyÑ�DW>UP�PWYi�\DU£MD�3��Q�^:^:Q�\_Uz�}Q<~�~PUP\DeºOGWTQ��PW>U7�eJQ��ÓMDOP~�W)h3UP~<WY^:^]��WzfdMDU¾�¯WdMD^6hPi:W4^:\_�¯W4�Pi6\d�7j���MDi:Q<MDOP~<WAeJ�PM�e ^6eJMDUP�P^ Q�U[eJ�3W^JM���W i6W>~�M2eJQ<\DU¡eJ\�® MD^sõå^6e8M�UP�P^}eJ\Ï5âQ<Ua§�Q��_hPi6W ¨Pm^�7mðTñp��ñ{ð ów»��Sþ%û ¼�� Õåö¬þ2�Sô±õ?¢��~�Sô±õPùL}WdMD�PWYi:^V�DW>i:^6W>�àQ<U�^ceJi6hPfYe:hPiJMD~�W Þ h%M�e:Q<\_U��¯\7�3W>~<^¡` x ºÛ�^Jo)�}Q<~�~�UP\�eJW£e:�%M�ed|Q<UÙ~�Q<UPW>MDi ^6j+^6e:W>�¯^>|FeJ�PW¯�PQ<i6W>f-e)W-�±WYfYe¡;£`{¢�?Wù�@C ¤»(' £ ¥T! o]Q<^ y�hP~�~�j­^6�XWYf>Q��%W>��O�jeJ�3W£�%M�e:��f>\+W�ѯfYQ<W>Uge;M2e:e8M�f8�PW>��e:\�eJ�3W¡~<Q�UP�[y�i:\D��� eJ\­¢ `{^6W>WÙ`/ä7m´³2¨�o y�\Di��M�eJ�PWY�£M2eJQ<f>MD~��PW-�%UPQ�e:Q<\_U%o)+ÓeJ�PWYi:WYy�\Di:WD|GeJ�PW4�PQ<i6W>fYe�WY�XW>f-e Q<^NQ<UP�3W>�GW>UP�PWYUge \DyeJ�3WN��MD~�hPW>^�»('X£ ¥T!ËM�eÈ�}�PQ<f8�¯��WN�P\_~��¯e:�PW}\DeJ�PWYi��PMDi:WYU�e:^�\�y�¢�m�¤¥U�UP\DUP~<Q�UPWdMDi^6j+^ceJW>�¯^>| e:�P\_^:WÙ��MD~<h3W>^l��\_h3~<�Ñ|]Q�U �DW>UPWYiJMD~/|]��\+�PQ�yEjØeJ�3W�W-�±WYfYe­\�y4� \_U

Page 36: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

b���/ Á¦ÂNÃ]Ä�Å�ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í�Î]Ï%ÐÑÄÈÒ�Ã]Î�ÏÓÐGÃ]ÎTÔsÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�Ï¢ MDU3�¿eJ��hP^V^6�P\_hP~���OXW�f8�P\_^6W>U®fdMDi6WYy�hP~�~�j¾e:\­i:WY�Pi:WY^:W>Uge;eJ�3W�e8M�i:�_W-eV�G\_~<Q�fYjhPUP�3W>i)MDU%MD~�j7^6Q<^Ym;§Ó\Di W-�3MD�¯�P~<W�|Se:�PW;�PQ<i6W>f-e)W-�±WYfYe)\Dy�Ma�PQ�~<~�\_U�e:�Pi:\D�VOX\D^:Q<^��\_hP~<�¿��\_^ce4~�Q<��W>~�jÙOGW£�3Q��XW>i:WYUge4y�\_i!�Pi:WY�_U%MDUgeVMDUP��UP\DUP�Pi:WY�_U%MDUgeV��\_�¯W>UÑm=�PQ<�3W>�¯Q<\_~�\_�_Q<^ceJ^�fdMD~�~P^:hPf8���PQ��XW>i6W>UPfYW>^)·6W-�±WYfYe���\+�PQ��PfdM�e:Q<\_UP¹�MDUP�¯Q<UP^6Q<^ceÈ\_U^:WY�%MDi:M�eJWY~�j°i6W>�G\_i6e:Q<UP�Ve:�PWAWY�XW>fYeTQ<UlW>MDf8��^6hPOP�G\_�PhP~<M�eJQ�\_UÑm

k�~�eJ�P\_h3�_�ãe:�PW��PQ<i6W>fYeaW-�±WYfYe�Q<^l^:WYUP^:Q�eJQ��DW�e:\àe:�PW�~<WY�DW>~�^[M�el�}�PQ�f8� ��W�P\_~��zeJ�PWN�%MDi6W>UgeJ^º\Dy±e:�PW}\_h3e:f>\_�¯W��DM�i:Q�M�OP~<W�|gQ�eÈQ<^=^6\_�¯WYeJQ���WY^=��W>MDUPQ�UP�Dy�hP~3e:\Md�_WYiJMD�_Wze:�PWz�PQ<i6W>f-eAWY�XW>fYeA\'�DW>i eJ�P\D^:Wz~<W-�_WY~<^>mV§Ó\Di]W-�3MD�¯�P~<W�|ÑQ�yÈ��WV�}Q<^6�Ùe:\MD^6^:W>^6^�eJ�3W4�PWY�_i:WYW4\Dy=�3Q<^:fYi:Q���Q�U%M�eJQ�\_U¡Q�U­M��DQ��_WYU[^6f8�P\7\D~F�}Q�eJ�P\_h7eTi6WYy�WYi:W>U3f>WeJ\;^6�XWYf>Q��Pf!�3W>�%MDiceJ�¯W>UgeJ^Y|%��W!fdMDUlfY\_���3h3eJW eJ�3W!�3Q��XW>i:WYUPf>W;�`/MD�P�¯Q<^6^:Q�\_UL?'·�£M�~<WUC!·�PWY�3e-o�¸ê;�`/MD�3��Q�^:^:Q�\_U@?¹·y�W>��MD~�WoCº·�PW>�3eÝo

MDUP�VM'�DW>i:MD�_W�e:�PQ<^F�PQ��±WYi:W>U3f>W�\'�DW>i=MD~�~g�PW>�PMDi6e:��WYU�e:^>m`���3Q<^¦Md�DW>iJM��_W��¯WdMD^6hPi:WY^eJ�3W£Q<U3f>i:W>MD^:W£Q<U�M��P��Q�^:^6Q<\_U�iJM�e:W£Q<U¿M[�gj+�X\De:�PWYe:Q<f>MD~�WÝ�7�XWYi:Q<�¯W>Uge4Q<U��}�3Q<f8���W�Q<UP^ceJi:h3fYeÈMD~�~+y�WY�£M�~<W�f>MDUP�PQ��%M�e:W>^�e:\ i:WYeJMDQ<UVe:�PW>Q�i��PWY�%MDiceJ�¯W>UgeÈ�Pi:W-y�W>i6W>UPfYW>^OPh3e)f8�%MDU3�_W¯e:�PW>Q�iA�_W>UP�3W>i!Q<�PWYU�e:Q��%f>M�eJQ�\_Uà`{\DU�eJ�3W�M��P�P~<Q�fdM�e:Q<\_U­y�\Di:�£oTy�i:\_�y�W>��MD~�W�eJ\;��MD~<W�m¤¥U°�_W>U3W>iJM�~0|7e:�PW Md�_WYiJMD�DW �PQ<i6W>fYe�WY�XW>fYe}f>MDU°OGW]�PWY�PUPW>�aMD^�M)^:WYe}\�yS�Pi:\_O+�MDOPQ�~<Q�eJQ<WY^

7»z¼¾½X¿ j ;�`>=@?Wù�uC,¤»(' £À¥T! o�;�` »(' £ ¥T! opC

\_UPW]y�\Di�WdMDf8�l~�WY�_WY~Ñ\DyS��m x W-�_WYiJMD~Ñ��MDi6Q�MDUgeJ^�\Dy�eJ�PQ�^��PWY�PUPQ�e:Q<\_U¡��Mdj£OGW hP^6W>��}�PWYU¯��M��XW>fYe:^È\DeJ�PWYi=�%MDi6W>UgeJ^º\DyX¢£m¦§%\_i=WÝ�3MD���3~<WD|_��W}�£Mdj)�}Q<^:�;eJ\!MD^6^:WY^:^eJ�3WAM'�DW>i:MD�_W f8�%MDU3�_W Q<UÇÈ£`/¢4o�Q<UP�3hPf>WY�aOgj£f8�%MDUP�_Q�UP�V��y�i:\D� �leJ\V� Î �}�3Q<~<W�DWYW>�PQ�UP�zeJ�PW \�eJ�PWYi��%MDi6W>UgeJ^}\DyF¢ÜfY\_UP^6eJMDUgeNM�e��}�%M2eJWY�DW>i���MD~<h3W�eJ�PW-j¡\_O7e8MDQ�UhPUP�3W>iN���3`E�Xo-mJ���PW!MD�P�Pi6\_�Pi6Q�M�e:W W-�7�Pi:WY^:^6Q<\_Uay�\_i�eJ�PQ�^�f8�%MDUP�DW)Q�^Á þ � þ��«`{¢Vo 6 7

»~¼ ½X¿ j� È£`/¢�? ���3`E� Î opC8���3` »(' £À¥T! o:o

¸sÈ¡`{¢±? ���7`��Xo)CJ�+�7` »z' £¦¥T! o:o � ;�` »(' £ ¥T! ? ���3`E�Xo:o ���PQ�^�WÝ�3�3i:W>^6^:Q�\_U°i:WY�Pi:WY^:W>Uge:^��}�%M�eº��W]MDfYe:h%MD~<~�j4�}Q�^:��e:\4�¯WdMD^6hPi:WTQ<U¯iJMDfYWT\Di^:WÝ�¡�3Q<^:fYi:Q���Q�U%M�eJQ�\_U;fdMD^6W>^Y|7�}�PWYi:W]��W]MDi6W�Q<U3^6eJi6hPfYe:W>�¡e:\VMD^6^:W>^6^�eJ�PW�WY�XW>fYe�\Dy\_UPW y{MDf-eJ\_i!`E��o��}�PQ�~<W �DWYW>�PQ�UP�¿·:MD~<~X\DeJ�3W>i�y{MDfYe:\_i:^}fY\_UP^6eJMDUgedm¼¹

Page 37: Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í Î]Ï%ÐÑÄÈÒ Ã]Î ÏÓÐGÃ]ÎTÔsÔ]Ì ...bayes.cs.ucla.edu/BOOK-99/ch4.pdf · b'ÀD¨ Á¦ÂNÃ]Ä Å ÆÈÇsÉ+ÊËÃ!ÁNÅFÌ-Í

É+Êt��ÊØÔ]̬ÇNÆNÁNÅ®ÆÈÕ¦Õ¦ÆNÁNÅ�ÏlÃ]Î�Ô Å�Â�Æ=̬Çå̬ԠÆÈÎ;ÅF̬Õ�Ì-Á�Ã�ÅFÌ-Í}Î b��Y�ð wàê ìfÄÖï>t]íÆÅ�Ç t ê°ëx WYfYeJQ�\_UP^�¨Pmp¶TMDUP�V¨3mn¨ M�i:WºO%MD^:WY�Ñ|gi:WY^:�GW>f-eJQ��DW>~�j_|_\_U4f>\_~�~�MDOG\_iJM2eJQ��DW¦��\Di:��^��}Q�e:�Z Md�+Q<�± !M�~<~<WY^�MDUP�±�_MD�¯W>^NL}\_OPQ�UP^>m