![Page 1: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/1.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
1nÙ £8ëê��O
Tianxiao Pang
Zhejiang University
September 28, 2016
Tianxiao Pang 1nÙ £8ëê��O
![Page 2: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/2.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 3: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/3.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 4: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/4.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 5: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/5.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 6: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/6.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 7: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/7.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 8: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/8.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 9: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/9.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 10: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/10.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 11: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/11.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
![Page 12: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/12.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�O£8ëê��Ä��{´���¦{(Least SquaresMethod). �Ùcn!?ØXÛA^���¦{¦£8ëê����¦�O, ¿ïÄù«�O�Ä�5�. 31oÚ1Ê!, ·�?Ø£8�.�Ä�b��·^5±9�ùb�Ø·^�, éêâAT��C�(Box-CoxC�). 318!·�ò?Ø2Â���¦�O. 31Ô!·�ò?Ø�«AÏ�gCþ, ¿?اXÛ����¦�O�5�³. 1l!Ú1Ê!·�ò?Øü«#��O�{: *�OṲ́©�O. ���!{ü0�SteinØ �O.
lùpm©, ·�b½gCþØ´�ÅCþ, Ï�§��� �±�<���. Ó�·��½: ���gCþ^��i1L«,���ÏCþ´�ÅCþ, �^��i1L«(k��L«��*ÿ�).
Tianxiao Pang 1nÙ £8ëê��O
![Page 13: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/13.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
���¦�O
^yL«ÏCþ, x1, · · · , xpL«(�U)éykK��p�gCþ. b�§��m÷vXe��5'Xª:
y = β0 + β1x1 + · · ·+ βpxp + e, (3.1.1)
Ù¥e´�ÅØ�, β0, · · · , βpL«�����ëê. ¡β0�£8~ê, ¡β1, · · · , βp�£8Xê. k�r§�Ú¡�£8Xê. b½®²k��
(xi1, · · · , xip, yi), i = 1, · · · , n,
K§�÷v
yi = β0 + β1xi1 + · · ·+ βpxip + ei, i = 1, · · · , n. (3.1.2)
b�Ø��ei, i = 1, · · · , n÷vGauss-Markovb�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 14: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/14.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
e^ÝL«, K���
Y =
y1
y2...yn
=
1 x11 · · · x1p
1 x21 · · · x2p...
......
...1 xn1 · · · xnp
β0
β1...βp
+
e1
e2...en
,
½��
Y = Xβ + e. (3.1.3)
Ù¥Y´n× 1��Å*ÿ�þ(ÏCþ�þ), X´n× (p+ 1)�®��OÝ(b�p+ 1 ≤ n), β´(p+ 1)× 1���ëê�þ, e´n× 1��ÅØ��þ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 15: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/15.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
òGauss-Markovb���¤Ý/ª:
E(e) = 0,Cov(e) = σ2In. (3.1.4)
ò(3.1.3)Ú(3.1.4)Ü�3�å, =����Ä���5£8�.:
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In. (3.1.5)
Tianxiao Pang 1nÙ £8ëê��O
![Page 16: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/16.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�^���¦�{Ïéβ��O, ù��OÏd�¡����¦�O(LSE: Least Squares Estimator). ù��{´Ïé��β��O, ¦� ��þe = Y −Xβ��Ý�²�‖Y −Xβ‖2����.
P
Q(β) = ‖Y −Xβ‖2 = (Y −Xβ)′(Y −Xβ)
= Y′Y − 2Y
′Xβ + β
′X
′Xβ.
éβ¦�, -Ù�u", ���§|
X′Xβ = X
′Y . (3.1.6)
¡ù��§|��5�§|(½�K�§|). ù��§|k��)�¿�^�´X
′X��´p+ 1, ù�duX��´p+ 1(=X
´�÷��). Ï�X ´�±<����, ¤±·�ob½X´�÷��. u´·���(3.1.6)���)
β = (X′X)−1X
′Y . (3.1.7)
Tianxiao Pang 1nÙ £8ëê��O
![Page 17: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/17.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
±þ�?Ø�U`²β´Q(β)���7:, ��7Ò´���:.eyβ(¢´Q(β)����:.
é?¿�β, k
‖Y −Xβ‖2 = ‖Y −Xβ +X(β − β)‖2
= ‖Y −Xβ‖2 + (β − β)′X
′X(β − β)
+2(β − β)′X
′(Y −Xβ). (3.1.8)
Ï�β÷v�5�§|(3.1.6), ¤±X′(Y −Xβ) = 0. ùy²
é?¿�β, k
‖Y −Xβ‖2 = ‖Y −Xβ‖2 + (β − β)′X
′X(β − β). (3.1.9)
duX′X´�K½Ý, ¤±(β − β)
′X
′X(β − β) ≥ 0. u´
‖Y −Xβ‖2 ≥ ‖Y −Xβ‖2.
�y.Tianxiao Pang 1nÙ £8ëê��O
![Page 18: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/18.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Pβ = (β0, β1, · · · , βp)′, ·��±��(²�)£8�§:
Y = Xβ ½ y = β0 + β1x1 + · · ·+ βpxp. (3.1.10)
ù��§´Ø´£ãy�x1, · · · , xp�ý¢'X, �I�?�Ú�ÚO©Û, 3�±�?n.
Tianxiao Pang 1nÙ £8ëê��O
![Page 19: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/19.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.1.1���5£8. b�gCþ�k��, P�x. ���(xi, yi), i = 1, · · · , n. u´k�5£8�.
yi = β0 + β1xi + ei, i = 1, · · · , n.
ù���5�§|�n
n∑i=1
xi
n∑i=1
xi
n∑i=1
x2i
(β0
β1
)=
n∑i=1
yi
n∑i=1
xiyi
,
��OÝX´�÷��, =xi, i = 1, · · · , nØ����,n∑i=1
(xi − x)2 6= 0, u´β0Úβ1�LSE�
β0 = y − β1x,
β1 =
∑ni=1(xi − x)(yi − y)∑n
i=1(xi − x)2=:
SxySxx
Tianxiao Pang 1nÙ £8ëê��O
![Page 20: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/20.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3£8©Û¥, ·�k�r�©êâ?1¥%zÚIOz. -
xj =1
n
n∑i=1
xij , j = 1, · · · , p.
ò(3.1.2)U��
yi = α+ β1(xi1 − x1) + · · ·+ βp(xip − xp) + ei,
i = 1, · · · , n. (3.1.11)
ùp, α = β0 + β1x1 + · · ·+ βpxp. ¡(3.1.11)�¥%z�.. P
Xc =
x11 − x1 x12 − x2 · · · x1p − xpx21 − x1 x22 − x2 · · · x2p − xp
......
......
xn1 − x1 xn2 − x2 · · · xnp − xp
. (3.1.12)
K(3.1.11)�U��
Y = α1n +Xcβ + e =(1n Xc
)(αβ
)+ e. (3.1.13)
Tianxiao Pang 1nÙ £8ëê��O
![Page 21: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/21.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ùp, β = (β1, · · · , βp)′. 5¿�
1′nXc = 0 (3.1.14)
Ïd�5�§|���(n 0
0 X′cXc
)(αβ
)=
(1
′nY
X′cY
), (3.1.15)
�d/�¤ {nα = 1
′nY ,
X′cXcβ = X
′cY
(3.1.16)
u´£8ëê�LSE�{α = y,
β = (X′cXc)
−1X′cY .
(3.1.17)
Tianxiao Pang 1nÙ £8ëê��O
![Page 22: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/22.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ïd, éu¥%z�5£8�.(3.1.11), £8~ê�LSEo´ÏCþ���þ�, £8Xêβ�LSE β = (X
′cXc)
−1X′cY�du
l�5£8�.Y = X′cβ + e¥O�β�LSE. 3¢SA^¥, O
�(X′cXc)
−1o´'O�(X ′X)−1��B�:, �·�o´AO'%£8Xê, ¤±¥%z´kÐ?�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 23: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/23.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.1.2���5£8(Y). ò~3.1.1¥����5£8�.?1¥%z, �
yi = α+ β1(xi − x) + ei, i = 1, · · · , n. (3.1.18)
dúª(3.1.17)�LSEα = y,
β1 =
∑ni=1(xi − x)yi∑ni=1(xi − x)2
=
∑ni=1(xi − x)(yi − y)∑n
i=1(xi − x)2.
(3.1.19)
Tianxiao Pang 1nÙ £8ëê��O
![Page 24: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/24.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
,, ·���±égCþ�IOz?n. P
s2j =
n∑i=1
(xij − xj)2, j = 1, · · · , p,
zij =xij − xjsj
, i = 1, · · · , n; j = 1, · · · , p.
-Z = (zij)n×p, §äk5�:
1′nZ = 0,
R = Z′Z = (rij),
Ù¥
rij =
∑nk=1(xki − xi)(xkj − xj)
sisj, i, j = 1, · · · , p (3.1.20)
�gCþxi�xj����'Xê. ¤±R´gCþ��'XêÝ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 25: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/25.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
égCþ?1IOz�Ð?: (1)�±^5©Û£8gCþ�m��''X; (2)IOz�Øþj�K�, Bué£8Xê�O��ÚO©Û.
²LIOz���5£8�.�
yi = α+xi1 − x1
s1β1 + · · ·+ xip − xp
spβp + ei, (3.1.21)
½�¤Ý/ª
Y = α1n +Zβ + e =(1n Z
)(αβ
)+ e.
��ëê�LSE�
α = y, β = (β1, · · · , βp)′
= (Z′Z)−1Z
′Y .
£8�§�
y = α+x1 − x1
s1β1 + · · ·+ xp − xp
spβp. (3.1.22)
Tianxiao Pang 1nÙ £8ëê��O
![Page 26: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/26.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.1.3��Á�Nì��ðøA9þ, ¦Ù�±ð§, eL¥,gCþxL«Nì±��íü �m�²þ§Ý(◦C), yL«ü �mS�Ñ��ðþ(L), �*ÿ25 ��mü . ã3.1.1´ùêâ�Ñ:ã, éù|êâ, A^¥%z�5£8�.(3.1.18),·���
y = 9.424, x = 52.6,
£8ëê�LSE�
α = y = 9.424, β1 = −0.0798.
¤±£8�§�
y = 9.424− 0.0798(x− 52.6),
½�¤y = 13.623− 0.0798x.
Tianxiao Pang 1nÙ £8ëê��O
![Page 27: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/27.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
L3.1.1: �ðêâ
SÒ y(L) x(◦C) SÒ y(L) x(◦C)1 10.98 35.3 14 9.57 39.12 11.13 29.7 15 10.94 46.83 12.51 30.8 16 9.58 48.54 8.40 58.8 17 10.09 59.35 9.27 61.4 18 8.11 706 8.73 71.3 19 6.83 707 6.36 74.4 20 8.88 74.58 8.5 76.7 21 7.68 72.19 7.82 70.7 22 8.47 58.1
10 9.14 57.5 23 8.86 44.611 8.24 46.4 24 10.36 33.412 12.19 28.9 25 11.08 28.613 11.88 28.1
Tianxiao Pang 1nÙ £8ëê��O
![Page 28: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/28.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ã3.1.1 Ñ:ã
Tianxiao Pang 1nÙ £8ëê��O
![Page 29: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/29.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
yx=read.table(”ex p33 data.txt”)y=yx[, 1]x=yx[, 2]mydata=data.frame(y,x)plot(y∼x)lm.sol=lm(y∼x,data=mydata)abline(lm.sol)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 30: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/30.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
����£8�§:
y = 13.623− 0.0798x.
Tianxiao Pang 1nÙ £8ëê��O
![Page 31: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/31.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
���¦�O�5�
���¦�O(LSE)äk�ûÐ�5�:
½n (3.2.1)
éu�5£8�.(3.1.5), LSE β = (X′X)−1X
′Yäke�5�:
(a) E(β) = β;(b) Cov(β) = σ2(X
′X)−1.
y²: (a)´�E(Y ) = Xβ, ¤±
E(β) = (X′X)−1X
′ · E(Y ) = β.
(b)Ï�Cov(Y ) = Cov(e) = σ2In, ¤±
Cov(β) = Cov((X′X)−1X
′Y )
= (X′X)−1X
′Cov(Y )X(X
′X)−1
= (X′X)−1X
′σ2InX(X
′X)−1 = σ2(X
′X)−1.
Tianxiao Pang 1nÙ £8ëê��O
![Page 32: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/32.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�c´p+ 1��~ê�þ, éu�5¼êc′β(ù´����ëê),
·�¡c′β�c
′β�LSE.
íØ (3.2.1)
(a) E(c′β) = c
′β;
(b) Cov(c′β) = σ2c
′(X
′X)−1c.
=é?¿��5¼êc′β, c
′β�c
′β�à �O, ���
σ2c′(X
′X)−1c. Ï�c
′β = c
′(X
′X)−1X
′Y�y1, · · · , yn��5
¼ê, ¤±c′β�c
′β����5à �O(�5�O��´*
ÿy1, · · · , yn��5¼ê). ·���±�EÑc′β�Ù§�5Ã
�O. ù�¤c′β��5à �Oa.
Tianxiao Pang 1nÙ £8ëê��O
![Page 33: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/33.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.2.2, Gauss-Markov)
éu�5£8�.(3.1.5), 3c′β�¤k�5à �O¥, ���
¦�Oc′β´��������5à �O(BLUE: best linear
unbiased estimator).
y²: �a′Y�c
′β����5à �O. u´é��p+ 1���
þβ,c′β = E(a
′Y ) = a
′Xβ,
Ïd
a′X = c
′. (3.2.1)
Ï�Var(a′Y ) = σ2a
′a = σ2‖a‖2, ·�é‖a‖2�©):
‖a‖2 = ‖a−X(X′X)−1c+X(X
′X)−1c‖2
= ‖a−X(X′X)−1c‖2 + ‖X(X
′X)−1c‖2
+2c′(X
′X)−1X
′(a−X(X
′X)−1c). (3.2.2)
Tianxiao Pang 1nÙ £8ëê��O
![Page 34: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/34.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
d(3.2.1)�í�
2c′(X
′X)−1X
′(a−X(X
′X)−1c) = 2c
′(X
′X)−1(X
′a−c′) = 0.
díØ3.2.1(b)�í�
σ2‖X(X′X)−1c‖2
= σ2c′(X
′X)−1X
′X(X
′X)−1c
= σ2c′(X
′X)−1c
= Var(c′β).
u´, ·�y²éc′β�?���5à �Oa
′Yk
Var(a′Y ) = Var(c
′β) + σ2‖a−X(X
′X)−1c‖2
≥ Var(c′β), (3.2.3)
�Ò¤á��=�‖a−X(X′X)−1c‖ = 0, =a = X(X
′X)−1c,
d�a′Y = c
′β. ½n�y.
Tianxiao Pang 1nÙ £8ëê��O
![Page 35: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/35.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3�5£8�.(3.1.5)¥�k��ëêσ2(Ø���), §�N�.Ø����, 3£8©Û¥åX���^. ·�y35�Oσ2.
e = Y −Xβ´Ø��þ, Ø�*ÿ. ^β�Oβ, ¡
e = Y −Xβ = Y − Y (3.2.4)
�í�(residual)�þ. Px′i��OÝX�1i1, K
ei = yi − x′iβ, i = 1, · · · , n (3.2.5)
�1ig*ÿ�í�. g,/, ·��±òew�e����O. ·�ò^
RSS = e′e =
n∑i=1
e2i (3.2.6)
5�Eσ2�à �Oþ.Tianxiao Pang 1nÙ £8ëê��O
![Page 36: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/36.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
RSS=Residual Sum of Squares, L«í�²�Ú. §�N¢Sêâ�nØ�.(3.1.5)� l§Ý½ö`[ܧÝ. RSS��L«êâ��.[Ü��Ð.
½n (3.2.3)
(a) RSS = Y′(In −X(X
′X)−1X
′)Y =: Y
′(In −H)Y ;
(b) σ2 = RSS/(n− p− 1)´σ2�à �Oþ.
5: ¡H = X(X′X)−1X
′�lf(hat)Ý, §´��é¡��
Ý.
Tianxiao Pang 1nÙ £8ëê��O
![Page 37: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/37.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
y²: (a)
RSS = e′e = (Y −Xβ)
′(Y −Xβ)
= [(In −X(X′X)−1X
′)Y ]
′[(In −X(X
′X)−1X
′)Y ]
= Y′(In −X(X
′X)−1X
′)Y .
(b)dE(Y ) = Xβ,Cov(Y ) = σ2In±9½n2.2.1�
E(RSS) = E[Y′(In −X(X
′X)−1X
′)Y ]
= β′X
′(In −X(X
′X)−1X
′)Xβ
+σ2tr(In −X(X′X)−1X
′)
= σ2[n− tr(X(X′X)−1X
′)].
�â,�5�tr(AB) = tr(BA)�
tr(X(X′X)−1X
′) = tr((X
′X)−1X
′X) = tr(Ip+1) = p+ 1.
u´E(RSS) = σ2(n− p− 1).
Tianxiao Pang 1nÙ £8ëê��O
![Page 38: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/38.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
XJb�Ø��þeÑl��©Ù, =e ∼ N(0, σ2In), @o·��±��βÚσ2�õ��5�.
½n (3.2.4)
éu�5£8�.(3.1.5), eØ��þe ∼ N(0, σ2In), K(a) β ∼ N(β, σ2(X
′X)−1);
(b) RSS/σ2 ∼ χ2(n− p− 1);(c) β�RSS�pÕá.
y²: (a) 5¿�Y ∼ N(Xβ, σ2In)±9β = (X′X)−1X
′Y´Y
��5C�, @od½n2.3.4B�í�(a).
Tianxiao Pang 1nÙ £8ëê��O
![Page 39: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/39.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(b) �â½Â
RSS = Y′(In −H)Y = Y
′(In −H)Y = Y
′NY ,
ùpN = In −H, §´��é¡��Ý. 5¿�NX = 0, ¤±
RSS = (Xβ + e)′N(Xβ + e) = e
′Ne. (3.2.7)
5¿�e ∼ N(0, σ2In), ¤±�â½n2.4.3·��Iy²N���n− p− 1. |^��Ý���u§�,ù�5�, ·���
rk(N) = tr(In −X(X′X)−1X
′)
= n− tr(X(X′X)−1X
′)
= n− tr((X′X)−1X
′X)
= n− p− 1.
Tianxiao Pang 1nÙ £8ëê��O
![Page 40: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/40.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(c) Ï�β = β + (X′X)−1X
′e, RSS = e
′Ne, 5¿�
(X′X)−1X
′N = 0,
¤±d½n2.4.5��(X′X)−1X
′e�RSS�pÕá, =β�RSS�
pÕá.
Tianxiao Pang 1nÙ £8ëê��O
![Page 41: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/41.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�β�1��©þ�~ê�β0�, �c = (0, · · · , 0, 1, 0, · · · , 0)′, Ù
¥1 3c�1i+ 1� �, Kc′β = βi. 2Pβ = (β0, · · · , βp)
′, u
´c′β = βi. 2^(A)iiL«ÝA�1(i, i)��, @o·Xeí
Ø:
íØ (3.2.2)
éu�5£8�.(3.1.5), ee ∼ N(0, σ2In), K(a) βi ∼ N(βi, σ
2[(X′X)−1]i+1,i+1);
(b)3βi, i = 1, · · · , p����5à �O¥, βi, i = 1, · · · , p´������ö.
Tianxiao Pang 1nÙ £8ëê��O
![Page 42: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/42.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ò½n3.2.1Ú½n3.2.4A^u¥%z�.(3.1.13), Kk
íØ (3.2.3)
éu¥%z�.(3.1.13), 5¿ùp�β = (β1, · · · , βp)′, k
(a) E(α) = α,E(β) = β, ùpα = y, β = (X′cXc)
−1X′cY ;
(b)
Cov
(α
β
)= σ2
(1n 0
0 (X′cXc)
−1
);
(c)e?�Úb�e ∼ N(0, σ2In), K
α ∼ N(α,σ2
n), β ∼ N(β, σ2(X
′cXc)
−1),
�α�β�pÕá.
Tianxiao Pang 1nÙ £8ëê��O
![Page 43: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/43.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â
R2 =ESS
TSS, (3.2.8)
Ù¥
ESS =
n∑i=1
(y − yi)2 = (Y − y1n)′(Y − y1n)
¡�£8²�Ú(½)º²�Ú: Explained Sum of Squares),
TSS =
n∑i=1
(yi − y)2 = (Y − y1n)′(Y − y1n)
¡�o �²�Ú(½¡�o²�Ú: Total Sum of Squares).¡R2��½Xê½ÿ½Xê, ¡R =
√R2�E�'Xê.
Tianxiao Pang 1nÙ £8ëê��O
![Page 44: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/44.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
d�5�§|�±y²
n∑i=1
(yi − y)2
=
n∑i=1
(yi − yi + yi − y)2
=
n∑i=1
(yi − yi)2 +
n∑i=1
(yi − y)2 + 2
n∑i=1
(yi − yi)(yi − y)
=n∑i=1
(yi − yi)2 +
n∑i=1
(yi − y)2.
=TSS = ESS + RSS.
ÏdR2Ýþ£8gCþx1, · · · , xpéÏCþy�[ܧÝ�Ð�. 0 ≤ R2 ≤ 1, §����, L²y�ÃgCþk�����'X.
Tianxiao Pang 1nÙ £8ëê��O
![Page 45: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/45.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
e¦^¥%z�.(3.1.13), @oESS�ÏLe�úªO�:
ESS = β′X
′cY = Y
′Xc(X
′cXc)
−1X′cY ,
ùpβ = (β1, · · · , βp)′. ¯¢þ, dY = α1n +Xcβ9úª
(3.1.17)��Y − y1n = Y − α1n = Xcβ.
¤±
ESS = (Y − y1n)′(Y − y1n) = (Xcβ)
′Xcβ
= β′X
′c ·Xc(X
′cXc)
−1X′cY
= β′X
′cY .
Tianxiao Pang 1nÙ £8ëê��O
![Page 46: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/46.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éu���5£8�.
yi = β0 + β1xi + ei, i = 1, · · · , n.
�±y²R2 = r2, Ù¥r��.¥gCþ�ÏCþ����'Xê.
¯¢þ, e·�r�.¥%z:
yi = α+ β1(xi − x) + ei, i = 1, · · · , n,
@o��
β1 =
∑ni=1(xi − x)yi∑ni=1(xi − x)2
,
Tianxiao Pang 1nÙ £8ëê��O
![Page 47: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/47.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
l
ESS = β1
n∑i=1
(xi − x)yi =
[∑ni=1(xi − x)yi
]2∑ni=1(xi − x)2
=
[∑ni=1(xi − x)(yi − y)
]2∑ni=1(xi − x)2
.
¤±
R2 =ESS
TSS=
[∑ni=1(xi − x)(yi − y)
]2∑ni=1(xi − x)2
∑ni=1(yi − y)2
= r2.
Tianxiao Pang 1nÙ £8ëê��O
![Page 48: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/48.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.2.1�â²��, 3<��p���^�e, ÙÉØ� Øy�Nx1!c#x2k'. yÂ813¶If�ÿþêâ, �eL. Áïáy'ux1, x2��5£8�§.
L3.2.1: ÉØêâ
SÒ xi1 xi2 yi SÒ xi1 xi2 yi1 152 50 120 8 158 50 1252 183 20 141 9 170 40 1323 171 20 124 10 153 55 1234 165 30 126 11 164 40 1325 158 30 117 12 190 40 1556 161 50 125 13 185 20 1477 149 60 123
Tianxiao Pang 1nÙ £8ëê��O
![Page 49: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/49.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
|^¥%z�.
yi = α+ β1(xi1 − x1) + β2(xi2 − x2) + ei, i = 1, · · · , 13.
²O���
x1 =1
13
13∑i=1
xi1 = 166.8, x2 =1
13
13∑i=1
xi2 = 38.85, y =1
13
13∑i=1
yi = 130,
Xc =
−14.08 11.1516.92 −18.854.92 −18.85−1.08 −8.85−8.08 −8.85−5.08 11.15−17.08 21.15−8.08 11.153.92 1.15−13.08 16.15−2.08 1.1523.92 1.1518.92 −18.85
,
Tianxiao Pang 1nÙ £8ëê��O
![Page 50: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/50.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�5�§|X′cXcβ = X
′cY�{
2078.92β1 − 1533.85β2 = 1607,−1533.85β1 + 2307.69β2 = −715.
)�β1 = 1.068, β2 = 0.4, α = y = 130. ¤±£8�§�
y = α+ β1(x1 − x1) + β2(x2 − x2)
= 130 + 1.068× (x1 − 166.8) + 0.4× (x2 − 38.85)
= −62.963 + 1.068x1 + 0.4x2.
d, ����
ESS = β′X
′cY = 1430.276, TSS = 1512,
¤±R2 = 1430.276/1512 = 0.9459.
Tianxiao Pang 1nÙ £8ëê��O
![Page 51: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/51.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
yx=read.table(”ex p39 data.txt”)x1=yx[, 1]x2=yx[, 2]y=yx[, 3]mydata=data.frame(y,x1,x2)lm.sol=lm(y∼x1+x2,data=mydata)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 52: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/52.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
dR$1(J�: y = −62.963 + 1.068x1 + 0.4x2, R2 = 0.9461.
Tianxiao Pang 1nÙ £8ëê��O
![Page 53: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/53.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�å���¦�O
éβØ\?Û�å^���/e, ·�?ا�LSE±9§�Ä�5�. �3�AÏ|Ü, ~Xb�u�¯K, ·�I�¦�k�½�å^��LSE.
b�
Aβ = b (3.3.1)
´���N�5�§|(=�§|k)), Ù¥A´k × (p+ 1)®�Ý, ��k, b´k × 1®��þ. ·�^Lagrange¦f{¦�5£8�.(3.1.5)3÷v�5�å(3.3.1)��LSE.=3(3.3.1)ù�^�e¦β¦Q(β) = ‖Y −Xβ‖2����.
Tianxiao Pang 1nÙ £8ëê��O
![Page 54: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/54.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�E9ϼê
F (β,λ) = ‖Y −Xβ‖2 + 2λ′(Aβ − b)
= (Y −Xβ)′(Y −Xβ) + 2λ
′(Aβ − b),
Ù¥λ = (λ1, · · · , λk)′�Lagrange¦f�þ. é¼êF (β,λ)'
uβ¦�¿-§�u0, �
−X ′Y +X
′Xβ +A
′λ = 0. (3.3.2)
y3, ·�I�)d(3.3.1)Ú(3.3.2)|¤�éá�§|.
·�^βcÚλcL«ù��§|�). d(3.3.2)�
Tianxiao Pang 1nÙ £8ëê��O
![Page 55: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/55.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
βc = (X′X)−1X
′Y − (X
′X)−1A
′λc
= β − (X′X)−1A
′λc. (3.3.3)
�\(3.3.1)�
b = Aβc = Aβ −A(X′X)−1A
′λc,
��d/�¤
A(X′X)−1A
′λc = Aβ − b. (3.3.4)
duA���k, ¤±A(X′X)−1A
′´k × k�_Ý, Ïd(3.3.4)
k��)λc = (A(X
′X)−1A
′)−1(Aβ − b),
2ò§�\(3.3.3)�
βc = β − (X′X)−1A
′(A(X
′X)−1A
′)−1(Aβ − b). (3.3.5)
Tianxiao Pang 1nÙ £8ëê��O
![Page 56: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/56.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
eyβc(¢´�5�åAβ = beβ�LSE. ù�Iy²(a) Aβc = b;(b)é��÷vAβ = b�β, Ñk‖Y −Xβ‖2 ≥ ‖Y −Xβc‖2.
d(3.3.5)�í�(a). �y(b), ò‖Y −Xβ‖2©)�
‖Y −Xβ‖2
= ‖Y −Xβ‖2 + ‖X(β − β)‖2
= ‖Y −Xβ‖2 + ‖X(β − βc + βc − β)‖2
= ‖Y −Xβ‖2 + ‖X(β − βc)‖2 + ‖X(βc − β)‖2,(3.3.6)
ùp�í�^�eã'X: (Y −Xβ)′X = 0±9é��÷
vAβ = b�β,
(β − βc)′X
′X(βc − β) = λ
′cA(βc − β) = λ
′c(Aβc −Aβ) = 0.
Tianxiao Pang 1nÙ £8ëê��O
![Page 57: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/57.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(3.3.6)L²: é��÷vAβ = b�β, ok
‖Y −Xβ‖2 ≥ ‖Y −Xβ‖2 + ‖X(β − βc)‖2, (3.3.7)
�Ò¤á��=�X(βc − β) = 0, =β = βc(Ï�X�÷�). Ïd3(3.3.7)¥^βc�Oβ, �Ò¤á. =
‖Y −Xβc‖2 = ‖Y −Xβ‖2 + ‖X(β − βc)‖2. (3.3.8)
y3, (Ü(3.3.7)Ú(3.3.8)B�í�(b).
Tianxiao Pang 1nÙ £8ëê��O
![Page 58: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/58.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�rβc¡�β��å���¦�O, u´ke��½n:
½n (3.3.1)
éu�5£8�.(3.1.5), ÷v(3.3.1)��åLSE�
βc = β − (X′X)−1A
′(A(X
′X)−1A
′)−1(Aβ − b),
Ù¥β = (X′X)−1X
′Y´Ã�å^�e�LSE.
Tianxiao Pang 1nÙ £8ëê��O
![Page 59: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/59.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.3.13U©ÿþ¥, éU�¥�n�( :�¤�n�/ABC�n�S�θ1, θ2, θ3?1ÿþ, ���y1, y2, y3, du�3ÿþØ�, ¤±I�éθ1, θ2, θ3?1�O, ·�|^�5�.L«k'�þ
y1 = θ1 + e1,y2 = θ2 + e2,y3 = θ3 + e3,θ1 + θ2 + θ3 = π.
�¤Ý/ª {Y = Xβ + e,Aβ = b,
ùpY = (y1, y2, y3)′, β = (θ1, θ2, θ3)
′, X = I3, A = (1, 1, 1),
b = π. 5¿�β = (X′X)−1X
′Y = Y , A^½n3.3.1, ²O��
Tianxiao Pang 1nÙ £8ëê��O
![Page 60: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/60.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
βc =
y1
y2
y3
− 1
3(
3∑i=1
yi − π)
111
,
=
θi = yi −1
3(
3∑i=1
yi − π), i = 1, 2, 3
�θi��åLSE.
Tianxiao Pang 1nÙ £8ëê��O
![Page 61: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/61.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
£8�ä
éu�5£8�.(3.1.5):
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In,
·�3?Ø��ëê����¦�O±9�Oþ�©Ù�, �Xe��Ä��b�:
(a)�5b�: ÏCþ�gCþäk�5�''X;(b)��à5b�: Var(ei) = σ2, i = 1, · · · , n;(c)Ø�'5b�: Cov(ei, ej) = 0, i 6= j.(d)��5b�: ei ∼ N(0, σ2), i = 1, · · · , n.
Tianxiao Pang 1nÙ £8ëê��O
![Page 62: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/62.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
XJùb�ؤá, @o·�±c?Ø����¦�O±9§�ÚO5�Òk�U´Ø¤á�. Ïd, 3¢S¯K¥, �·�k�1êâ�, I��·��êâ´Ä÷v½öÄ�÷vùb�. ù´�.�ä�SN.
Ï�ùb�Ñ��ÅØ�ek', í��þe�w¤´e����O, Ïd·��±ÏLí�5©Û4�Ä�b�´Ä¤á. �Ï�ù��Ï, ùÜ©SN��¡�í�©Û.
Tianxiao Pang 1nÙ £8ëê��O
![Page 63: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/63.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ØI�é�.?1�ä, ·��I�éêâ��?1�ä, §�¹É~:�äÚrK�:�ä.
3£8©Û¥, ¤¢É~:(q�l+:)´�éQ½�. lé��êâ:. �§e��O(�½Â´��(J�, �8�vkÚ��½Â. 8c, éÉ~:��61�w{´: (1)É~:´�@�ý�õêêâ:²wØ�N�êâ:. ù�É~:�n)�¤b½©Ù¥�4à:;(2)É~:Ò´@À/:, =´��ý�õêêâ:Ø´5gÓ�©Ù��Oêâ:.
É~:�·\òéëê��OE¤K�, Ïd·�I�uÿÑÉ~:¿ò§íØ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 64: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/64.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
êâ8¥�rK�:´�éÚOíä(ëê�O!b�u��)�)��K��êâ:, 3d�§¥, A�é£8ëêβ����¦�Ok��K��êâ:.
éz�|êâ:(x′i, yi), ·�F"§é£8ëê��Ok�½�
K�, �qF"ù«K�ØU��. ù�, ·����²�£8�§Òk�½�½5. ÄK, XJ�O�ü|êâé�OkÉ~��K�, �·�GØùêâ��, ÒU����5�Éé��²�£8�§.
Ïd3£8©Û¥, ·�I��z|êâéëê�O�K���. ùÜ©SN�¡�K�©Û.
uÿÑrK�:�, ·�I�Ø�ùêâ´Ä�~, eØ�~KíØ�, ÄK�ÄÂ8�õ�êâ½æ^�è�O�{± �/DºrK�:é�O�K�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 65: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/65.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
É~:ÚrK�:´ü�ØÓ�Vg. l�¡�©Û�±wÑ,§��mQk�½�éX�k�½�«O.
rK�:�UÓ�´É~:��UØ´, ��, É~:�UÓ�q´rK�:��UØ´.
Tianxiao Pang 1nÙ £8ëê��O
![Page 66: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/66.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ïd, £8�ä�¹êâ��ä��.��äùüÜ©SN. Ù¥êâ��ä�)É~:�äÚrK�:�ä, �.��ä�¹�5�ä!��à5�ä!Ø�'5�äÚ��5�ä4 Ü©SN.
Tianxiao Pang 1nÙ £8ëê��O
![Page 67: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/67.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
k5?Øí�©Û:
éu�5£8�.:
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In, (3.4.1)
·�^
ei = yi − x′iβ, i = 1, · · · , n (3.4.2)
L«1ig*ÿ�í�, ¿r§w¤´ei����O. e�.(3.4.1)�(, @oeiAäkei��5G.
PY = Xβ, ¡Y�[Ü��þ, ¡yi = x′iβ�1i�[Ü�. ´�
Y = X(X′X)−1X
′Y = HY , (3.4.3)
Ù¥H = X(X′X)−1X
′�¡�lfÝ, §´é¡��Ý.
Tianxiao Pang 1nÙ £8ëê��O
![Page 68: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/68.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
í�e��L«�
e = Y − Y = (I −H)Y = (I −H)e. (3.4.4)
I −H�´��é¡��Ý.
½n (3.4.1)
e(3.4.1)¤á, K(a) E(e) = 0, Cov(e) = σ2(I −H);(b) Cov(Y , e) = 0.(c)e?�Úb�e ∼ N(0, σ2I), K
e ∼ N(0, σ2(I −H)).
y²: N´, Ñ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 69: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/69.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�Var(ei) = σ2(1− hii)(hiiL«ÝH�1i�é����), �à5, ùkNuei�¢SA^. Ïd·��Ĥ¢�Æ)zí�
ri =ei
σ√
1− hii, i = 1, · · · , n, (3.4.5)
ùpσ2 = RSS/(n− p− 1) = e′e/(n− p− 1).
=¦3e ∼ N(0, σ2I)�^�e, ri�©ÙE,'�E,, ��Cq/@�ri�pÕá�ÑlN(0, 1)(��FW(1987)). d½n3.4.1��{ri, i ≥ 1}�{yi, i ≥ 1}Õá({ei, i ≥ 1}�{yi, i ≥ 1}�Õá).
Tianxiao Pang 1nÙ £8ëê��O
![Page 70: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/70.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
í�ã´±,«í�(Æ)zí�ri½ÊÏí�ei)�p�I, ±?ÛÙ§�þ�î�I�Ñ:ã. c®�Ñí���Ø�ei��OAT�ei��Ø�, ��âí�ã5G´Ä�Ak�5����,Ò�±é�.b��Ün5Jø�k^�&E.
e¡·�±[Ü�yi�î�I, Æ)zí�ri�p�I�í�ã�~?Øí�ã�äNA^. ���J�´, Ï~�¹e, ±ÊÏí�ei�p�IÚ±Æ)zí�ri�p�I�í�ã/G���Ó, ±,�gCþxj�î�I½ö±SÒi�î�IÚ±[Ü�yi�î�I�í�ã/G����Ó.
Tianxiao Pang 1nÙ £8ëê��O
![Page 71: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/71.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�5�ä : e�5b�¤á, @oeiØ�¹5ggCþ�?Û&E, Ïdí�ãØA¥y?Ûk5K�/G, ÄKknd~¦�5b�ؤá.
Tianxiao Pang 1nÙ £8ëê��O
![Page 72: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/72.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��à5�ä : e��à5, @oí�ãþ�:´/þ!0ÑÙ�, ÄK, í�ãÏ~¬¥y//i.0½/�/i.0½üöo k��/G.
Tianxiao Pang 1nÙ £8ëê��O
![Page 73: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/73.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ø�'5�ä : eÕá5¤á, @oí�ãþ�:Ø¥y5K5, ÄK, Ñ:ãò¥y”8ì5”½/ì���50�/G.
Tianxiao Pang 1nÙ £8ëê��O
![Page 74: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/74.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��5�ä : e��5¤á, @oÆ)zí�ri�Cqw¤´�pÕá�ÑlN(0, 1). ¤±, 3±ri�p�Iyi�î�I�í�ãþ, ²¡þ�:(yi, ri), i = 1, · · · , n ��Aá3°Ý�4�Y²�|ri| ≤ 2«�S(ù�ªÇA395%�m), �Ø¥y?Ûª³.
·���±^Æ)zí��QQ ã5���5�ä. �|Nþ�n�êâ'u,�©ÙF (x)�QQãÒ´±êâ�i/n© ê�p�I, ±F (x)�i/n© ê�î�I�Ñ:ã.
XJêâ´5gF (x)���{ü�Å��, KùÑ:A��3�^��þ,ÏdÏL�äQQã´ÄCq3�^��þ���¤��êâ8´ÄÑl¤�½©Ù��ÚO�â.
d, ·���±Shapiro-Wilk�{5���5u�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 75: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/75.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��5�ä : e��5¤á, @oÆ)zí�ri�Cqw¤´�pÕá�ÑlN(0, 1). ¤±, 3±ri�p�Iyi�î�I�í�ãþ, ²¡þ�:(yi, ri), i = 1, · · · , n ��Aá3°Ý�4�Y²�|ri| ≤ 2«�S(ù�ªÇA395%�m), �Ø¥y?Ûª³.
·���±^Æ)zí��QQ ã5���5�ä. �|Nþ�n�êâ'u,�©ÙF (x)�QQãÒ´±êâ�i/n© ê�p�I, ±F (x)�i/n© ê�î�I�Ñ:ã.XJêâ´5gF (x)���{ü�Å��, KùÑ:A��3�^��þ,ÏdÏL�äQQã´ÄCq3�^��þ���¤��êâ8´ÄÑl¤�½©Ù��ÚO�â.
d, ·���±Shapiro-Wilk�{5���5u�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 76: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/76.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−2
−1
01
2
y
r
©Û: Ù5G���5b�Ä���, Ïd�@���5b�e ∼ N(0, σ2I)´Ün�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 77: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/77.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−1.
5−
1.0
−0.
50.
00.
51.
0
y
r
©Û: í�ã¥y/i., Ïd@���à5b�ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
![Page 78: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/78.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−6
−4
−2
02
46
y
r
©Û: í�ã¥y�/i., Ïd@���à5b�ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
![Page 79: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/79.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−0.
15−
0.10
−0.
050.
000.
050.
10
y
r
©Û: ã/¥yk5K�/G, Ïd@��5b�ØÜn, í�¥A¹kgCþ�&E.
Tianxiao Pang 1nÙ £8ëê��O
![Page 80: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/80.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−3
−2
−1
01
23
y
r
©Û: í�ã¥y8ì5, Ïd@�Ø�'5ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
![Page 81: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/81.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−1.
0−
0.5
0.0
0.5
1.0
y
r
©Û: í�ã¥yì���5, Ïd@�Ø�'5ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
![Page 82: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/82.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
lí�ã�äÑ5�U�/;¾0, �Ò´,b�^�ؤá,·�I�é¯K/éwe�0:
XJkwG¦·�~¦�5b�ؤá, @o·��±�Ä3£8gCþ¥O\,gCþ��g�, Xx2
1, x22½���x1x2�,
äNO\=gCþ�=�, IÀ¢S�J ½;
XJkwG¦·�~¦��à5b�ؤá, @o�±�ÄéÏCþ�·��C�¦#CþäkCq�����(��½5C�), ½öæ^2Â���¦�O;
XJkwG¦·�~¦Ø�'5b�ؤá, @o�±�ÄéÏCþ�/�©0, ¦#CþäkCqÕá5;
XJkwG¦·�~¦��5b�ؤá, @·��±�ÄéÏCþ���5C�–Box-CoxC�. ���J�´, Box-CoxC�´�«nÜ£n�Y, e�!2�[0�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 83: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/83.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��½5C� :
��ÅCþY�þ��µ, ���σ2, b����þ��¼ê, =b�σ2 = g(µ), Ù¥g´��®��¼ê. ~X, eY ∼ B(n, p),KE(Y ) = np, Var(Y ) = np(1− p) = µ(1− µ/n). yI�{���C�Z = f(Y ), ¦�Var(Z)�~ê. ùI�éѼêf�L�ª. (ùp, ·�^��i1YÚZL«�ÅCþ, ��i1yÚzL«��Å�Cþ)
Pz = f(y), ¿-§3y = µ?TaylorÐm, �Cqª
f(y) = f(µ) + f′(µ)(y − µ).
òyU¤�ÅCþ, k
Z = f(Y ) = f(µ) + f′(µ)(Y − µ),
Ù��c = Var(Z) = [f′(µ)]2Var(Y ) = [f
′(µ)]2g(µ). �f
′(µ) =√
c/g(µ), K��
f(y) =
∫ √c/g(y)dy.
Tianxiao Pang 1nÙ £8ëê��O
![Page 84: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/84.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
e¡´A�A~:
(1) g(µ)�µ¤�'�, Pg(µ) = aµ, K∫ √c
aydy = 2
√c
ay + c
′,
Ñ�~êØO, ��²��C�: Z =√Y .
(2)�g(µ)�µ2¤�'�, Pg(µ) = aµ2, �y > 0, K∫ √c
ay2dy =
√c
aln y + c
′.
Ñ�~êØO, ��éêC�: Z = lnY .
(3)�g(µ)�µ4¤�'�, Pg(µ) = aµ4, K∫ √c
ay4dy = −
√c
a
1
y+ c
′.
Ñ�~êØO, ���êC�: Z = 1/Y .Tianxiao Pang 1nÙ £8ëê��O
![Page 85: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/85.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3A^þ, Äklí�ãoÑ/��eσ2�µ�U�3�A«'X(=�O¼êg(·)), ,�lúª
f(y) =
∫ √c/g(y)dy
¦ÑéA�C�. éA«C�L�êâ©O����¦?n, �#�í�ã, w=�«C��í�ãÃ���à5��î. l¥ÀÑ�Ð���½5C�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 86: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/86.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.4.1�ïÄ^>p¸z���^>þy�z�o^>þx�'X, yÂ8,�53rêâ.
L3.4.1: ^>þêâ
i x y yi ei z =√y zi ei
1 679 0.790 1.669 -0.879 0.889 1.229 -0.3402 292 0.440 0.244 0.196 0.663 0.860 -0.1973 1012 0.560 2.896 -2.336 0.748 1.547 -0.7984 493 0.790 0.984 -0.194 0.889 1.052 -0.1635 582 2.700 1.312 1.388 1.643 1.137 0.5066 1156 3.640 3.426 0.214 1.908 1.684 0.2247 997 4.730 2.840 1.890 2.175 1.532 0.6438 2189 9.500 7.230 2.270 3.082 2.668 0.4149 1097 5.340 3.209 2.131 2.311 1.628 0.68310 2078 6.850 6.822 0.028 2.617 2.562 0.05511 1818 5.840 5.864 -0.024 2.417 2.315 0.10212 1700 5.210 5.430 -0.220 2.283 2.202 0.08013 747 3.250 1.920 1.330 1.803 1.294 0.50914 2030 4.430 6.645 -2.215 2.105 2.517 -0.412
Tianxiao Pang 1nÙ £8ëê��O
![Page 87: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/87.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
i x y yi ei z =√y zi ei
15 1643 3.160 5.220 -2.060 1.778 2.148 -0.37016 414 0.500 0.693 -0.193 0.707 0.977 -0.27017 354 0.170 0.472 -0.302 0.412 0.920 -0.50718 1276 1.880 3.868 -1.988 1.371 1.798 -0.42719 745 0.770 1.912 -1.142 0.877 1.292 -0.41520 435 1.390 0.771 0.619 1.179 0.997 0.18221 540 0.560 1.157 -0.597 0.748 1.097 -0.34822 874 1.560 2.388 -0.828 1.249 1.415 -0.16623 1543 5.280 4.851 0.429 2.298 2.052 0.24524 1029 0.640 2.958 -2.318 0.800 1.563 -0.76325 710 4.000 1.784 2.216 2.000 1.259 0.74126 1434 0.310 4.450 -4.140 0.557 1.949 -1.39227 837 4.200 2.251 1.949 2.049 1.380 0.67028 1748 4.880 5.606 -0.726 2.209 2.248 0.03929 1381 3.480 4.255 -0.775 1.865 1.898 -0.03330 1428 7.580 4.428 3.152 2.753 1.943 0.81031 1255 2.630 3.791 -1.161 1.622 1.778 -0.15632 1777 4.990 5.713 -0.723 2.234 2.275 -0.04233 370 0.590 0.531 0.059 0.768 0.935 -0.16734 2316 8.190 7.698 0.492 2.862 2.789 0.073
Tianxiao Pang 1nÙ £8ëê��O
![Page 88: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/88.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
i x y yi ei z =√y zi ei
35 1130 4.790 3.330 1.460 2.189 1.659 0.53036 463 0.510 0.874 -0.364 0.714 1.023 -0.30937 770 1.740 2.004 -0.264 1.319 1.316 0.00338 724 4.100 1.835 2.265 2.025 1.272 0.75339 808 3.940 2.144 1.796 1.985 1.352 0.63340 790 0.960 2.078 -1.118 0.980 1.335 -0.35541 783 3.290 2.052 1.238 1.814 1.328 0.48642 406 0.440 0.664 -0.224 0.663 0.969 -0.30643 1242 3.240 3.743 -0.503 1.800 1.766 0.03444 658 2.140 1.592 0.548 1.463 1.209 0.25445 1746 5.710 5.599 0.111 2.390 2.246 0.14446 468 0.640 0.892 -0.252 0.800 1.028 -0.22847 1114 1.900 3.271 -1.371 1.378 1.644 -0.26548 413 0.510 0.690 -0.180 0.714 0.976 -0.26249 1787 8.330 5.750 2.580 2.886 2.285 0.60150 3560 14.940 12.280 2.660 3.865 3.974 -0.10951 1495 5.110 4.675 0.435 2.261 2.007 0.25452 2221 3.850 7.348 -3.498 1.962 2.699 -0.73653 1526 3.930 4.789 -0.859 1.982 2.036 -0.054
Tianxiao Pang 1nÙ £8ëê��O
![Page 89: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/89.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A^���¦{, �£8�§
y = −0.83130 + 0.00368x.
R§S:
yx=read.table(”ex p47 data.txt”)x=yx[, 1]y=yx[, 2]mydata=data.frame(y,x)lm.sol=lm(y∼x,data=mydata)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 90: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/90.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 91: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/91.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�í�©Û:
y.fit=predict(lm.sol)e.hat=y-y.fit /*½öe.hat=residuals(lm.sol)*/e.std=rstandard(lm.sol)plot(e.hat∼y.fit)plot(e.std∼y.fit)plot(e.hat∼x)
ÊÏí�ã��±ÏLXe�ª��:
plot(lm.sol,which=1)
Tianxiao Pang 1nÙ £8ëê��O
![Page 92: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/92.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 93: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/93.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 94: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/94.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 95: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/95.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 96: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/96.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
lí�ã�wÑ, ù´��/i.í�ã, ´��à5Ø�ÎÜ���wG. �ÄéÏCþy�C�, }Áz =
√y, �£8�§
z = 0.5822 + 0.000953x.
R§S:
z=sqrt(y)mydata2=data.frame(z,y,x)lm.sol2=lm(z∼x,data=mydata2)summary(lm.sol2)z.fit=predict(lm.sol2)e.hat=z-z.fit /*½öe.hat=residuals(lm.sol2)*/plot(e.hat∼z.fit) /*½ö^plot(lm.sol2,which=1)*/
Tianxiao Pang 1nÙ £8ëê��O
![Page 97: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/97.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 98: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/98.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 99: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/99.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 100: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/100.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
#�í�ãØ¥y?Û²w5K5, ùL²·�¤^�C�´Ü·�. ���£8�§�
y = z2 = (0.5822+0.000953x)2 = 0.339+0.0011x+0.00000091x2.
Tianxiao Pang 1nÙ £8ëê��O
![Page 101: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/101.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�e5·��êâ��ä: É~:�äÚrK�:�ä.
É~:�ä : duÆ)zí�ri�Cqw¤´�pÕá�ÑlN(0, 1), @o|ri| > 2´��Vǯ�, u)�VÇ��0.05. Ïd, ek,�|ri| > 2, ·�Òknd~¦éA���:(x
′i, yi)´
É~:.
Tianxiao Pang 1nÙ £8ëê��O
![Page 102: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/102.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
rK�:�ä : kÚ?�PÒ, ^Y(i), X(i)Úe(i)©OL«lY , XÚe¥GØ1i1¤����þ½Ý. GØ1i|êâ�,�e�n− 1|êâ��5£8�.�
Y(i) = X(i)β + e(i), E(e(i)) = 0, Cov(e(i)) = σ2In−1. (3.4.6)
rlù��.¦��β�LSEP�β(i), K
β(i) = (X′
(i)X(i))−1X
′
(i)Y(i). (3.4.7)
�þβ − β(i)�N1i|êâé£8Xê�O�K���, �§´���þ, ØBuA^©Û, A�ħ�,«êþz¼ê. Cookål´Ù¥A^�2���«.
Tianxiao Pang 1nÙ £8ëê��O
![Page 103: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/103.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Äk·�5¦β − β(i)�°(L�ª. �d, Ik0���ð�ª:
Ún
�A�n× n�_Ý, uÚvþ�n× 1�þ, @ok
(A− uv′)−1 = A−1 +
A−1uv′A−1
1− v′A−1u.
Tianxiao Pang 1nÙ £8ëê��O
![Page 104: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/104.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Px′i��OÝX�1i1. @oX = (x1, · · · ,xn)
′. |^þãÚ
n, ��
(X′
(i)X(i))−1 = (X
′X − xix
′i)−1
= (X′X)−1 +
(X′X)−1xix
′i(X
′X)−1
1− hii,(3.4.8)
Ù¥hii = x′i(X
′X)−1xi�lfÝH�1i�é����, �¡
�m\:. e,hii���, K�¡�pm\:. qÏ�
X′
(i)Y(i) =∑j 6=ixjyj =
n∑j=1
xjyj − xiyi = X′Y − xiyi,
¤±
Tianxiao Pang 1nÙ £8ëê��O
![Page 105: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/105.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
β(i) = (X′
(i)X(i))−1X
′
(i)Y(i)
=[(X
′X)−1 +
(X′X)−1xix
′i(X
′X)−1
1− hii
](X
′Y − xiyi)
= β − (X′X)−1xiyi +
(X′X)−1xix
′iβ
1− hii
−(X′X)−1xihiiyi1− hii
= β − (X′X)−1xiyi1− hii
+(X
′X)−1xix
′iβ
1− hii
= β − (X′X)−1xiei1− hii
. (3.4.9)
¤±
β − β(i) =(X
′X)−1xiei1− hii
.
Tianxiao Pang 1nÙ £8ëê��O
![Page 106: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/106.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
CookÚ?e��ål:
Di(M , c) =(β(i) − β)
′M(β(i) − β)
c,
Ù¥M´�½��½Ý, c´�½��~ê. N´wÑ
Di(M , c) =e2i
c(1− hii)2· x′
i(X′X)−1M(X
′X)−1xi.
�M = X′X, c = (p+ 1)σ2, K
Di =e2i
(p+ 1)σ2(1− hii)2· hii =
1
p+ 1· hii
1− hii· r2i .
Tianxiao Pang 1nÙ £8ëê��O
![Page 107: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/107.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.4.2)
Cookål
Di =(β(i) − β)
′X
′X(β(i) − β)
(p+ 1)σ2
=e2i
(p+ 1)σ2(1− hii)2· hii
=1
p+ 1· hii
1− hii· r2i , i = 1, · · · , n, (3.4.10)
Ù¥hii�lfÝH�1i�é��, ri´Æ)zí�.
ù�½nL², 3O�Cookål��ÿ, ·��I�l��êâ��5£8�.�ÑÆ)zí�ri ÚlfÝ�é��hiiÒ�±, Ø7é?Û��Ø��êâ��5£8�.(3.4.6) ?1O�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 108: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/108.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
hii�¹Â: hiiÝþ1i|êâxi�¥%x = 1n
∑ni=1 xi�ål.
��.
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In,
b�gCþ®¥%z, K
X =
1 x11 − x1 · · · x1p − xp...
......
...1 xn1 − x1 · · · xnp − xp
=:
1 (x1 − x)′
......
1 (xn − x)′
,
ùpx = (x1, · · · , xp)′, xi = (xi1, · · · , xip)
′. ²{üO���
X′X =
(n 00 L
),
Tianxiao Pang 1nÙ £8ëê��O
![Page 109: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/109.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ù¥L =∑n
i=1(xi − x)(xi − x)′, �p��. ?�Ú��
hii =1
n+ (xi − x)
′L−1(xi − x).
e´��¥%z�5£8�., K
X =
1 x1 − x...
...1 xn − x
, X′X =
(n 00 Sxx
), hii =
1
n+
(xi − x)2
Sxx.
Tianxiao Pang 1nÙ £8ëê��O
![Page 110: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/110.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3úª(3.4.10)¥, 1/(p+ 1)�iÃ', Pi = hii1−hii´hii�üNO¼
ê, Ï�hiiÝþ1i|êâxi�¥%x = 1n
∑ni=1 xi�ål. Ï
dPi�x1i|êâålÙ§êâ��C. Cookål�PiÚr2i�
��¤û½. ½n3.4.2w�·�, pm\:�U´rK�:, ��UØ´; É~:�U´rK�:, ��UØ´; �XJ�|êâQ´pm\:q´É~:, @o§Ò´rK�:.
��Cookål��^±�½rK�:��.�´é(J�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 111: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/111.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A^�&ý¥�±éCookålí�¥�M , c�À����½�nØ|±. 3Ø���b�e,
β ∼ N(β, σ2(X′X)−1),
|^íØ2.4.1�
(β − β)′X
′X(β − β)
σ2∼ χ2(p+ 1),
,��¡(n− p− 1)σ2
σ2∼ χ2(n− p− 1),
��ö�pÕá. ¤±
(β − β)′X
′X(β − β)
(p+ 1)σ2∼ F (p+ 1, n− p− 1). (3.4.11)
Tianxiao Pang 1nÙ £8ëê��O
![Page 112: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/112.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
8Ü
S ={β :
(β − β)′X
′X(β − β)
(p+ 1)σ2≤ Fα(p+ 1, n− p− 1)
}¡�β��&Y²�1− α��&ý¥. (3.4.10)�(3.4.11)¥�ÚOþé�q, �cö¿ØÑlF©Ù. , /Ïu�ö·��±éDi�����ѱe�VÇ)º.
~X, eDi = F0.50(p+ 1, n− p− 1), KL²1i|êâ(x′i, yi)�
GØ�, β��Oβ(i)£Ä�β��&Y²�0.5��&ý¥>.
þ; eDj = F0.80(p+ 1, n− p− 1), Kβ(j)£Ä�β��&Y²�0.2��&ý¥>.þ. ��1i|êâé�O�K�'1j|êâ5��.
Tianxiao Pang 1nÙ £8ëê��O
![Page 113: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/113.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éu~3.4.1, ·��Xe�êâ�ä©Û:
R§S:
yx=read.table(”ex p47 data.txt”)x=yx[,1]y=yx[,2]z=sqrt(y)mydata2=data.frame(z,x)lm.sol2=lm(z∼x,data=mydata2)summary(lm.sol2)z.fit=predict(lm.sol2)r.std=rstandard(lm.sol2)plot(r.std∼z.fit)text(z.fit,r.std,type=”1:53”)r.stdinfluence.measures(lm.sol2)
Tianxiao Pang 1nÙ £8ëê��O
![Page 114: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/114.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
©Û: 126Ò��:�É~:. ek”�cü�”�:, K�ÑÆ)zí
���?1?�Ú(@.Tianxiao Pang 1nÙ £8ëê��O
![Page 115: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/115.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 116: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/116.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 117: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/117.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 118: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/118.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
o(: 126Ò��:�É~:, ·�I�u�êâ5 ´ÄkL�, eÃL�KíØd��:. d, 126Ò��:�rK�:,IÚå5¿.
Tianxiao Pang 1nÙ £8ëê��O
![Page 119: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/119.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.4.2�åÿÁêâ. L3.4.2´��Æ[ÿÁ�21��Ö�P¹, Ù¥x´�Ö�c#(ü : �), y´,«�å�I, ÏLùêâ, ·��ïá�å�c#Cz�'X.
L3.4.2�åÿÁêâ
SÒ x y SÒ x y1 15 95 12 9 962 26 71 13 10 833 10 83 14 11 844 9 91 15 11 1025 15 102 16 10 1006 20 87 17 12 1057 18 93 18 42 578 11 100 19 17 1219 8 104 20 11 86
10 20 94 21 10 10011 7 113
Tianxiao Pang 1nÙ £8ëê��O
![Page 120: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/120.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R §S:
x=c(15,26,10,9,15,20,18,11,8,20,7,9,10,11,11,10,12,42,17,11,10)y=c(95,71,83,91,102,87,93,100,104,94,113,96,83,84,102,100,
105,57,121,86,100)lm.sol=lm(y∼x)summary(lm.sol)influence.measures(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 121: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/121.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 122: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/122.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 123: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/123.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
plot(lm.sol,which=2) /*QQã*/y.fit=predict(lm.sol)e=y-y.fitr.std=rstandard(lm.sol)plot(r.std∼y.fit,xlab=expression(hat(y)),ylab=“r”)text(y.fit,r.std,type=“1:25”)
Tianxiao Pang 1nÙ £8ëê��O
![Page 124: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/124.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 125: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/125.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 126: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/126.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
o(: 119Ò��:�É~:, 118,19Ò���:´rK�:.dQQãÚí�ã, ·��É�5b�!��à5b�!Ø�'5b�Ú��5b�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 127: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/127.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Box-CoxC�
éu*ÿêâ, e²L£8�ä���, §�Ø÷v�5b�!��à5b�!Ø�'5b�Ú��5b�¥���½eZ�,@o·�Ò�éêâæ�”£�”��. ¢�y², êâC�´?nk¯Kêâ��«Ð�{.
�!0�Box-CoxC�, §�Ì�A:´Ú\��ëê, ÏLêâ���OTëê, l (½Aæ��êâC�/ª. ¢�y²,Box-CoxC�éNõ¢SêâÑ´1�k��.
Tianxiao Pang 1nÙ £8ëê��O
![Page 128: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/128.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Box-CoxC�´éÏCþ�XeC�:
y(λ) =
{yλ−1λ , λ 6= 0,
ln y, λ = 0,(3.5.1)
ùpλ´���½�C�ëê.
Box-CoxC�´�xC�, §�)Nõ~��C�, ÃXéêC�(λ = 0), �êC�(λ = −1)Ú²��C�(λ = 1/2)��.
Tianxiao Pang 1nÙ £8ëê��O
![Page 129: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/129.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éÏCþ�n�*ÿ�y1, · · · , ynA^þãC�, ��C����þ
Y (λ) = (y(λ)1 , · · · , y(λ)
n )′.
·��(½C�ëêλ¦�Y (λ)÷v
Y (λ) = Xβ + e, e ∼ N(0, σ2I). (3.5.2)
=�¦C����þY (λ)�£8gCþ�mäk�5�''X,Ø�÷v��©Ù!��à5!�pÕá.
Ïd, Box-CoxC�´ÏLéëêλ�ÀJ, ��é�5êâ�”nÜ£n”, ¦Ù÷v�����5£8�.�¤kb�^�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 130: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/130.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�^4�q,�{5(½λ. é�½�λ, βÚσ2, Y (λ)�q,¼ê�
1
(√
2πσ)nexp
{− 1
2σ2(Y (λ) −Xβ)
′(Y (λ) −Xβ)
},
¤±Y�q,¼ê�
L(β, σ2) =1
(√
2πσ)nexp
{− 1
2σ2(Y (λ) −Xβ)
′(Y (λ) −Xβ)
}|J |,
ùpJ�C��Jacobi1�ª
J =
n∏i=1
dy(λ)i
dyi=
n∏i=1
yλ−1i .
lnL(β, σ2)'uβÚσ2¦�¿-Ù�u", ��βÚσ2�MLE�{β(λ) = (X
′X)−1X
′Y (λ),
σ2(λ) = 1nY
(λ)′(I −H)Y (λ) ∆
= 1nRSS(λ,Y (λ)).
(3.5.3)
Tianxiao Pang 1nÙ £8ëê��O
![Page 131: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/131.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éA�q,¼ê����
Lmax(λ) = L(β(λ), σ2(λ))
= (2πe)−n/2 · |J | ·(RSS(λ,Y (λ))
n
)−n/2.(3.5.4)
ù´λ�¼ê,·�ÏL¦§����5(½λ. ù�duÏL¦lnLmax(λ)����5(½λ. Äk, k
lnLmax(λ) = −n2· ln[RSS(λ,Y (λ))
]+ ln |J |+ C
= −n2
ln[Y (λ)
′
|J |1/n(I −H)
Y (λ)
|J |1/n]
+ C
∆= −n
2ln[RSS(λ,Z(λ))
]+ C, (3.5.5)
Ù¥Z(λ) = (z(λ)1 , · · · , z(λ)
n )′
= Y (λ)/|J |1/n,
Tianxiao Pang 1nÙ £8ëê��O
![Page 132: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/132.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
RSS(λ,Z(λ)) = Z(λ)′(I −H)Z(λ). (3.5.6)
(3.5.5)ªéBox-CoxC�3O�Åþ�¢y�5é��B, Ï��¦lnLmax(λ)����, �I¦RSS(λ,Z(λ))����. �,éJé�¦RSS(λ,Z(λ))������λ�)ÛL�ª, �é�X��½�λ�, ÏL¦���¦�O�£8§S, N´O�ÑéA�RSS(λ,Z(λ)). ,�xÑRSS(λ,Z(λ))'uλ�ã, lãþ�±Cq�éѦRSS(λ,Z(λ)) ������λ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 133: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/133.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Box-CoxC��äNÚ½:
1. é�½�λ�,O�z(λ)i , i = 1 · · · , n.
2. U(3.5.6)ªO�í�²�ÚRSS(λ,Z(λ)).
3. é�X��½�λ�, EþãÚ½, ���A�í�²�ÚRSS(λ,Z(λ))��G�. ±λ�î¶, RSS(λ,Z(λ)) �p¶, xÑ�A��. ^�*�{, éѦRSS(λ,Z(λ))������λ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 134: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/134.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.5.13~3.4.1¥, ·�éÏCþy�²��C�, ù��u¦^λ = 0.5�Box-CoxC�. ·�y35y¢ù��C�´Ü·�.eL�Ñ12�ØÓ�λ�éA�í�²�ÚRSS(λ,Z(λ)), {ü'���uy�λ = 0.5�í�²�ÚRSS(λ,Z(λ))����. ÏdCq/@�0.5Ò´C�ëêλ��`ÀJ.
L3.5.1
λ -2 -1 -0.5 0 0.125 0.25RSS 34101.04 986.04 291.59 134.10 119.20 107.21λ 0.375 0.5 0.625 0.75 1 2
RSS 100.26 96.95 97.29 101.69 127.87 1275.56
Tianxiao Pang 1nÙ £8ëê��O
![Page 135: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/135.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
x<-c(679,292,1012,493,582,1156,997,2189,1097,2078,1818,1700,747,2030,1643,414,354,1276,745,435,540,874,1543,1029,710,1434,837,1748,1381,1428,1255,1777,370,2316,1130,463,770,724,808,790,783,406,1242,658,1746,468,1114,413,1787,3560,1495,2221,1526)y<-c(0.79,0.44,0.56,0.79,2.70,3.64,4.73,9.50,5.34,6.85,5.84,5.21,3.25,4.43,3.16,0.50,0.17,1.88,0.77,1.39,0.56,1.56,5.28,0.64,4.00,0.31,4.20,4.88,3.48,7.58,2.63,4.99,0.59,8.19,4.79,0.51,1.74,4.10,3.94,0.96,3.29,0.44,3.24,2.14,5.71,0.64,1.90,0.51,8.33,14.94,5.11,3.85,3.93)lm.sol<-lm(y∼x)library(MASS)boxcox(lm.sol,plotit=T,lambda=seq(-2,2,by=0.05))
Tianxiao Pang 1nÙ £8ëê��O
![Page 136: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/136.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 137: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/137.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
2Â���¦�O
3c¡�?Ø¥, ·�o´b��5£8�.�Ø�´��à5�Ø�'�, =Cov(e) = σ2In.�´3Nõ¢S¯K¥, êâ Ø÷vù�b�(�ÏLí�ã�ä). ù�·�Ib�Ø��þ����Ý�Cov(e) = σ2Σ, ùpΣ´���½Ý. ù�Σ�¹��ëê, �·�ùpb�Σ ´��®��.
·��?Ø��5£8�.�
Y = Xβ + e, E(e) = 0, Cov(e) = σ2Σ. (3.6.1)
Ì�8�´�Oβ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 138: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/138.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�Σ´é¡�½Ý, ¤±�3n× n���P¦�
Σ = PΛP′,
ùpΛ = diag(λ1, · · · , λn), λi > 0, i = 1, · · · , n, ´Σ�A��.P
Σ12 = P diag(λ
121 , · · · , λ
12n )P
′,
K��(Σ12 )2 = Σ, ¡Σ
12´Σ�²��. Σ−
12L«Σ
12�_Ý
.
·�r�5£8�.(3.6.1)?1��C�. ^Σ−12�¦(3.6.1). P
Z = Σ−12Y , U = Σ−
12X, ε = Σ−
12e.
Tianxiao Pang 1nÙ £8ëê��O
![Page 139: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/139.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�Cov(ε) = Σ−12σ2ΣΣ−
12 = σ2In, u´·��Xe��5£
8�.
Z = Uβ + ε, E(ε) = 0, Cov(ε) = σ2In. (3.6.2)
ù´·�®?ØL��.. 3ù#�.¥, ��β�LSE�
β∗ = (U′U)−1U
′Z = (X
′Σ−1X)−1X
′Σ−1Y . (3.6.3)
·�¡β∗�β�2Â���¦�O(GLSE). ù��OäkûÐ�ÚO5�.
½n (3.6.1)
(a) E(β∗) = β;(b) Cov(β∗) = σ2(X
′Σ−1X)−1;
(c)é?¿�p+ 1���þc, c′β∗�c
′β��������5Ã
�O.
Tianxiao Pang 1nÙ £8ëê��O
![Page 140: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/140.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
y²: (a)
E(β∗) = (X′Σ−1X)−1X
′Σ−1E(Y ) = (X
′Σ−1X)−1X
′Σ−1Xβ = β.
(b)|^½n2.1.3,
Cov(β∗) = Cov[(X′Σ−1X)−1X
′Σ−1Y ]
= (X′Σ−1X)−1X
′Σ−1Cov(Y )((X
′Σ−1X)−1X
′Σ−1)
′
= σ2(X′Σ−1X)−1X
′Σ−1Σ((X
′Σ−1X)−1X
′Σ−1)
′
= σ2(X′Σ−1X)−1.
(c)�b′Y´c
′β�?��5à �O. éu�.(3.6.2),
c′β∗ = c
′(U
′U)−1U
′Z, b
′Y = b
′Σ
12 Σ−
12Y = b
′Σ
12Z,
=c′β∗�c
′β�LSE, b
′Y = b
′Σ
12Z�c
′β��5à �O. ¤±
é�.(3.6.2)A^Gauss-Markov½n�
Var(c′β∗) ≤ Var(b
′Σ
12Z) = Var(b
′Y ),
�Ò¤á��=�c′β∗ = b
′Σ
12Z = b
′Y .
Tianxiao Pang 1nÙ £8ëê��O
![Page 141: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/141.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n3.6.1(c)Ò´���/e�Gauss-Markov½n, §L²3���5£8�.(3.6.1)¥, GLSE β∗´�`�(eΣ = In, KGLSEòz�LSE β). éu�.(3.6.1), N´y²βE´Ã �O, ��7´�`��5à �O, Ï�Var(c
′β∗) ≤ Var(c
′β). ùÒ´`,
éu���5£8�.(3.6.1), GLSEo´`uLSE.
Tianxiao Pang 1nÙ £8ëê��O
![Page 142: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/142.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�.(3.6.1)�{ü�~f´ÏCþ�ØÓ*ÿäkØ�����/, =
Cov(e) = diag(σ21, · · · , σ2
n),
ùp�σ2i , i = 1, · · · , nØ���. Px
′1, · · · ,x
′n©O´�OÝ
X�n�1�þ. N´íÑ,
β∗ =( n∑i=1
xix′i
σ2i
)−1( n∑i=1
xiyiσ2i
). (3.6.4)
ü�Úª©O´xix′iÚxiyi�\�Ú(�Ñ�1/σ2
i ), Ïd�¡β∗�\����¦�O(WLSE).
σ2i ´���, ù�·�I��{¦�§���Oσ2
i , ,�3(3.6.4)¥^σ2
i�Oσ2i . ù«�O�{¡�üÚ�O(two-stage
estimate).
Tianxiao Pang 1nÙ £8ëê��O
![Page 143: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/143.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.6.1b�·�^�«°�¤ì3ü�¢�¿éÓ��þµ©O?1n1gÚn2gÿþ, Pùÿþ�©O�y11, · · · , y1n1Úy21, · · · , y2n2 . r§��¤�5£8�./ª{
y1i = µ+ e1i, i = 1, · · · , n1,y2i = µ+ e2i, i = 1, · · · , n2.
duü�¢�¿��*^�9¤ì�°ÝØÓ, �§��ÿþØ����Ø�. �
Var(e1i) = σ21, Var(e2i) = σ2
2, σ21 6= σ2
2.
Pe = (e11, · · · , e1n1 , e21, · · · , e2n2)′, K
Cov(e) =
(σ2
1In1 00 σ2
2In2
)= σ2
2
(θIn1 0
0 In2
)∆= σ2
2Σ,
ùpθ = σ21/σ
22. b��, KӮ�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 144: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/144.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5¿�ùp��OÝX = (1, · · · , 1)′, u´µ�GLSE�
µ∗ =(n1
θ+ n2
)−1(1
θ
n1∑i=1
y1i +
n2∑i=1
y2i
).
P
y1 =1
n1
n1∑i=1
y1i, y2 =1
n2
n2∑i=1
y2i,
ω1 =1
Var(y1)=n1
σ21
, ω2 =1
Var(y2)=n2
σ22
.
Kµ∗�U��
µ∗ =ω1
ω1 + ω2y1 +
ω2
ω1 + ω2y2.
Tianxiao Pang 1nÙ £8ëê��O
![Page 145: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/145.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
=µ∗´ü�¢�¿*ÿ�þ��\�²þ, §��� ω1ω1+ω2
Úω2
ω1+ω2��¢�¿ÿþ�Ø���Úÿþgêk'. Ø����
�, ÿþgê��, éA��Ò�.
µ∗�¹��ëêσ21Úσ
22, ØUGâSA^. ·��±�{�
Eσ21Úσ
22��O. ¯¢þ, ùü�¢�¿�*ÿêâ©O�¤�
5£8�.Yi = µ1ni + ei, i = 1, 2,
ùpYi = (y1i, · · · , y1ni)′, ei = (ei1, · · · , eini)
′. Ï�Cov(ei) =
σ2i Ini , ¤±ei, i = 1, 2÷vGauss-Markovb�. ¤±σ2
i�LSE�
σ2i =
1
ni − 1‖Yi − yi1ni‖2 i = 1, 2.
^σ2i , i = 1, 2�Oµ∗¥�σ2
i , i = 1, 2, =���µ�üÚ�O.
Tianxiao Pang 1nÙ £8ëê��O
![Page 146: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/146.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
õ��5
£8Xê�LSEkNõ`û�5�, Ù¥���´Gauss-Markov½n, §L²3�5à �Oa¥, LSE´���äk������O. �´ù�`:, ¦�LSE3�5ÚO�.��OnØÚ¢SA^¥Ókýé��/ .
�´·�±c?Ø�LSEI�b��OÝX´�÷��, =�¦ÝX���þ�m´�5Ã'�. , , 3¢S¯K¥, du²~�?n¹k�õgCþ��.£8¯K, �²LCþ�m Ø´�á� ´�péX�, ùÑ�¦�OÝX���þ�mØ�U���5Ã'. éõ�¹e, �OÝX���þ�m�3õ��5/E��5(multi-collinearity)'X.
Tianxiao Pang 1nÙ £8ëê��O
![Page 147: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/147.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â
e�3Ø��0�p+ 1�~êc0, c1, · · · , cp¦�
c0 + c1xi1 + · · ·+ cpxip = 0, i = 1, · · · , n,
K¡gCþx1, · · · , xp�m�3X���õ��5/E��5'X.
3¢S¯K¥, ���õ��5/E��5'X¿Øõ�, ��Ñy�´�½§Ýþ���5.
Tianxiao Pang 1nÙ £8ëê��O
![Page 148: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/148.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â (õ��5/E��5'X)
e�3Ø��0�p+ 1�~êc0, c1, · · · , cp¦�
c0 + c1xi1 + · · ·+ cpxip ≈ 0, i = 1, · · · , n,
K¡gCþx1, · · · , xp�m�3Xõ��5/E��5'X.
Tianxiao Pang 1nÙ £8ëê��O
![Page 149: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/149.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
é²Lêâï��, õ��5��/éõ, õ��5�/¬�õ��5£8©Û�5�oK�!XÛ�ägCþ�m�õ��5±9XÛ�Ñõ��5�K��¯Kò´�!�?Ø�Ì�SN.
Tianxiao Pang 1nÙ £8ëê��O
![Page 150: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/150.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�kÚ\��Vg: þ�Ø�(MSE: Mean Squared Errors), §´^5µd���O`��IO�.
½Â
�θ����þ. θ�θ����O. ½Âθ�þ�Ø��
MSE(θ) = E‖θ − θ‖2 = E[(θ − θ)′(θ − θ)].
Tianxiao Pang 1nÙ £8ëê��O
![Page 151: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/151.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.7.1)
MSE(θ) = tr[Cov(θ)] + ‖Eθ − θ‖2.
y²: ØJwÑ
MSE(θ) = E[(θ − θ)′(θ − θ)]
= E[(θ − Eθ) + (Eθ − θ)]′[(θ − Eθ) + (Eθ − θ)]
= E(θ − Eθ)′(θ − Eθ) + E(Eθ − θ)
′(Eθ − θ)
∆= ∆1 + ∆2.
|^,�5�,
∆1 = E{tr[(θ − Eθ)′(θ − Eθ)]}
= E{tr[(θ − Eθ)(θ − Eθ)′]}
= tr[E(θ − Eθ)(θ − Eθ)′] = tr[Cov(θ)].
∆2 = E[(Eθ − θ)′(Eθ − θ)] = ‖Eθ − θ‖2´w,�. y..
Tianxiao Pang 1nÙ £8ëê��O
![Page 152: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/152.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ePθ = (θ1, · · · , θp+1)′, K
∆1 =
p+1∑i=1
Var(θi),
§´θ�©þ����Ú.
∆2 =
p+1∑i=1
(Eθi − θi)2,
§´θ�©þ� ²��Ú. ¤±, ���O�þ�Ø�d§���Ú �¤û½. ��Ð��OAk�����Ú �.
Tianxiao Pang 1nÙ £8ëê��O
![Page 153: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/153.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n
3�5£8�.(3.1.5)¥, éβ�LSE β, k(a) MSE(β) = σ2
∑p+1i=1
1λi;
(b) E‖β‖2 = ‖β‖2 + σ2∑p+1
i=11λi,
Ù¥λ1, · · · , λp+1 > 0�X′X�A��.
y²: (a)Ï�LSE β´Ã �O, ¤±∆2 = 0,
MSE(β) = ∆1 = tr[Cov(β)] = σ2tr[(X′X)−1].
Ï�X′X´é¡�½Ý, ¤±�3��P¦�
X′X = P diag(λ1, · · · , λp+1)P
′,
ùpλ1, · · · , λp+1 > 0�X′X�A��. ¤±
(X′X)−1 = P diag(
1
λ1, · · · , 1
λp+1)P
′.
Tianxiao Pang 1nÙ £8ëê��O
![Page 154: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/154.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
|^,�5�êþ��
tr(X′X)−1 = tr
(diag(
1
λ1, · · · , 1
λp+1))
=
p+1∑i=1
1
λi.
¤±MSE(β) = σ2∑p+1
i=11λi
.
(b) �
MSE(β) = E[(β − β)′(β − β)]
= E(β′β − 2β
′β + β
′β)
= E‖β‖2 − β′β,
u´
E‖β‖2 = ‖β‖2 + MSE(β) = ‖β‖2 + σ2p+1∑i=1
1
λi.
Tianxiao Pang 1nÙ £8ëê��O
![Page 155: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/155.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(Ø(a)w�·�, XJX′X��k��A���~�, =�~�
Cu", @oMSE(β)Ò¬é�. lþ�Ø��IO5w, ���¦�OβØ´��Ð��O. ùÚGauss-Markov½n¿Øgñ, Ï�Gauss-Markov½n==�y���¦�O3�5à �Oa¥�����5. �3X
′X��k��A���~��, ù��
�������%é�, Ï ��é��þ�Ø�.
(Ø(b)w�·�, XJX′X��k��A���~�, @o��
�¦�Oβ��ݲþ`5�'ý��β��Ý�éõ. ùÒ��β�,©þ�ýé�L�.
o�, �X′X��k��A���~��, ���¦�OØ2´
����O.
Tianxiao Pang 1nÙ £8ëê��O
![Page 156: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/156.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�5©Û”��k��A���~�”3�OÝX½ö£8gCþþ¿�X�o.
PX = (1,x1, · · · ,xp), =xi�X�1i+ 1�. �λ�X′X���
A��, φ�ÙéA�A��þ, Ù�Ý�1, =φ′φ = 1.
eλ ≈ 0, K
‖Xφ‖2 = φ′X
′Xφ = λφ
′φ = λ ≈ 0.
u´Xφ ≈ 0. Pφ = (c0, c1, · · · , cp)′, K
c0 + c1x1 + · · ·+ cpxp ≈ 0. (3.7.1)
=�OÝX���þ�m(=gCþ�m)kõ��5'X.
Tianxiao Pang 1nÙ £8ëê��O
![Page 157: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/157.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��, e�OÝX���þ�mkõ��5'X, =(3.7.1)¤á, d�X
′XE´�½Ý�|X ′
X| ≈ 0. dd��
p+1∏i=1
λi = |X ′X| ≈ 0,
¤±��k��A���~�, �Cu".
�Ò´`��k��A���~��X���þ�mkõ��5'X´�d�. ù�¡�OÝX´¾�Ý.
Tianxiao Pang 1nÙ £8ëê��O
![Page 158: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/158.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
õ��5��ä :
(1) ��)äÏf(VIF: variance inflation factor)�ä{
PR2j�gCþxjéÙ{p− 1�gCþ��½Xê, ½Â
VIFj =1
1−R2j
, j = 1, · · · , p.
duR2jÝþgCþxjéÙ{p− 1�gCþ�m��5�'§
Ý, x1, · · · , xp�m�õ��5�î, R2j��Cu1, VIFj�Ò
��. Ïd^VIF 5Ýþõ��5�§Ý´Ün�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 159: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/159.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ýþ�OK:
�k,�VIFj ≥ 10
�, @�gCþ�m�3î�õ��5; ½ö�
VIF =1
p
p∑i=1
VIFj � 1
�, @�gCþ�m�3î�õ��5.
Tianxiao Pang 1nÙ £8ëê��O
![Page 160: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/160.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(2)A���^�ê(CI: Condition Index)�ä{
��Øþj�K�, ·�b�gCþ�ÏCþ�*ÿ�þ®IOz. d��@��5£8�.vk�å�, �OÝX´n× p�Ý, X
′X´p�gCþ����'XêÝ.
A���ä{: XJX′Xkm�A��Cq�", @oXÒkm
�õ��5'X, ¿�ùm�õ��5'X�Xê�þÒ´ùm��Cu"�A��¤éA�IO��zA��þ.
^�ê�ä{: b�X′X�p�A��©O�λ1, · · · , λp, Ù¥�
�A���λmax, ��A���λmin, ¡
κj =λmaxλj
, j = 1, · · · , p
�A��λj�^�ê.
Tianxiao Pang 1nÙ £8ëê��O
![Page 161: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/161.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
P
κ = maxjκj =
λmaxλmin
.
§�±^5ÝþÝX′X�A��Cq�"�§Ý, Ïd�±^
5�äõ��5´Ä�3±9õ��5�î§Ý.
^��OK:
e0 < κ < 100, K@�Ø�3õ��5;
e100 ≤ κ ≤ 1000, K@��3�r�õ��5;
eκ > 1000, K@��3î�õ��5.
Tianxiao Pang 1nÙ £8ëê��O
![Page 162: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/162.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.7.1.�Ä��k8�£8gCþ��5£8¯K, �©êâ�eL.
L3.7.1
SÒ y x1 x2 x3 x4 x5 x61 10.006 8 1 1 1 0.541 -0.0992 9.737 8 1 1 0 0.130 0.0703 15.087 8 1 1 0 2.116 0.1154 8.422 0 0 9 1 -2.397 0.2525 8.625 0 0 9 1 -0.046 0.0176 16.289 0 0 9 1 0.365 1.5047 5.958 2 7 0 1 1.996 -0.8658 9.313 2 7 0 1 0.228 -0.0559 12.960 2 7 0 1 1.380 0.502
10 5.541 0 0 0 10 -0.798 -0.39911 8.756 0 0 0 10 0.257 0.10112 10.937 0 0 0 10 0.440 0.432
Tianxiao Pang 1nÙ £8ëê��O
![Page 163: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/163.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
yx=read.table(”ex p62 data.txt”)y=yx[, 1]x1=yx[, 2]x2=yx[, 3]x3=yx[, 4]x4=yx[, 5]x5=yx[, 6]x6=yx[, 7]mydata=data.frame(y,x1,x2,x3,x4,x5,x6)lm.sol=lm(y∼ x1+x2+x3+x4+x5+x6,data=mydata)summary(lm.sol)library(DAAG)vif(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 164: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/164.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
VIFÑÑ(J
x1 x2 x3 x4 x5 x6
182.0500 161.3600 266.2600 297.7100 1.9200 1.4553
Ï�co�VIPÑØ�u10, ¤±@�gCþ�m�3î�õ��5.
Tianxiao Pang 1nÙ £8ëê��O
![Page 165: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/165.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A���^�ê�ä{:
R§S:
X=cbind(x1,x2,x3,x4,x5,x6)Xrho=cor(X)rhoeigen(rho)kappa(rho,exact=TRUE) /*%@´exact=FALSE, ù�k���O�Ø�*/
Tianxiao Pang 1nÙ £8ëê��O
![Page 166: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/166.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A��!A��þ�^�ê:
Tianxiao Pang 1nÙ £8ëê��O
![Page 167: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/167.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
λmin = 0.001106051, éA�A��þ�
(0.44768, 0.42114, 0.541689, 0.57337, 0.00605, 0.00217)′.
¤±IOzgCþx∗i , i = 1, · · · , 6�m�3Xe�õ��5'X
0.44768x∗1 + 0.42114x∗2 + 0.541689x∗3 + 0.57337x∗4 + 0.00605x∗5
+0.00217x∗6 ≈ 0.
ù`²éu�©gCþxi, i = 1, · · · , 6, �3~êc0¦�
c0 + 0.44768x1 + 0.42114x2 + 0.541689x3 + 0.57337x4
+0.00605x5 + 0.00217x6 ≈ 0.
½��
c0 + 0.44768x1 + 0.42114x2 + 0.541689x3 + 0.57337x4 ≈ 0.
Tianxiao Pang 1nÙ £8ëê��O
![Page 168: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/168.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
^�êκ = 2195.908 > 1000, �`²gCþ�m�3î�õ��5'X.
�Øõ��5��{:
O���Nþ, �ؽ�)gCþ��5�'5.
ã+à 5, Ïék �O, ·�ò0�*�OṲ́©�O.
Tianxiao Pang 1nÙ £8ëê��O
![Page 169: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/169.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�O
�gCþ�mäkõ��5�, ��ÑLSE²wC��¯K,Hoerlu1962cJÑ�«U?����¦�O�{, =*�O(Ridge Estimate). ��, HoerlÚKennard31970céT�O�?�Ú��[?Ø.
*�O�g�: �gCþ�m�3õ��5�, �OÝX´¾��, =|X ′
X| ≈ 0, l (X′X)−1�CÛÉ. �;�ù�y�,
�X′X \þ���~êé�ÝkI(k > 0),KÝ
(X′X + kI)−1
�CÛÉ��U5�'(X′X)−1�CÛÉ��U5��õ. Ïd
^
β(k) = (X′X + kI)−1X
′Y (3.8.1)
����ëêβ��OAT'���¦�O�½�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 170: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/170.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â
é�½�0 < k <∞, ¡β(k) = (X′X + kI)−1X
′Y�£8X
êβ�*�O. d*�O¤ïá�£8�§¡�*£8�§.¡k�*ëê. éuβ(k)�©þβj(k), r3²¡���IX
¥βj(k)�kCz¤LyÑ5��¡�*,(ridge trace).
5: kØÓ, ·���ØÓ��O. Ïd*�Oβ(k)´���Oa.�k = 0�, β(k)Ò´Ï~�LSE. ���¹e, ·�Jå*�O,´Ø�)LSE�.
3?1*�O�c, ��Øþj�K�, ·�ob�gCþ�ÏCþþ®IOz, Ïdùp��OÝX´n× pÝ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 171: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/171.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�O�5� :
5�1: β(k)´β�k �O, =é?¿�0 < k <∞, E(β(k)) 6=β.
k 5´*�O����¦�O�����ØÓ�?. ���O�þ�Ø�d���ÚÚ ��²�Ú|¤. ��3õ��5�, ���¦�O�,�± �Ü©�", �§���Ü©%é�, �ª��§�þ�Ø�é�. ·�Ú?*�O�8�´ã+à 5, ����Ü©��ÌÝ~�, �ªü$Ùþ�Ø�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 172: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/172.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5�2: β(k)´���¦�Oβ����5C�.
y²: �I5¿�
β(k) = (X′X + kI)−1X
′Y
= (X′X + kI)−1X
′X · (X ′
X)−1X′Y
= (X′X + kI)−1X
′Xβ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 173: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/173.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5�3: é?¿�k > 0, e‖β‖ 6= 0, K·�ok‖β(k)‖ < ‖β‖.=*�O´r���¦�Oβ��:�·Ý�Ø ���, *�O´��Ø k �O.
y²: �Äõ��5£8�.Y = Xβ + e, -
Z = XP , α = P′β,
Ù¥P���Ý÷v
P′(X
′X)P = Λ = diag(λ1, · · · , λp),
λ1, λ2, · · · , λp > 0�X′X�A��. ù�, õ��5£8�.�
��
Y = Zα+ e, E(e) = 0, Cov(e) = σ2In. (3.8.2)
·�¡(3.8.2)��5£8�.�;K/ª, ¡α�;K£8Xê.
Tianxiao Pang 1nÙ £8ëê��O
![Page 174: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/174.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5¿�Z′Z = P
′X
′XP = Λ, ¤±
α = (Z′Z)−1Z
′Y = Λ−1Z
′Y .
β = (X′X)−1X
′Y = PΛ−1P
′X
′Y = PΛ−1Z
′Y = Pα.
§��A�*�O©O�
α(k) = (Z′Z + kI)−1Z
′Y = (Λ + kI)−1Z
′Y ,
β(k) = (X′X + kI)−1X
′Y
= PP′(X
′X + kI)−1PP
′X
′Y
= Pα(k).
Ïd
‖β(k)‖ = ‖α(k)‖ = ‖(Λ + kI)−1Λα‖ < ‖α‖ = ‖β‖.
Tianxiao Pang 1nÙ £8ëê��O
![Page 175: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/175.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5: ´�þ�Ø�3�OÚëê���C�e�±ØC, ¤±;K£8Xê����¦�O(½*�O)Ú�£8Xê����¦�O(½*�O)k�Ó�þ�Ø�:
MSE(α) = MSE(β), MSE(α(k)) = MSE(β(k)). (3.8.3)
Tianxiao Pang 1nÙ £8ëê��O
![Page 176: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/176.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.8.1, *�O�35½n)
�3k > 0¦�MSE(β(k)) < MSE(β).
=�3k > 0, ¦�3þ�Ø�¿Âe, *�O`u���¦�O.
y²: d(3.8.3), �Iy²�3�3k > 0¦�
MSE(α(k)) < MSE(α). (3.8.4)
Pf(k) = MSE(α(k)), k ≥ 0. 5¿f(0) = MSE(α). e·�Uy²f(k)3[0,∞)þ´ëY¼ê�f
′(0) < 0, K7�3�����
k > 0 ¦�(3.8.4)¤á.
Tianxiao Pang 1nÙ £8ëê��O
![Page 177: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/177.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5?Øf(k). 5¿�
E(α(k)) = (Λ + kI)−1Z′E(Y )
= (Λ + kI)−1Z′Zα
= (Λ + kI)−1Λα,
�
Cov(α(k)) = σ2(Λ + kI)−1Z′Z(Λ + kI)−1
= σ2(Λ + kI)−1Λ(Λ + kI)−1.
¤±
f(k) = MSE(α(k)) = tr[Cov(α(k))
]+ ‖E(α(k))−α‖2
= σ2p∑i=1
λi(λi + k)2
+ k2p∑i=1
α2i
(λi + k)2
∆= f1(k) + f2(k). (3.8.5)
Tianxiao Pang 1nÙ £8ëê��O
![Page 178: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/178.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
w,f(k)´[0,∞)þ�ëY¼ê. q
f′1(k) = −2σ2
p∑i=1
λi(λi + k)3
, f′1(0) = −2σ2
p∑i=1
1
λ2i
< 0 (3.8.6)
±9
f′2(k) = 2k
p∑i=1
λiα2i
(λi + k)3, f
′2(0) = 0, (3.8.7)
¤±f′(0) = f
′1(0) + f
′2(0) = −2σ2
∑pi=1
1λ2i< 0. y..
Tianxiao Pang 1nÙ £8ëê��O
![Page 179: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/179.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�O�35½n3nØþy²�3,�*�O`u���¦�O, ��éÑù�*ëêk´ØN´�. N´wÑ, nØþ, ù�k´e��§�):
f′(k) = −2σ2
p∑i=1
λi(λi + k)3
+ 2k
p∑i=1
λiα2i
(λi + k)3
= 2
p∑i=1
λi(kα2i − σ2)
(λi + k)3= 0. (3.8.8)
ù�)�6u��ëêαi, i = 1, · · · , pÚσ2, ¤±Ø�Ul)�§��ݼ�*ëêk. ÚOÆ[�lÙ§å»JÑÀJ*ëêk��{.
Tianxiao Pang 1nÙ £8ëê��O
![Page 180: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/180.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*ëêÀJ�{ :
(1) Hoerl-Kennardúª
HoerlÚKennardJÑ�ÀJk�úª´
k =σ2
maxiα2i
. (3.8.9)
¼�ù�úª��{Xe: d(3.8.8)�, ekα2i − σ2 < 0éi = 1,
· · · , pѤá, Kf′(k) < 0. u´�
k∗ =σ2
maxiα2i
,
�0 < k < k∗�, f′(k)o´�u", Ï f(k)3(0, k∗)þ´üN4
~¼ê, 2df(k)3[0,∞)�ëY5�f(k∗) < f(0). ,�2^αiÚσ2 �Oαi Úσ
2, B�(3.8.9).
Tianxiao Pang 1nÙ £8ëê��O
![Page 181: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/181.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(2) *,{
òβ1(k), · · · , βp(k)�*,x3�Üãþ, �â*,�Czª³ÀJk. ±e´A^ÀJk�OK:
�£8Xê�*�O��'�½;
^���¦�O�ÎÒØÜn�£8Xê, Ù*�O�ÎÒòC�Ün;
£8XêvkØÜn�ÎÒ;
í�²�ÚØ�þ,�õ.
���¹e, ·�ÀJ���k�U¦��^*,Ñm©ªu½.
Tianxiao Pang 1nÙ £8ëê��O
![Page 182: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/182.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.8.1{I²Ló�öF"ÏLISo��x1,�;þx2,o�¤þx3�ýÿ?�o�y, ±þCþ�ü þ��·{J. �dÂ81949-1959�11c�êâ, �eL.
c° x1 x2 x3 y1949 149.3 4.2 108.1 15.91950 161.2 4.1 114.8 16.41951 171.5 3.1 123.2 19.01952 175.5 3.1 126.9 19.11953 180.8 1.1 132.1 18.81954 190.7 2.2 137.7 20.41955 202.1 2.1 146.0 22.71956 212.4 5.6 154.1 26.51957 226.1 5.0 162.3 28.11958 231.9 5.1 164.3 27.61959 239.0 0.7 167.6 26.3
Tianxiao Pang 1nÙ £8ëê��O
![Page 183: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/183.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
library(MASS)yx=read.table(”ex p68 data.txt”)x1=yx[, 1]x2=yx[, 2]x3=yx[, 3]y=yx[, 4]mean(x1);mean(x2);mean(x3);mean(y)sd(x1);sd(x2);sd(x3);sd(y)mydata=data.frame(y, x1, x2, x3)lm.sol=lm(y∼x1+x2+x3)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 184: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/184.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
¼��Cþ���þ�!��IO�êâ:
Tianxiao Pang 1nÙ £8ëê��O
![Page 185: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/185.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
¦^���¦�{�(J:
£8�§: y = −10.128− 0.051x1 + 0.587x2 + 0.287x3.
Tianxiao Pang 1nÙ £8ëê��O
![Page 186: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/186.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
£8©Û�(JL²: x1�£8Xê��O´Kê, ùØÎÜÙ²L¿Â, ù´Ï�{I´���á�?�I, �ISo��x1
O\�, ?�o�y�AO\. ¤±, £8Xê�ÎÒ�¢SØÎ.Ù�Ï´n�gCþ�m�3Xõ��5, ù�{ü/lx1,x2, x3����'XêÝwÑ:
Ïdx1�x3�3pÝ��'5.
Tianxiao Pang 1nÙ £8ëê��O
![Page 187: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/187.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�õ��5�ä, �±²(uygCþ�m�3Xõ��5:
Tianxiao Pang 1nÙ £8ëê��O
![Page 188: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/188.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
^*�O�{Ïé*£8�§:
kéêâ?1IOz, R§S:
yxs=scale(yx)x1=yxs[, 1]x2=yxs[, 2]x3=yxs[, 3]y=yxs[, 4]mydata2=data.frame(x1,x2,x3,y)mydata2
Tianxiao Pang 1nÙ £8ëê��O
![Page 189: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/189.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
rr.sol=lm.ridge(y∼0+x1+x2+x3,data=mydata2,lambda=c(seq(0,0.01,by=0.001),seq(0.02,0.1,by=0.01),seq(0.2,1,by=0.1)))rr.sol /∗w«*�O∗/plot(rr.sol) /∗�*,ã∗/matplot(rr.sol$lambda,t(rr.sol$coef),type=”l”,col=c(”red”,”blue”,
”black”),main=”ridge trace”,xlab=expression(lambda),ylab=expression(hat(beta)(lambda))) /∗�*,ã∗/
Tianxiao Pang 1nÙ £8ëê��O
![Page 190: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/190.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�Oêâ:
Tianxiao Pang 1nÙ £8ëê��O
![Page 191: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/191.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 192: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/192.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 193: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/193.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�â*,ãÀJ*ëêk = 0.4, IOzCþ�*£8�§�
v = 0.416u1 + 0.213u2 + 0.531u3.
��, I�=�¤�©Cþ�*£8�§:
y − 21.891
4.544= 0.416× x1 − 194.591
30.000+ 0.213× x2 − 3.300
1.649
+0.531× x3 − 139.736
20.634,
=y = −8.655 + 0.063x1 + 0.587x2 + 0.117x3.
Tianxiao Pang 1nÙ £8ëê��O
![Page 194: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/194.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�¤kCþ�þjÑ���, �±��é�©êâ?1*�O:
rr.sol=lm.ridge(y∼x1+x2+x3,data=mydata,lambda=c(seq(0,0.01,by=0.001),seq(0.02,0.1,by=0.01),seq(0.2,1,by=0.1)))rr.solplot(rr.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 195: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/195.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
![Page 196: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/196.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ÀJ*ëêk = 0.4, �*£8�§:
y = −8.621 + 0.063x1 + 0.588x2 + 0.117x3.Tianxiao Pang 1nÙ £8ëê��O
![Page 197: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/197.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
2Â*�O :
PK = diag(k1, · · · , kp), ki ≥ 0, i = 1, · · · , p. ¡
β(k) = (X′X +K)−1X
′Y
�β�2Â*�O.
Tianxiao Pang 1nÙ £8ëê��O
![Page 198: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/198.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�O
̤©(Principle Component)�O´dW.F.Massyu1965cJÑ�,�«k �O, 8�´��Ñ�OÝX�¾�Ý����¦�O�½5òC�é�ù�"�. ̤©�O�Ä�g�´: Äk/Ïu��C�ò£8gCþC�éA�̤©(̤©�*ÿ�þ´pØ�'�, l �Øõ��5¯K), ,�l¤k�̤©¥À��Ü©��̤©(å�ü���^)¿±§���#�£8gCþïá#�£8�., ^���¦�{�O#�.¥�£8Xê¿��£8�§. Äu���£8�§2ò§�=���©Cþ�£8�§.
Tianxiao Pang 1nÙ £8ëê��O
![Page 199: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/199.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��Øþj�K�, b�gCþ�ÏCþþ®IOz.
�ģ8�.:
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In, (3.9.1)
Ù¥X´n× p�OÝ. Pλ1 ≥ · · · ≥ λp > 0�X′X�A��,
φ1, · · · ,φp�éA�IO��zA��þ. K
Φ = (φ1, · · · ,φp)
�p× p��Ý�
Φ′X
′XΦ = diag(λ1, · · · , λp)
∆= Λ.
2PZ = XΦ, α = Φ′β, K�.(3.9.1)�U��
Y = Zα+ e, E(e) = 0, Cov(e) = σ2In. (3.9.2)
Tianxiao Pang 1nÙ £8ëê��O
![Page 200: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/200.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3þã��5£8;K�.(3.9.2)¥, #��OÝ
Z∆= (z1, · · · , zp) = (Xφ1, · · · ,Xφp),
ePX = (x1, · · · ,xp), K#gCþ�*ÿ�þ��©gCþ�*ÿ�þ�'X�
zj = φ1jx1 + · · ·+ φpjxp, j = 1, · · · , p.
ù´é�©gCþ�*ÿ�þ����5Cþ, C��Xê�þ�A��λj¤éA�IO��zA��þ.
ÚOþ, ¡*ÿ�þzj , j = 1, · · · , péA�#gCþzj , j = 1, · · · ,p�p�̤©. z�̤©Ñ´�©gCþ��5|Ü:
zj = φ1jx1 + · · ·+ φpjxp, j = 1, · · · , p.
Tianxiao Pang 1nÙ £8ëê��O
![Page 201: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/201.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�5� :?¿ü��̤©�*ÿ�þÑ´pØ�'�,�1j�̤©� �²�Ú
∑ni=1(zij − zj)2 = λj .
y²: Ï�Z′Z = Φ
′X
′XΦ = Λ = diag(λ1, · · · , λp), ¤±
z′jzk = 0, ∀j 6= k
�z′jzj = λj , j = 1, · · · , p. qÏ�X´I5z�OÝ, ¤±
zj =1
n
n∑i=1
zij =1
n
n∑i=1
p∑k=1
φkjxik =1
n
p∑k=1
φkj
n∑i=1
xik = 0.
Ïdk
n∑i=1
(zij − zj)2 =
n∑i=1
z2ij = z
′jzj = λj , j = 1, · · · , p.
Tianxiao Pang 1nÙ £8ëê��O
![Page 202: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/202.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
λjÝþ1j�̤©zj���CÄ��. Ï�λ1 ≥ · · · ≥ λp > 0,¤±·�¡z1�1�̤©, z2�1�̤©, · · · . ùp�̤©�*ÿ�þ´pØ�'�, ¤±#�gCþz1, · · · , zp(½�OÝZ)Ø�3õ��5¯K.
d5�1��, z1 éÏCþ�)ºUå�r, z2 g�, · · · , zp�f.e�OÝX´¾�Ý, @ok�X
′X�A��é�, Ø�
b�λr+1, · · · , λp ≈ 0.
ù�, �¡�p− r�̤©���CÄÒé��þ3"NC��.¤±ùp− r�̤©éÏCþ�K�Ò�±�ÑK, �ò§�l£8�.¥GØ. ^���¦{é�e�r�̤©(=r�#�gCþ)�£8=�. ��2C£��©Cþ�£8�§.
5:̤©£8�Ì�8�: ��(�Øõ��5), ü�(~�O�þ), ,�ïá£8�§.
Tianxiao Pang 1nÙ £8ëê��O
![Page 203: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/203.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éΛ,α,Z,Φ�©¬:
Λ =
(Λ1 00 Λ2
), α =
(α1
α2
), Z = (Z1
...Z2), Φ = (Φ1...Φ2),
Ù¥Λ1�r × rÝ, α1�r × 1�þ, Z1�n× rÝ, Φ1�p× rÝ. GØZ2α2, �.(3.9.2)C�
Y ≈ Z1α1 + e, E(e) = 0, Cov(e) = σ2In. (3.9.3)
Z1Ø´¾�Ý, ¤±���A^���¦�O�α1�LSE
α1 = (Z1′Z1)−1Z1
′Y .
Tianxiao Pang 1nÙ £8ëê��O
![Page 204: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/204.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
c¡·�l�.¥GØ�¡�p− r�̤©, ù��u·�^α2 = 0��Oα2. |^'Xβ = Φα, �β�̤©�O�
β = Φ
(α1
α2
)= (Φ1,Φ2)
(α1
0
)= Φ1Λ
−11 Z
′1Y
= Φ1Λ−11 Φ
′1X
′Y .
�A�̤©£8�§�Y = Xβ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 205: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/205.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�O�L§ :
Step1:���C�Z = XΦ, ¼�#�gCþ, ¡�̤©.
Step2:�£8gCþÀJ, GØéA�A��'���@̤©.
Step3:ò�{�̤©éy����¦£8, 2�£��5�ëê�O, ��'u�©Cþ�̤©£8�§.
Tianxiao Pang 1nÙ £8ëê��O
![Page 206: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/206.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�O�5� :
5�1. β = Φ1Φ′1β, =̤©�O´���¦�O����5C
�.
y²: �âe�'X:
Z = (Z1...Z2) = (XΦ1
...XΦ2), Φ′1Φ1 = Ir, Φ
′1Φ2 = 0
9X
′X = ΦΛΦ
′= Φ1Λ1Φ
′1 + Φ2Λ2Φ
′2
¿5¿�X′(I −H) = 0(H�lfÝ), ��
β = Φ1Λ−11 Φ
′1X
′Y = Φ1Λ
−11 Φ
′1X
′Xβ
= Φ1Λ−11 Φ
′1Φ1Λ1Φ
′1β + Φ1Λ
−11 Φ
′1Φ2Λ2Φ
′2β
= Φ1Λ−11 Φ
′1Φ1Λ1Φ
′1β
= Φ1Φ′1β.
Tianxiao Pang 1nÙ £8ëê��O
![Page 207: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/207.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5�2. E(β) = Φ1Φ′1β. =��r < p, ̤©�OÒ´k �O.
y²: �I5¿�E(β) = β=�.
5�3. ‖β‖ ≤ ‖β‖, =̤©�O´Ø �O.
y²: -I = diag(Ir,0), KdΦ�½Â�
Φ1Φ′1 = ΦIΦ
′.
l k
‖β‖ = ‖ΦIΦ′β‖ = ‖IΦ
′β‖ ≤ ‖Φ′
β‖ = ‖β‖.
Tianxiao Pang 1nÙ £8ëê��O
![Page 208: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/208.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.9.1)
��©gCþ�3vî�õ��5�, ·�ÀJ�3�̤©�ê�¦Ì¤©�O'���¦�Ok���þ�Ø�, =
MSE(β) < MSE(β).
y²: b�X′X��p− r�A��λr+1, · · · , λpé�Cu". Ø
JwÑ
MSE(β) = MSE
(α1
0
)= tr
[Cov
(α1
0
)]+∥∥∥E
(α1
0
)−α
∥∥∥2
= σ2tr(Λ−11 ) + ‖α2‖2.
Tianxiao Pang 1nÙ £8ëê��O
![Page 209: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/209.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�
MSE(β) = σ2tr(Λ−1) = σ2tr(Λ−11 ) + σ2tr(Λ−1
2 ),
¤±MSE(β) = MSE(β) + (‖α2‖2 − σ2tr(Λ−1
2 )).
u´MSE(β) < MSE(β)
��=�
‖α2‖2 < σ2tr(Λ−12 ) = σ2
p∑i=r+1
1
λi. (3.9.4)
�õ��5vî��ÿ, λr+1, · · · , λp�±¿©�Cu".Ïdþªmà�±v�¦�Ø�ª(3.9.4)¤á.
Tianxiao Pang 1nÙ £8ëê��O
![Page 210: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/210.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�α2 = Φ′2β, (3.9.4)���(β
σ
)′
Φ2Φ2′(βσ
)< tr(Λ−1
2 ), (3.9.5)
ùÒ´`, �βÚσ÷v(3.9.5)�, ̤©�Oâ'���¦�Ok���þ�Ø�. (3.9.5)L«ëê�m¥(Àβ/σ�ëê)��¥%3�:�ý¥. u´l(3.9.5)��Xe(Ø:
(a)é�½�ëêβÚσ2, �X′X��p− r�A��'���, Ì
¤©�O'���¦�Ok���þ�Ø�.
(b)é�½�X′X, =�½�Λ2, é�é���β/σ, ̤©�O
'���¦�Ok���þ�Ø�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 211: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/211.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�êr�À� :
(1)Ñ�A���Cu"�@̤©.
(2)ÀJr¦�cr�A���Ú3p�A��oÚ¥¤Ó�'~(¡�\O�zÇ)��ýk�½��. �X, ÀJ���r¦�∑r
i=1 λi∑pi=1 λi
> 0.85.
Tianxiao Pang 1nÙ £8ëê��O
![Page 212: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/212.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.9.1 (Y~3.8.1){I²Lêâ©Û¯K.
k?1~1����¦�O�õ��5�ä:
yx=read.table(”ex p68 data.txt”)x1=yx[, 1]x2=yx[, 2]x3=yx[, 3]y=yx[, 4]economy=data.frame(x1,x2,x3,y)economylm.sol=lm(y∼x1+x2+x3,data=economy)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 213: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/213.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
X=cbind(x1,x2,x3)rho=cor(X)rholibrary(DAAG)vif(lm.sol)eigen(rho)kappa(rho,exact=TRUE)
Tianxiao Pang 1nÙ £8ëê��O
![Page 214: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/214.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��Øõ��5�K�, �̤©£8:
economy.pr=princomp(∼x1+x2+x3,data=economy,cor=TRUE)summary(economy.pr,loadings=TRUE)
1n�A��λ3 = 0.05187378392 = 0.00269 ≈ 0.
Tianxiao Pang 1nÙ £8ëê��O
![Page 215: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/215.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éA�n�IO��zA��þ�
φ1 = (−0.706, 0,−0.707)′,
φ2 = (0,−0.999, 0)′,
φ3 = (0.707, 0,−0.707)′.
n�̤©©O�
z1 = −0.706x1 − 0.707x3,
z2 = −0.999x2,
z3 = 0.707x1 − 0.707x3.
Ï�1��A���\O�zÇ�0.666385 ≤ 0.85, cü�A���\O�zÇ0.9991030 > 0.85, ¤±·�í�1n�̤©,��3cü�̤©.
Tianxiao Pang 1nÙ £8ëê��O
![Page 216: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/216.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
O�̤©�©(=#Cþ�*ÿ��þ):
pre=predict(economy.pr)pre
Tianxiao Pang 1nÙ £8ëê��O
![Page 217: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/217.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
?1̤©�O:
z1=pre[, 1];z2=pre[, 2]yxs=scale(yx)y=yxs[, 4]mydata=data.frame(y,z1,z2)pc.sol=lm(y∼0+z1+z2,data=mydata)summary(pc.sol)
Tianxiao Pang 1nÙ £8ëê��O
![Page 218: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/218.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�̤©£8�§:
u = −0.65787z1 − 0.1824z2
= −0.65787× (−0.706x∗1 − 0.707x∗3)− 0.1824× (−0.999x∗2)
= 0.46446x∗1 + 0.18222x∗2 + 0.46511x∗3.
Tianxiao Pang 1nÙ £8ëê��O
![Page 219: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/219.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5¿±þ�IOzCþ�£8�§. =z��©Cþ�£8�§,�
y − 21.891
4.544= 0.46446× x1 − 194.591
30.000+ 0.18222× x2 − 3.300
1.649
+0.46511× x3 − 139.736
20.634,
=y = −7.768 + 0.070x1 + 0.502x2 + 0.102x3.
ùp�y, x1, x2, x3L«�©Cþ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 220: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/220.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
eL�Ñ���¦�O!*�OṲ́©�O�'�:
�{ ~ê� x1 x2 x3
���¦�O -10.128 -0.051 0.587 0.287
*�O(k = 0.04) -8.655 0.063 0.587 0.117
̤©�O(r = 2) -7.768 0.070 0.502 0.102
o�5`, *�OṲ́©�O'��C. ����¦�O�',*�OṲ́©�OÑ�ؽ�)õ��5¤�5�K�,¤±x1�£8Xê�ÎÒ�u)Cz.
Tianxiao Pang 1nÙ £8ëê��O
![Page 221: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/221.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SteinØ �O
*�OṲ́©�Oùü«k �O´é���¦�Oβ��:�·��Ø . �¦�éëê�Oβ��©þ��´�þ!Ø .�!òÄuStein�Ø g�?Ø���¦�Oβ�þ!Ø �O, T�O´Stein31955cJÑ5�.
Tianxiao Pang 1nÙ £8ëê��O
![Page 222: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/222.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â (Stein�O)
3�5£8�.(3.1.5)¥, Ù£8Xêβ����¦�O�β =(X
′X)−1X
′Y . ·�¡
βs(c) = cβ
�Stein�O, Ù¥0 ≤ c ≤ 1�¡�Ø Xê.
w,, βs(c)´éβ�z�©þ����Ø , ¤±Stein�O´�«þ!Ø �O.
Tianxiao Pang 1nÙ £8ëê��O
![Page 223: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/223.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SteinØ �O�5� :
5�1.�c 6= 1�, βs(c)´β�k !Ø �O.
5�2.�30 < c < 1, ¦�MSE(βs(c)) < MSE(β).
y²: βs(c)�þ�Ø��
MSE(βs(c)) = tr[Cov(βs(c))] + ‖E(βs(c))− β‖2
= c2σ2tr[(X′X)−1] + (c− 1)2‖β‖2
= c2σ2p∑i=1
λ−1i + (c− 1)2‖β‖2
∆= g(c).
ùp, ��Øþj�K�, ·�b�gCþÚÏCþþ®IOz, Ïd�OÝX�n× pÝ.
Tianxiao Pang 1nÙ £8ëê��O
![Page 224: Zhejiang Universitypan.hohoweiya.xyz/regression-slides/chap 3.pdf · Zhejiang University September 28, 2016 Tianxiao Pang 1nÙ £8ºŒ˙ O. LSELSE˙5 åLSE ä BCCƒ GLSE ı› ‡5](https://reader033.vdocuments.site/reader033/viewer/2022052817/60b11229a904c85e0e6d8d4d/html5/thumbnails/224.jpg)
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ég(c)'uc¦�¿-Ù�u"�)�c��`��
c∗ =‖β‖2
σ2∑p
i=1 λ−1i + ‖β‖2
< 1. (3.10.1)
dug(c)'uc����ê�u
2σ2p∑i=1
λ−1i + 2‖β‖2 > 0,
Ïdg(c) = MSE(βs(c))3c∗?����, ¿��c∗ ≤ c < 1�,
kMSE(βs(c)) < MSE(β).
Tianxiao Pang 1nÙ £8ëê��O