projectors and linear estimation in general linear models
TRANSCRIPT
This article was downloaded by: [Tufts University]On: 02 December 2014, At: 10:35Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK
Communications in Statistics - Theory andMethodsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/lsta20
Projectors and linear estimation in general linearmodelsRadostaw Kala aa Department of Mathematical and Statistical Methods , Academy of Agriculture ,Poznań, 60-637, PolandPublished online: 27 Jun 2007.
To cite this article: Radostaw Kala (1981) Projectors and linear estimation in general linear models, Communicationsin Statistics - Theory and Methods, 10:9, 849-873, DOI: 10.1080/03610928108828078
To link to this article: http://dx.doi.org/10.1080/03610928108828078
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy, completeness, or suitabilityfor any purpose of the Content. Any opinions and views expressed in this publication are the opinionsand views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy ofthe Content should not be relied upon and should be independently verified with primary sources ofinformation. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands,costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantial orsystematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distributionin any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions
COMMUN. STATIST.-THEOR. METII., A10(9), 849-873 (1981)
PROJECTORS AND LINEAR ESTIMATION I N GENERAL LINEAR MODELS
Department o f Ma themat i ca l and S t a t i s t i c a l Methods Academy o f A g r i c u l t u r e , 60-637 Poznai5, Po land
Key b?ords and Phrases: general Gauss-Markov model; minimm disper7sion linear unbiased estimator; simple least squares estimator.
ABSTRACT
The paper g i v e s a s e l f - c o n t a i n e d accoun t o f minimum d i s p e r -
s i o n l i n e a r unb iased e s t i m a t i o n o f t h e e x p e c t a t i o n v e c t o r i n a
l i n e a r model w i t h t h e d i s p e r s i o n m a t r i x b e l o n g i n g t o some, r a t h e r
a r b i t r a r y , s e t o f nonnega t i ve d e f i n i t e m a t r i c e s . The approach t o
l i n e a r e s t i m a t i o n i n genera l l i n e a r models recommended h e r e i s a
d i r e c t g e n e r a l i z a t i o n o f some ideas and r e s u l t s p r e s e n t e d by Rao
( 1 9 7 3 , 1 9 7 4 ) f o r t h e case o f a genera l Gauss-Markov model.
A new i n s i g h t i n t o t h e n a t u r e o f some e s t i m a t i o n prob lems
o r i g i n a l y a r i s i n g i n t h e c o n t e x t o f a genera l Gauss-Markov model
as we1 l as t h e cor respondence o f r e s u l t s known i n t h e 1 i t e r a t u r e
t o those o b t a i n e d i n t h e p r e s e n t paper f o r genera l l i n e a r models
a r e a l s o g i v e n . As p r e l i m i n a r y r e s u l t s t h e t h e o r y o f p r o j e c t o r s
d e f i n e d by Rao ( 1 9 7 3 ) i s ex tended.
1 . INTRODUCTION AND SUMMARY
The t h e o r y o f l i n e a r e s t i m a t i o n i n l i n e a r models has been
t r e a t e d i n t h e l i t e r a t u r e by u s i n g v a r i o u s methods. The deve lop -
Copyright O 198 1 by Marcel Dekker, Inc.
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
KALA
ment o f the methods s t a r t s f rom the most e lementary techniques,
reviewed r e c e n t l y by Kempthorne (1976) , and con t inues through the
v a r i o u s a l g e b r a i c a l methods proposed by Goldman and Zelen (19641,
Zysk ind and M a r t i n (1969) and Rao and M i t r a (1971) , u n t i l t he
f u l l y geometr ica l methods i n i t i a t e d by Kruskal (1961) . I t seems,
however, t h a t the most a p p r o p r i a t e method i s t h a t which b r i n g s t o -
ge ther the s i m p l i c i t y o f e lementary c o n s i d e r a t i o n s , the p r a c t i c a l
usefu lness o f a l g e b r a i c a l s o l u t i o n s and the elegance o f the geo-
m e t r i c a l approach. For the case o f a general Gauss-Markov model,
such a p r e s e n t a t i o n o f e s t i m a t i o n theory has been ob ta ined by Rao
(1973, 19741, who used s u i t a b l y d e f i n e d p r o j e c t o r s as a main t o o l .
The o b j e c t o f the p resen t paper i s t o g i v e a s e l f - c o n t a i n e d
account o f the geomet r i ca l method o f l i n e a r e s t i m a t i o n o f the ex-
p e c t a t i o n v e c t o r i n a l i n e a r model w i t h the d i s p e r s i o n m a t r i x be-
long ing t o some, r a t h e r a r b i t r a r y , s e t o f nonnegat ive d e f i n i t e ma-
t r i c e s , and thus t o extend t o a w ider c l a s s o f l i n e a r models the
approach proposed by Rao.
Sec t ion 2 i s devoted t o an ex tens ion o f the theory o f p r o j e c -
t o r s in t roduced by Rao (1973) . Sec t ion 3 comprises the main re -
s u l t s on e s t i m a t i o n o f the e x p e c t a t i o n v e c t o r i n a general Gauss
-Markov model expressed, f o l l o w i n g Rao ( 1973, 1 9 7 4 ) ~ i n the lan -
guage o f p r o j e c t o r s . I n Sec t ion 4 and 5 t h i s geometr ica l approach
i s adopted i n d e r i v i n g a necessary and s u f f i c i e n t c o n d i t i o n f o r
the ex is tence o f the minimum d i s p e r s i o n l i n e a r unbiased e s t i m a t o r
o f the e x p e c t a t i o n v e c t o r i n a genera l l i n e a r model and i n charac-
t e r i z i n g the c l a s s o f a l l p r o j e c t o r s lead ing t o such an e s t i m a t o r ,
i f i t e x i s t s i n the model. Moreover, i n Sec t ion 5, the correspon-
dence i s shown between t h e r e s u l t s ob ta ined i n the p resen t paper
and those known i n the l i t e r a t u r e . The l a s t s e c t i o n , Sec t ion 6,
g i ves a new i n s i g h t i n t o the na tu re o f s o l u t i o n s o f th ree est ima-
t i o n problems f r e q u e n t l y a r i s i n g i n the c o n t e x t o f a genera l Gauss
-Markov model, one o f which i s d iscussed i n t h e l i t e r a t u r e s i n c e
1948.
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS
2. PROJECTORS AND THEIR PROPERTIES
I n t h i s s e c t i o n we extend the theory o f l i n e a r t rans fo rm-
a t i o n s c a l l e d p r o j e c t o r s , which appears t o be very u s e f u l i n s o l v -
i n g the e s t i m a t i o n problems i n l i n e a r models. The a t t r a c t i v e n e s s
o f such t rans fo rmat ions f o r the l i n e a r s t a t i s t i c a l i n f e r e n c e has
been p o i n t e d o u t by Rao ( 1 9 7 4 ) , who expressed the b a s i c r e s u l t s o f
the l i n e a r e s t i m a t i o n i n a Gauss-Markov model i n the e legan t geo-
m e t r i c a l language o f p r o j e c t o r s .
I n the sequel we wi 1 l say t h a t X ( A ) and X ( B ) , the column
space o f the nxp m a t r i x A and the nxq m a t r i x B , r e s p e c t i v e l y , a r e
d i s j o i n t subspaces o f Rn, t h e n-d iment ional Eucl idean space, i f
t h e i r i n t e r s e c t i o n i s the n u l l v e c t o r , i .e . , i f
Moreover, we w i l l say t h a t X ( A ) and X ( B ) are complementary i f they
a re d i s j o i n t and t h e column space o f the p a r t i t i o n m a t r i x ( A : B )
co inc ides w i t h Rn, i .e. , i f
X ( A ) n X ( B ) = { O ) and X ( A : B ) = Rn.
Now we i n t r o d u c e the n o t i o n o f a p r o j e c t o r , as considered by
Rao ( 1 9 7 3 ) .
Def i n i t i on 2 . 1 . Let X( A ) and X ( B ) be d i s j o i n t subspaces o f Rn
and l e t for every x E X ( A : B ) ,
represents the unique decomposition, such t h a t y E X ( A ) and z E
R( B ) . Then y i s sa id t o be t he pro jec t ion o f x onto X ( A ) along
X( B ) , and a matr ix P which transforms any z E X ( A : B ) i n t o i t s
p ro j ec t i on y f X( A ) i s sa id t o be a pro jec tor onto R ( A ) along X ( B ) .
v The d i f f e r e n c e between a p r o j e c t o r descr ibed i n D e f i n i t i o n
2 . 1 and t h a t considered i n the l i t e r a t u r e ( e . g . i n Ben- Israel and
G r e v i l l e , 1 9 7 4 , p. 50) l i e s i n the assumption on the subspaces
X ( A ) and X ( B ) . I n our case these subspaces a r e d i s j o i n t , w h i l e
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
852 KALA
cus tomar i l y they a re assumed t o be complementary. The weakening
o f t h i s assumption causes (see Rao, 1574) t h a t , i n genera l , such
an extended p r o j e c t o r need no t be unique o r idempotent. Neverthe-
less , i t can be s imply c h a r a c t e r i z e d as a s o l u t i o n o f a m a t r i x
equat ion, as shown i n the f o l l o w i n g l e m a , due t o Rao ( 1 9 7 3 ) .
Lemma 2 . 1 . Let R( A ) and R( B ) be d isc jo in t subspaces o f Rn.
Then a matrix P i s a projector onto X(A) along R ( s ) i f and only
i s PA = A , PB = O . ( 2 . 1 )
Proo f . Since X ( A ) and R ( R ) a r e d i s j o i n t , any vec to r x €
X(A:h) can be decomposed un ique ly as x
z = Bb f o r some vec to rs a and b. Then,
P i s a p r o j e c t o r on to X(A) a long X ( B ) i
Aa f o r a l l vec to rs a and b . Th is r e l a t i '
t o the equa t ion ( 2 . 1 ) , which completes
= y + z, where y = Aa and
by D e f i n i t i o n 2 . 1 , a m a t r i x
f and o n l y i f P ( A a + Bb) =
on, however, i s e q u i v a l e n t
the p r o o f . V
The s e t o f a l l s o l u t i o n s o f
symbol { P A i B } , i .e.,
{PA,,} = {P: PA = A , PB = 0)
The correspondence between the se
( 2 . 1 ) wi l l be denoted by the
j e c t o r s on to R ( A ) a long R ( B ) i s g iven i n the nex t lemma
with the s e t o f a l l projecto
Proo f . The s e t { P A I B } i s
( 2 . 1 ) has a s o l u t i o n , f o r wh
t h a t
Lemma 2 . 2 . The s e t {PA I B } i s ncnerpty i f and only i f the
s-&spaces X( A ) and R( B) are d i s j o i n t , i n which case it coincides
r s onto R ( A ) along JlZ(B).
nonempty i f and o n l y i f the equa t ion
i c h i t i s necessary and s u f f i c i e n t
where pr imes denote transposes o f mat r i ces . The i n c l u s i o n (2.31,
however, i s e q u i v a l e n t t o the statement t h a t f o r any v e c t o r a and
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS
b such t h a t Aa + Bb = 0 t h e r e i s a l s o Aa = 0. Since t h i s s t a t e -
ment i s a c t u a l l y the necessary and s u f f i c i e n t c o n d i t i o n f o r d i s -
j o i n t e n e s s o f B ( A ) and X ( U ) , the f i r s t p a r t o f t h e lemma i s es-
t a b l i s h e d . For the r e s t o f the p r o o f we observe t h a t t h e second
p a r t o f the lemma f o l l o w s d i r e c t l y f rom Lemma 2 .1 . V
An e x p l i c i t e r e p r e s e n t a t i o n o f elements o f the s e t ( ' L I B "
o r e q u i v a l e n t l y o f the s e t o f a l l p r o j e c t o r s o n t o X ( A ) a long
X ( E ) , revea ls the lemma below. I n what f o l l o w s we use A- and A~
t o des igna te , r e s p e c t i v e l y , a g - inverse o f A and a m a t r i x o f
maximum rank such t h a t L'AL = P .
Lemma 2 . 3 . I f X ( A ) and K( B 1 ure d is , jo in t subspaces o.f Rn,
then P (1 { P A , Y} if and only if
P = A ( S ~ ' A ) - & ' + u ( A : B ) ' ' ( 2 . 4 )
Proo f . The r e s u l t f o l l o w s f rom Lemma 2 . 6 o f Rao ( 1 9 7 4 ) and
the o b s e r v a t i o n t h a t the second term on t h e r i g h t hand s i d e i n
' 2 . 4 i s a genera l s o l u t i o n o f the equa t ion ?(AL:?) = 0, w i t h r e -
spect t o T. 0
I n view o f the r e p r e s e n t a t i o n ( 2 . 4 ) i t i s easy t o c o n f i r m
the f a c t , mentioned a f t e r D e f i n i t i o n 2 . 1 , t h a t a p r o j e c t o r here
considered i s n e i t h e r un ique nor idempotent. However, i n t h e case
o f complementary subspaces X ( A ) and X ( B ) t h e f o l l o w i n g w e l l known
p r o p e r t i e s ho ld .
Lemma 2 .4 . I f jll( A ) and JR( B ) are complementary subspaces o f
Rn, then
( a ) there e x i s t s e xac t l y one projec tor onto IR(A) along X( B);
t h i s p r ~ ~ j e c t o r w i Z 2 be denoted by P A I R , ( b ) p A I H = A ( B ~ ' A ) - B ~ ' , ( c ) PA is idempotent.
Proo f . ( a ) and ( c j f o l l o w f rom Theorem 8 o f Ben- Is rae l and
G r e v i l l e (1974, p. 5 0 ) . ( b ) i s a consequence o f formula ( 2 . 4 ) and
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
KALA
the observa t ion t h a t i f X ( A ) and X ( B ) a r e complementary, then
( A : B ) ~ = 0 . v
The nex t lemma prov ides the c o n d i t i o n under which t h e i n c l u -
s i o n r e l a t i o n between two se ts o f p r o j e c t o r s can be e s t a b l i s h e d .
Lemma 2 . 5 . L e t X ( C ) and X ( D ) be d i s j o i n t subspaces of Rn.
Then
Proo f . I f ( 2 . 5 ) ho lds , then, on account o f Lemma 2 .3 and 2.1,
every p r o j e c t o r
s a t i s f i e s a l s o ( 2 . 1 ) . S u b s t i t u t i n g ( 2 . 7 ) , w i t h U = 0 , i n t o the
f i r s t equa t ion i n ( 2 . 1 ) g i v e s c ( D ~ ' c ) - D ~ ' A = A , and t h e r e f o r e
X ( A ) - c R ( c ) .
On the o t h e r hand, s u b s t i t u t i n g ( 2 . 7 ) i n t o the second equa-
t i o n i n ( 2 . 1 ) g i v e s
Since U i s an a r b i t r a r y m a t r i x , the l a s t e q u a l i t y i m p l i e s , i n
p a r t i c u l a r , t h a t
and t h a t
From ( 2 . 9 ) i t f o l l o w s t h a t B has a r e p r e s e n t a t i o n B = CR + D S ,
f o r some mat r i ces R and S, which combined w i t h ( 2 . 8 ) and t h e f a c t
D ~ ~ D = O g i v e s
Therefore, the i n c l u s i o n X ( B ) c X ( D ) i s e s t a b l i s h e d . -
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS
Conversely, l e t (2.6) h o l d and l e t P E {PCID}, which se t i s
nonempty, s ince R(C) and R(D) a r e d i s j o i n t subspaces. Then, f o r
some mat r i ces R and S, A = CR, B = DS and, i n v iew o f Lemma 2.1,
PC = C (2.10)
and
PD = 0. (2.11)
Thus, p o s t m u l t i p l y i n g (2.10) by R and (2.11) by S i m p l i e s PA = A
and PB = 0 , which, by Lemma 2 . 1 again, i s e q u i v a l e n t t o the s t a t e -
ment t h a t P E {PAIB}. V
I t i s o f some i n t e r e s t t o no te t h a t t h e assumption on t h e
d i s j o i n t e n e s s o f subspaces R(C) and X(D) i s n o t necessary t o
show the i m p l i c a t i o n from (2.6) t o (2.5). T h i s i s a consequence
o f the f a c t t h a t i f R(C) f! X(D) # {0} then, on account o f Lemma
2 . 2 , {PCID} i s an empty se t , and thus, the r e l a t i o n (2.5) o b v i -
o u s l y ho lds .
Using the p reced ing lemma and the remark above, we can now
prove the f o l l o w i n g r e s u l t .
Lemma 2 .6 . Let A be an nxp matr ix and for r = I , . . . , t Zet
B, be an nxq, matr ix . Then t
{'AI(B~: ... : B ~ ) } s r 2 1 { P ~ ~ ~ , } y (2.12)
and i f X( A ) and X( B1 : . . . : Bt ) are d i s j o i n t subspaces o f Rn, then t n_ {PAIB,J # 0. r-1
Proo f . I n v iew o f the obv ious i n c
r = I , ..., t, i t f o l l o w s f rom Lemma 2
p r o o f o f i t t h a t
{'AI(R~:.. . : B ~ ) } 2 { P ~ ~ ~ , } ' =
us ions X((B,) c X ( B 1 : ... :Bt), 5 and t h e remark below the
T h i s e s t a b l i s h e s (2.12). To p rove (2.13) i t su
t h a t , i f subspaces X(A) and N(B1: . . . :Bt) a r e d
Lemma 2.2, the s e t on the l e f t hand s i d e i n (2
f f i ces t o observe
i s j o i n t , then, by
.12) i s nonempty.
v
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
85 6
The nex t resu
se ts o f p r o j e c t o r s
Lemma 2.7 . Le
KALA
I t i s a l s o concerned w i t h t h e i n t e r s e c t i o n o f
t A be an nxp matrix and f i r r = I , . . . , t l e t
B, and C , be, r e spec t i ue l y , t he nxq, and g,xs matr ices . Then
Proof . I f the se t on the l e f t hand s ide i n ( 2 . 1 4 ) i s an empty
s e t , then the i n c l u s i o n holds. Thus, we assume t h a t t h i s set i s
nonempty and t h a t P i s any o f i t s elements. Then, i t f o l l o w s from
Lemma 2.1 t h a t PA = A and P B , = 0 f o r r = 1 , . . . , t. The l a s t
equat ions imply P ( B I C 1 + ...+ B T C t ) = 0 , which combined w i t h
PA = A g ives
Th is e s t a b l i s h e s ( 2 . 1 4 1 , thus complet ing the p r o o f . V
An i n t e r e s t i n g s i m p l i f i c a t - i o n o f Lemma 2.7 can be ob ta ined i f
we l e t s = q , and q , = q f o r r = 1 , . . . , t , and a l s o l e t C, = a,I,
where a , represents a s c a l a r w h i l e I, the i d e n t i t y m a t r i x . A f t e r
such m o d i f i c a t i o n s the lemma above can be expressed as f o l l o w s .
Lemma 2.8. Let A be an nxp matr ix and for r = I , ..., t l e t
B, be an nxq matr ix . Then
for any scalars a l , ..., a t - V
For the end o f t h i s s e c t i o n we r e c a l l the n o t i o n o f the o r -
thogonal p r o j e c t o r and then we d e r i v e the w e l l known p r o p e r t i e s o f
such p r o j e c t o r s .
D e f i n i t i o n 2 .2 . Let R ( A ) be a subspace o f Rn. Then the pro-
jec tor onto X i A ) along J Z ( A ~ ) i s said t o be the orthogonal projec-
t o r onto R i A ) . V
The n o t i o n o f o r t h o g o n a l i t y used i n D e f i n i t i o n 2.2 i s l i n k e d
w i t h the concept o f o r t h o g o n a l i t y o f subspaces X ( A ) and x (A ' ) , which, i n our case, a r e or thogonal i n the usual sense o f the
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS 857
s t a n d a r d i n n e r p r o d u c t i n k n . The common p r o p e r t i e s o f t h e o r t h o -
gona l p r o j e c t o r s a r e c o l l e c t e d i n t h e f o l l o w i n g .
Lemma 2 . 9 . Let ? ; ( A ) be c subspace of Rn. Then
( u i there e x i s t s c r u c t l y one orthogonal pro jec tor o n t o ?(A!;
t h i s proj'eczor uiZ? be denoted by PA , ( s ) = A ( A I A ) - A I , ( C ) PA is i&yote l ; t an2 s y m e t i ~ i o .
P r o o f . ( a ! i s an immedia te consequence o f Lemma 2 . 4 ( a ) and
t h e f a c t t h a t R ( h ) and x ( A ~ ) a r e complementary subspaces o f R".
( b ) f o l l o w s fromi Lemma 2 . 4 ( b l and t h e o b s e r v a t i o n t h a t one p o s s i b l e
c h o i c e c f ( h i ) i i s A i t s e l f . !c) i s easy t o v e r i f y u s i n g ( b ) and
t h e b a s i c p r o p e r t i e s o f g - i n v e r s e s . V
3. ESTIMATION OF THE EXPECTATION VECTOR IN GENERAL GAUSS-MARKOV MODELS
A genera l Gauss-Markov model i s o r d i n a r i l y denoted by t h e
t r i p l e t
where y i s an o b s e r v a b l e random ? - v e c t o r w i t h t h e e x p e c t a t i o n
E ( y ) = Yg and t h e d i s p e r s i o n m a t r i x D(: l ' = - . I n t h i s se t -up * i s
an q x p known m a t r i x o f a r b i t r a r y rank , [ i s a p - v e c t o r o f unknown
parameters and V i s an n x a nonnega t i ve d e f i n i t e m a t r i x known en-
t i r e l y o r e x c e p t f o r a p o s i t i v e s c a l a r m u l t i p l i e r .
One o f t h e main prob lems c o n s i d e r e d i n t h e l i t e r a t u r e devo ted
t o t h e t h e o r y o f Gauss-Markov model i s t h e p rob lem o f e s t i m a t i n g
l i n e a r l y t h e e x p e c t a t i o n o f p . I t s s o l u t i o n i s w e l l known n o t o n l y
f o r v a r i o u s p a r t i c u l a r cases o f t h e model d e s c r i b e d b u t a l s o f o r
t h e case o f i t s g e n e r a l f o r m u l a t i o n . D e s c r i p t i o n s o f t h e s o l u t i o n
can be found i n t h e s t a n d a r d t e x t s on s t a t i s t i c a l i n f e r e n c e ( s e e
e.g. A l b e r t , 1 9 7 2 ; Rao, 1 9 7 3 ~ ' . N e v e r t h e l e s s , we c o l l e c t he re main
r e s u l t s o f t h e t h e o r y , t r e a t i n g t h i s s e c t i o n as a base f o r f u r t h e r
genera l i z i n g c o n s i d e r a t i o n s , f o r wh ich t h e model {y, XP, V ) ap-
pea rs t o be a s p e c i a l case.
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
KALA
D e f i n i t i o n 3 . 1 . In a genera2 Zinear Gauss-Markov mode2
{ y , X B , V ) a s t a t i s t i c Py, where P i s an nxn matr ix , i s cg22ed the
Minimwn Dispersion Linear Unbiased Estimator (MDLUE) o f X4 i f i t
i s unbiased .for XP and i f for any o ther Zinear unbiased es t imator
o f XP, say By,
D(By) - D(?I) 2 0,
the i nequa l i t y meaning t h a t the matr ix D(By) - D(P!y) i s nonnega-
t i v e d e f i n i t e . v
The key t o determine the c l a s s o f a l l t rans fo rmat ions P
l e a d i n g t o the MDLUE o f XP i s the c r i t e r i o n which appears i n the
paper by Seely and Zysk ind ( 1 9 7 1 ) , a t t r i b u t e d t o E . Lehmann and
H. Schef fe by t h e former au thors . Th is c r i t e r i o n can a l s o be found
i n the paper by Rao (19731, who, however, a t t r i b u t e s i t s o r i g i n t o
R . A. F i sher . We p resen t i t here w i t h o u t p r o o f .
Theorem 3.1 . I n a genera2 Gauss-Markov model { y , X B , V i , Pg
i s the MDLUE o f X B i f and only i f
I t should be observed t h a t the cons is tency o f ( 3 . 1 ) i s ac-
t u a l l y a necessary and s u f f i c i e n t c o n d i t i o n f o r the e x i s t e n c e o f
the MDLUE o f X p , which, i n view o f Lemma 2 . 2 , i s e q u i v a l e n t t o the
d i s j o i n t e n e s s o f subspaces X ( X ) and JK(vxL). I n the case o f the gen-
e r a l Gauss-Markov model { y , XP, V ) , however, t h e MDLUE o f X C- a l -
ways e x i s t s i n v iew o f the f o l l o w i n g lemma g iven by Rao (1974) .
Lemma 3 . 1 . Let X be an nxp matrix and l e t V be an nxn nomega-
t i v e d e f i n i t e matrix . Then X ( X ) and TZ( VX') are d i s j o i n t subspaces
o f Rn. v
Taking i n t o account the remark above and D e f i n i t i o n 2.1, we
can express the r e s u l t o f Theorem 3.1 as f o l l o w s .
Theorem 3.2. Let { y , X B , V ) be a genera2 Gauss-Markov model.
Then
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS
( a ) the MDLUE o f X4 e x i s t s ,
( b ) Py i s t he MDLUE o f XR i f and only i f P E {PXI v x ~ } . V
The genera l r e p r e s e n t a t i o n o f t rans fo rmat ions l e a d i n g t o the
MDLUE o f XP can now e a s i l y be d e r i v e d u s i n g Lemma 2.3, by which a
ge:leral form o f p r o j e c t o r s f rom t h e s e t {P I) i s o b t a i n a b l e . XI VX Never theless, we c h a r a c t e r i z e elements o f t h i s s e t w i t h the h e l p
o f Theorem 8(b) o f Rao and Yanai ( 1 9 7 9 ) ( s e e a l s o Rao, 1973, 1978),
which appears t o be more f r u i t f u l i n f u r t h e r c o n s i d e r a t i o n s .
Theorem 3 .3 . Let X be an nxp ma t r i x and l e t V be an nxn non-
negative d e f i n i t e matrix . Then t he general soZution o f the
eqilation
can be expressed as
P = x(x~T-x)-x~T- + A(I - TT-1,
where A i s an a rb i t ra ry matr ix , T = V + XUX' and U i s any symmetric
mat&x such t h a t R ( T ) = X(V:X). V
Al though Theorem 3 . 3 i s fo rmu la ted i n t h e sense o f g i v i n g a
genera l s o l u t i o n o f the m a t r i x equa t ion (3.2), i t f o l l o w s f rom
Lemma 2.2 and t h e d i s j o i n t e n e s s o f subspaces X(X) and XZ(vXL), t h a t
the theorem g ives a t t h e same t ime a genera l form o f a l l p r o j e c t o r s
o n t o R(X) a long X(VxL) and thus c h a r a c t e r i z e s the elements o f the
se t IPXIvx~J.
4 . ESTIMATION OF THE EXPECTATION VECTOR IN GENERAL LINEAR MODELS
A model, i n which t h e problem o f the l i n e a r e s t i m a t i o n o f t h e
e x p e c t a t i o n v e c t o r w i l l now be d iscussed, i s determined by t h e
usual moment represen ta t i o n
where y i s an observable random n - v e c t o r , X i s an nxp known m a t r i x ,
p i s a p - v e c t o r o f unknown parameters, V1, . . . , Vt a r e known non-
n e g a t i v e d e f i n i t e nxn m a t r i c e s and Al, ..., A t a r e unknown s c a l a r s .
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
860 KALA
I t i s easy t o observe t h a t the d i f f e r e n c e between t h e general
Gauss-Markov model considered i n the p rev ious s e c t i o n and the
model desc r ibed by ( 4 . 1 ) l i e s i n the s t r u c t u r e o f the d i s p e r s i o n
m a t r i x D ( y ) , which ranges now over the set
D = { V : V = hlVl + . . . + htVt 2 0, h, E R} ( 4 . 2 )
o f nonnegative d e f i n i t e mat r i ces , r a t h e r than i s p r o p o r t i o n a l t o
a g iven m a t r i x V. I n the sequel the s e t o f c o n d i t i o n s ( 4 . 1 ) t o -
ge ther w i t h ( 4 . 2 ) w i l l be denoted by the t r i p l e t
{Y, XD, D}
and w i l l be c a l l e d a general l i n e a r model. To avo id redundant e l -
ements i n the s p e c i f i c a t i o n o f { y ,
t h a t the mat r i ces V 1 , . . ., Vt a r e
t h a t
dim {V: V = alVl + . . . + atVt
XD, D}, we assume moreover
i n e a r l y independent, i . e . ,
Under the genera l set -up descr ibed above we w i l l now be con-
cerned w i t h the two r e l a t e d problems d iscussed b r i e f l y i n t h e p r e -
v ious s e c t i o n f o r the case o f a general Gauss-Markov model. The
f i r s t o f them p e r t a i n s t o the ex is tence o f t h e MDLUE o f Xa, which
i s meant here i n accordance w i t h the f o l l o w i n g
Def i n i t i o n 4.1 . In a genera2 l i near mode2 { y , X P , D) a s t a t -
i s t i c Py, where P i s an nxn m a t ~ i x , i s ca22ed t he minimum disper-
s ion l i near unbiased est imator o f XR i f for any V E D FLJ i s t he
MDLUE o f X p i n the mode2 { y , X B , If}. V
The second problem can be fo rmu la ted as a genera l c h a r a c t e r -
i z a t i o n o f a l l t rans fo rmat ions lead ing t o the MDLUE o f X P , pro -
v ided such e s t i m a t o r i n { y , X P , D} e x i s t s .
For the f i r s t problem, i n c o n t r a r y t o t h e second, t h e r e a r e
a l r e a d y known s o l u t i o n s ob ta ined e i t h e r i n v a r i o u s p a r t i c u l a r
cases o f the 1 i n e a r model { y , X P , D) (see Eaton, 1970; Seely and
Zyskind, 1971; M i t r a and Moore, 1973) o r i n the genera l framework
o f the model (see M i t r a and Moore, 1976). Never the less, we so lve
i t again here, p r e s e n t i n g e s s e n t i a l y new geometr ica l approach
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS 86 1
which i s a n a t u r a l e x t e n s i o n o f the c o n s i d e r a t i o n s o f t h e p rev ious
s e c t i o n f o r the Gauss-Markov model. The second problem mentioned
above w i l l a l s o be d iscussed here.
Theorem 4.1. Let {y, X 8 D) be a genernl linear model. Then
(a) the MDLUE of X D exists if and only if t
C' WXlV & # @, r=l
( 4 . 3 )
! b ) if (a! is the case, Py 7:s the MDLUE of X D if an2 only if
p c n { p X l V Xl}. (4.4 r=1 l-
Proo f . Since f o r r = I, ..., t V, E G, the e x i s t e n c e o f the
MDLUE o f i? i n model {y, XO, D) i m p l i e s , i n v iew o f D e f i n i t i o n 4.
t h a t t h e r e e x i s t s a m a t r i x P such t h a t Py i s the MDLUE o f X E i n
each genera l Gauss-Markov model {y, XD, V,), f o r r = 1 , . .., t. Therefore, i t f o l l o w s f rom Theorem 3.2 t h a t
E { P , f o r r = 1 , . . . , t , r"
thus e s t a b l i s h i n g ( 4 . 4 ) and ( 4 . 3 ) .
To prove the converse i m p l i c a t i o n l e t assume ( 4 . 3 ) and l e t P
be any m a t r i x s a t i s f y i n g ( 4 . 4 ) . Then, from Lemma 2 .8 , w i t h A = X
and f o r r = I, . . . , t w i t h B, = Vrzi and a, = h,, i t f o l l o w s t h a t
which i s t r u e f o r any s e t o f s c a l a r s {Al, ..., A*}. I n consequence
P E { P X j m ~ ) f o r a l l V E D and thus, i n v iew o f Theorem 3.2 and
D e f i n i t i o n 4.1, % i s the MDLUE o f XO i n the genera l l i n e a r model
{y, XO, Dl. v
More p r e c i s e i n s p e c t i o n o f the
t h a t Theorem 4.1 e s t a b l i s h e s a c t u a l
t
"5, = ,$ {PXI V r X ~ ) ' VED
I t i s i n t e r e s t i n g t o no te t h a t
p r o o f above a l l o w s us t o s t a t e
l y the a q u a l i t y
(4.5)
nonemptiness o f t h e se t on t h e
l e f t hand s i d e i n ( 4 . 5 ) i s , by D e f i n i t i o n 4.1 and Theorem 3.2,
e q u i v a l e n t t o the e x i s t e n c e o f the MDLUE o f X[3 i n a genera l l i n e a r
model {y, XP, D), and, moreover, i f t h i s i s t h e case t h e s e t c o i n -
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
862 KALA
c ides w i t h the s e t o f a l l p r o j e c t o r s , which a p p l i e d t o y g i v e the
requ i red MDLUE . Comparing Theorem 4.1 and 3.2 i t i s easy t o observe t h a t the
l a t e r f o l l o w s f rom t h e former by s e t t i n g t = I and us ing Lemma 3.1
toge ther w i t h Lemma 2.2 t o s i m p l i f y the statement ( a ) . On t h e
o t h e r hand, i n t h e case o f a Gauss-Markov model { y , XP, V) Theorem
3.2 g ives , by a p p l y i n g Lemma 2.3, a genera l form o f t rans fo rm-
a t i o n s p r o v i d i n g the MDLUE o f Xg, b u t i n t h e case o f t h e ex is tence
o f the MDLUE o f Xp i n a genera l l i n e a r model {y, XP, D} Theorem
4.1 mere ly shows t h a t a f u l l c h a r a c t e r i z a t i o n o f t rans fo rmat ions
lead ing t o t h e MDLUE o f X P i s g i ven by the se t o f p r o j e c t o r s t h a t
appears on t h e r i g h t hand s i d e i n ( 4 . 5 ) . Thus, the problem o f
f i n d i n g a genera l formula f o r the p r o j e c t o r s lead ing t o t h e MDLUE
i n a general l i n e a r model { y , XP, D), when t > I, i s no t answered
y e t . A l though the s o l u t i o n o f t h i s problem w i l l be g iven i n t h e
next s e c t i o n , we can now g i v e q u i c k answers f o r two s p e c i a l cases
o f t h e model {y, XP, Dl.
C o r o l l a r y 4.1. Let i n a l i near model { y , XB, D) t he MDLUE of
XP e x i s t and l e t V o E D be a pos i t i v e d e f i n i t e matr i z . Then the
s e t o f a22 transformations leading t o the MDLUE of XP contains
the unique projec tor PX , V O X ~ only . Proo f . I f Vo i s p o s i t i v e d e f i n i t e , then TUX) and X(V~X') a r c
complementary subspaces o f Rn and thus, by Lemma 2.4, t h e se t
{PXlvoXL} c o n s i s t s o f one element P X I V o X ~ o n l y . T h i s combined w i t h
the e q u a l i t y ( 4 . 5 ) and the f a c t s t h a t Vo E D and t h a t the MDLUE o f
X R e x i s t s , i m p l i e s t h e r e s u l t . V
The nex t c o r o l l a r y i s an immediate consequence o f C o r o l l a r y
4.1 and D e f i n i t i o n 2.2.
C o r o l l a r y 4.2. Let i n a Linear model { y , XB, 11) the MDLUE of
X B e x i s t and Zet I E D. Then the s e t o f a l l transformations lead-
ing t o the MDLUE o f X b contains the orthogonal pro jec tor PX only .
v
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS
5. PROJECTORS LEADING TO THE MDLUE OF X P
Before d e r i v i n g the general formula f o r t rans fo rmat ions an-
nounced i n t h e t i t l e , we g i v e t h r e e e q u i v a l e n t s o f the c o n d i t i o n
( 4 . 3 ) f o r the e x i s t e n c e o f the MDLUE o f XP. To t h i s end i t w i l l be
convenient t o i n t r o d u c e the f o l l o w i n g concept, which p res8mab ly
was f i r s t used i n t h e con tex t o f a l i n e a r model by LaMotte ( 1 9 7 7 ) .
D e f i n i t i o n 5 . 1 . A matr ix Va E D i s said t o be a
ement i n the s e t D if R( V ) g X I ( V:,:) for a l l I/ E Do
When 3 i s meant as the s e t desc r ibed by (4.2), i
observe t h a t one o f the mat r i ces f u l f i l l i n g the condi
n i t
I t
s e t
maximal e l -
L'
t i s easy t o
t i o n s o f D e f i -
on 5.1 i s the m a t r i x
V l + ... + V, . ( 5 . 1 )
s a l s o easy t o no te t h a t t o determine a maximal element i n t h e
D i t i s n o t , i n genera l , necessary t o use formula ( 5 . I ) , and
sometimes the maximal element can be found on a s imp le r way. For
instance, when U c o n t a i n s a p o s i t i v e d e f i n i t e m a t r i x , then t h i s
m a t r i x i s a maximal element i n D.
Now we a re i n a p o s i t i o n t o g i v e the e q u i v a l e n t s o f the con-
d i t i o n ( 4 . 3 ) .
Theorem 5.1. Let { y , XP, D) be a general l i near model, l e t V..
be a maximal element i n the s e t D and l e t
where U i s an arbi t rary symmetric matr ix such t h a t XI(T,) = X I ( V,,,:X).
Then the following statements are equivalen t :
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
864 KALA
Proof . I f the c o n d i t i o n (5.3) i s s a t i s f i e d then, on account
o f Lemma 2 . 1 , t h e r e e x i s t s a m a t r i x Po such t h a t
POX = X (5.7)
and
P~V~X' = 0 , f o r r = I , . .., t. (5.8)
Since V* E D, Va = alV, + ... + a,Vt f o r some s c a l a r s al, ..., at 9
which combined w i t h (5.8) and (5.7) shows t h a t Po i s a s o l u t i o n o f
the equa t ion
On the o t h e r hand, i t f o l l o w s f rom Theorem 3 .3 , w i t h V rep laced by
V;:, t h a t a general s o l u t i o n o f (5.9) takes the form
where A i s an a r b i t r a r y m a t r i x and T:,: i s as de f ined i n (5.2).
Therefore, f o r some A0
x(x'T,x)-X'T~ + A ~ ( I - T:,:T;) = po . (5.10)
S u b s t i t u t i n g (5.10) i n t o (5.8) leads t o (5.41, s ince
- T?:T:-:V, = V , , f o r r = I , . . . , t,
and x'T~x(x'T~x)-X'T~ = X ' T ~ . Now assume (5.4). Then
f o r some m a t r i x R. P r e m u l t i p l y i n g (5.12) by T?: and us ing (5.11)
again, shows t h a t
m(v,xL: . . . :vtxL) - c X(T;;X').
Hence (5.5) f o l l o w s , as TStXL = V;:XL, which i s obv ious i n view o f
(5.2).
The nex t i m p l i c a t i o n can be e s t a b l i s h e d by observ ing t h a t ,
s ince V?: € D, V9: 2 0. Therefore, by Lemma 3 .1 , X(X) and R(V;.:XL)
are d i s j o i n t subspaces, and thus the c o n d i t i o n (5.5) i m p l i e s t h a t
R(X) and X ( V ~ X ~ : . . . : V ~ X ~ ) a r e a l s o d i s j o i n t . But t h i s i s a c t u a l l y
the statement (5.6).
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS 865
To comp le te t h e p r o o f i t s u f f i c e s t o n o t e t h a t t h e i m p l i c a -
t i o n f r o m ( 5 . 6 ) t o ( 5 . 3 ) i s an immediate consequence o f Lemma 2.6.
v From t h e above c o l l e c t i o n o f necessa ry and s u f f i c i e n t c o n d i -
t i o n s f o r t h e e x i s t e n c e o f t h e MDLUE o f t h e e x p e c t a t i o n v e c t o r i n
a genera l l i n e a r model {y, XE, TI) t h e c o n d i t i o n ( 5 . 4 ) i s a d i r e c t
g e n e r a l i z a t i o n o f t h e r e s u l t due t o M i t r a and Moore ( 1 9 7 6 ) . T h e i r
c o n d i t i o n f o l l o w s f r o m ( 5 . 4 ) by u s i n g V:.: = V , + . . . + I/, as a
maximal e lement i n D and s u b s t i t u t i n g i t i n Y?: = 7:: + Xu', wh ich
i s ( 5 . 2 ) w i t h U = I. The c o n d i t i o n ( 5 . 6 ) , i n t u r n , i s a s i m p l i f i -
c a t i o n o f t h e c r i t e r i o n deve loped by Drygas ( 1 9 7 2 ) .
The n e x t theorem e s t a b l i s h e s a base f o r d e r i v i n g a genera l
f o r m u l a f o r t r a n s f o r m a t i o n s l e a d i n g t o t h e MDLUE o f XC, i f i n t h e
model iy, Xi3, C) such e s t i m a t o r e x i s t s .
Theorem 5.2 . L e t {y, XB, D} b e n genera l l i n e a r nodcl on2 Let
V?: be u rrazirml e l e v e n t i n t h e s e t 3. Then eac4 of t h e ,corzdit.ions
from ( 5 .3) t o ( 5 . 6 ) .is 7:ecessni.y m i se i f f i c ien t f o r the eqmlitli
P r o o f . I n v iew o f Theorem 5.1 , i t s u f f i c e s t o show t h a t any
one o f t h e c o n d i t i o n s l i s t e d t h e r e i s a necessa ry and s u f f i c i e n t
f o r t h e e q u a l i t y ( 5 . 1 3 ) . I n t h e p r o o f we e x p l o i t t h e c o n d i t i o n
(5.5), w h i c h seems t o be t h e most o p e r a t i v e .
Now obse rve t h a t , on accoun t o f t h e o b v i o u s r e l a t i o n s
Lemma 2.5 t o g e t h e r w i t h t h e remark g i v e n a f t e r t h e p r o o f o f i t
i m p l y t h e i n c l u s i o n
Moreover , i t f o l l o w s . f rom Lemma 2.8 t h a t
t
{PXI V,XL} C V$;& ' r= l
s i n c e V s = a l V l + ... + ntVt f o r some s c a l a r s al, ..., at. On t h e o t h e r hand, s i n c e V;,: 2 0, t h e subspaces X(X) and
~ L ( v : , : x ~ ) a r e d i s j o i n t . Thus, u s i n g Lemma 2 . 5 , we can s t a t e t h a t
( 5 . 5 ) i s e q u i v a l e n t t o
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
866 KALA
1 1, { p ~ ~ ~ f i ~ L J c { p ~ ~ ( ~ l ~ L : .. .:VtX ) (5.16)
which, i n view o f (5.14) and (5.15), completes the p r o o f . V
The r e s u l t o f Theorem 5.2 combined w i t h the statement ( b ) o f
Theorem k . 1 and Lemma 2.3 g ives , i n p r i n c i p l e , a d e f i n i t e answer
t o t h e f o l l o w i n g ques t ion : What i s a general representa t io , : o f
pro jec tors ieading t o the MDLUE of Xp i n a general l i near model
{y, Xp, D ) ? Never theless, i t seems t h a t the most e legan t form o f
such a r e p r e s e n t a t i o n can be ob ta ined by the use o f Theorem 3 .3 ,
thus lead ing t o a g e n e r a l i z a t i o n o f the r e s u l t o f Rao ( 1973) (see
h i s Corol l a r y 3 . 7 ) .
Theorem 5.3. Let { y , XP, D) be a general l i near model, l e t
be a maximal element i n t he s e t D and l e t Tk be a matr ix de f ined
i n (5.2). Then any one o f t h e condit ions from (5.3) t o (5.6) i s
necessary and s u f f i c i e n t for the ex i s t ence o f t he MDLUE o f Xp, and
i f it does e x i s t t he general representa t ion o f pro jec tors P provid-
i ng t he MDLUE o f XP i s
P = X(X'T~X)-X'T~ + A(I - T ~ ~ T ~ ) , (5.17)
where A i s an arbi t rary matr ix .
Proo f . I n view o f Theorem 4 . 1 , 5.1 and 5.2 we search f o r a
general form o f p r o j e c t o r s i n the s e t {PXI v.,.XL). By Lemma 2.1,
t h i s genera l r e p r e s e n t a t i o n i s e q u i v a l e n t t o a genera l s o l u t i o n o f
the equa t ion
PX = X , P V ~ ~ X ~ = 0. (5.18) . But, on account o f Theorem 3 . 3 , (5.17) i s j u s t such a s o l u t i o n ,
and t h i s concludes the p r o o f . V
I t should be emphasized t h a t t h e set { P X , v , X ~ J e x p l o i t e d i n
the p roo f above i s always nonempty, and t h a t i t co inc ides w i t h the
se t o f a l l p r o j e c t o r s lead ing t o the MDLUE o f XP i f and o n l y i f
any o f t h e c o n d i t i o n s l i s t e d i n Theorem 5 .1 i s s a t i s f i e d . However,
we can say more i n regard t o the se t {P xl(vlxL:. . .:vtx+ which,
by Theorem 4 .1 and 5.2, can a l s o be used t o c h a r a c t e r i z e t h e c l a s s
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS 867
o f a l l t r a n s f o r m a t i o n s p r o v i d i n g t o t h e MDLUE o f XP i n t h e genera l
l i n e a r model {y, Xb, D}. Namely, by t h e use o f Lemma 2.2 and i n
view o f the c o n d i t i o n ( 5 . 6 ) i t i s easy t o no te t h a t t h e se t
{l=z,(v,xL:.. .:v+xL) } i s nonempty i f and o n l y i f the MDLUE o f XP i n
the model { y , XP, D) e x i s t s . There fo re , nonemptiness o f the se t
I } i s i n f a c t an a l t e r n a t i v e c r i t e r i o n f o r t h e iPm ( vlxL: . . . : vtx ) e x i s t e n c e o f t h e MDLUE o f XR, w h i l e the same i s n o t t r u e i n t h e
case o f t h e s e t {PXI v , X ~ } .
Concluding t h i s s e c t i o n l e t us throw some
l a r i e s o f Sec t ion 4. As i t i s easy t o see, t h i
based on a common assumption t h a t D con ta ins a
element, which, i n view o f D e f i n i t i o n 5 .1 , imp
1 i ght on the c o r o
s c o r o l l a r i e s a r e
p o s i t i v e d e f i n i t e
l i e s t h a t a maxima
element i n D, V$:, i s a l s o p o s i t i v e d e f i n i t e . As t h e consequence,
R(X) and x(v:~:x~) a r e complementary subspaces, and thus, by Lemma
2.4, t h e r e e x i s t s t h e unique p r o j e c t o r o n t o R(X) a long R(v;:x~).
There fo re , t h e f o l l o w i n g r e s u l t i s e s t a b l i s h e d .
C o r o l l a r y 5.1. Let i n a general l i near model {y, Xp, D} the
MDLUE o f XP e x i s t and l e t a maximal element i n D, V* , be p o s i t i v e
d e f i n i t e . Then t he MDLUE o f X@ i s provided by t he unique projec tor
px I vaxL. v
The r e s u l t s o f C o r o l l a r y 4 .1 and 4.2 can now e a s i l y be ob-
t a i n e d from the c o r o l l a r y above. To t h i s end i t s u f f i c e s t o r e -
p lace Va by Vo o r by I, r e s p e c t i v e l y .
The c o n d i t i o n o f C o r o l l a r y 5.1 i s o n l y a s u f f i c i e n t c o n d i t i o n
f o r the uniqueness o f t h e p r o j e c t o r l e a d i n g t o t h e MDLUE o f XB, i f
the e s t i m a t o r i n t h e model { y , XP, D} e x i s t s . However, by t h e Note
1 . 1 o f Rao (1978), t h e equa t ion (5.18) and i t s genera l s o l u t i o n
(5.17), i t i s easy t o e s t a b l i s h a c o n d i t i o n t h a t i s n o t o n l y s u f f i -
c i e n t b u t a l s o necessary.
Corol l a r y 5.2. Let i n a general Linear model {y, XP, D} t he
MDLUE o f Xb e x i s t and Let be a maximal element i n t he s e t D.
Then a pro jec tor leading t o the MDLUE o f X B i s unique i f and onZy
i f JR(Va:X) = Rn. V
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
KALA
6. THREE RELATED PROBLEMS
I n t h i s s e c t i o n we consider th ree e s t i m a t i o n problems t h a t ,
by t h e i r p r a c t i c a l importance, focus much a t t e n t i o n o f many
authors. These problems a r e here so lved by adop t ing the approach
o f the p rev ious s e c t i o n s , which a l s o g ives a new i n s i g h t i n t o t h e
na tu re o f the s o l u t i o n s ob ta ined .
The f i r s t problem can be s t a t e d b r i e f l y as f o l l o w s : f i a t are
necessary and s u f f i c i e n t condi t ions for the orthogonal projector
P t o provide, by pro jec t ing y, the MDLUE of X B i n a X -Markov model { y , X p, V } ?
The problem has a long h i s t o r y , presumably o r i g i
the paper by Anderson (19481, who n o t i c e d t h a t i f i n
model {y, Xb, V } the mat r i ces X and V a re o f f u l l c o l
genera2 Gauss
n a t i n g from
a l i n e a r
umn ranks and
the columns o f X are a l l or thogonal e igenvec to rs o f the m a t i x V,
then the Simple Least Squares Es t imato r (SLSE) o f Xp, i . e . P X g , i s
i d e n t i c a l w i t h the MDLUE o f XB. Anderson's s u f f i c i e n t c o n d i t i o n
was l a t e r red iscovered by Magness and McGuire ( 1 9 6 2 ) , who showed
t h a t i t i s b o t h necessary and s u f f i c i e n t f o r the e q u a l i t y between
the d i s p e r s i o n mat r i ces o f t h e SLSE o f Xp and o f the MDLUE o f XP,
which a c t u a l l y i s e q u i v a l e n t t o the e q u a l i t y between these two
s t a t i s t i c s themselves. Independently, Zysk ind ( 1 9 6 2 ) , ex tend ing
cons idera t ions t o a model {y, XP, V ) w i t h X o f no t f u l l column
rank, has g iven e i g h t e q u i v a l e n t necessary and s u f f i c i e n t cond i -
t i o n s . Another statement o f the " i f and o n l y i f " c o n d i t i o n f o r
the considered e q u a l i t y has been e s t a b l i s h e d by Rao (1967) . A l -
though Rao's r e s u l t has been proved under the assumption t h a t X
and V a r e o f f u l l column ranks, he has remarked on i t t h a t i t i s
a l s o v a l i d i n the case when these mat r i ces a r e o f a r b i t r a r y ranks.
T h i s f a c t has then been conf i rmed by Zysk ind (1967) (see a l s o Rao,
1968), who has extended t h e se t o f h i s e a r l i e r e i g h t c o n d i t i o n s
by d e l e t i n g t h e r e s t r i c t i v e rank assumption on t h e m a t r i x V, and
who a l s o d iscussed t h e equiva lence between h i s extended r e s u l t s
and the c o n d i t i o n due t o Rao (1967) . From the r e s u l t s o f Zysk ind
l e t us r e c a l l t h a t which seems t o be o f the s imp les t form.
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS
Theorem 6 . 1 . In n general Gums-Markov mode2 {y, Xi? , V} t h e
SLSE o f .YE coinc ides u i t h t he MDLUF of XP i f and o n l y i$
Proo f . F i r s t n o t e t h a t ( 6 . 1 ) i s e q u i v a l e n t t o
Now, f rom Lemma 2 . 5 , D e f i n i t i o n 2 . 2 and Lemma 2.9 , i t f o l i o w s t h a t
( 6 . 2 ) i s a necessary and s u f f i c i e n t c o n d i t i o n f o r t h e r e l a t i o n
P X t { P X I V X & ( 6 . 3 )
which, i n view o f Theorem 3 . 2 , completes the p r o o f . V
We remark now t h a t t h e c o n d i t i o n ( 6 . 3 ) can e q u i v a l e n t l y be
w r i t t e n i n t h e form
{pLly1? {F X I 7~ 11 # @, ( 6 . 4 )
which i s ve ry a t t r a c t i v e by i t s s i m i l a r i t y w i t h the c o n d i t i o n (4 .31
o f Theorem 4 . 1 . T h i s o b s e r v a t i o n leads immediate ly t o t h e conc lu -
s i o n t h a t t h e c o n d i t i o n ( 6 . 1 ) assures n o t o n l y e q u a l i t y between
the SLSE o f X B and the MDLUE o f Xp i n the model {y, X f , V}, b u t i t
a l s o assures t h e e x i s t e n c e o f t h e MDLUE o f XP i n t h e genera l
l i n e a r model { ~ g , XP, So}, w i t h
Co = i v : v = hlV + hZI 2 0, h l , E R!.
Moreover, s i n c e I t i t f o l l o w s f rom C o r o l l a r y 4 . 2 t h a t Px i s
the unique p r o j e c t o r lead ing t o t h e MDLUE o f XP i n t h i s genera l
l i n e a r model.
The second problem i s a s imp le g e n e r a l i z a t i o n o f t h e p r e v i o u s .
I t can be fo rmu la ted as f o l l o w s : idhat are necessary an? s u f f f c i m t
condi t ions for each projec tor leading t o t he MDLUC of XP i n the
Gauss-Markov modeZ { z j , XP, Vl} t o proz)ide a l so t h e MDLUE o f X3 7'n
t he Gauss-Markou mode2 { z j , XP, V2}?
The f i r s t a t tempt t o answer t h i s q u e s t i o n was made by Thomas
(1968) , who has g iven two necessary and s u f f i c i e n t c o n d i t i o n s , b u t
o n l y f o r the case when t h e d i s p e r s i o n mat r i ces i n b o t h models a r e
nons ingu la r . Th is a d d i t i o n a l requirement i s n o t assumed i n the
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
KALA 870
cons i de
Moore (
r a t ions conducted by Rao ( 197
1973). From among v a r i o u s equ
I) and l a t e r
i v a l e n t cond
by M i t r a and
i t i o n s we r e c a l l
here the f o l l o w i n g due t o Rao (1971 ) .
Theorem 6.2 . Each projector leading t o t he MDLUE o f XB i n a
general Gauss-Markov model {y, XP, V1} provides a l so t he MDLUE o f
XP i n a Gauss-Markov model {y, XP, V2} i f and only i f
X( v2x1) 2 X( v1xL) . (6.5)
Proof . I t f o l l o w s f rom Lemma 2.5 t h a t (6.5) i s e q u i v a l e n t t o
iPx I v1xL} s. IPx I v2xL} (6.6)
On account o f Theorem 3.2, the r e l a t i o n (6.6) proves t h e r e s u l t .
v I t i s easy t o observe t h a t the c o n d i t i o n (6.6) can a l s o be
w r i t t e n i n the e q u i v a l e n t form
ipx, vlxL} "IPxl v2xL} = IPx, vlxl} I n view o f Lemma 3.1 and 2.1, the se t on the r i g h t hand s i d e i n
t h e above e q u a l i t y i s nonempty. Thus, w i t h the use o f Theorem 4.1,
we can s t a t e t h a t each p r o j e c t o r i n t h e s e t { P X l y l X ~ } prov ides
n o t o n l y the MDLUE o f X@ i n the model { y , XP, V 2 } but a l s o the
MDLUE o f XP i n the genera l l inear model {y, XP, D+}, where
I n the t h i r d problem we a r e a l s o concerned w i t h two d i f f e r e n t
Gauss-Markov models. The problem can be s t a t e d i n t h e f o l l o w i n g
form: What are necessary and s u f f i c i e n t condi t ions for t he e x i s t -
ence o f a s t a t i s t . 7 : ~ t h a t i s t he MDLUE o f Xi3 whichever o f t he models
iy, XB, Vli and {y, XD, 1/21 i s ac tua l l y t rue?
T h i s ques t ion was answered by Rao ( 1968). However, we r e c a l l
here a l a t e r r e s u l t o f the same au thor , Rao (1971), which seems t o
be much s imp le r than t h e o r i g i n a l s o l u t i o n .
Theorem 6.3. Let {y, XR, Vl 1 and {y, XB, V2] be two general
Gauss-Markov models. Then there e x i s t s a s t a t i s t i c t h a t i s the
MDLUE o f XP whichever model i s t rue i f and only i f
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS 871
Ins tead o f showing the p r o o f , which can be e a s i l y e s t a b l i s h e d ,
we no te t h a t c o n d i t i o n ( 6 . 8 ) a c t u a l l y c o i n c i d e s w i t h the c o ~ d i t i o n
( 5 . 6 ) o f Theorem 5.1 when t = 2 . Thus, by the use o f t h e r e s u l t s
o f Theorem 5.1 and 4.1, we can a s c e r t a i n t h a t ( 6 . 8 ) assures n o t
o n l y the e x i s t e n c e o f a p r o j e c t o r lead ing t o the MDLUE o f Xi3 i n
bo th models, { y , X P , V1) and {y, Xlj, V 2 ) , b u t i t i s a l s o the
necessary and s u f f i c i e n t c o n d i t i o n f o r the e x i s t e n c e o f the MDLUE
o f X l j i n the genera l l i n e a r model { y , Xp, I l t } , where D i s the se t t d e f i n e d i n ( 6 . 7 ) . Moreover, Theorem 4 . 1 combined w i t h Theorem 5.2
a l l o w us t o s t a t e t h a t i f c o n d i t i o n ( 6 . 8 ) i s s a t i s f i e d , then the
se t {PXI(v,X~:v2X~)} con ta ins a l l p r o j e c t o r s p r o v i d i n g the MDLUE.
ACKNOWLEDGMENT
T h i s research was completed w h i l e the au thor was a t the
Mathematical l n s t i
The au thor i s ve ry
i n v i t a t i o n t o the
t u t e o f the P o l i s h Academy o f Sciences i n Warsow.
indebted t o Pro fessor C . Olech f o r h i s k i n d
I n s t i t u t e .
REFERENCES
A l b e r t , A. ( 1 9 7 2 ) . Regress ion and t h e Moore-Penrose Pseudoinverse . New York: Academic Press.
Anderson, T. W . ( 1 9 4 8 ) . On the theory o f t e s t i n g s e r i a l c o r r e l a - t ion. Skand. A k t u a r i e t i d s k . 31 , 88-1 16.
Ben- Is rae l , A. and G r e v i l l e , T. N. E . ( 1 9 7 4 ) . Genera l i zed I n u e r s e s : Theor2y and A p p l i c a t i o n s . New York: John W i l ey .
Drygas, H. ( 3 9 7 2 ) . A no te on Gauss-Markov es t imat ion . , in m u l t i v a r i - a t e l i n e a r models. Tech. Report, I n s t i t u t e f u r Okonometrie und Operat ions Research, U n i v e r s i t s t Bonn.
Eaton, M. L. ( 1 9 7 0 ) . Gauss-Markov e s t i m a t i o n f o r m u l t i v a r i a t e l inear models: a coord ina te f r e e approach. Ann. Math. S t a t i s t . 41, 528-538.
Goldman, A. J . and Zelen, M. ( 1 9 6 4 ) . Weak genera l i zed inverses and minimum var iance l i n e a r unbiased e s t i m a t i o n . J . Res. Nat. Bur. S tand. 688, 151-172.
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
872 KALA
Kempthorne, 0. ( 1 9 7 6 ) . Bes t l i n e a r unb iased e s t i m a t i o n w i t h a r b i t - r a r y v a r i ance mat r i x . Essays i n P r o b a b i l i t y and S t a t i s t i c s ( S . Ikeda, Ed . ) . Tokyo: Sh inko Tsusho, 203-225.
K r u s k a l , W . ( 1 9 6 1 ) . The c o o r d ; , i a t e - f r e e approach t o Gauss-Markov e s t i m a t i o n , and i t s a p p l i c a t i o n t o m i s s i n g and e x t r a obe rva - t i o n s . F o . ~ r t h Rerke ley Symp. Math. S t a t i s t . Prob. 1 , 435-451
LaMot te , L . R. ( 1977) . A c a n o n i c a l f o r m f o r t h e gene ra l 1 i n e a r model. Ann. S t a t i s t . 5 , 787-789.
Magness, T . A . and McGuire, J . B. ( 1 9 6 2 ) . Comparison o f l e a s t squares and minimum v a r i a n c e e s t i m a t e s o f r e g r e s s i o n pa ra - me te rs . Ann. Math. S t a t i s t . 2 , 462-470.
M i t r a , S . K. and Moore, 6. J. ( 1 9 7 3 ) . Gauss-Markov e s t i m a t i o n w i t h an i n c o r r e c t d i s p e r s i o n m a t r i x . S'cnkhya Ser.A 35, 139-152.
M i t r a , S. K. and Moore Thorne, B. J . ( 1 9 7 6 ) . L i n e a r e s t i m a t i o n i n gene ra l 1 i n e a r models . Essays i n ProbabiZi ty and S t a t i s t i c s (S. I keda , E d . ) . Tokyc: Sh inko Tsusho, 189-201.
Rao, C . R . ( 1967 ) . Leas t squares t h e o r y u s i n g an e s t i m a t e d d i s p e r - s i o n m a t r i x and i t s a p p l i c a t i o n t o measurement o f s i g n a l s . F i f t h Berakeley Symp. '.lath. S t a t i s t . Prob. 1, 355-372.
Rao, C. R. ( 1 9 6 8 ) . A n o t e on a p r e v i o u s lemma i n t h e t h e o r y o f l e a s t squares and some f u r t h e r r e s u l t s . Sankhya Ser . A 30, 259-266.
Rao, C . R. ( 1 9 7 1 ) . U n i f i e d t h e o r y o f l i n e a r e s t i m a t i o n . scnkhya Ser. A 33, 371-394.
Rao, C . R. ( 1 9 7 3 ) . R e p r e s e n t a t i o n s o f b e s t l i n e a r unb iased e s t i - ma to rs i n t h e Gauss-Markof f model w i t h a s i n g u l a r d i s p e r s i o n m a t r i x . J . M u l t i v a r i a t e Ana l . . 3, 276-292.
Rao, C . R. ( 1 9 7 3 ~ ) . Linear S t a t i s t i c a l I n f e r e n c e and i t s Appl ica- t i o n s . (Second e d i t i o n ) . New York : John W i l e y .
Rao, C . R. ( 1 9 7 4 ) . P r o j e c t o r s , gene ra l i z e d i n v e r s e s and t h e BLUE'S. J. Roy. S t a t i s t . Soc. B 36, 442-448.
Rao, C. R . ( 1 9 7 8 ) . Cho ice o f b e s t l i n e a r e s t i m a t o r s i n t h e Gauss- M a r k o f f model w i t h a s i n g u l a r d i s p e r s i o n . Commxn. S t u t i s t . - Theqr. Math. A8, 1199-1208.
Rao, C . R. and M i t r a , S . K. ( 1971 ) . General ized I n v e r s e o f Matr ices and i t s A p p l i c a t i o n s . New York : John W i l e y .
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014
LINEAR ESTIMATION IN GENERAL LINEAR MODELS 873
Rao, C . R. and Yana i , H . , 1979 ' . General d e f i n i t i o n and decomposi- t i o n o f p r o j e c t o r s and some a p p l i c a t i o n s t o s t a t i s t i c a l p r o b - lems. J . Stat. F12.n. Irz~*~ere;nce 3 , -17.
See ly , J . and ance unb
Thomas, D . H . ( Abst r a c
Z y s k i n d , G. ( 1 9 7 1 ) . L i n e a r spaces and minimum v a r i - i a s e d e s t irnat i o n . Ann. :vkcJ;. 5tatiJst . 42, 691-703.
( 1 9 6 8 ) . When do minimum v a r i a n c e e s t i m a t o r s c o i n c i d e ? 0. 4r v . 'lac4. L t a t i s t . 39, 1365.
Z y s k i n d , G . ; I 9 6 2 l i n e a r l e a s t 33, 1502-150
Zysk ind , G. (1967 m a t r i c e s and
. On c o n d i t i o n s f o r e q u a l i t y o f b e s t and s imp le squares e s t i rnators . ( A b s t r a c t ) . Ann. "rlrh. : z r . r : s ~ .
. On c a n o n i c a l fo rms, non -nega t i ve c o v a r i a n c e b e s t and s i m p l e l e a s t squares l i n e a r e s t i m a t o r s
i n l i n e a r models. Ann. ,"dath. S t a t z ' s t . 38, 1092-1 109.
Zysk i n d , G. and M a r t i n , F . 5 . ( 1 9 6 9 ) . On b e s t l i n e a r e s t i m a t i o n and a genera l Gauss-Markov theorem i n l i n e a r models w i t h a r b i t r a r y nonnega t i ve c o v a r i a n c e s t r u c t u r e . STA'" J. AppLI. " l u t k . 17, 1190-1202.
Dow
nloa
ded
by [
Tuf
ts U
nive
rsity
] at
10:
35 0
2 D
ecem
ber
2014