inequality restricted maximum entropy estimation in stat

8/12/2019 Inequality Restricted Maximum Entropy Estimation in STAT

1/26

IntroductionJaynes Dice Problem

GME Estimation in Linear Regression ModelGME Command with User Supplied Parameter Support Matrix

Sign and Cross-Parameter RestrictionsConclusion

Inequality restricted maximum entropy estimationin Stata

Randall Campbelly, R. Carter Hillz

Mississippi State Universityy, Louisiana State Universityz

Stata Conference New Orleans July 18, 2013

Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata

1


2/26




Generalized Maximum Entropy Estimation

GME estimator developed by Golan, Judge, and Miller (1996)

Campbell and Hill (2006) impose inequality restrictions onGME estimator in a linear regression model

We develop new STATA commands to obtain GME parameterestimates, with or without inequality restrictions

Our commands utilizes the optimize() function in MATA


2


3/26




Maximum Entropy Problem

Entropy is the amount of uncertainty represented by a discreteprobability distribution

Entropy is measured as H (p ) = p k ln(p k )

Jaynes dice problem: Estimate unknown probabilities of rolling each value on a die

Maximize H (p ) subject to: k p k = y and p k = 1,

where k is the number of sides on die and y is the mean fromprior rolls


3


4/26




Maximum Entropy Solution

L = p k ln(p k ) + (y k p k ) + (1 p k )

dL/ dp k = 1 ln(p k ) k = 0

Which implies bp k = exp( b k ) exp( b k )Substitute into Lagrangian and minimize (Golan, Judge, and

Miller (1996)):

L( ) = bp k ( b ) ln( bp k ( b )) + b (y k bp k ( b ))Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata

4


5/26

I d i


6/26




STATA MEDICE Command

The part that evaluates the function:void mydice_eval(todo, lam, x, k, y, L, g, H){a = J(k,1,.)b = J(k,1,.)for (i=1; i< =k; i++)

{a[i] = exp(-x[i]*lam)}L = lam*y + ln(sum(a))


6

I t d ti


7/26




Example

. scalar k = 6

. scalar y = 4.5

. medice k y

entropy function = 1.6135811

r6 .34749406

r5 .23977444

r4 .1654468r3 .11415998

r2 .07877155

r1 .05435317

c1

r(phat)[6,1]


7

Introduction


8/26




Model Specication for GME Estimation

Model: y = X + e

Set up such that unknown parameters in form of probabilities(GJM, 1996):

= Zp =2664

z 01 0 ... 00 z

02 ... 0

. . . .

0 0 ... z 0k

3775

.

2664

p 1p 2

.

p k

3775

Do same thing with error term


8

Introduction


9/26




Generalized Entropy Function

Reparameterized Model: y = XZp + Vw

Z and V are parameter and error support matrices,respectivelyMax H (p , w ) = p 0 ln(p ) w 0 ln(w ), subject toy = XZp + Vw (I K

Oi

0

M )p = i K

(I N Oi 0J )w = i N ,where M is # support points for each parameter; J is #support points for errors


9

Introduction


10/26




GME Solution in Linear Regression Model

L = p 0 ln(p ) w 0 ln(w ) + 0(XZp + Vw y )+

0[i K (I K Oi 0M )p ] + 0[i N (I N Oi

0J )w ]

Solution:

bp km = exp(z km x

0k

b ) /

M

m = 1

exp(z km x 0k

b ), and

bw nj = exp(v nj b n ) /J

j = 1

exp(v nj b n )


10


11/26

Introduction


12/26

Jaynes Dice ProblemGME Estimation in Linear Regression Model

GME Command with User Supplied Parameter Support MatrixSign and Cross-Parameter Restrictions

Conclusion

Example using Generated Data

With the automated command, the user types gmereg y x1 x2x3 ...

As an example, we generate 7 standard normal randomvariables (x1-x7) and a standard normal error

y = 2x 1 + 1. 2x 2 + 0. 35x 3 + 0. 4x 4 0. 05x 5 + 0. 8x 6 3x 7 + e


12

Introduction


13/26



Conclusion

Results

Results obtained using REG and GMEREG in STATA:true reg gmereg

x1 2.000 1.974 1.948x2 1.200 1.149 1.160x3 0.350 0.218 0.203x4 0.400 0.134 0.108

x5 -0.050 0.060 0.016x6 0.800 0.822 0.791x7 -3.000 -3.198 -3.194const 0.000 -0.125 -0.110


13

Introduction


14/26



Conclusion

STATA GMEREGM Command

GMEREG automatically sets up the parameter support matrix

based on initial OLS estimates

A second version, GMEREGM, allows the user to specify theirown support matrix

This allows the user to specify wider or narrower bounds and,as we will show, to impose cross-parameter restrictions


14

Introductionbl


15/26



Conclusion

Specifying the Parameter Support Matrix

Suppose we wish to impose wide, uninformative bounds; alloweach parameter to fall between -20 and 20

In our example, we have K = 8 parameters to estimate andhave M = 5 support points. We specify the K x M supportmatrix:. matrix zmat = (-20,-20,-20,-20,-20,-20,-20,-20 n

-10,-10,-10,-10,-10,-10,-10,-10 n0,0,0,0,0,0,0,0 n10,10,10,10,10,10,10,10 n20,20,20,20,20,20,20,20)

. gmeregm y x1 x2 x3 x4 x5 x6 x7


15

IntroductionJ Di P bl


16/26



Conclusion

Alternative Specications

Suppose we wish a more informative prior:. matrix zmat = (-5,-5,-1,-1,-1,-1,-5,-5 n

-2.5,-2.5,-0.5,-0.5,-0.5,-0.5,-2.5,-2.5 n0,0,0,0,0,0,0,0 n2.5,2.5, 0.5,0.5,0.5,0.5,2.5,2.5 n5,5,1,1,1,1,5,5)

Or one that is not symmetric about zero:

. matrix zmat = (-10,-10,-1,-1,-1,-1,-15,-10 n0,0,0,0,-0.5,0,10,-5 n5,5,1,1,0,1,-5,0 n10,10, 2,2,0.5,2,0,5 n15,15,3,3,1,3,10,10)


16


17/26



18/26



Conclusion

Sign Restrictions

Our estimates all took correct signs except the estimate for 5

Parameter sign restrictions are easily accomplished throughsupport matrix. For example,. matrix zmat = (0,0,0,0,0.4,0,-10,-10 n

2.5,1,0.25,0.25,-0.3,0.5,7.5,-5 n5,2,0.5,0.5,-0.2,1,-5,0 n7.5,3,0.75,0.75,-0.1,1.5,-2.5,5 n10,4,1,1,0,2,0,10)


18



19/26



Conclusion

Cross-Parameter Restrictions

Note that while the true 4 > 3, this does not hold for ourestimates

Suppose we wish to restrict b 4 >

b 3Campbell and Hill (2006) impose such restrictions through thesupport matrix:

3 4 = Z

p 3p 4 =

z 03 0

z 03 z 04

p 3p 4

Specifying z 04 to take only non-negative values ensures that

b 4 >

b 3


19



20/26



Conclusion

GME Solution with Cross-Parameter Restrictions

The problem becomes harder computationally; the GMEsolution is

bp km = exp(z km x

01 b + z km x

02 b + ... + z km x

0K b )M

m = 1

exp(z km x 01 b + z km x

02 b + ... + z km x

0K b )

When parameter support is block diagonal, cross-productterms drop out


20



21/26

Jay es ce ob eGME Estimation in Linear Regression Model


Conclusion

Restricted GME Estimation

A third version, GMEREGMR, performs restricted estimation.However, must enter larger zmat (KM x K)Enter separate support column vector for each parameter (and

a vector of 0s). matrix z0 = (0 n0n0n0n0). matrix z1 = (0 n5n10n15n20). matrix z2 = z1

. matrix z3 = z1. matrix z4 = (0 n0.5n1n1.5n2)

. matrix z5 = (-20 n-15n-10n-5n0)

. matrix z6 = z1

. matrix z7 = z5

. matrix z8 = (-20 n-10n0n10n20)Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata

21



22/26

yGME Estimation in Linear Regression Model


Conclusion

STATA GMEREGMR Command

Now combine into zmat; matrix is block diagonal except forcolumn 4 which include z3 and z4. matrix zmat = (z1,z0,z0,z0,z0,z0,z0,z0 n

z0,z2,z0,z0,z0,z0,z0,z0 nz0,z0,z3,z3 ,z0,z0,z0,z0 nz0,z0,z0,z4 ,z0,z0,z0,z0 nz0,z0,z0,z0,z5,z0,z0,z0 nz0,z0,z0,z0,z0,z6,z0,z0 nz0,z0,z0,z0,z0,z0,z7,z0 nz0,z0,z0,z0,z0,z0,z0,z8)

Now enter command gmeregmr = y x1 x2 x3 x4 x5 x6 x7


22



23/26



Final Example - Multiple Restrictions

Suppose we wish to impose 4 > 3 and 6 > 3 + 4 and 2 > 3 + 6The parameter support vectors we chose:

. matrix z0 = (0 n0n0n0n0)

. matrix z1 = (0 n5n10n15n20)

. matrix z2 = (0 n0.5n1n1.5n2)

. matrix z3 = z1

. matrix z4 = z2

. matrix z5 = (-20 n-15n-10n-5n0)

. matrix z6 = z2

. matrix z7 = z5

. matrix z8 = (-20 n-10n0n10n20)Randall Campbelly , R. Carter Hillz Inequality restricted maximum entropy estimation in Stata

23



24/26



Zmat - Multiple Restrictions

The zmat to ensure restrictions hold. matrix zmat = (z1,z0,z0,z0,z0,z0,z0,z0 n

z0,z2 ,z0,z0,z0,z0,z0,z0 nz0,3*z3 ,z3,z3 ,z0,2*z3 ,z0,z0 nz0,z4 ,z0,z4 ,z0,z4 ,z0,z0 nz0,z0,z0,z0,z5,z0,z0,z0 nz0,z6 ,z0,z0,z0,z6 ,z0,z0 nz0,z0,z0,z0,z0,z0,z7,z0 nz0,z0,z0,z0,z0,z0,z0,z8)

Note: b 4 = b 3 + z 04p 4 > b 3 ; b 6 = b 3 + b 4 + z

06p 6 > b 3+ b 4 ; and b 2 = b 3 + b 6 + z

02p 2 > b 3+ b 6


24


GME E i i i Li R i M d l


25/26



Results

Results based on dierent support matrices:true sign only single multiple

x1 2.000 1.961 2.009 2.067x2 1.200 1.217 1.250 1.324x3 0.350 0.363 0.165 0.006x4 0.400 0.347 0.385 0.224x5 -0.050 -0.150 -0.021 -0.041x6 0.800 0.849 0.952 0.860x7 -3.000 -3.220 -3.186 -3.194const 0.000 -0.125 -0.139 -0.163


25


GME E ti ti i Li R g i M d l


26/26



Work to Do

Our commands provide GME estimates and can be used with

either no user input or a user-specied parameter support

We are working to output additional statistics such as priormean and standard errors

We are also developing STATA GME commands for binaryand multinomial choice models, censroed regression, ..


26

inequality restricted maximum entropy estimation in stat

Documents