** Chapter 9 ** for W.H. Greene, Econometric Analysis 6th ed. *****************
* (c) Noel Roy 2003, 2008
*
*				               MODELS FOR PANEL DATA
*
* This tutorial will cover estimation and testing of fixed-effect and 
* random-effect models using the OLS command.
*
* SHAZAM does not handle panel data models very well, so for serious work
* students are advised to utilize a program such as LIMDEP or STATA that is
* designed to do so. But with some effort, panel data models can be estimated
* in SHAZAM.

* The following example utilizes the dataset described in Table F6.1, Costs
* for U.S. Airlines, 90 observations on 6 firms for 1970-1984. A simple 
* loglinear model for the cost of production is estimated, where log cost is
* a function of log output, log fuel price, and load factor (average
* capacity utilization).
*
*===============================================================================
* 9.3 THE POOLED REGRESSION MODEL
*
* Cost Function for Airline Production.
*
READ (TableF6-1.prn)I YEAR C Q PFuel LoadFact /SKIPLINES=1
*
GENR LNC=LOG(C)
GENR LNQ=LOG(Q)
GENR LNPF=LOG(PFuel)
*
*===============================================================================
* 9.3.1 Estimate the model by Ordinary Least Squares. 
*
OLS LNC LNQ LNPF LoadFact /COEF=BETA LOGLOG RESID=E
*
* Keep the SSE and the SigmaSq for use in Section 9.5 (Random Effects Model).
*.
GEN1 SSE=$SSE
GEN1 SIGPOOL2=$SIG2
GEN1 K=$K
*
* Calculate the economies of scale parameter.
*
GEN1 ES=1/BETA(1) - 1
PRINT ES
*
* There appears to be evidence of substantial economies of scale.
*
* Resestimate using the HETCOV option to obtain the White heteroskedaticity
* consistent standard errors.
*
OLS LNC LNQ LNPF LoadFact / LOGLOG HETCOV
*
* The POOL command provides features for estimating pooled cross-section
* time-series models. The NCROSS= option can  be used to specify the number
* of cross-sectional units in the data. The Beck-Katz robust standard errors
* (9-3) can be obtained by using the HETCOV option.
*
POOL LNC LNQ LNPF LoadFact / LOGLOG HETCOV NCROSS=6
*
* The test statistics indicate the presence both of individual heteroskedasticity
* an cross-observation correlation.
*===============================================================================
* 9.3.4 ROBUST ESTIMATION USING GROUP MEANS
*
* Set parameters of the panel. Denote the number of groups as N, and the
* number of observations for each group as T.
*
GEN1 N=6
GEN1 T=15
GEN1 NT=N*T
*
* First, calculate the group means for the variables.
* A DO loop will be used to do this for each of the N firms.
*
SET NODOECHO
DO #=1,N
*
* The SAMPLE range will be altered to calculate the means in each group.
* The STAT command is used to store the average values of the variables. 
*
GEN1 TA=(#-1)*T+1
GEN1 TB=#*T
SAMPLE TA TB
?STAT LNC LNQ LNPF LoadFact E / MEAN=AVG#
ENDO
SAMPLE 1 5
COPY AVG1 AVG2 AVG3 AVG4 AVG5 AVG6 AVG
MATRIX AVGP=AVG'
*
* Now set up the group means.
*
MATRIX MLNC=AVGP(0,1)
MATRIX MLNQ=AVGP(0,2)
MATRIX MLNPF=AVGP(0,3)
MATRIX MLoadFac=AVGP(0,4)
MATRIX ME=AVGP(0,5)
*
* Do the Group mean regression. 
*
SAMPLE 1 N
OLS MLNC MLNQ MLNPF MLoadFac /LOGLOG
* The group means of the OLS residuals SIGGRP2 is stored for use in
* Section 9.5.3.
GEN1 SIGGRP2=$SIG2
*
*===============================================================================
* 9.4 THE FIXED EFFECTS MODEL
*
* A DO loop is ed to initialize the dummy variables for the 
* firm effects regression. The following DO-loop will
* generate six vectors of dummy variables identifying the individual firms. 
*
SAMPLE 1 NT
DO #=1,N
GENR A#=0
IF (I .EQ. #) A#=1
ENDO
*
* Recall that, when variables are listed in a series, the first and last 
* variable in the range separated by a hyphen will denote the entire series 
* (see the OLS command below). Recall that the NOCONSTANT option must be
* used to avoid the "dummy variable trap". The coefficient vector and the 
* asymptotic covariance matrix are saved for computation of the Hausman test
* in Section 9.5, and the residuals for use in Section 9.6.2.
*
OLS LNC LNQ LNPF LoadFact A1-A6 / NOCONSTANT LOGLOG COEF=BETAFE COV=COVFE RESID=EFE
*
* Now construct the F-test statistic testing the difference across firms.
*
TEST 
TEST A1=A2
TEST A1=A3
TEST A1=A4
TEST A1=A5
TEST A1=A6
END
*
* The evidence is strongly in favour of a firm specific effect in the data.
*
* Keep the variance of the estimate for use in Section 9.5.
*
GEN1 SIGMAE2=$SIG2
*
* Panel Data models are often characterized by a large (thousands) of microunits,
* requiring a large number of dummy variables to capture the fixed effects. This
* is not an efficient use of computer resources, and may in fact require more
* memory than is available. Fortunately, the same results can be obtained by
* recasting the regression in terms of deviations from the group means, as outlined
* in Section 9.3.6. Note that the R-squared in invalid because the dependent 
* variable has been transformed.
*
* Set up the variables.
GENR DLNC=LNC-MLNC(I)
GENR DLNQ=LNQ-MLNQ(I)
GENR DLNPF=LNPF-MLNPF(I)
GENR DLoadFac=LoadFact-MLoadFac(I)
OLS DLNC DLNQ DLNPF DLoadFac / NOCONSTANT LOGLOG COEF=B
*
* The results are the same as for the dummy variable regression.
*===============================================================================
* 9.4.3 Testing the Significance of the Group Effects

* We can test for significance of the firm  effects using equation (9-20).
*
GEN1 F=((SSE-$SSE)/(N-1))/(($SSE)/(NT-N-$K))
GEN1 DF1=N-1
GEN1 DF2=NT-N-$K
DISTRIB F /TYPE=F DF1=DF1 DF2=DF2

* Now calculate the fixed effects from the group means, as in (9-4b).
* This can be obtained from a group-mean-deviation regression.
SAMPLE 1 N
GENR ALPHA=MLNC-MLNQ*B(1)-MLNPF*B(2)-MLoadFac*B(3)
PRINT ALPHA
DELETE B
*
*===============================================================================
* 9.4.4 Fixed Time and Group Effects
*
* Now do the same with the time effects regression. 
*
SAMPLE 1 NT
DO #=1,T
GENR T#=0
IF (YEAR .EQ. #) T#=1
ENDO
OLS LNC LNQ LNPF LoadFact T1-T15 / NOCONSTANT LOGLOG 
TEST 
TEST T1=T2
TEST T1=T3
TEST T1=T4
TEST T1=T5
TEST T1=T6
TEST T1=T7
TEST T1=T8
TEST T1=T9
TEST T1=T10
TEST T1=T11
TEST T1=T12
TEST T1=T13
TEST T1=T14
TEST T1=T15
END
*
* There does not appear to be a significant cost difference across the two periods 
* that is not accounted for by fuel price, output, and load factor.
*
* For the regression with both firm and time effects, we can impose the restrictions 
* implicitly be substituting for the coefficients of a6 and t15 respectively.
* The dummy variables are then redefined as follows:
*
DO #=1,5
GENR DA#=A#-A6
ENDO
DO #=1,14
GENR DT#=T#-T15
ENDO
OLS LNC LNQ LNPF LoadFact DA1-DA5 DT1-DT14 /  LOGLOG COEF=B
*
* Testing for the significance of the time effects now requires that all the
* time dummies be zero, since the time dummies are now contrasts.
*
TEST
TEST DT1
TEST DT2
TEST DT3
TEST DT4
TEST DT5
TEST DT6
TEST DT7
TEST DT8
TEST DT9
TEST DT10
TEST DT11
TEST DT12
TEST DT13
TEST DT14
END
*
* For completeness, solve for the coefficients of a6 and t15.
*
GEN1 BDA6=-B(4)-B(5)-B(6)-B(7)-B(8)
GEN1 BDT15=-B(9)-B(10)-B(11)-B(12)-B(13)-B(14)-B(15)-B(16)-B(17)-B(18)-B(19)-B(20)-B(21)-B(22)
PRINT BDA6 BDT15
*
*===============================================================================
* 9.5 RANDOM EFFECTS
*
* 9.5.3 Testing for Random Effects
*
* Now compute the LM statistic (9-39). e'e has been saved in Section 9.3.1
* as SSE. ebar has been saved in Secion 9.3.4 as ME.
*
PRINT ME
MATRIX ME2=ME'*ME
GEN1 LM = 0.5*NT/(T-1)*(T**2*ME2/SSE-1)**2
PRINT LM
DISTRIB LM /TYPE=CHI DF=1
*
* We can conclude that the classical regression model with a single constant term 
* is inappropriate for these data. We can reject this null hypothesis in favour
* of the random effects model. However, the fixed effects model is still in
* contention.
*===============================================================================
* Estimating the Random Effects Model
*
* First, estimate the variance of the individual effects SigmaSquared u (p. 204),  
* and the weighting parameter theta (p.202). 
*
GEN1 SIGMAU2=SIGPOOL2-SIGMAE2
* Check whether SIGMAU2 is non-negative, and if so, remove the degrees
* of freedom corrections as in p. 205 of the textbook.
IF (SIGMAU2 .LT. 0) SIGMAU2=(NT-K-1)/NT*SIGPOOL2-(NT-N-K)/NT*SIGMAE2
*
GEN1 THETA=1-SQRT(SIGMAE2/(SIGMAE2+T*SIGMAU2))
PRINT SIGMAE2 SIGPOOL2 SIGMAU2 THETA
*
* Now transform the variables, including the constant term,
* in accordance with equation (9-30).
*
GEN1 SIGMAE=SQRT(SIGMAE2)
GENR LCSTAR=(LNC-THETA*MLNC(I))/SIGMAE
GENR LQSTAR=(LNQ-THETA*MLNQ(I))/SIGMAE
GENR LPFSTAR=(LNPF-THETA*MLNPF(I))/SIGMAE
GENR LFSTAR=(LoadFact-THETA*MLoadFac(I))/SIGMAE
GENR CON=(1-THETA)/SIGMAE
*
* Random effects estimation of the Firm Effects Model.
*
OLS LCSTAR LQSTAR LPFSTAR LFSTAR CON /NOCONSTANT COEF=BETARE COV=COVRE
*
* The coefficient vector and the 
* asymptotic covariance matrix are saved for computation of the Hausman test.

*
*===============================================================================
* 9.5.4 Hausman's Specification Test for the Random Effects Model
*
* Extract the parts of the coefficient vectors and the asymptotic covariance
* matrices of the fixed effects and random effects estimators that correspond
* to the slopes in the models. These correspond to the first three rows
* and columns.
*
SAMPLE 1 3
COPY BETAFE BFE /FROW=1;3
COPY BETARE BRE /FROW=1;3
DIM VARBFE 3 3 VARBRE 3 3
DO #=1,3
DO !=1,3
MATRIX VARBFE(#,!)=COVFE(#,!)
MATRIX VARBRE(#,!)=COVRE(#,!)
ENDO
ENDO
PRINT VARBFE VARBRE
*
* Now calculate the Wald statistic.
*
MATRIX W=(BFE-BRE)'*INV(VARBFE-VARBRE)*(BFE-BRE)
PRINT W
DISTRIB W /TYPE=CHI DF=3 
*
* The null hypothesis that the individual effects are uncorrelated
* with the other independent variables cannot be rejected. The test
* strongly supports the random effects model. However,
* the value of this test is uncertain. The matrix (VARBFE-VARBRE)
* should be positive definite, but two of the characteristic roots
* are negative. It is almost singular, so inversion probably involves
* significant roundoff error.
.
MATRIX EIG=EIGVAL(VARBFE-VARBRE)
MATRIX DET=DET(VARBFE-VARBRE)
PRINT DET EIG
*
STOP
*===============================================================================
* Updated September 26, 2008