1、Econometric Analysis of Panel Data,William GreeneDepartment of EconomicsStern School of Business,Econometric Analysis of Panel Data,20. Sample Selection and Attrition,Dueling Selection Biases From two emails, same day.,“I am trying to find methods which can deal with data that is non-randomised and
2、suffers from selection bias.”“I explain the probability of answering questions using, amongother independent variables, a variable which measures knowledge breadth.Knowledge breadthcan be constructed only for those individuals that fill in a skill description in the company intranet. This is where t
3、heselection bias comes from.,The Crucial Element,Selection on the unobservablesSelection into the sample is based on both observables and unobservablesAll the observables are accounted forUnobservables in the selection rule also appear in the model of interest (or are correlated with unobservables i
4、n the model of interest)“Selection Bias”=the bias due to not accounting for the unobservables that link the equations.,A Sample Selection Model,Linear model2 stepML Murphy & TopelBinary choice applicationOther models,Canonical Sample Selection Model,Applications,Labor Supply model: y*=wage-reservati
5、on waged=labor force participationAttrition model: Clinical studies of medicinesSurvival bias in financial dataIncome studies value of a college applicationTreatment effectsAny survey data in which respondents self select to reportEtc,Estimation of the Selection Model,Two step least squaresInefficie
6、ntSimple exists in current softwareSimple to understand and widely used Full information maximum likelihoodEfficientSimple exists in current softwareNot so simple to understand widely misunderstood,Heckmans Model,Two Step Estimation,The “LAMBDA”,FIML Estimation,Classic Application,Mroz, T., Married
7、womens labor supply, Econometrica, 1987.N =753N1 = 428A (my) specificationLFP=f(age,age2,family income, education, kids)Wage=g(experience, exp2, education, city)Two step and FIML estimation,Selection Equation,+-+| Binomial Probit Model | Dependent variable LFP | Number of observations 753 | Log like
8、lihood function -490.8478 |+-+-+-+-+-+-+-+|Variable| Coefficient | Standard Error |b/St.Er.|P|Z|z| Mean of X|+-+-+-+-+-+-+-+Index function for probability Constant| -4.15680692 1.40208596 -2.965 .0030 AGE | .18539510 .06596666 2.810 .0049 42.5378486 AGESQ | -.00242590 .00077354 -3.136 .0017 1874.548
9、47 FAMINC | .458045D-05 .420642D-05 1.089 .2762 23080.5950 WE | .09818228 .02298412 4.272 .0000 12.2868526 KIDS | -.44898674 .13091150 -3.430 .0006 .69588313,Heckman Estimator and MLE,Extension Treatment Effect,Sample Selection,Extensions Binary Data,Panel Data and Selection,Panel Data and Sample Se
10、lection Models: A Nonlinear Time Series,I. 1990-1992: Fixed and Random Effects ExtensionsII. 1995 and 2005: Model Identification through Conditional Mean AssumptionsIII. 1997-2005: Semiparametric Approaches based on Differences and Kernel WeightsIV. 2007: Return to Conventional Estimators, with Bias
11、 Corrections,Panel Data Sample Selection Models,Zabel Economics Letters,Inappropriate to have a mix of FE and RE modelsTwo part solutionTreat both effects as “fixed”Project both effects onto the group means of the variables in the equationsResulting model is two random effects equationsUse both rand
12、om effects,Selection with Fixed Effects,Practical Complications,The bivariate normal integration is actually the product of two univariate normals, because in the specification above, vi and wi are assumed to be uncorrelated. Vella notes, however, “ given the computational demands of estimating by m
13、aximum likelihood induced by the requirement to evaluate multiple integrals, we consider the applicability of available simple, or two step procedures.”,Simulation,The first line in the log likelihood is of the form Evd=0() and the second line is of the form EwEv()()/. Using simulation instead, the
14、simulated likelihood is,Correlated Effects,Suppose that wi and vi are bivariate standard normal with correlation vw. We can project wi on vi and write wi = vwvi + (1-vw2)1/2hiwhere hi has a standard normal distribution. To allow the correlation, we now simply substitute this expression for wi in the
15、 simulated (or original) log likelihood, and add vw to the list of parameters to be estimated. The simulation is then over still independent normal variates, vi and hi.,Conditional Means,A Feasible Estimator,Estimation,Kyriazidou - Semiparametrics,Bias Corrections,Val and Vella, 2007 (Working paper)
16、Assume fixed effectsBias corrected probit estimator at the first stepUse fixed probit model to set up second step Heckman style regression treatment.,Postscript,What selection process is at work?All of the work examined here (and in the literature) assumes the selection operates anew in each periodA
17、n alternative scenario: Selection into the panel, once, at baseline.Why arent the time invariant components correlated? (Greene, 2007, NLOGIT development)Other modelsAll of the work on panel data selection assumes the main equation is a linear model.Any others? Discrete choice? Counts?,Attrition,In
18、a panel, t=1,T individual I leaves the sample at time Ki and does not return.If the determinants of attrition (especially the unobservables) are correlated with the variables in the equation of interest, then the now familiar problem of sample selection arises.,Application of a Two Period Model,“Hem
19、oglobin and Quality of Life in Cancer Patients with Anemia,”Finkelstein (MIT), Berndt (MIT), Greene (NYU), Cremieux (Univ. of Quebec)1998With Ortho Biotech seeking to change labeling of already approved drug erythropoetin. r-HuEPO,QOL Study,Quality of life study i = 1, 1200+ clinically anemic cancer
20、 patients undergoing chemotherapy, treated with transfusions and/or r-HuEPOt = 0 at baseline, 1 at exit. (interperiod survey by some patients was not used)yit = self administered quality of life survey, scale = 0,100xit = hemoglobin level, other covariatesTreatment effects model (hemoglobin level)Ba
21、ckground r-HuEPO treatment to affect Hg levelImportant statistical issuesUnobservable individual effectsThe placebo effectAttrition sample selectionFDA mistrust of “community based” not clinical trial based statistical evidenceObjective when to administer treatment for maximum marginal benefit,Deali
22、ng with Attrition,The attrition issue: Appearance for the second interview was low for people with initial low QOL (death or depression) or with initial high QOL (dont need the treatment). Thus, missing data at exit were clearly related to values of the dependent variable.Solutions to the attrition
23、problemHeckman selection model (used in the study)ProbPresent at exit|covariates = (z) (Probit model)Additional variable added to difference model i = (zi)/(zi) The FDA solution: fill with zeros. (!),An Early Attrition Model,Methods of Estimating the Attrition Model,Heckman style “selection” modelTw
24、o step maximum likelihoodFull information maximum likelihoodTwo step method of moments estimatorsWeighting schemes that account for the “survivor bias”,Selection Model,Maximum Likelihood,A Model of Attrition,Nijman and Verbeek, Journal of Applied Econometrics, 1992Consumption survey (Holland, 1984 1
25、986)Exogenous selection for participation (rotating panel)Voluntary participation (missing not at random attrition),Attrition Model,Selection Equation,Estimation Using One Wave,Use any single wave as a cross section with observed lagged values.Advantage: Familiar sample selection modelDisadvantagesL
26、oss of efficiency“One can no longer distinguish between state dependence and unobserved heterogeneity.”,One Wave Model,Maximum Likelihood Estimation,See Zabels model in slides 20 and 23.Because numerical integration is required in one or two dimensions for every individual in the sample at each iter
27、ation of a high dimensional numerical optimization problem, this is, though feasible, not computationally attractive.The dimensionality of the optimization is irrelevantThis is much easier in 2008 than it was in 1992 (especially with simulation) The authors did the computations with Hermite quadratu
28、re.,Testing for Selection?,Maximum Likelihood ResultsCovariances were highly insignificant. LR statistic=0.46.Two step results produced the same conclusion based on a Hausman testML Estimation results looked like the two step results.,A Study of Health Status in the Presence of Attrition,“THE DYNAMI
29、CS OF HEALTH IN THE BRITISH HOUSEHOLD PANEL SURVEY,”Contoyannis, P., Jones, A., N. RiceJournal of Applied Econometrics, 19, 2004, pp. 473-503.Self assessed healthBritish Household Panel Survey (BHPS)1991 1998 = 8 wavesAbout 5,000 households,Data,Variable of Interest,Dynamics,Attrition,Random Effects
30、 Dynamic Ordered Probit Model,Probability Weighting Estimators,A Patch for Attrition(1) Fit a participation probit equation for each wave.(2) Compute p(i,t) = predictions of participation for each individual in each period.Special assumptions needed to make this workIgnore common effects and fit a weighted pooled log likelihood: i t dit/p(i,t)logLPit.,
