1、1,Multiple Regression Analysis,y = b0 + b1x1 + b2x2 + . . . bkxk + u2. Inference,2,Assumptions of the Classical Linear Model (CLM),So far, we know that given the Gauss-Markov assumptions, OLS is BLUE, In order to do classical hypothesis testing, we need to add another assumption (beyond the Gauss-Ma
2、rkov assumptions)Assume that u is independent of x1, x2, xk and u is normally distributed with zero mean and variance s2: u Normal(0,s2),3,CLM Assumptions (cont),Under CLM, OLS is not only BLUE, but is the minimum variance unbiased estimatorWe can summarize the population assumptions of CLM as follo
3、wsy|x Normal(b0 + b1x1 + bkxk, s2)While for now we just assume normality, clear that sometimes not the caseLarge samples will let us drop normality,4,.,.,x1,x2,The homoskedastic normal distribution with a single explanatory variable,E(y|x) = b0 + b1x,y,f(y|x),Normal distributions,5,Normal Sampling D
4、istributions,6,The t Test,7,The t Test (cont),Knowing the sampling distribution for the standardized estimator allows us to carry out hypothesis testsStart with a null hypothesisFor example, H0: bj=0If accept null, then accept that xj has no effect on y, controlling for other xs,8,The t Test (cont),
5、9,t Test: One-Sided Alternatives,Besides our null, H0, we need an alternative hypothesis, H1, and a significance levelH1 may be one-sided, or two-sidedH1: bj 0 and H1: bj 0 are one-sidedH1: bj 0 is a two-sided alternativeIf we want to have only a 5% probability of rejecting H0 if it is really true,
6、then we say our significance level is 5%,10,One-Sided Alternatives (cont),Having picked a significance level, a, we look up the (1 a)th percentile in a t distribution with n k 1 df and call this c, the critical value We can reject the null hypothesis if the t statistic is greater than the critical v
7、alueIf the t statistic is less than the critical value then we fail to reject the null,11,yi = b0 + b1xi1 + + bkxik + uiH0: bj = 0 H1: bj 0,c,0,a,(1 - a),One-Sided Alternatives (cont),Fail to reject,reject,12,Examples 1,Hourly Wage EquationH0: bexper = 0 H1: bexper 0,13,One-sided vs Two-sided,Becaus
8、e the t distribution is symmetric, testing H1: bj than c then we fail to reject the nullFor a two-sided test, we set the critical value based on a/2 and reject H1: bj 0 if the absolute value of the t statistic c,14,yi = b0 + b1Xi1 + + bkXik + uiH0: bj = 0 H1: bj 0,c,0,a/2,(1 - a),-c,a/2,Two-Sided Al
9、ternatives,reject,reject,fail to reject,15,Summary for H0: bj = 0,Unless otherwise stated, the alternative is assumed to be two-sidedIf we reject the null, we typically say “xj is statistically significant at the a % level”If we fail to reject the null, we typically say “xj is statistically insignif
10、icant at the a % level”,16,Examples 2,Determinants of College GPAcolGPAcollege GPA(great point average), hsGPAhigh school GPAskippedaverage numbers of letures missed per week.,17,Testing other hypotheses,A more general form of the t statistic recognizes that we may want to test something like H0: bj
11、 = aj In this case, the appropriate t statistic is,18,Examples 3,Campus Crime and EnrollmentH0: benroll = 1 H1: benroll 1,19,Examples 4,Housing Prices and Air PollutionH0: blog(nox) = -1 H1: blog(nox) - 1,20,Confidence Intervals,Another way to use classical statistical testing is to construct a conf
12、idence interval using the same critical value as was used for a two-sided testA (1 - a) % confidence interval is defined as,21,Computing p-values for t tests,An alternative to the classical approach is to ask, “what is the smallest significance level at which the null would be rejected?”So, compute
13、the t statistic, and then look up what percentile it is in the appropriate t distribution this is the p-valuep-value is the probability we would observe the t statistic we did, if the null were true,22,Most computer packages will compute the p-value for you, assuming a two-sided testIf you really wa
14、nt a one-sided alternative, just divide the two-sided p-value by 2 Many software,such as Stata or Eviews provides the t statistic, p-value, and 95% confidence interval for H0: bj = 0 for you,23,Testing a Linear Combination,Suppose instead of testing whether b1 is equal to a constant, you want to tes
15、t if it is equal to another parameter, that is H0 : b1 = b2Use same basic procedure for forming a t statistic,24,Testing Linear Combo (cont),25,Testing a Linear Combo (cont),So, to use formula, need s12, which standard output does not haveMany packages will have an option to get it, or will just per
16、form the test for you More generally, you can always restate the problem to get the test you want,26,Examples 5,Suppose you are interested in the effect of campaign expenditures on outcomesModel is voteA = b0 + b1log(expendA) + b2log(expendB) + b3prtystrA + uH0: b1 = - b2, or H0: q1 = b1 + b2 = 0b1
17、= q1 b2, so substitute in and rearrange voteA = b0 + q1log(expendA) + b2log(expendB - expendA) + b3prtystrA + u,27,Example (cont):,This is the same model as originally, but now you get a standard error for b1 b2 = q1 directly from the basic regressionAny linear combination of parameters could be tes
18、ted in a similar mannerOther examples of hypotheses about a single linear combination of parameters: b1 = 1 + b2 ; b1 = 5b2 ; b1 = -1/2b2 ; etc,28,Multiple Linear Restrictions,Everything weve done so far has involved testing a single linear restriction, (e.g. b1 = 0 or b1 = b2 )However, we may want
19、to jointly test multiple hypotheses about our parametersA typical example is testing “exclusion restrictions” we want to know if a group of parameters are all equal to zero,29,Testing Exclusion Restrictions,Now the null hypothesis might be something like H0: bk-q+1 = 0, . , bk = 0The alternative is
20、just H1: H0 is not trueCant just check each t statistic separately, because we want to know if the q parameters are jointly significant at a given level it is possible for none to be individually significant at that level,30,Exclusion Restrictions (cont),To do the test we need to estimate the “restr
21、icted model” without xk-q+1, , xk included, as well as the “unrestricted model” with all xs includedIntuitively, we want to know if the change in SSR is big enough to warrant inclusion of xk-q+1, , xk,31,The F statistic,The F statistic is always positive, since the SSR from the restricted model cant
22、 be less than the SSR from the unrestrictedEssentially the F statistic is measuring the relative increase in SSR when moving from the unrestricted to restricted modelq = number of restrictions, or dfr dfurn k 1 = dfur,32,The F statistic (cont),To decide if the increase in SSR when we move to a restr
23、icted model is “big enough” to reject the exclusions, we need to know about the sampling distribution of our F statNot surprisingly, F Fq,n-k-1, where q is referred to as the numerator degrees of freedom and n k 1 as the denominator degrees of freedom,33,0,c,a,(1 - a),f(F),F,The F statistic (cont),r
24、eject,fail to reject,Reject H0 at asignificance levelif F c,34,Example:Major League Baseball Players Salary,35,Relationship between F and t Stat,The F statistic is intended to detect whether any combination of a set of coefficients is different from zero, The t test is best suited for testing a sing
25、le hypothesis. Group a bunch of insignificant varialbes with a significant variable, it is possible conclude that the entire set of variables is jointly insignificant. Often, when a variable is very statistically significant and it is tested jointly with another set of variables, the set will be joi
26、ntly significant.,36,The R2 form of the F statistic,Because the SSRs may be large and unwieldy, an alternative form of the formula is usefulWe use the fact that SSR = SST(1 R2) for any regression, so can substitute in for SSRu and SSRur,37,Overall Significance,A special case of exclusion restriction
27、s is to test H0: b1 = b2 = bk = 0Since the R2 from a model with only an intercept will be zero, the F statistic is simply,38,General Linear Restrictions,The basic form of the F statistic will work for any set of linear restrictionsFirst estimate the unrestricted model and then estimate the restricte
28、d modelIn each case, make note of the SSRImposing the restrictions can be tricky will likely have to redefine variables again,39,Example:,Use same voting model as beforeModel is voteA = b0 + b1log(expendA) + b2log(expendB) + b3prtystrA + unow null is H0: b1 = 1, b3 = 0Substituting in the restriction
29、s: voteA = b0 + log(expendA) + b2log(expendB) + u, soUse voteA - log(expendA) = b0 + b2log(expendB) + u as restricted model,40,F Statistic Summary,Just as with t statistics, p-values can be calculated by looking up the percentile in the appropriate F distribution If only one exclusion is being tested, then F = t2, and the p-values will be the same,
