Describing Relationships

Conditional Expectation Functions (CEF)

Line Fitting I

Line Fitting II

#library(stargazer)
# install.packages("wooldridge") if you don't have the package
# install.packages("stargazer") if you don't have the package
#library(stargazer)
library(wooldridge)
data("wage1", package = "wooldridge")   # load data
reg<-lm(wage~educ, data=wage1)
summary(reg)
#stargazer(reg, header = F, single.row = TRUE, no.space = T, dep.var.labels.include = FALSE,dep.var.caption = "Hourly Wages", type='html', omit.stat=c("rsq", "ser", "f"))

The CEF is All You Need

library(stargazer)
wage2<-wage1%>%group_by(educ)%>%summarize(wage=mean(wage), count=n())
reg2<-lm(wage~educ, weights = count, data=wage2)
summary(reg2)
#stargazer(reg2, header = F, single.row = TRUE, no.space = T, dep.var.labels.include = FALSE,dep.var.caption = "Hourly Wages", type='html', omit.stat=c("rsq", "ser", "f"))

Fits and Residuals

As you just saw, regression breaks any dependent variable into two pieces:

$$Y_{i}=\hat{Y_{i}}+e_{i}$$ $\hat{Y}_{i}$ is the fitted value or the part of $Y_{i}$ that the model explains. The residual $e_{i}$ is what is left over.

Some of the residuals properties:

1. Regression residuals have expectation zero (i.e., $E(e_{i})=0$)

2. Regression residuals are uncorrelated with all the regressors that made them and with the corresponding fitted value: $E(X_{ik}e_{i})=0$ for each regressor $X_{ik}$. In other words, if you regress $e_{i}$ on $X_{1i}, X_{2i}, \dots$, the estimated coefficients will be equal to zero.

3. Regression residuals are uncorrelated with the corresponding fitted values: $E(\hat{Y}_{i}e_{i})=0$

Those properties can be derived using the first-order conditions we used to get $\hat{\beta}_{0}$ and $\hat{\beta}_{1}$.

Regression for Dummies

An important case using regression is the bivariate regression with a dummy regressor (we saw it here). Say $Z_{i}$ takes on two values: 0 and 1. Hence, one can write it as

$$E[Y_{i}|Z_{i}=0]=\beta_{0} \\ E[Y_{i}|Z_{i}=1]=\beta_{0}+\beta_{1}$$

so that $\beta_{1}=E[Y_{i}|Z_{i}=1]-E[Y_{i}|Z_{i}=0]$.

Using this notation, we can write

$$E[Y_{i}|Z_{i}]=E[Y_{i}|Z_{i}=0]+(E[Y_{i}|Z_{i}=1]-E[Y_{i}|Z_{i}=0])Z_{i} = \beta_{0}+\beta_{1}Z_{i}$$

Does it look familiar? This shows that $E[Y_{i}|Z_{i}]$ is a linear function of $Z_{i}$, with slope $\beta_{1}$ and intercept $\beta_{0}$. Because the CEF with a single dummy variable is linear, regression fits this CEF perfectly.

Controlling for Variables

Building Models with Logs I

Frequently, economists transform variables using the natural logarithm. For instance, wages are approximately log-normally distributed, and you might want to use $ln(Wage)$ as a dependent variable instead of $Wage$ to model the relationship between earnings vs. schooling - in that way, $ln(Wage)$ would be approximately normally distributed. Taking the log also reduces the impact of outliers, and the estimates have a convenient interpretation.

To see what happens, let's go back to the bivariate regression with dummies example, modifying the dependent variable:

$$ln(Y_{i})=\beta_{0}+\beta_{1}Z_{i}+\varepsilon_{i}$$ where $Z_{i}$ takes on two values: 0 and 1. We can rewrite the equation as $$E[ln(Y_{i})|Z_{i}]=\beta_{0}+\beta_{1}Z_{i}$$ and the regression, in this case, fits the CEF perfectly.

Suppose we engineer a ceteris paribus change in $Z_{i}$ for individual $i$. This reveals potential outcome $Y_{0i}$ when $Z_{i}=0$ and $Y_{1i}$ when $Z_{i}=1$ (cont.)

Building Models with Logs II

Regression Standard Errors

options(scipen=999)
data("wage1", package = "wooldridge")   # load data
### Standard deviation of educ
sdX<-sd(wage1$educ)
### Number of observations in the sample
n<-nrow(wage1)
### Regression residuals
reg<-lm(lwage~educ, data=wage1)
sde<-sd(residuals(reg))
### Standard error of beta1
SE_educ<-sde/(sqrt(n-2)*sdX)
SE_educ

Example: Private School Effects

• Students who attended a private four-year college in America paid an average of about $31,875 in tuition and fees in the 2018-19 period, while students who went to a public university in their home state paid, on average, 9,212 dollars - the yearly difference in tuition is considerable. Is it worth it?

• Comparisons between earnings of those who went to different schools may reveal a significant gap favoring elite-college alumni. Indeed, on average, the wage difference is 14% in favor of those who went private. Is that a fair comparison?

  • Differences in test scores, family background, motivation, and perhaps other skills and talents affect future earnings. Without dealing with selection bias, one cannot claim that there is a causal effect of private education on salaries

• What if we use regression to control for family income, SAT scores, and other covariates that we can observe?

  • Still, there are factors complicated to quantify that might affect both attendance decisions and later earnings. For instance, how do we take into account diligence and family connections?

Dale and Krueger (2002) I

• Instead of focusing on everything that might matter for college choice and earnings, Stacy Dale and Alan Krueger¹ came up with a compelling shortcut: the characteristics of colleges to which students applied and were admitted

• Consider the case of two students who both applied to and were admitted to UMass and Harvard, but one goes to Harvard, and the other goes to UMass. The fact that those students applied to the same universities suggests they have similar ambition and motivation. Since both were admitted to the same places, one can assume that they might be able to succeed under the same circumstances. Hence, comparing earnings of those two similar students that took different paths would be fair

• Dale and Krueger analyzed a large data set called College and Beyond (C&B). The C&B data contains information about thousands of students who enrolled in a group of selective U.S. universities, together with survey information collected multiple times from the year students took the SAT to long after most had graduated from college

[1] Stacy Berg Dale and Alan B. Krueger, "Estimating the Payoff to Attending a More Selective College: An Application of Selection on Observables and Unobservables," Quarterly Journal of Economics, vol. 117, no. 4, November 2002.

College Matching Matrix I

College Matching Matrix II

Dale and Krueger (2002) II

Using the C&B dataset, Dale and Krueger matched 5,583 students into 151 similar selectivity groups containing students who went to both public and private universities. Besides the "group variables" that capture the relative selectivity of the schools to which students applied and were admitted, the researchers also controlled for other variables such as SAT scores and parental income.

The resulting regression model looks like this:

$$ln(Earnings_{i})=\beta_{0}+\beta_{1}Private_{i}+ \sum_{j=1}^{150}\gamma_{j}GROUP_{ji}+\delta_{1}SAT_{i}+\delta_{2}PI_{i}+\varepsilon_{i}$$

where $\beta_{1}$ is the treatment effect of interest: the extent to which earnings differ for students who attend a private school compared to students who went to public universities. The model also controls for 151 selectivity groups: the variable $GROUP_{ji}$ equals to one when student $i$ is in the group $j$ and is zero otherwise. The idea is to control for the sets of schools to which students applied and were admitted to bring the comparison as close as possible to an apples-to-apples comparison.

Dale and Krueger (2002) III

Regression and Causality I

Regression reduces - maybe even eliminates - selection bias as long as you have credible identification strategy

Let's go back to our potential outcomes framework: $$Y_{i}=Y_{0i}+D_{i}(Y_{1i}-Y_{0i})$$ where we get to see either $Y_{1i}$ or $Y_{0i}$, but never both. We hope to measure the average $Y_{1i}-Y_{0i}$. The naive comparison gives us

In the example of private versus public education, students that go to private schools would have higher future earnings after college anyway, and the positive bias exacerbates the benefits of private education.

Regression and Causality II

The conditional independence assumption (CIA) asserts that conditional on observed characteristics, $X_{i}$, the selection bias disappears. Formally,

$$Y_{1i}, Y_{0i} \perp\!\!\!\perp D_{i}|X_{i}$$

Given the CIA, $conditional-on-X_{i}$ comparisons have a causal interpretation:

$$E[Y_{i}|X_{i}, D_{i}=1]-E[Y_{i}|X_{i}, D_{i}=0]=E[Y_{1i}-Y_{0i}|X_{i}]$$

Back to the private versus public education:

$$E[Y_{0i}|Private_{i};GROUP_{i},SAT_{i}, lnPI_{i}]=E[Y_{0i}|GROUP_{i},SAT_{i}, lnPI_{i}]$$ and the CIA makes this causal.

OVB I

Regression is a way to make other things equal. However, you can only generate fair comparisons for variables included on the right-hand side - fail to include what matters still leaves us with selection bias. The "regression version" of selection bias is what we call omitted variable bias (OVB). Let us go back to our example of earnings versus schooling. We already ran both the long regression

$$\underbrace{Y_{i}}_{Wage_{i}}=\beta_{0}^{l}+\beta_{1}^{l}\underbrace{X_{1i}}_{Educ_{i}}+\beta_{2}\underbrace{X_{2i}}_{Exper_{i}}+\varepsilon_{i}^{l}$$ and the short regression

$$\underbrace{Y_{i}}_{Wage_{i}}=\beta_{0}^{s}+\beta_{1}^{s}\underbrace{X_{1i}}_{Educ_{i}}+\varepsilon_{i}^{s}$$

The OVB formula describes the relationship between short and long coefficients as follows:

$$\beta^{s}=\beta^{l}+\pi_{21} \beta_{2}$$

where $\beta_{2}$ is the coefficient is the coefficient of $X_{2i}$ in the long regression, and $\pi_{21}$ is the coefficient of $X_{1i}$ in a regression of $X_{2i}$ on $X_{1i}$.

OVB II

library(wooldridge)
# load data
data("wage1", package = "wooldridge")   
short_reg<-lm(lwage~educ, data=wage1)
beta_short<-short_reg$coefficients[2]
beta_short

  educ 

long_reg<-lm(lwage~educ+exper, data=wage1)
beta_long<-long_reg$coefficients[2]
beta2<-long_reg$coefficients[3]
exper_on_educ<-lm(exper~educ, data=wage1)
pi12<-exper_on_educ$coefficients[2]
beta_long+pi12*beta2

  educ 

## which is equal to beta_short

OVB III

In other words, the omitted variable bias (OVB) formula connects regression coefficients in models with different controls. Consider the following long regression of wages on schooling $(s_{i})$, controlling for ability $(A_{i})$:

$$Y_{i} = \alpha+\rho s_{i}+ A^{'}_{i} \gamma + \varepsilon_{i}$$ Since ability is hard to measure, what are the consequences of omitting that variable?

$$\dfrac{Cov(Y_{i}, s_{i})}{V(s_{i})}=\rho+\gamma^{'}\delta_{As}$$ where $\delta_{As}$ is the vector of coefficients from regressions of the elements of $A_{i}$ on $s_{i}$.

In English, short equals long plus the effect of omitted times the regression of omitted on included. When omitted and included are uncorrelated, short $=$ long.

Regression Sensitivity Analysis

Your Standard Errors are Probably Wrong

Fixing Standard Errors

library(wooldridge)
data("wage1", package = "wooldridge")   # load data
library(fixest)
reg1<-feols(wage~educ, data=wage1, se="standard")
reg2<-feols(wage~educ, data=wage1, se="hetero")
etable(reg1, reg2)

##                               reg1               reg2
## Dependent Var.:               wage               wage
##                                                      
## (Intercept)       -0.9049 (0.6850)   -0.9049 (0.7255)
## educ            0.5414*** (0.0532) 0.5414*** (0.0613)
## _______________ __________________ __________________
## S.E. type                 Standard Heteroskedas.-rob.
## Observations                   526                526
## R2                         0.16476            0.16476
## Adj. R2                    0.16316            0.16316