21.Technical_Dif_in_Dif_Premand_Holla_ENG_PP

in Practice" book. This additional material is made freely but

Technical Track Session III

IN

& PANEL DATA

DIFFERENCE

DD)
Estimation strategy: 3 ways to look at DD
Examples:
Extension of education

1

2

3

impact of a “past” program
As always, we need to identify
which

We can try to find a “natural experiment” that allows us to identify the impact of a policy

E.g. An unexpected change in policy
E.g. A policy that only affects 16 year-olds but not 15 year-olds
In general, exploit variation of policies in time and space

The quality of the comparison group determines the quality of the evaluation.

T=0
T=1
Time
Enrolled
Not enrolled
Estimated ATE
Y1 | Di=1
DD=[(Y̅1|D=1)-(Y̅0|D=1)] - [(Y̅1|D=0)-(Y̅0|D=0)]

regression with fixed effects for time and group…

on the inter-temporal variation between the groups.
I.e. Changes in the

I.e. Jumps in trends in the outcome variable, that happen only for the treatment groups, not for the comparison groups, exactly at the time that the treatment kicks in.

own their land, farmers who don’t own their land
Effects that

Valid only when the policy change has an immediate impact on the outcome variable.

If there is a delay in the impact of the policy change, we do need to use lagged treatment variables.

control groups, that occur at the same time as the

If there are other factors that affect the difference in trends between the two groups, then the estimation will be biased!

Di=1
Y1 | Di=0
Y0 | Di=0
Estimated Impact
Y1 |

Bias

“fake” treatment group
Ex. for previous years (e.g. Years -2, -1).
Or

Use a different comparison group
The two DDs should give the same estimates.

Use an outcome variable Y2 which you know is NOT affected by the intervention:

Using the same comparison group and treatment year
If the DD estimate is different from zero, we have a problem

outcomes prior to the intervention. E.g. “Ashenfelter dip”: selection into

If program impact is heterogeneous across individual characteristics, pre-treatment differences in observed characteristics can create non-parallel outcome dynamics (Abadie, 2005).

Similarly, bias would occur when the size of the response depends in a non-linear way on the size of the intervention, and we compare a group with high treatment intensity, with a group with low treatment intensity

When outcomes within the unit of time/group are correlated, OLS standard errors understate the st. dev. of the DD estimator (Bertrand et al., 2004).

in Indonesia: evidence from an unusual policy experiment
Esther Duflo, MIT American

on schooling?

one school per 500 children between 5 and 14 years

The enrollment rate increased from 69% to 85% between 1973 and 1978

The number of schools built in each region depended on the number of children out of school in those regions in 1972, before the start of the program.

number of schools received in each region.
There are 2 sources

By age

Children who were older than 12 years in 1972 did not benefit from the program.
The younger a child was 1972, the more it benefited from the program –because she spent more time in the new schools.

date
1995 salary level
1995 level of education
The intensity of the building

Sample: men born between 1950 and 1972.

problem and estimate the impact of the program.
We simplify

Young cohort of children who benefitted
Older cohort of children who did not benefit

We simplify the groups of children affected by the program

of schooling”

of schooling”

groups whom you know did not benefit, compute a DD,

achievement increase by 0.12- 0.19 years
The average salaries increased

Making sure the DD estimation is accurate:

A placebo DD gave 0 estimated effect
Use various alternative specifications
Check that the impact estimates for each age cohort make sense.

of Water Services on Child Mortality
Sebastián Galiani, Universidad de San

governed the municipality
Federal, Peronist y Provincial parties: allowed privatization
Radical party:

Which party was in power/whether the water got privatized did not depend on:

Income, unemployment, inequality at the municipal level
Recent changes in infant mortality rates

in the two types of municipalities before privatization
You can do

Check that privatization only affects mortality through reasons that are logically related to water and sanitation issues.

For example, there is no effect of privatization on death rate from cardiovascular disease or accidents.

support

municipalities than in non-poor municipalities

infant mortality of 5-7%.
Using a combination of methods, we found

The reduction of mortality is:

Due to fewer deaths from infectious and parasitic diseases.
Not due to changes in death rates from reasons not related to water and sanitation

The largest decrease in infant mortality occurred in low income municipalities.

72.
Ashenfelter, O. (1978): “Estimating the Effect of Training Programs on

Chay, Ken, McEwan, Patrick and Miguel Urquiola (2005): “The central role of noise in evaluating interventions that use test scores to rank schools,” American Economic Review, 95, pp. 1237-58.

Gertler, Paul (2004): “Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA’s Control Randomized Experiment,” American Economic Review, 94, pp. 336-41.

Duflo, E. (2001). “Schooling and Labor Market Consequences of School Construction in Indonesia: Evidence From an Unusual Policy Experiment,” American Economic Review, Sept 2001

Galiani, S., Gertler, P. and E. Schargrodsky (2005): “Water for Life: The Impact of the Privatization of Water Services on Child Mortality,” Journal of Political Economy, Volume 113, pp. 83-120.

Bertrand, M., Duflo, E. and S. Mullainathan (2004). “How much should we trust differences-in-differences Estimates?,” Quarterly Journal of Economics.