SUMMARY OF RIGOROUS IMPACT EVALUATION …



Summary of Rigorous Impact Evaluation Methodologies

| |Identifying Assumption |Data Requirements |Advantages |Disadvantages |

|Randomized Controlled Trial |No systematic differences in unobservable|At least post-intervention data on |Weakest identifying assumption of 4 |Spillovers can invalidate control group |

| |characteristics between treated & control|randomly assigned members of treated & |methods (as long as randomization |Might be politically difficult to argue |

| |groups (bolstered by showing that there |control groups (baseline helps make |successful) |in favor of randomization |

| |are no systematic differences in |identifying assumption plausible) |When budget prevents program from being |External validity can be questionable |

| |observable characteristics between | |provided to everyone simultaneously, |(was the study population representative |

| |treated & control groups) | |randomizing (either timing or in the |of the population served at scale?) |

| | | |absolute sense of who does and does not | |

| | | |get the program) is sometimes viewed as | |

| | | |the fairest way to allocate scarce | |

| | | |resources | |

|Difference-in-Difference |Whatever happened to the control group |At least pre- & post-intervention cross |Might be possible to do with existing |Identifying assumption is relatively |

| |over time is what would have happened to |sections including both treated & control|survey data |strong (need to consider all other |

| |the treated group in the absence of the |(additional pre-intervention data helps | |possible differences between the groups |

| |program |make identifying assumption plausible) | |that could have led to the observed |

| | | | |outcomes) |

|Regression Discontinuity |Those just above the threshold (the |Data on threshold qualification (already |When threshold is built into program |Only identifies program impact for those |

| |treated) and those just below (the |exists) & outcomes |design there is no need to do anything |at the threshold |

| |control) are otherwise identical | |special for evaluation | |

|Matching |After controlling for observables, |Rich data on as many observable |Might be possible to do with existing |Strong identifying assumption (even if |

| |treated & control are not systematically |characteristics as possible, large sample|survey data |they’re otherwise identical, why did some|

| |different |size (so that it is possible to find | |get program and others not?) |

| | |appropriate match) | | |

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download