Uncertainty in Empirical Economics
Uncertainty in Empirical Economics was written by Frank Schorfheide and Zhiheng You in 2025. It was published as part of the National Bureau of Economic Research working paper series.
The authors explore the validity of standard errors reported for point estimates. In particular, they explore how empirical studies can produce highly variable point estimates yet report comparatively smaller standard errors.
The authors suppose 4 explanations for this:
- Estimates are very noisy
- The studies may estimate different parameters, because the parameter actually varies over time and countries.
- The models have different parameters.
- The models have different assumptions.
The authors then study explanations 1 and 2.
The authors explore 2 by assuming that the parameters estimated by each study are not equivalent. They use a Bayesian hierarchical model to account for their correlations. Studies (indexed by j) are divided equally into groups (indexed by i). In other words, a particular study's parameter is θij. Within a group, the parameters are assumed to be normally distributed about a common mean τi and vary according to v. If v=0, then each study's parameter actually is equivalent.
Each study's estimate ˆθij is interpretted as being drawn from a normal distribution about the true parameter θij and varying by standard errors σ2.
Within each group, all but one study are considered 'baseline' studies, and their joint likelihood distribution given hyperparameter v is calculated to give a posterior distribution of τi. The final study is a 'validation' study, testing the posterior probability. The key metric here is coverage probability.
The authors discuss tuning the hyperparameters. (J denotes the number of baselines studies in a groups, so adjusting the group sizes this can be expressed as tuning the hyperparameter J.)
Reading Notes
This one is a bit too advanced for me to follow. I can't say I'm familiar enough with Bayesian statistics to work with this.