Hostname: page-component-686fd747b7-n7dl9 Total loading time: 0 Render date: 2025-02-13T12:05:16.208Z Has data issue: false hasContentIssue false
  • English
  • Français

CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?

Published online by Cambridge University Press:  30 November 2007

Hannes Leeb  and
Benedikt M. Pötscher
Hannes Leeb
Affiliation:
Yale University
Benedikt M. Pötscher
Affiliation:
University of Vienna
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider the problem of estimating the unconditional distribution of a post-model-selection estimator. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model-selection criterion such as the Akaike information criterion [AIC] or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for the distribution; performance is here measured by the probability that the estimation error exceeds a given threshold. These lower bounds are shown to approach ½ or even 1 in large samples, depending on the situation considered. Similar impossibility results are also obtained for the distribution of linear functions (e.g., predictors) of the post-model-selection estimator.The research of the first author was supported by the Max Kade Foundation and by the Austrian National Science Foundation (FWF), Grant no. P13868-MAT. A preliminary draft of the material in this paper was written in 1999.

Type
Research Article
Information
Econometric Theory , Volume 24 , Issue 2 , April 2008 , pp. 338 - 376
Copyright
© 2008 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Ahmed, S.E. & A.K. Basu (2000) Least squares, preliminary test and Stein-type estimation in general vector AR(p) models. Statistica Neerlandica 54, 47–66.Google Scholar
Bauer, P., B.M. Pötscher, & P. Hackl (1988) Model selection by multiple test procedures. Statistics 19, 39–44.Google Scholar
Billingsley, P. (1995) Probability and Measure, 3rd ed. Wiley.
Brownstone, D. (1990) Bootstrapping improved estimators for linear regression models. Journal of Econometrics 44, 171–187.Google Scholar
Danilov, D.L. & J.R. Magnus (2004) On the harm that ignoring pre-testing can cause. Journal of Econometrics 122, 27–46.Google Scholar
Dijkstra, T.K. & J.H. Veldkamp (1988) Data-driven selection of regressors and the bootstrap. Lecture Notes in Economics and Mathematical Systems 307, 17–38.Google Scholar
Dukić, V.M. & E.A. Peña (2005) Variance estimation in a model with Gaussian submodel. Journal of the American Statistical Association 100, 296–309.Google Scholar
Freedman, D.A., W. Navidi, & S.C. Peters (1988) On the impact of variable selection in fitting regression equations. Lecture Notes in Economics and Mathematical Systems 307, 1–16.Google Scholar
Hansen, P.R. (2003) Regression Analysis with Many Specifications: A Bootstrap Method for Robust Inference. Working paper, Department of Economics, Brown University.
Hjort, N.L. & G. Claeskens (2003) Frequentist model average estimators. Journal of the American Statistical Association 98, 879–899.Google Scholar
Kabaila, P. (1995) The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537–549.Google Scholar
Kapetanios, G. (2001) Incorporating lag order selection uncertainty in parameter inference for AR models. Economics Letters 72, 137–144.Google Scholar
Kilian, L. (1998) Accounting for lag order uncertainty in autoregressions: The endogenous lag order bootstrap algorithm. Journal of Time Series Analysis 19, 531–548.Google Scholar
Knight, K. (1999) Epi-convergence in Distribution and Stochastic Equi-semicontinuity. Working paper, Department of Statistics, University of Toronto.
Kulperger, R.J. & S.E. Ahmed (1992) A bootstrap theorem for a preliminary test estimator. Communications in Statistics: Theory and Methods 21, 2071–2082.Google Scholar
Leeb, H. (2002) On a differential equation with advanced and retarded arguments. Communications on Applied Nonlinear Analysis 9, 77–86.Google Scholar
Leeb, H. (2005) The distribution of a linear predictor after model selection: Conditional finite-sample distributions and asymptotic approximations. Journal of Statistical Planning and Inference 134, 64–89.Google Scholar
Leeb, H. (2006) The distribution of a linear predictor after model selection: Unconditional finite-sample distributions and asymptotic approximations. IMS Lecture Notes-Monograph Series 49, 291–311.Google Scholar
Leeb, H. & B.M. Pötscher (2003) The finite-sample distribution of post-model-selection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100–142.Google Scholar
Leeb, H. & B.M. Pötscher (2005a) Model selection and inference: Facts and fiction. Econometric Theory 21, 21–59.Google Scholar
Leeb, H. & B.M. Pötscher (2005b) Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators? Working paper, Department of Statistics, University of Vienna.
Leeb, H. & B.M. Pötscher (2006a) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk bound results. Econometric Theory 22, 69–97. (Corrigendum, Econometric Theory, forthcoming.)Google Scholar
Leeb, H. & B.M. Pötscher (2006b) Can one estimate the conditional distribution of post-model-selection estimators? Annals of Statistics 34, 2554–2591.Google Scholar
Lehmann, E.L. & G. Casella (1998) Theory of Point Estimation, 2nd ed. Springer Texts in Statistics. Springer-Verlag.
Nickl, R. (2003) Asymptotic distribution theory of post-model-selection maximum likelihood estimators. Master's thesis, Department of Statistics, University of Vienna.
Pötscher, B.M. (1991) Effects of model selection on inference. Econometric Theory 7, 163–185.Google Scholar
Pötscher, B.M. (1995) Comment on “The effect of model selection on confidence regions and prediction regions” by P. Kabaila. Econometric Theory 11, 550–559.Google Scholar
Pötscher, B.M. & A.J. Novak (1998) The distribution of estimators after model selection: Large and small sample results. Journal of Statistical Computation and Simulation 60, 19–56.Google Scholar
Rao, C.R. & Y. Wu (2001) On model selection. IMS Lecture Notes-Monograph Series 38, 1–57.Google Scholar
Robinson, G.K. (1979) Conditional properties of statistical procedures. Annals of Statistics 7, 742–755.Google Scholar
Sen, P.K. (1979) Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 1019–1033.Google Scholar
Sen, P.K. & A.K.M.E. Saleh (1987) On preliminary test and shrinkage M-estimation in linear models. Annals of Statistics 15, 1580–1592.Google Scholar
van der Vaart, A.W. (1998) Asymptotic Statistics. Cambridge University Press.