Hostname: page-component-6dbcb7884d-64ff6 Total loading time: 0 Render date: 2025-02-14T08:16:45.588Z Has data issue: false hasContentIssue false
  • English
  • Français

The UMP Exact Test and the Confidence Interval for Person Parameters in IRT Models

Published online by Cambridge University Press:  01 January 2025

Xiang Liu Open the ORCID record for Xiang Liu [Opens in a new window] ,
Zhuangzhuang Han  and
Matthew S. Johnson
Xiang Liu*
Affiliation:
Teachers College of Columbia University
Zhuangzhuang Han
Affiliation:
Teachers College of Columbia University
Matthew S. Johnson
Affiliation:
Teachers College of Columbia University
*
Correspondence should be made to Xiang Liu, Department of Human Development, Teachers College of Columbia University, 525 West 120th Street, New York, NY 10027-6696, USA. Email: xl2438@tc.columbia.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

In educational and psychological measurement when short test forms are used, the asymptotic normality of the maximum likelihood estimator of the person parameter of item response models does not hold. As a result, hypothesis tests or confidence intervals of the person parameter based on the normal distribution are likely to be problematic. Inferences based on the exact distribution, on the other hand, do not suffer from this limitation. However, the computation involved for the exact distribution approach is often prohibitively expensive. In this paper, we propose a general framework for constructing hypothesis tests and confidence intervals for IRT models within the exponential family based on exact distribution. In addition, an efficient branch and bound algorithm for calculating the exact p value is introduced. The type-I error rate and statistical power of the proposed exact test as well as the coverage rate and the lengths of the associated confidence interval are examined through a simulation. We also demonstrate its practical use by analyzing three real data sets.

Keywords

IRTexponential familyUMPexact distributionalgorithmp valuesufficient statisticconfidence interval
Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 1 , March 2018 , pp. 182 - 202
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agresti, A., (2003). Dealing with discreteness: Making ‘exact’ confidence intervals for proportions, differences of proportions, and odds ratios more exact, Statistical Methods in Medical Research, 12(1) 3–21.CrossRefGoogle ScholarPubMed
Baker, F.B., Kim, S-H, (2004). Item response theory: Parameter estimation techniques, 2Boca Raton: CRC Press.CrossRefGoogle Scholar
Biehler, M., Holling, H., Doebler, P., (2014). Saddlepoint approximations of the distribution of the person parameter in the two parameter logistic model, Psychometrika, 80(3) 665–688.CrossRefGoogle ScholarPubMed
Bock, D.R., Lieberman, M., (1970). Fitting a response model for n dichotomously scored items, Psychometrika, 35(2) 179–197.CrossRefGoogle Scholar
Brent, R.P., (1973). Algorithms for minimization without derivatives, Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Casella, G., Berger, R., (2001). Statistical inference, Textbook Binding: Duxbury Resource Center.Google Scholar
Doebler, A., Doebler, P., Holling, H., (2012). Optimal and most exact confidence intervals for person parameters in item response theory models, Psychometrika, 78(1) 98–115.CrossRefGoogle ScholarPubMed
Fisher, R.A., (1935). The design of experiments, Edinburgh: Oliver and Boyd.Google Scholar
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B., (2013). Bayesian data analysis, 3 Boca Raton: CRC.CrossRefGoogle Scholar
Hagell, P., Westergren, A., (2011). Measurement properties of the SF-12 health survey in Parkinson’s disease, Journal of Parkinson’s Disease, 1, 185–196.CrossRefGoogle ScholarPubMed
Hambleton, R.K., Swaminathan, H., (1985). Item response theory: Principles and applications, Boston: Kluwer-Nijhoff Publishing.CrossRefGoogle Scholar
Johnson, M. S., (2004). Item response models and their use in measuring food insecurity and hunger. In Paper presented at the workshop on the measurement of food insecurity and hunger. The national academy of science panel to review USDA’s measurement of food insecurity and hunger.Google Scholar
Klauer, K.C., (1991). Exact and best confidence intervals for the ability parameter of the Rasch model, Psychometrika, 56(3) 535–547.CrossRefGoogle Scholar
Land, A.H., Doig, A.G., (1960). An automatic method of solving discrete programming problems, Econometrica, 28(3) 497–520.CrossRefGoogle Scholar
Leiserson, C.C.E., Rivest, R.R.L., Stein, C., Cormen, T.H., (2009). Introduction to algorithms, 3Cambridge: The MIT Press.Google Scholar
Liou, M., Chang, C-H, (1992). Constructing the exact significance level for a person fit statistic, Psychometrika, 57(2) 169–181.CrossRefGoogle Scholar
Little, J.D.C., Murty, K.G., Sweeney, D.W., Karel, C., (1963). An algorithm for the traveling salesman problem, Operations Research, 11(6) 972–989.CrossRefGoogle Scholar
Lord, F.M., (1980). Applications of item response theory to practical testing problems, Broadway, NJ: Lawrence Erlbaum Associates, Inc..Google Scholar
Lord, F.M., (1983). Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability, Psychometrika, 48(2) 233–245.CrossRefGoogle Scholar
Lugannani, R., Rice, S., (1980). Saddle point approximation for the distribution of the sum of independent random variables, Advances in Applied Probability, 12(2) 475.CrossRefGoogle Scholar
Mair, P., Hatzinger, R., (2007). CML based estimation of extended Rasch models with the eRm package in R, Psychology Science, 49(1) 26–43.Google Scholar
Molenaar, I.W., Hoijtink, H., (1990). The many null distributions of person fit indices, Psychometrika, 55(1) 75–106.CrossRefGoogle Scholar
Rizopoulos, D., (2006). ltm: An R package for latent variable modeling and item response theory analyses, Journal of Statistical Software, 17(5) 1–25.CrossRefGoogle Scholar
Thissen, D., (2016). Bad questions: An essay involving item response theory, Journal of Educational and Behavioral Statistics, 41(1) 81–89.CrossRefGoogle Scholar
Ware, J., Kosinski, M., Keller, S.D., (1996). A 12-item short-form health survey: Construction of scales and preliminary tests of reliability and validity, Medical Care, 34(3) 220–33.CrossRefGoogle ScholarPubMed
Warm, T.A., (1989). Weighted likelihood estimation of ability in item response theory, Psychometrika, 54(3) 427–450.CrossRefGoogle Scholar
Wasserman, L., (2004). All of statistics New York, NY: Springer.CrossRefGoogle Scholar