Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-02-05T23:34:22.162Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian Estimation of the DINA Q matrix

Published online by Cambridge University Press:  01 January 2025

Yinghan Chen ,
Steven Andrew Culpepper ,
Yuguo Chen  and
Jeffrey Douglas
Yinghan Chen
Affiliation:
University of Nevada, Reno
Steven Andrew Culpepper*
Affiliation:
University of Illinois at Urbana-Champaign
Yuguo Chen
Affiliation:
University of Illinois at Urbana-Champaign
Jeffrey Douglas
Affiliation:
University of Illinois at Urbana-Champaign
*
Correspondence should be made to Steven Andrew Culpepper, Department of Statistics, University of Illinois at Urbana-Champaign, 725 South Wright Street, Champaign, IL 61820, USA. Email: sculpepp@illinois.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Cognitive diagnosis models are partially ordered latent class models and are used to classify students into skill mastery profiles. The deterministic inputs, noisy “and” gate model (DINA) is a popular psychometric model for cognitive diagnosis. Application of the DINA model requires content expert knowledge of a Q matrix, which maps the attributes or skills needed to master a collection of items. Misspecification of Q has been shown to yield biased diagnostic classifications. We propose a Bayesian framework for estimating the DINA Q matrix. The developed algorithm builds upon prior research (Chen, Liu, Xu, & Ying, in J Am Stat Assoc 110(510):850–866, 2015) and ensures the estimated Q matrix is identified. Monte Carlo evidence is presented to support the accuracy of parameter recovery. The developed methodology is applied to Tatsuoka’s fraction-subtraction dataset.

Keywords

cognitive diagnosis modelsdeterministic inputsnoisy “and” gate (DINA) modelQ matrixBayesian statisticsfraction-subtraction data
Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 1 , March 2018 , pp. 89 - 108
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-017-9579-4) contains supplementary material, which is available to authorized users.

References

Chen, Y., Liu, J., Xu, G., Ying, Z., (2015). Statistical analysis of Q-matrix based diagnostic classification models, Journal of the American Statistical Association, 110(510) 850–866.CrossRefGoogle Scholar
Chiu, C-Y, (2013). Statistical refinement of the Q-matrix in cognitive diagnosis, Applied Psychological Measurement, 37(8) 598–618.CrossRefGoogle Scholar
Chiu, C-YDouglas, J.A., Li, X., (2009). Cluster analysis for cognitive diagnosis: Theory and applications, Psychometrika, 74(4) 633–665.CrossRefGoogle Scholar
Chiu, C-YKöhn, H-F, (2016). The reduced RUM as a logit model: Parameterization and constraints, Psychometrika, 81(2) 350–370.CrossRefGoogle ScholarPubMed
Chiu, C-YKöhn, H-FWu, H-M, (2016). Fitting the reduced RUM with Mplus: A tutorial, International Journal of Testing, 16, 1–21.CrossRefGoogle Scholar
Chung, M., (2014). Estimating the Q-matrix for Cognitive Diagnosis Models in a Bayesian Framework (Unpublished doctoral dissertation). Columbia University.Google Scholar
Cowles, M. K., & Carlin, B. P., (1996). Markov chain Monte Carlo convergence diagnostics: A comparative review. Journal of the American Statistical Association, 91, 883–904.CrossRefGoogle Scholar
Culpepper, S.A., (2015). Bayesian estimation of the DINA model with Gibbs sampling, Journal of Educational and Behavioral Statistics, 40(5) 454–476.CrossRefGoogle Scholar
DeCarlo, L.T., (2010). On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the Q-matrix, Applied Psychological Measurement, 35, 8–26.CrossRefGoogle Scholar
DeCarlo, L.T., (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model, Applied Psychological Measurement, 36, 447–468.CrossRefGoogle Scholar
de la Torre, J., (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications, Journal of Educational Measurement, 45(4) 343–362.CrossRefGoogle Scholar
de la Torre, J., (2009). Estimation code for the G-DINA model. In Presentation at the meeting of the American Educational Research Association. San Diego, CA.Google Scholar
de la Torre, J., Chiu, C-Y, (2016). A general method of empirical Q-matrix validation, Psychometrika, 81(2) 253–273.CrossRefGoogle ScholarPubMed
de la Torre, J., Douglas, J.A., (2004). Higher-order latent trait models for cognitive diagnosis, Psychometrika, 69(3) 333–353.CrossRefGoogle Scholar
de la Torre, J., Douglas, J.A., (2008). Model evaluation and multiple strategies in cognitive diagnosis: An analysis of fraction subtraction data, Psychometrika, 73(4) 595–624.CrossRefGoogle Scholar
Geweke, J., (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. Bayesian Statistics, 4, 169–188.CrossRefGoogle Scholar
Henson, R. & Templin, J. (2007). Importance of Q-matrix construction and its effects cognitive diagnosis model results. In Annual meeting of the national council on measurement in education. Chicago, IL.Google Scholar
Henson, R., Templin, J., Willse, J., (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables, Psychometrika, 74, 191–210.CrossRefGoogle Scholar
Huff, K., Goodman, D.P., eds.Leighton, J.P., Gierl, M.J., (2007). The demand for cognitive diagnostic assessment, Cognitive diagnostic assessment for education: Theory and applications, Cambridge: Cambridge University Press pp(19–60.CrossRefGoogle Scholar
Junker, B.W., Sijtsma, K., (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory, Applied Psychological Measurement, 25(3) 258–272.CrossRefGoogle Scholar
Leighton, J.P., Gierl, M.J. eds.Leighton, J.P., Gierl, M.J., (2007). Why cognitive diagnostic assessment, Cognitive diagnostic assessment for education: Theory and applications, Cambridge: Cambridge University Press pp(3–18.CrossRefGoogle Scholar
Liu, J., (2017). On the consistency of Q-matrix estimation: A commentary, Psychometrika, 82(2) 523–527.CrossRefGoogle ScholarPubMed
Liu, J., Xu, G., Ying, Z., (2012). Data-driven learning of Q-matrix, Applied Psychological Measurement, 36(7) 548–5643733574.CrossRefGoogle ScholarPubMed
Liu, J., Xu, G., Ying, Z., (2013). Theory of the self-learning Q-matrix, Bernoulli, 19(5A) 1790–18174011940.CrossRefGoogle ScholarPubMed
Mislevy, R.J., Wilson, M., (1996). Marginal maximum likelihood estimation for a psychometric model of discontinuous development, Psychometrika, 61(1) 41–71.CrossRefGoogle Scholar
Norris, S.P., Macnab, J.S., Phillips, L.M. eds.Leighton, J.P., Gierl, M.J., (2007). Cognitive modeling of performance on diagnostic achievement tests, Cognitive diagnostic assessment for education: Theory and applications Cambridge: Cambridge University Press pp(61–84.CrossRefGoogle Scholar
Rupp, A. A., (2009). Software for calibrating diagnostic classification models: An overview of the current state-of-the-art. In Symposium conducted at the meeting of the American Educational Research Association, San Diego, CA.Google Scholar
Rupp, A.A., Templin, J.L., (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model, Educational and Psychological Measurement, 68(1) 78–96.CrossRefGoogle Scholar
Rupp, A.A., Templin, J.L., Henson, R.A., (2010). Diagnostic measurement: Theory, methods, and applications New York: Guilford Press.Google Scholar
Tatsuoka, C., (2002). Data analytic methods for latent partially ordered classification models, Journal of the Royal Statistical Society: Series C (Applied Statistics), 51(3) 337–350.Google Scholar
Tatsuoka, K. K.,(1984). Analysis of errors in fraction addition and subtraction problems. Computer-Based Education Research Laboratory, University of Illinois at Urbana-Champaign.Google Scholar
Templin, J.L., & Henson, R. A., (2006). A Bayesian method for incorporating uncertainty into Q-matrix estimation in skills assessment. In Symposium conducted at the meeting of the American Educational Research Association, San Diego, CA.Google Scholar
Templin, J.L., Hoffman, L., (2013). Obtaining diagnostic classification model estimates using Mplus, Educational Measurement: Issues and Practice, 32(2) 37–50.CrossRefGoogle Scholar
von Davier, M., (2014). The DINA model as a constrained general diagnostic model: Two variants of a model equivalency. British Journal of Mathematical and Statistical Psychology, 67(1), 49–71.CrossRefGoogle Scholar
Xiang, R., (2013). Nonlinear Penalized Estimation of True Q-matrix in Cognitive Diagnostic Models (Unpublished doctoral dissertation). Columbia University.Google Scholar
Xu, G., Zhang, S., (2016). Identifiability of diagnostic classification models, Psychometrika, 81(3) 625–649.CrossRefGoogle ScholarPubMed
Supplementary material: File

Chen et al. supplementary material

Chen et al. supplementary material 1
Download Chen et al. supplementary material(File)
File 2.2 KB
Supplementary material: File

Chen et al. supplementary material

Chen et al. supplementary material 2
Download Chen et al. supplementary material(File)
File 28.9 KB
Supplementary material: File

Chen et al. supplementary material

Chen et al. supplementary material 3
Download Chen et al. supplementary material(File)
File 1.4 KB