Study of statistical procedures for diagnostic evaluation in educational contexts research team:

Some results:

7 Ma, W., Sorrel, M. A., Ge, Y., & Zhai, X. (manuscript accepted for publication). A dual-purpose model for estimating ability and misconceptions. Journal of Educational Measurement.

Existing diagnostic models mainly identify students’ mastery of skills, with limited focus on uncovering specific scientific misconceptions. This article introduces a dual-purpose model (GDPM) to concurrently estimate overall ability and detect misconceptions. An expectation-maximization algorithm is devised for parameter estimation. A simulation study assesses parameter recovery accuracy under diverse conditions. Real data from science education validates the model’s practical viability.

6 de la Torre, J., & Sorrel, M. A. (2023). Cognitive Diagnosis Models. In F. Ashby, H. Colonius, & E. Dzhafarov (Eds.), New Handbook of Mathematical Psychology (Cambridge Handbooks in Psychology, pp. 385-420). Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108902724.010

Cognitive diagnosis models, originating in educational measurement, offer detailed insights for formative assessment by classifying examinees on binary attributes. This chapter provides a concise overview of original models, extensions, and recent methodological developments. Topics covered include model estimation, Q-matrix specification, model fit evaluation, and procedures for validity and reliability. The chapter concludes with a discussion on future trends in the field.

5 Sorrel, M. A., Escudero, S., Nájera, P., Kreitchmann, R. S., & Vázquez-Lira, R. (2023). Exploring approaches for estimating parameters in cognitive diagnosis models with small sample sizes. Psych, 5(2), 336-349. https://doi.org/10.3390/psych5020023

Commonly used MMLE-EM estimation in CDMs faces issues with small sample sizes. This study compares various estimation methods for CDMs across sample sizes, using simulated and real data. Methods include MMLE-EM, Bayes modal, Markov chain Monte Carlo, non-parametric, and parsimonious parametric models like Restricted DINA. Findings show alternatives outperform MMLE-EM with small samples, yielding comparable results with larger samples. Practitioners should consider alternatives for accurate CDM parameter estimates with small samples. This study guides parameter estimation to maximize CDM potential.

4 Nájera, P., Abad, F.J., Chiu, C-Y., & Sorrel, M.A. (manuscript accepted for publication). A Comprehensive Cognitive Diagnostic Method for Classroom-Level Assessments. Journal of Educational and Behavioral Statistics.

Here we propose a one-parameter CDM that can provides accurate classifications and posterior probabilities under small sample scenarios. The model is already available at the cdmTools R package, please check the RDINA() function at the package version at Github (https://github.com/Pablo-Najera/cdmTools/). We’ll submit shortly an updated version of the package to CRAN.

3 Sanz, S., Kreitchmann, R. S., Nájera, P., Moreno, J. D., Martínez-Huertas, J. A., & Sorrel, M. A.. FoCo: A Shiny app for formative assessment using cognitive diagnosis modeling. Psicología Educativa.

In this paper we examine the FoCo Shiny app to conduct CDM analysis. This is a user-friendly menu-based software in Spanish. The goal is to make CDM analysis more accesible to the broader audience and potential users.

You can find the app here: https://psychometricmodelling.shinyapps.io/FoCo/

2 Kreitchmann, R. S., de la Torre, J., Sorrel, M. A., Nájera, P., & Abad, F. J. (2022). Improving reliability estimation in cognitive diagnosis modeling. Behavior Research Methods, 1-15. https://doi.org/10.3758/s13428-022-01967-5

Item parameter estimates under small sample settings might be unreliable. Thus, in this paper we proposed a multiple imputation procedure to account for the item parameter uncertainty in the computation of classification accuracy measures.

This new procedure is available at the cdmTools R package. The only required input is a calibrated CDM using the GDINA R package:

cdmTools::CA.MI(GDINA.obj)

1 Franco-Martínez, A., Alvarado, J., & Sorrel, M. A. (in press). Range restriction affects factor analysis: Normality, estimation, fit, loadings, and reliability. Educational and Psychological Measurement. https://doig.org/10.1177/00131644221081867

Range restriction is likely to happen when the sampling procedure is not random. In this paper we illustrate the implications that this might have in the context of factor analysis.