Hostname: page-component-6dbcb7884d-fw45b Total loading time: 0 Render date: 2025-02-14T08:54:38.910Z Has data issue: false hasContentIssue false
  • English
  • Français

Machine Learning Analysis of Digital Clock Drawing Test Performance for Differential Classification of Mild Cognitive Impairment Subtypes Versus Alzheimer’s Disease

Published online by Cambridge University Press:  23 March 2020

Russell Binaco ,
Nicholas Calzaretto ,
Jacob Epifano ,
Sean McGuire ,
Muhammad Umer ,
Sheina Emrani Open the ORCID record for Sheina Emrani [Opens in a new window] ,
Victor Wasserman Open the ORCID record for Victor Wasserman [Opens in a new window] ,
David J. Libon  and
Robi Polikar Open the ORCID record for Robi Polikar [Opens in a new window]
Russell Binaco
Affiliation:
Signal Processing and Pattern Recognition Laboratory, Rowan University Glassboro, Glassboro, NJ, USA
Nicholas Calzaretto
Affiliation:
Signal Processing and Pattern Recognition Laboratory, Rowan University Glassboro, Glassboro, NJ, USA
Jacob Epifano
Affiliation:
Signal Processing and Pattern Recognition Laboratory, Rowan University Glassboro, Glassboro, NJ, USA
Sean McGuire
Affiliation:
Signal Processing and Pattern Recognition Laboratory, Rowan University Glassboro, Glassboro, NJ, USA
Muhammad Umer
Affiliation:
Signal Processing and Pattern Recognition Laboratory, Rowan University Glassboro, Glassboro, NJ, USA
Sheina Emrani
Affiliation:
Department of Psychology, Rowan University, Glassboro, NJ, USA
Victor Wasserman
Affiliation:
Department of Psychology, Rowan University, Glassboro, NJ, USA
David J. Libon
Affiliation:
Department of Psychology, Rowan University, Glassboro, NJ, USA New Jersey Institute for Successful Aging, School of Osteopathic Medicine, Rowan University, Stratford, NJ, USA
Robi Polikar*
Affiliation:
Signal Processing and Pattern Recognition Laboratory, Rowan University Glassboro, Glassboro, NJ, USA
*
*Correspondence and reprint requests to: Robi Polikar, PhD, Signal Processing and Pattern Recognition Laboratory, Rowan University, Glassboro, NJ, USA. E-mail: polikar@rowan.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

To determine how well machine learning algorithms can classify mild cognitive impairment (MCI) subtypes and Alzheimer’s disease (AD) using features obtained from the digital Clock Drawing Test (dCDT).

Methods:

dCDT protocols were administered to 163 patients diagnosed with AD(n = 59), amnestic MCI (aMCI; n = 26), combined mixed/dysexecutive MCI (mixed/dys MCI; n = 43), and patients without MCI (non-MCI; n = 35) using standard clock drawing command and copy procedures, that is, draw the face of the clock, put in all of the numbers, and set the hands for “10 after 11.” A digital pen and custom software recorded patient’s drawings. Three hundred and fifty features were evaluated for maximum information/minimum redundancy. The best subset of features was used to train classification models to determine diagnostic accuracy.

Results:

Neural network employing information theoretic feature selection approaches achieved the best 2-group classification results with 10-fold cross validation accuracies at or above 83%, that is, AD versus non-MCI = 91.42%; AD versus aMCI = 91.49%; AD versus mixed/dys MCI = 84.05%; aMCI versus mixed/dys MCI = 84.11%; aMCI versus non-MCI = 83.44%; and mixed/dys MCI versus non-MCI = 85.42%. A follow-up two-group non-MCI versus all MCI patients analysis yielded comparable results (83.69%). Two-group classification analyses were achieved with 25–125 dCDT features depending on group classification. Three- and four-group analyses yielded lower but still promising levels of classification accuracy.

Conclusion:

Early identification of emergent neurodegenerative illness is criterial for better disease management. Applying machine learning to standard neuropsychological tests promises to be an effective first line screening method for classification of non-MCI and MCI subtypes.

Keywords

Clock drawingThe Digital Clock Drawing TestMild cognitive impairmentMachine learningCognitive assessment
Type
Regular Research
Information
Journal of the International Neuropsychological Society , Volume 26 , Issue 7 , August 2020 , pp. 690 - 700
Copyright
Copyright © INS. Published by Cambridge University Press, 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

These authors contributed equally to the work.

References

REFERENCES

Alzheimer’s Association. (2019). 2019 Alzheimer’s disease facts and figures. Alzheimer’s Dementia, 15(3), 321–87. Retrieved from https://www.alz.org/facts/CrossRefGoogle Scholar
Aerts, L., Heffernan, M., Kochan, N.A., Crawford, J.D., Draper, B., Trollor, J.N., Sachdev, P.S., & Brodaty, H. (2017). Effects of MCI subtype and reversion on progression to dementia in a community sample. Neurology, 88(23), 2225–2232. doi: 10.1212/WNL.0000000000004015CrossRefGoogle Scholar
Ahmed, S., Brennan, L., Eppig, J., Price, C.C., Lamar, M., Delano-Wood, L., Bangen, K.J., Edmonds, E.C., Clark, L., Nation, D.A., Jak, A., Au, R., Swenson, R., Bondi, M.W., & Libon, D.J. (2016). Visuoconstructional impairment in subtypes of mild cognitive impairment. Applied Neuropsychology. Adult, 23(1), 43–52. doi: 10.1080/23279095.2014.1003067CrossRefGoogle ScholarPubMed
Ainslie, N.K., & Murden, R.A. (1993). Effect of education on the clock drawing dementia screen in non-demented elderly persons. Journal of the American Geriatric Society, 1993, 41, 249–252.CrossRefGoogle ScholarPubMed
Amato, F., López, A., Peña-Méndez, E.M., Vaňhara, P., Hampl, A., & Havel, J. (2013). Artificial neural networks in medical diagnosis. Journal of Applied Biomedicine, 11(2), 47–58. doi: 10.2478/v10136-012-0031-xCrossRefGoogle Scholar
Autio, L., Juhola, M., & Laurikkala, J. (2007). On the neural network classification of medical data and an endeavour to balance non-uniform data sets with artificial data extension. Computers in Biology and Medicine, 37(3), 388–397. doi: 10.1016/j.compbiomed.2006.05.001CrossRefGoogle Scholar
Beretta, L., & Santaniello, A. (2016). Nearest neighbor imputation algorithms: a critical evaluation. BMC Medical Informatics and Decision Making, 16(S3), 74. doi: 10.1186/s12911-016-0318-zCrossRefGoogle ScholarPubMed
Brodaty, H., & Moore, C.M. (1997). The clock drawing test for dementia of the Alzheimer’s type: a comparison of three scoring methods in a memory disorders clinic. International Journal of Geriatric Psychiatric, 12, 619–627.3.0.CO;2-H>CrossRefGoogle Scholar
Brown, G. (2009). A new perspective for information theoretic feature selection. Journal of Machine Learning Research – Proceedings Track, 5(1), 49–56.Google Scholar
Caballero, J., Ownby, R.L., Jacobs, R.J., Pandya, N., Hardigan, P.C. & Ricabal, L.C. (2018). Predicting medication adherence in older Hispanic patients with type 2 diabetes. American Journal of Health Systems Pharmacy, 75(9):e194–e201. doi: 10.2146/ajhp170067.CrossRefGoogle ScholarPubMed
Cahn-Weiner, D.A., Williams, K., Grace, J., Tremont, G., Westervelt, H., & Stern, R.A. (2003). Discrimination of dementia with lewy bodies from Alzheimer disease and Parkinson disease using the clock drawing test. Cognitive and Behavioral Neurology : Official Journal of the Society for Behavioral and Cognitive Neurology, 16(2), 85–92.CrossRefGoogle ScholarPubMed
Carew, T.G., Lamar, M., Cloud, B.S., Grossman, M., & Libon, D.J. (1997). Impairment in category fluency in ischemic vascular dementia. Neuropsychology, 11(3), 400–412. doi: 10.1037/0894-4105.11.3.400CrossRefGoogle ScholarPubMed
Chawla, N.V., Bowyer, K.W., Hall, L.O., & Kegelmeyer, W.P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. doi: 10.1613/jair.953CrossRefGoogle Scholar
Cohen, J., Penney, D.L., Davis, R., Libon, D.J., Swenson, R.A., Ajilore, O., Kumar, A., & Lamar, M. (2014). Digital clock drawing : differentiating “Thinking” versus “Doing” in younger and older adults with depression. Journal of the International Neuropsychological Society, 20, 1–9. doi: 10.1017/S1355617714000757CrossRefGoogle ScholarPubMed
Cosentino, S., Jefferson, A., Chute, D.L., Kaplan, E., & Libon, D.J. (2004). Clock drawing errors in dementia: neuropsychological and neuroanatomical considerations. Cognitive and Behavioral Neurology : Official Journal of the Society for Behavioral and Cognitive Neurology, 17(2), 74–84. doi: http://dx.doi.org/10.1097/01.wnn.0000119564.08162.46CrossRefGoogle ScholarPubMed
Davis, R., Libon, D.J., Au, R., Pitman, D., & Penney, D.L. (2014). THink: Inferring Cognitive Status from Subtle Behaviors. In Proceedings of the Conference on Innovative Applications of Artificial Intelligence 2014 (pp. 2898–2905). Québec City, Québec, Canada.Google Scholar
Davis, R., Penney, D.L., Pittman, D., Libon, D.J., Swenson, R., & Kaplan, E. (2010). The Digital Clock Drawing Test (dCDT)-I: Development of a New computerize Quantitative System. InAnnual Meeting of The International Neuropsychological Society. Montreal, Canada.Google Scholar
Delis, D.C., Kramer, J., Kaplan, E., & Ober, B. (2000). The California Berbal Learning Test-II. New York, NY: Psychology Corporation.Google Scholar
Díaz-Mardomingo, M., García-Herranz, S., Rodríguez-Fernández, R., Venero, C., & Peraita, H. (2017). Problems in Classifying Mild Cognitive Impairment (MCI): One or multiple syndromes? Brain Sciences, 7(12), 111. doi: 10.3390/brainsci7090111CrossRefGoogle ScholarPubMed
Dykiert, D., Der, G., Starr, J.M., & Deary, I.J. (2016). Why is Mini-Mental state examination performance correlated with estimated premorbid cognitive ability? Psychological Medicine, 46(12), 2647–2654. doi: 10.1017/S0033291716001045CrossRefGoogle ScholarPubMed
Ehreke, L., Luppa, M., Luck, T., Wiese, B., Weyerer, S., Eifflaender-Gorfer, S., Weeg, D., Olbrich, J., van den Bussche, H., Bachmann, C., Eisele, M., Maier, W., Jessen, F., Wagner, M., Fuchs, A., Pentzek, M., Angermeyer, M.C., König, H., & Riedel-Heller, S.G. (2009). Is the clock drawing test appropriate for screening for mild cognitive impairment? – Results of the German study on ageing, cognition and dementia in primary care patients (AgeCoDe). Dementia and Geriatric Cognitive Disorders, 28, 365–372.CrossRefGoogle Scholar
Emrani, S., Libon, D.J., Lamar, M., Price, C.C., Jefferson, A.L., Gifford, K.A., Hohman, T.J., Nation, D.A., Delano-Wood, L., Jak, A., Bangen, K.J., Bondi, M.W., Brickman, A.M., Manly, J., Swenson, R., & Au, R. (2018). Assessing working memory in Mild Cognitive Impairment with serial order recall. Journal of Alzheimer’s Disease : JAD, 61(3), 917–928. doi: 10.3233/JAD-170555CrossRefGoogle ScholarPubMed
Emrani, S., Wasserman, V., Matusz, E., Miller, D., Lamar, M., Price, C.C., Ginsberg, T.B., Au, R., Swenson, R., & Libon, D.J. (2019). Visual versus verbal working memory in statistically-determined patients with mild cognitive impairment – On behalf of the consortium for Clinical and Epidemiological Neuropsychological Data Analysis (CENDA). Journal of the International Neuropsychological Society, 25(10), 1001-1010. doi: 10.1017/S1355617719000808CrossRefGoogle Scholar
Eppig, J., Wambach, D., Nieves, C., Price, C.C., Lamar, M., Delano-Wood, L., Giovannetti, T., Bettcher, B.M., Penney, D.L., Swenson, R., Lippa, C., Kabasakalian, A., Bondi, M.W., & Libon, D.J. (2012). Dysexecutive functioning in mild cognitive impairment: Derailment in temporal gradients. Journal of the International Neuropsychological Society, 18(1), 20–28. doi: 10.1017/S1355617711001238CrossRefGoogle ScholarPubMed
Feeney, J., Savva, G.M., O’Regan, C., King-Kallimanis, B., Cronin, H., & Kenny, R.A. (2016). Measurement error, reliability, and minimum detectable change in the Mini-Mental state examination, montreal cognitive assessment, and color trails test among community living middle-aged and older adults. Journal of Alzheimer’s Disease : JAD, 53(3), 1107–1114. doi: 10.3233/JAD-160248CrossRefGoogle ScholarPubMed
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189–198. doi: 10.1016/0022-3956(75)90026-6CrossRefGoogle ScholarPubMed
Freedman, M., Leach, L., Kaplan, E., Winocur, G., Shulman, K.I., & Delis, D.C. (1994). Clock drawing: A neuropsychological analysis. New York, NY, US: Oxford University Press.Google Scholar
Hetland, A.J., Carr, D.B., Wallendorf, M.J., & Barco, P.P., (2014). Potentially driver-impairing (PDI) medication use in medically impaired adults referred for driving evaluation. Annals of Pharmacotherapy, 48, 476–482.CrossRefGoogle ScholarPubMed
Holsinger, T., Plassman, B.L., Stechuchak, K.M., Burke, J.R., Coffman, C.J., & Williams, J.W. Jr. (2015). Stability of diagnoses of cognitive impairment, not dementia in a veterans affairs primary care population. Journal of the American Geriatric Society, 63, 1105–1111.CrossRefGoogle ScholarPubMed
Jak, A.J., Bondi, M.W., Delano-Wood, L., Wierenga, C., Corey-Bloom, J., Salmon, D.P., & Delis, D.C. (2009). Quantification of five neuropsychological approaches to defining mild cognitive impairment. The American Journal of Geriatric Psychiatry : Official Journal of the American Association for Geriatric Psychiatry, 17(5), 368–75. doi: 10.1097/JGP.0b013e31819431d5CrossRefGoogle ScholarPubMed
Kaplan, E., Goodglass, H., & Weintraub, S. (1983). The Boston Naming Test. Philadelphia, PA: Lea and Febiger.Google Scholar
Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. In International Conference on Learning Representations.Google Scholar
Kirby, M., Denihan, A., Bruce, I., Coakley, D., & Lawlor, B.A. (2001). The clock drawing test in primary care: sensitivity in dementia detection and specificity against normal and depressed elderly. International Journal of Geriatric Psychiatry 16, 935–940.CrossRefGoogle ScholarPubMed
Lamar, M., Ajilore, O., Leow, A., Charlton, R., Cohen, J., GadElkarim, J., Yang, S., Zhang, A., Davis, R., Penney, D., Libon, D.J., & Kumar, A. (2016). Cognitive and connectome properties detectable through individual differences in graphomotor organization. Neuropsychologia, 85, 301–309. doi: 10.1016/j.neuropsychologia.2016.03.034CrossRefGoogle ScholarPubMed
Lamar, M., Price, C.C., Cynthia, D., Kaplan, E., & Libon, D.J. (2002). Capacity to maintain mental set in dementia. Neuropsychologia, 40(4), 435–445. doi: 10.1016/s0028-3932(01)00125-7CrossRefGoogle ScholarPubMed
Libon, D.J., Bondi, M.W., Price, C.C., Lamar, M., Eppig, J., Wambach, D.M., Nieves, C., Delano-Wood, L., Giovannetti, T., Lippa, C., Kabasakalian, A., Cosentino, S., Swenson, R., & Penney, D.L. (2011). Verbal serial list learning in mild cognitive impairment: A profile analysis of interference, forgetting, and errors. Journal of the International Neuropsychological Society, 17, 905–914.CrossRefGoogle ScholarPubMed
Libon, D.J., Malamut, B.L., Swenson, R., Sands, L.P., & Cloud, B.S. (1996). Further analyses of clock drawings among demented and nondemented older subjects. Archives of Clinical Neuropsychology, 11(3), 193–205. doi: 10.1016/0887-6177(95)00026-7CrossRefGoogle ScholarPubMed
Libon, D.J., Penney, D.L., Davis, R., Tabby, D.S., Eppig, J., Nieves, C., Bloch, A., Donohue, J.B., Brennan, L., Rife, K.L., Wicas, G., Lamar, M., Price, C.C., Au, R., Swenson, R., & Garrett, K.D. (2014). Deficits in processing speed and decision making in relapsing-remitting multiple sclerosis : the Digital Clock Drawing Test (dCDT), Journal of Multiple Sclerosis, 1(2), 1–8. doi: 10.4172/jmso.1000113Google Scholar
McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, E.M. (1984). Clinical diagnosis of Alzheimer’s disease: Report of the NINCDS-ADRDA Work Group* under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology, 34(7), 939–939. doi: 10.1212/WNL.34.7.939CrossRefGoogle ScholarPubMed
Nasreddine, Z.S., Phillips, N.A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J.L., & Chertkow, H, (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatric Society, 53, 695–699.CrossRefGoogle ScholarPubMed
Pandya, S.Y., Lacritz, L.H., Weiner, M.F., Deschner, M., & Woon, F.L. (2017). Predictors of reversion from mild cognitive impairment to normal cognition. Dementia and Geriatric Cognitive Disorders, 43(3–4), 204–214. doi: 10.1159/000456070CrossRefGoogle ScholarPubMed
Penney, D.L., Davis, R., Libon, D.J., Lamar, M., Price, C.C., Swenson, R., Garrett, K.D., Freedland, A., Weninger, C., Scala, S., Giovannetti, T., & Kaplan, E. (2010). The Digital Clock Drawing Test (dCDT)-II: A New computerized Quantitative System. In Annual Meeting of The International Neuropsychological Society. Montreal, Canada.Google Scholar
Penney, D.L., Libon, D.J., Lamar, M., Swenson, R., Price, C.C., Weninger, C., Freedland, A., Garrett, K.D., Scala, S., & Davis, R. (2010). The Digital Clock Drawing Test (dCDT)-III: Clinician reliability for a New Quantitative System. In Annual Meeting of The International Neuropsychological Society. Montreal, Canada.Google Scholar
Price, C.C., Cunningham, H., Coronado, N., Freedland, A., Cosentino, S., Penney, D.L., Penisi, A., Bowers, D., Okun, M.S., & Libon, D.J. (2011). Clock drawing in the montreal cognitive assessment: Recommendations for dementia assessment. Dementia and Geriatric Cognitive Disorders, 31(3), 179–187. doi: 10.1159/000324639CrossRefGoogle ScholarPubMed
Rabin, L.A., Barr, W.B., & Burton, L.A. (2005). Assessment practices of clinical neuropsychologists in the United States and Canada : A survey of INS, NAN, and APA Division 40 members. Archives of Clinical Neuropsychology, 20(October 2017), 33–65. doi: 10.1016/j.acn.2004.02.005CrossRefGoogle ScholarPubMed
Reinera, K., Eichlera, T., Hertela, J., Hoffmann, W., Thyriana, J.R. (2018). The clock drawing test: A reasonable instrument to assess probable dementia in primary care? Current Alzheimer Research, 15, 38–43.Google Scholar
Reitan, R.M., & Wolfson, D. (1985). The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation (Vol. 4). Tucson, AZ: Neuropsychology Press.Google Scholar
Rouleau, I., Salmon, D.P., Butters, N., Kennedy, C., & McGuire, K. (1992). Quantitative and qualitative analyses of clock drawings in Alzheimer’s and Huntington’s disease. Brain and Cognition, 18(1), 70–87.CrossRefGoogle ScholarPubMed
Royall, D.R., Cordes, J.A., & Polk, M. (1998). CLOX: an executive clock drawing task. Journal of Neurology, Neurosurgery & Psychiatry, 64(5), 588–594. doi: 10.1136/jnnp.64.5.588CrossRefGoogle Scholar
Royall, D.R., Palmer, R.F., & Markides, K.S. (2017). Exportation and validation of latent constructs for dementia case finding in a Mexican American population-based cohort. Journals of Gerontology – Series B Psychological Sciences and Social Sciences, 72(6), 947–955. doi: 10.1093/geronb/gbw004Google Scholar
Schillerstrom, J.E., Sawyer Baker, P., Allman, R.M., Rungruang, B., Zamrini, E., & Royall, D.R. (2007). Clock drawing phenotypes in community-dwelling African Americans and Caucasians: Results from the University of Alabama at Birmingham study of aging. Neuroepidemiology, 28(3), 175–180. doi: 10.1159/000104095CrossRefGoogle ScholarPubMed
Shimada, H., Doi, T., Lee, S., & Makizako, H. (2019). Reversible predictors of reversion from mild cognitive impairment to normal cognition: A 4-year longitudinal study. Alzheimer’s Research and Therapy, 11(1), 1–9. doi: 10.1186/s13195-019-0480-5Google ScholarPubMed
Shulman, K.I. (1993). Clock-drawing and dementia in the community: A longitudinal study. International Journal of Geriatric Psychiatry, 8(6), 487–496.CrossRefGoogle Scholar
Souillard-Mandar, W., Davis, R., Rudin, C., Au, R., Libon, D.J., Swenson, R., Price, C.C., Lamar, M., & Penney, D.L. (2016). Learning classification models of cognitive conditions from subtle behaviors in the Digital Clock Drawing Test. Machine Learning: Special issue for Healthcare and Medicine, 102, 393–441.CrossRefGoogle ScholarPubMed
Spreen, O., & Strauss, E.A. (1990). Compendium of Neuropsychological Tests. New York: Oxford University Press.Google Scholar
Tuokko, H., Hadjistavropoulos, T., Miller, J.A., & Beattie, B.L. (1992). The clock test: A sensitive measure to differentiate normal elderly from those with Alzheimer disease. Journal of the American Geriatrics Society, 40(6), 579–584. doi: 10.1111/j.1532-5415.1992.tb02106.xCrossRefGoogle ScholarPubMed
Wasserman, V., Emrani, S., Matusz, E.F., Miller, D., Garrett, K.D., Gifford, K.A., Hohman, T.J., Jefferson, A.L., Au, R., Swenson, R., & Libon, D.J. (2019). Visual and verbal serial list learning in patients with statistically-determined mild cognitive impairment. Innovation in Aging, 3(2), 1–12. doi: 10.1093/geroni/igz009CrossRefGoogle ScholarPubMed
Wasserman, V., Emrani, S., Matusz, E.F., Peven, J.C., Cleary, S., Price, C.C., Ginsberg, T.B., Swenson, R., Heilman, K.M., Lamar, M., & Libon, D.J. Visuospatial Performance in Patients with Statistically-Defined Mild Cognitive Impairment (submitted for publication).Google Scholar
Wechsler, D. (1997). WAIS-III: Administration and scoring manual: Wechsler adult intelligence scale. Psychological Corporation.Google Scholar
Supplementary material: File

Binaco et al. supplementary material

Tables S1-S10

Download Binaco et al. supplementary material(File)
File 38.5 KB