Significant outcomes
-
∙ We have presented an adaptation of the Alzheimer’s Disease Assessment Scale – Cognitive (ADAS-Cog) suitable for use in a low-literacy rural population in sub-Saharan Africa (SSA).
-
∙ Initial validation results indicate good sensitivity and specificity for dementia, high concurrent validity and excellent internal consistency and inter-rater reliability.
-
∙ This adaptation of the ADAS-Cog could be used to assess cognitive outcomes of intervention for dementia in this and similar environments. Further validation and field testing in similar settings in SSA would be useful.
Limitations
-
∙ This study included 66 individuals and was therefore relatively small.
-
∙ Cognitively normal control subjects were oversampled for illiteracy or lower educational level in order to ensure similar educational level to participants with dementia, resulting in a non-random sample.
-
∙ The cross-sectional design resulted in spectrum bias as those with mild cognitive impairment (MCI), and other conditions affecting cognition, such as severe depression or learning disability, were excluded.
-
∙ The reported sensitivity, specificity and area under the receiver operating characteristic curve (AUC) of the ADAS-Cog in this cohort may therefore be artificially high.
-
∙ Nevertheless, the primary purpose of the ADAS-Cog is not dementia diagnosis, but to measure cognitive change.
-
∙ Accurate dementia subtype diagnoses were not possible in this cohort due to a lack of neuroimaging facilities locally.
-
∙ The profile of cognitive deficits in individuals with dementia in this study may therefore differ from those with Alzheimer’s disease dementia (ADD) for which the ADAS-Cog was originally developed.
Introduction
The prevalence of dementia in SSA is set to rise by almost two and a half times the rate of increase in high-income countries (HICs) by 2040, as populations age and lifestyles change (Reference George-Carey, Adeloye and Chan1). Research interest in this area is increasing, with several recent epidemiological studies indicating prevalence in SSA similar to that reported in HICs (Reference Olayinka and Mbuyi2).
Cognitive assessment tools used in HICs for screening and diagnosis of dementia are not valid in many settings across SSA due to differences in cultural and educational background (Reference Paddick, Longdon and Gray3), but appropriate cognitive assessment tools for this setting have recently been developed (Reference Gray, Paddick and Kisoli4,Reference Toure, Coume, Ndiaye and Thiam5). These same cultural and educational differences between older adults in HICs and SSA would be expected to result in differing needs for dementia intervention, but to date no treatment trials have taken place outside of South Africa (Reference Ritchie, Ames, Clayton and Lai6). One difficulty is a lack of validated cognitive assessment tools designed to measure efficacy of interventions in this setting.
The Alzheimer’s Disease Assessment Scale – Cognitive (ADAS-Cog) is a comprehensive cognitive assessment tool designed to assess cognitive change in dementia treatment trials (Reference Rosen, Mohs and Davis7). It has been extensively used in trials of pharmacological and non-pharmacological interventions for dementia in HICs, and is therefore useful in allowing comparisons between studies. The ADAS-Cog has been translated and validated in a number of languages including Icelandic (Reference Hannesdóttir and Snaedal8), Turkish (Reference Mavioglu, Gedizlioglu, Akyel, Aslaner and Eser9,Reference Kolibas, Korinkova, Novotny, Vajdickova and Hunakova10), Slovak (Reference Kolibas, Korinkova, Novotny, Vajdickova and Hunakova10), Greek (Reference Tsolaki, Fountoulakis, Nakopoulou, Kazis and Mohs11), Italian, French, Portuguese (Reference Schultz, Siviero and Bertolucci12), Spanish (Reference Pena-Casanova, Aguilar and Santacruz13), Korean(Reference Youn, Lee and Kim14), and Chinese, but has been validated in few low and middle income country (LMIC) settings where low-literacy rates may result in some bias for tasks involving reading, drawing or writing. In fact, only one adaptation specifically accounting for illiteracy has been published. This study took place in China and included older people with a wide range of educational backgrounds, therefore adaptations were intended to accommodate both higher and lower educated individuals (Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). Performance on the adapted ADAS-Cog was unaffected by age or educational level, except amongst those with minimal or no education, suggesting that difficulties in cognitive assessment remained despite these adaptations. Our own work in rural Tanzania noted problems in administering cognitive screening assessments to low-literacy populations, where the drawing tasks within the Community Screening Instrument for Dementia (CSI-D) were poorly performed, with a high proportion of refusals (Reference Paddick, Longdon and Kisoli16). This is of particular note since the CSI-D was developed for use in low-literacy settings, including one site in Nigeria (Reference Hall, Ogunniyi, Hendrie and Osuntokun17). Other studies involving individuals with less than primary education have specifically excluded those without basic literacy (Reference Tsolaki, Fountoulakis, Nakopoulou, Kazis and Mohs11,Reference Schultz, Siviero and Bertolucci12).
Across much of SSA a high proportion of older adults are illiterate (Reference Paddick, Longdon and Gray3). This is due, in part, to the historical scarcity of formal education. The problem is more prevalent in rural areas and in older women. As a result, cognitive assessment tools designed for use in older adults in HICs are less effective in SSA. Surprisingly, even tools designed to be ‘culture-fair’ in other LMIC settings present problems in rural SSA, with high numbers of false positives and low specificity (Reference Gray, Paddick and Kisoli4,Reference Paddick, Longdon and Kisoli16). The ADAS-Cog includes a number of literacy-dependent items, including word recognition (where individuals are required to read and recall words presented on flashcards) and construction (requiring individuals to copy four simple drawings).
Aims of the study
We aimed to adapt the ADAS-Cog for use in low-literacy settings in SSA and conduct a feasibility study of this low-literacy ADAS-Cog in individuals with and without dementia in rural Tanzania. This was intended as the first step towards a valid cognitive assessment tool for intervention studies of pharmacological and non-pharmacological dementia treatment in low-literacy settings in SSA. Adaptation of a well-recognised and validated tool (ADAS-Cog) would also allow comparison of intervention efficacy with that reported in trials in HICs.
Materials and methods
Ethics
Ethical approval was obtained from the local research ethics committee at Kilimanjaro Christian Medical University College and from the National Institute of Medical Research (NIMR) in Dar-es-Salaam, Tanzania. Written informed consent was sought from participants. Those unable to read and write indicated consent/assent through a thumbprint. Where there were any concerns regarding capacity to consent due to cognitive impairment, written assent was obtained from the primary carer. Assent/consent was checked regularly during assessments.
Setting
The study took place in the Hai district of northern Tanzania, a demographic surveillance site (DSS) on the slopes of Mount Kilimanjaro. The majority of this rural population are subsistence farmers, with some producing cash crops, such as coffee and tomatoes. The overall educational level of older adults is low, and over two-thirds of older women are illiterate (Reference Paddick, Longdon and Gray3). Previous epidemiological data from Hai demonstrate an age-adjusted prevalence of dementia of 6.4% (Reference Longdon, Paddick and Kisoli18), similar to that reported in HICs. Swahili, the lingua franca for much of East Africa, is widely spoken along with a tribal language (Chagga).
Recruitment
Participants were recruited through a community dementia screening programme conducted in six villages in the rural Hai district as part of a separate published study (Reference Collingwood, Paddick and Kisoli19). In brief, all those with intermediate or high probability of dementia following screening, alongside a random sample of those with low probability of dementia, underwent a detailed interview, neurological examination and informant history with a research doctor, and diagnoses of dementia were made according to DSM-IV dementia criteria (20). Only those with mild or moderate dementia, as classified by the clinical dementia rating scale (CDR; categories 1 and 2), were included in this study.
Individuals with no cognitive impairment (controls) were recruited from those individuals with low probability of cognitive impairment following screening, who also had no evidence of significant functional impairment assessed using an instrumental activities of daily living (IADL) scale developed and validated for use in this setting (Reference Collingwood, Paddick and Kisoli19). We used convenience sampling to select participants from those with normal screening scores, but oversampled for illiteracy and less than primary education. This was done to allow comparison with scores achieved by those with known mild and moderate dementia, without bias due to differing educational level. Illiteracy was defined as the inability to read or write a short note, and education was recorded in number of years of school attendance.
Assessments were carried out jointly by a research doctor (S.-M.P.) and research nurse (A.K.) in September 2014. Both clinicians were experienced in working with people with dementia in Hai. Assessments took place either in individuals own homes, or a nearby primary healthcare building depending on participant preference and mobility.
Adaptation of the ADAS-Cog
The ADAS-Cog includes 11 items and is scored out of 70, with lower scores indicating less impairment. Three main subscales are produced; language, memory and new learning, and praxis. Where adaptation was required (e.g. choice of words for recall, ordering of questions and choice of objects), the adaption was discussed by an expert panel drawn from staff at Kilimanjaro Christian Medical University College and Kilimanjaro Christian Medical Centre and chaired by a UK-based doctor (S.-M.P.). All panel members had a professional interest in dementia. Panel members included a local social scientist/anthropologist (D.M.), a local dementia nurse specialist (A.K.), two local occupational therapists (S.M. and G.M.) and local doctors with a specialist interest in psychiatry or neurology.
Language subscale
The ADAS-Cog assesses a number of elements of language ability. Spoken language ability, word finding and comprehension are assessed through open-ended conversation on neutral topics at the beginning of the interview. Comprehension is also assessed through response to five increasingly complex commands. No changes to administration or scoring of these items were necessary, although primary/preferred language was clarified and these sections completed in the Chagga tribal language, rather than Swahili, where appropriate. Naming ability is assessed by naming high and low-frequency objects and parts of the body and a number of changes were required due to cultural, environmental and educational factors. Because individual names for fingers do not exist in Swahili, naming of fingers of the dominant hand was replaced by naming five high to low-frequency body parts (neck, knee, shoulder, thumb, fingernails). High-, medium- and low-frequency items for the naming task were altered where, in the opinion of the expert panel, these were likely to be unfamiliar to older adults who did not travel. In contrast to some other adaptations, real objects were presented rather than photographs or line drawings. The rationale for this was that although cognitive screening tests utilising line drawings and photographs have been used in lower-literacy settings (Reference Nitrini, Caramelli and Herrera21,Reference Verghese, Noone and Johnson22), there is evidence that in illiterate people, naming of real objects is less educationally biased than naming of either line drawings or photographs (Reference Rosselli, Ardila and Rosas23). We hoped that use of real objects might be a fairer test of naming ability in this group.
Praxis subscale
In the original ADAS-Cog, ideational praxis and constructional praxis are both assessed. Ideational praxis is assessed through ability to correctly fold and place a letter in an envelope, seal and address the envelope and affix a stamp. Constructional praxis assesses ability to copy four simple shapes correctly. It was agreed by the panel that even illiterate individuals would be familiar with the process of posting a letter, but that it would be usual to request help with writing. This task was marked as correct if individuals correctly indicated where they would like the assessor to write the name and address, and showing where they should place a stamp. This is similar to the adaptation used for illiterate individuals in the Chinese ADAS-Cog adaptation (Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). Previous experience in this setting indicated that most illiterate individuals would be reluctant to attempt a drawing task, and that this often resulted in embarrassment and distress (Reference Gray, Paddick and Kisoli4). Furthermore, drawing is recognised to be educationally biased in very low-literacy settings (Reference Ostrosky-Solis, Ardila, Rosselli, Lopez-Arango and Uriel-Mendoza24). Therefore, this task was replaced by the stick design test, a similar four-item construction task validated in low-literacy settings in Nigeria (Reference Baiyewu, Unverzagt and Lane25) and Brazil (Reference de Paula, Costa, Bocardi, Cortezzi, De Moraes and Malloy-Diniz26) where individuals are asked to copy four simple shapes constructed with matchsticks. One item from this test is included in a validated cognitive screening test in this setting (Reference Gray, Paddick and Kisoli4).
Memory and new learning subscale
Memory is assessed through tests of orientation, learning of a high-imagery familiar word list, recognition of a visually presented word list and need for reminders of the word recognition task. Other adaptations have included a variety of words for the word list, based on published word frequency lists where these exist in the language (Reference Youn, Lee and Kim14). No frequency list exists in Swahili, but the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) 10 word list has been extensively used in LMICs (Reference Sosa, Albanese and Prince27). In SSA, it has been used in Tanzania and Nigeria for cognitive assessment in epidemiological studies, and forms part of a brief screening tool for dementia developed and validated by our team for use in Tanzania and Nigeria (the Identification and Intervention for Dementia in Elderly Africans (IDEA) cognitive screen) (Reference Gray, Paddick and Kisoli4,Reference Gureje, Ogunniyi and Kola28). Therefore, we used this same word list. As for the Chinese ADAS-Cog adaptation, we used photographs of low-frequency words for the word recognition task, but also added the printed words alongside the photographs (Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). Although, as described above, illiterate individuals have greater difficulty in identifying two dimensional rather than three-dimensional objects, the use of flashcards with photographs and words was intended to create a similar encoding task to that in the original ADAS-Cog. Minimal adaptation of the orientation task was necessary to ensure cultural relevance, although one item ‘season’ had to be removed as this was not well understood during pilot work.
The adapted ADAS-Cog used is available in the supplementary materials for this study.
Inter-rater reliability
Inter-rater reliability was assessed using a separate sample of older adults who had taken part in the community dementia screening programme. Individuals were assessed by a research nurse (A.K.) and clinical officer (J.K.) who had been trained by our team in completion of the adapted ADAS-Cog. Assessments took place in a rural primary healthcare centre or, where necessary, in the participants home. The ADAS-Cog interview was administered on one occasion to each participant and each rater completed and scored the assessment independently and blinded to the other rater’s score and to the group allocation (dementia or cognitively normal). Administration of the interview on one occasion was intended to minimise practice and carry over effects.
Statistical analysis
Data were analysed using IBM SPSS version 21 for Windows (IBM Corporation, Armonk, NY, USA). All data were found to be non-normally distributed and therefore reported in terms of frequency or median and interquartile range. Non-parametric tests (χ2 test, Mann–Whitney U-test) were used throughout. Correlation was assessed using Spearman’s test for continuous or ordinal data and the point biserial test used for dichotomous data. Significance tests were two-tailed and significance set at the 5% level. When comparing dementia patients and controls, we adjusted for the influence of age on ADAS-Cog score using logistic regression, with age and ADAS-Cog score treated as continuous variables and forced into the model.
An area under the receiver operating characteristic (AUROC) curve was constructed, with dementia as the dependent variable, to assess the performance of the adapted ADAS-Cog in differentiating those with and without dementia. Concurrent validity was assessed through correlation with the CDR scale and the IDEA cognitive screen score. Internal consistency was measured using Cronbach’s α for the entire cohort. In addition, multivariable linear regression was used to assess for any relationship between age, gender and educational level and total ADAS-Cog score. Inter-rater reliability was measured using the intra-class correlation coefficient (ICC) using a two-way mixed effects model for absolute agreement between raters.
Results
In total, 34 individuals with dementia and 32 controls completed the adapted ADAS-Cog assessments. Demographic and cognitive testing data for both groups are presented in Table 1. Controls were significantly younger than individuals with dementia (median age 74 vs. 80), but there were no significant differences in gender, educational background or literacy between groups. Differences between groups in total ADAS-Cog score and in the individual items were significant, see Table 1. Total ADAS-Cog scores for dementia patients and controls are presented as box-plots in Fig. 1. The difference in total ADAS-Cog score remained highly significant, even after adjusting for the potential confounding effect of age (Odds ratio 1.792, 95% confidence interval (CI) 1.264–2.539, p=0.001).
Fig. 1 Boxplot comparison of total adapted ADAS-Cog scores in those with and without dementia. Lower scores indicate better cognitive performance.
Table 1 Demographic and clinical data for patients with dementia and controls
ADAS-Cog, Alzheimer’s Disease Assessment Scale – Cognitive; CERAD, consortium to establish a registry for Alzheimer's disease; IDEA, identification and intervention for dementia in elderly Africans; IQR, interquartile range.
In AUROC curve analysis (Fig. 2), the optimum cut-off for differentiating between dementia patients from controls using this adapted ADAS-Cog was <19. This yielded a sensitivity of 100% and specificity of 94.0%. The AUROC curve was 0.973 (95% CI 0.936–1.000), suggesting excellent criterion validity.
Fig. 2 Area under the receiver operating characteristic curve (AUC) measure of diagnostic accuracy of the adapted ADAS-Cog total score with presence of DSM-IV dementia as the dependent variable.
Internal consistency, as measured by Cronbach’s α, was 0.737 for people with dementia, 0.711 for controls and 0.884 for the whole cohort. Analysis of individual items indicated that none were redundant. Concurrent validity assessment demonstrated very high correlation between the ADAS-Cog score and the IDEA cognitive screen score (r=0.833, p<0.001) and CDR score (r=0.795, p<0.001). Within dementia cases, higher ADAS-Cog score was not correlated with greater age (r=0.129, p=0.496), female gender (r=0.279, p=0.110), lower education level (r=−0.159, p=0.369) or illiteracy (r=0.208, p=0.253).
In total, 21 individuals [13 (61.9%) with dementia] were independently assessed by two raters for measurement of inter-rater reliability. Inter-rater reliability as measured by the ICC was excellent at 0.905 (95% CI 0.804–0.964) for total ADAS-Cog score. Similarly, there was excellent agreement for the language 0.977 (95% CI 0.945–0.991) and praxis sub-scores 0.956 (95% CI 0.895–0.982), with slightly lower agreement on the memory subscale 0.730 (95% CI 0.520–0.901).
Discussion
Criterion validity, concurrent validity, internal consistency and inter-rater reliability
Comparison of people with and without dementia, but with a similar low educational background, demonstrated a significant difference in performance between the groups on each of the 11 cognitive tasks included in the ADAS-Cog. We demonstrated excellent correlations between severity of dementia (as assessed on the CDR), performance on the previously validated IDEA brief cognitive screening tool, and total ADAS-Cog score. The correlation with CDR compares favourably to that reported for the Chinese ADAS-Cog adaptation, designed for a similar population within an LMIC (Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). Other adaptations of the ADAS-Cog have demonstrated correlation with mini-mental state examination (MMSE) score (Reference Mavioglu, Gedizlioglu, Akyel, Aslaner and Eser9), but previous work carried out by our team suggests that the MMSE is not valid or particularly meaningful in this setting (Reference Paddick, Dotchin and Gray29).
Our internal consistency of 0.884 was good and comparable or better than that reported for other ADAS-Cog adaptations (Reference Tsolaki, Fountoulakis, Nakopoulou, Kazis and Mohs11,Reference Youn, Lee and Kim14,Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). Likewise, the high level of inter-rater reliability indicates a low rate of measurement error when used by trained raters in this setting.
Role of education and literacy
The lack of association between ADAS-Cog score and illiteracy or education within dementia cases suggests that any differences in performance are primarily due to cognition rather than educational background. However, the relationship between ADAS-Cog score and educational level should be explored further in larger cohorts and in other low-literacy settings. In our cohort only four people had either competed primary school or attended secondary school and the performance of our adapted ADAS-Cog in those with this level of education should be explored further. Other adaptations have reported mixed results in relation to the effect of education on total ADAS-Cog score. Studies in Greece and Turkey reported no association (Reference Mavioglu, Gedizlioglu, Akyel, Aslaner and Eser9,Reference Tsolaki, Fountoulakis, Nakopoulou, Kazis and Mohs11), in contrast to Brazil (Reference Schultz, Siviero and Bertolucci12). Interestingly, the Chinese adaptation specifically designed for a low-literacy population did note an educational effect, although only in those with less than primary education (Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). Comparisons are difficult, however, because very few studies have included individuals with similarly low educational background to those included in our study. Furthermore, the lack of educational bias reported in our study may be due to our attempts to oversample controls for lower educational level, and if this were repeated in individuals with secondary level education our results might be different. Nevertheless our aim was to produce an instrument for use in rural SSA, and the educational background of those included was typical of that usually found amongst rural-dwelling older adults in Tanzania (Reference Paddick, Longdon and Gray3,Reference Longdon, Paddick and Kisoli18).
Comparison with other studies
The median score for normal controls in our study was 12.8 (indicating impaired cognitive function), far higher than that expected for those with normal cognition in a HIC setting where individuals with cognition in the normal range might be expected to score 5 or below (Reference Graham, Snow, Massman and Doody30). However, our median scores are similar to those reported with minimally adapted translations of the ADAS-Cog, such as that used in Brazil for individuals with <4 years of education (mean 10.9) (Reference Schultz, Siviero and Bertolucci12) and in China for individuals with no education (mean 11.9) (Reference Liu, Teng, Chuang, Lin, Fuh and Wang15). It is therefore, likely that the relatively high scores reported are, in part, due to the factors inherent within LMIC populations, including education level and socioeconomic status. If so, direct comparison with scores in HICs should be done cautiously.
Direct comparisons with interventional studies in HIC settings using the original ADAS-Cog may also be problematic for other reasons. Previous dementia prevalence studies in Hai conducted by our research group suggested a high prevalence of vascular dementia in this region; diagnosis was supported by neuroimaging (Reference Paddick, Longdon and Kisoli31). The ADAS-Cog was originally developed for use in Alzheimer’s disease dementia (ADD), but has been utilised in other dementia subtypes (Reference Graham, Snow, Massman and Doody30). Unfortunately, neuroimaging was not available locally at the time of the current study, and therefore subtype diagnoses were not possible. However, it seems likely that the individuals included in this study have a different pattern of underlying cognitive impairments to that commonly seen in ADD, with a greater contribution of vascular factors to the clinical presentation. The ADAS-Cog nevertheless differentiated those with and without dementia by standard diagnostic criteria in this setting.
Limitations
This adaptation attempts to match all cognitive domains included in the original ADAS-Cog. Nevertheless, alterations to cognitive tasks were necessary to allow these to be completed by illiterate individuals. For example, the process of encoding a visual image in our low-literacy adaptation differs from encoding of a visually presented written word, required in the original version of the word recognition task, although both our adaptation and the original similarly assess recognition memory. As there are evidenced neurocognitive differences between literate and illiterate individuals in both performance on neuropsychological tasks and cognitive processes (Reference Ostrosky-Solis, Ardila, Rosselli, Lopez-Arango and Uriel-Mendoza24,Reference Brucki32,Reference Ardila, Bertolucci and Braga33), some of these differences may be unavoidable if the aim is to develop an assessment tool suitable for measurement of cognition in a low-literacy setting.
Conclusions
We have developed and validated an adaptation of the ADAS-Cog which demonstrates good performance in discriminating low-literate individuals with and without dementia and which has excellent concurrent validity, internal consistency and inter-rater reliability. Further assessment of the ability of this instrument to detect cognitive change in a low-literate population with differing level of baseline cognition would help determine its utility in interventional studies in other settings in SSA.
Acknowledgements
The authors wish to acknowledge the help of all healthcare workers, officials, carers, and family members who assisted in this study. Authors’ contributions: design and conception of the study and adapted cognitive assessment tool – S.-M.P., W.K.G., C.D., R.W., A.O., S.M., D.M., A.K., O.O. Data collection – S.-M.P., A.K., G.M., S.M., J.K. Data analysis S.-M.P., W.K.G. Interpretation of data analysis S.-M. P., C.D., W.K.G., R.W. Writing of the first draft, S.-M. P. All authors contributed to the manuscript and approved the final version for publication.
Financial Support
This work was supported by Grand Challenges Canada (GCC) Grand Challenges in Mental Health programme as part of the IDEA (Identification and Interventions for Dementia in Elderly Africans) study. Grant number 0086-04.
Conflicts of Interest
None.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/neu.2016.65
