Introduction
People with aphasia are known to communicate better in daily life than would be anticipated by results of standardized language tests, administered by professionals (Manochiopinig, Sheard, & Reed, Reference Manochiopinig, Sheard and Reed1992; Rautakoski, Korpijaakko-Huuhka, & Klippi, Reference Rautakoski, Korpijaakko-Huuhka and Klippi2008). This indicates the importance of information on the daily functioning of the person with aphasia by a proxy: a significant other who is familiar with the person with aphasia, such as a partner, friend, or caregiver. To broaden our understanding of the impact of aphasia on everyday communication, data from multiple sources—such as the affected people themselves, proxies and clinicians—should be gathered. A proxy has knowledge of the premorbid functioning of the person with aphasia, and spends substantially more time with the person with aphasia in various situations than the professional, providing ample opportunity for daily observations. A professional has encountered many people with aphasia in various degrees of severity and in a diversity of personalities; a professional therefore is experienced in anticipating the impact of aphasia on different affected persons and different proxies, also in various stages after stroke onset.
Several factors may influence the proxy's perception of the communicative ability of the person with aphasia. The age of the person with aphasia is generally associated with the type and number of social and communication situations in which the patient still participates. The relation with the proxy (e.g., spouse, child, or friend) determines how much time they spend together. The severity of the aphasia noticeable in spontaneous speech and the profile of aphasic impairment, for example, verbal versus nonverbal communicative abilities and productive skills versus language comprehension level, may influence proxies’ assessment of verbal communication. Neurological and neuropsychological impairments apart from aphasia are often related to the level of dependence and quality of life of the person with aphasia, and influence how the proxy views the affected partner. The time elapsed since onset is also considered to have an impact on proxies’ perception because adaptation to and acceptance of possible changes in daily functioning of the person with aphasia is a process that evolves over time (Hilari, Owen, & Farrelly, Reference Hilari, Owen and Farrelly2007). The psychological coping mechanisms used by relatives and their personality characteristics may also influence their estimation of the communicative ability of the person with aphasia. Finally, a significantly better agreement was found between experts and spouses of persons receiving aphasia treatment (Rautakoski et al., Reference Rautakoski, Korpijaakko-Huuhka and Klippi2008; Shewan & Cameron, Reference Shewan and Cameron1984).
Obviously, the judgments of professionals are also influenced by various factors. Experts have worked with many people with aphasia and this reference might introduce a bias when rating an individual with aphasia. Also, experts have little knowledge of the individual's premorbid functioning and possibly rate strictly by comparing communication to an ill-defined ideal of “normal” behavior (Douglas, O'flaherty, & Snow, Reference Douglas, O'flaherty and Snow2000). Finally, experts might tend to rate strictly as they know how to recognize and categorize linguistic errors.
Most studies comparing experts’ and proxies’ perception of the activity limitations and participation restrictions of people with aphasia have concluded that family members make valid and reliable evaluations of both linguistic and nonlinguistic communicative abilities (Aftonomos, Steele, Appelbaum, & Harris, Reference Aftonomos, Steele, Appelbaum and Harris2001; Holland, Reference Holland1980; Linebaugh & Young-Charles, Reference Linebaugh and Young-Charles1981; Lomas et al., Reference Lomas, Pickard, Bester, Elbard, Finlayson and Zoghaib1989; Long, Hesketh, & Bowen, Reference Long, Hesketh and Bowen2009). However, a difference between experts’ and proxies’ perceptions of aphasia was also found (Oxenham, Sheard, & Adams, Reference Oxenham, Sheard and Adams1995). As for the nature of this disagreement, various authors reported that proxies overestimated the linguistic ability of their aphasic partner compared with speech-language therapists (SLTs) (Helmick, Watamori, & Palmer, Reference Helmick, Watamori and Palmer1976; Sarno, Reference Sarno1993; Taylor, Reference Taylor1965). The opposite, an underestimation of linguistic ability by proxies was also observed (Hesketh, Long, & Bowen, Reference Hesketh, Long and Bowen2011; Vogel & Costello, Reference Vogel and Costello1985).
In conducting trials and planning and providing care for people with aphasia, it is crucial to have insight into the extent to which the judgments of experts and proxies align. It determines for example the guidance and information SLTs can best provide to family members. The first aim of this study was to examine the association between the rating of everyday communication by the SLT (expert) and by the proxy of people with aphasia due to stroke, and the influence of time post onset (3 vs. 6 months) on this association. The second aim of this study was to determine if aphasia severity, quality of life, level of independence, age, sex, education level, stroke type, and relation with the proxy influence the level of agreement between both ratings at 6 months, a point in time at which language ability is more stable.
Method
The people included in the present study participated in the Rotterdam Aphasia Therapy Study-2 (RATS-2; n = 80), a multicenter, randomized controlled trial on the efficacy of cognitive-linguistic therapy in the acute stage of aphasia after stroke. The design and results of RATS-2 have been published elsewhere (de Jong-Hagelstein et al., Reference de Jong-Hagelstein, van de Sandt-Koenderman, Prins, Dippel, Koudstaal and Visch-Brink2011). RATS-2 was approved by the ethics committee of Erasmus MC and completed in accordance with the Helsinki Declaration (http://www.wma.net/e/policy/17-c_e.html). Written informed consent was obtained from all participants and their proxy before inclusion in the study.
Participants
Participants were between 18 and 85 years and had aphasia due to acute stroke. Aphasia resulted in disruption of everyday verbal communication and an overt semantic and/or phonological disorder. All participants were assessed within 3 weeks after stroke and at 3 and 6 months. This study centers on the 3 and 6 month outcomes only.
Measurements
ANELT
The Amsterdam-Nijmegen Everyday Language Test (ANELT) (Blomert, Koster, & Kean, Reference Blomert, Koster and Kean1995) measures functional verbal communication, an important clinical outcome measure in randomized controlled trials. It was the only Dutch functional test available at that time. The ANELT scale A (understandability) was scored from audio recording by two independent, experienced SLTs, blinded to test moment and treatment allocation. The means of both raters’ scores were used in the analyses. More details have been published elsewhere (de Jong-Hagelstein et al., Reference de Jong-Hagelstein, van de Sandt-Koenderman, Prins, Dippel, Koudstaal and Visch-Brink2011). The ANELT was administered within 3 weeks and at 3 and 6 months by a member or trainee of the RATS team, of the Aphasia Team in Rotterdam, a colleague of the treating SLT, or the treating SLT at that time. It consists of 10 scenarios to which the person with aphasia is asked to respond verbally (see Appendix for examples). The verbal responses are rated for informational content on a scale from 1 (bad) to 5 (good) so the total score ranges from 10 to 50 points. The ANELT is both a valid test, with strong criterion-related, construct and ecological validity (no differences between expert ratings and ratings by naive persons without any experience with aphasia), and a reliable test with high inter-rater and test–retest reliability (Blomert, Kean, Koster, & Schokker, Reference Blomert, Kean, Koster and Schokker1994). It is increasingly being used as an outcome measure in treatment studies, both in The Netherlands (Bastiaanse, Hurkmans, & Links, Reference Bastiaanse, Hurkmans and Links2006; Doesborgh et al., Reference Doesborgh, Van De Sandt-Koenderman, Dippel, Van Harskamp, Koudstaal and Visch-Brink2004a, Reference Doesborgh, van de Sandt-Koenderman, Dippel, van Harskamp, Koudstaal and Visch-Brink2004b; Links, Hurkmans, & Bastiaanse, Reference Links, Hurkmans and Bastiaanse2010) and elsewhere (Laska, Kahan, Hellblom, Murray, & Von Arbin, Reference Laska, Kahan, Hellblom, Murray and Von Arbin2008; Laska, Von Arbin, Kahan, Hellblom, & Murray, Reference Laska, Von Arbin, Kahan, Hellblom and Murray2005).
PCQ
One of the few assessment tools for functional communicative performance that are designed to be used by proxies is the Partner Communication Questionnaire (PCQ) (Blomert, Reference Blomert1995). The PCQ is an observational instrument with which partners can assess the verbal communicative abilities of their aphasic partner in everyday situations (see Appendix for examples). The partners are asked to indicate to what extent they think the person with aphasia is able to handle verbally every situation on a 5-point scale ranging from 1: never succeeds to 5: always succeeds. The PCQ consists of 46 questions so total score ranges from 46 to 230 points, with a higher score indicating better verbal communication. The partner is instructed to take into account only the ability to verbally convey a message, without the help of hand gestures and facial expressions. In case they did not actually experience the given situation, the partner is instructed to imagine the verbal ability of his partner in such a situation. The PCQ is not suitable in the hospital phase; therefore, it was administered only at 3 and 6 months.
Aphasia Severity Rating Scale
We used the (Goodglass) Aphasia Severity Rating Scale from the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, Reference Goodglass and Kaplan1983) to rate communicative behavior of the person with aphasia. Since functional verbal communication—measured with the ANELT and PCQ—was the focus of this study, we selected a similar measure (communication) to explore the influence of its severity on agreement of ratings. Spontaneous speech is collected through a conversation in which the person with aphasia is encouraged to speak for 10 min, guided by standard questions about what happened, family, occupation, and hobbies. The capacity for oral communication is rated on a 6-point scale, ranging from 0: no comprehensible speech production to 5: no or minimal noticeable speech difficulty. The rating was performed within 3 weeks and at 3 and 6 months by a member or trainee of the RATS team or of the Aphasia Team in Rotterdam.
EQ-5D
The European quality of life-5 dimensions (EQ-5D) (The EuroQol Group, 1990) was used to assess self-reported health status of the person with aphasia at the time of completion (within 3 weeks and at 3 and 6 months). The EQ-5D is widely used in stroke populations, available in Dutch and relatively simple. It was administered together with the participant by the treating SLT at that time, a member or trainee of the RATS team, of the Aphasia Team in Rotterdam, or by a colleague of the treating SLT. All cues were used to help the person with aphasia understand the items. The EQ-5D consists of five subscales that assess mobility, self-care, usual activities, pain, and anxiety/depression. Each subscale is scored on a 3-point scale: no problems, some problems or severe problems. A utility score was calculated with population-based preference weights for combined health scores (Lamers, Stalmeier, McDonnell, Krabbe, & van Busschbach, Reference Lamers, Stalmeier, McDonnell, Krabbe and van Busschbach2005). Utility scores range from -0.33 to 1.00; a score of 1 represents perfect health, a score of 0 represents death, and negative scores represent health states considered worse than death.
Barthel Index
The Barthel Index (Mahoney & Barthel, Reference Mahoney and Barthel1965) was used to measure the degree of physical activity limitations and the level of independence in activities of daily living (ADL). The scale was filled out by a nurse within 3 weeks after stroke (standard procedure in hospitals) or by the SLT together with the participant and often with help from a proxy at 3 and 6 months. It consists of 10 items all representing physical abilities, for example, climbing the stairs, using the toilet, getting dressed. The rater is asked to evaluate the dependency on others of the person with aphasia for each activity on a 3-point scale with 0 indicating complete dependence and 2 indicating complete independence. Scores range from 0 to 20, with a higher score indicating more independence in ADL.
Statistical Analyses
For comparability we rescaled the ANELT-A and PCQ by dividing the ANELT-A scores by 10 and the PCQ scores by 46 (the number of scenarios/questions) so that total scores on both measures range from 1 to 5.
First aim
The association between the rating of the SLT (expert) and the rating of the proxy both at 3 and at 6 months after stroke was examined through scatter plots of the data and with Pearson correlation coefficients. We labeled the points in the scatter plots to explore agreement between both ratings as a function of aphasia type.
Second aim
To determine which factors affect the level of agreement between expert and proxy judgment at 6 months after stroke, we performed a hierarchical multiple regression analysis with the absolute difference between ANELT and PCQ scores (range, 0 to 4) as dependent variable. This provides us with information about the influence of the factors on the extent to which both ratings align. The predictors considered were (Goodglass) Aphasia Severity Rating Scale, Barthel Index, EQ-5D, age, sex, and education level of the person with aphasia, stroke type (ischemic or hemorrhagic), and type of relation with the proxy (whether the proxy was the partner or a non-partner, that is, child, parent or friend of the person with aphasia). First, we used a base model that consisted of age, sex, education level, and stroke type. Then, one variable at a time, we added the Aphasia Severity Rating Scale, Barthel Index, EQ-5D and type of relation with the proxy. Analyses were performed in SPSS 17.0 for Windows.
Results
For 53 of 80 participants, data on all the predictor variables (Aphasia Severity Rating Scale, Barthel Index, EQ-5D, age of the person with aphasia, sex, education level, stroke type and type of relation with the proxy) and ANELT and PCQ were complete at 6 months after stroke. For 39 of these 53 people, ANELT and PCQ were obtained both at 3 and 6 months after stroke (not all proxies returned both questionnaires). Baseline characteristics of the 53 people included in this study are summarized in Table 1. Of the four people with a right-hemisphere stroke, one was left-handed, two were ambidextrous, and one was dextral with crossed aphasia. The intraclass correlation coefficients between the two independent raters of the ANELT indicated excellent agreement (at baseline 0.95, at 3 months 0.97 and at 6 months 0.96).
Table 1 Baseline characteristics of the participants (n = 53)
a“low” is unfinished elementary school up to sophomore high school or lower vocational education; “high” is junior high school or middle vocational education up to university.
bAmsterdam-Nijmegen Everyday Language Test, scale A (understandability).
cPartner Communication Questionnaire; not suitable in acute stroke, thus only used 3 and 6 months after stroke.
dAachen Aphasia Test (Graetz, De Bleser, & Willmes, Reference Graetz, De Bleser and Willmes1991); not suitable in acute stroke, thus administered 8 weeks after stroke.
eMildest aphasia; minimal problems such as slowness in word finding and difficulties discussing abstract topics.
Association ANELT – PCQ
The first question concerned the association between the rating of SLTs on the ANELT and the rating of the proxies on the PCQ. Figure 1 shows scatter plots of expert versus proxy ratings at 3 and 6 months. It appears that the higher the scores (both on ANELT and PCQ), the better the agreement. Figure 1 also shows that proxies tended to rate the verbal communicative ability of the person with aphasia somewhat higher than experts. We tested this by means of paired samples t tests: mean difference at 3 months = .38 (95% CI = .05 to .71), p = .023; mean difference at 6 months = .35 (95% CI = .005 to .71) and p = .053. The correlation was moderate: at 3 months r = .662 (p ≤ .0001) and at 6 months r = .565 (p ≤ .0001). The mean improvement from 3 to 6 months on the ANELT was 0.155 points (3.9% of the maximum improvement of 4 points), the mean improvement on the PCQ was 0.124 points (3.1% of the maximum improvement of 4 points). Neither measure showed a statistically significant improvement (ANELT: p = .074; PCQ: p = .188).
Fig. 1 Scatter plots of the ratings by the proxy (Partner Communication Questionnaire, PCQ) and the expert (Amsterdam-Nijmegen Everyday Language Test, ANELT) of the verbal communicative ability of a person with aphasia 3 and 6 months after stroke (n = 39).
Points in the scatter plots are labeled for aphasia type (classification according to the Aachen Aphasia Test). It appears that for all aphasia types, proxies more often give higher ratings than experts than vice versa. Also, the aforementioned effect of severity is reflected in the aphasia types: in the more severe types (Global, Wernicke, and Broca), ratings diverge more strongly than in less severe types.
Predictors of the Level of Agreements
The second aim was to examine which factors influenced the level of agreement between expert and proxy ratings at 6 months.
We performed a hierarchical multiple regression analysis with the absolute difference between ANELT and PCQ scores (range, 0 to 4) as the dependent variable. The base model, which included age, sex, education, and stroke type, was not significant (Table 2).
Table 2 Hierarchical multivariate regression analysis of agreement between proxy and expert rating of the verbal communicative ability of a person with aphasia (absolute difference between ANELT and PCQ) at 6 months, range 0 to 4 (n = 53)
Note. EQ-5D = European quality of life-5 dimensions.
aReference is Base model.
Addition of Aphasia Severity Rating significantly increased the explained variance by 21.2% (p = .001; Table 2). A higher (more favorable) Aphasia Severity Rating was significantly associated with better agreement between proxy and expert judgment of everyday verbal communicative ability (unstandardized regression coefficient B = −0.261; p = .001). None of the other independent variables tested, including the Barthel Index, EQ-5D, or type of relationship between the person with aphasia and the proxy, accounted for additional variance relative to the base + Aphasia Severity Rating model.
Discussion
In this study of people with aphasia after stroke we found moderate agreement between the rating by the SLT and by the proxy of the everyday verbal communicative ability of the person with aphasia, both at 3 and 6 months. A higher Aphasia Severity Rating of the person with aphasia was associated with better agreement between proxy and expert rating at 6 months.
This study was conducted within the setting of a randomized clinical trial. An advantage of this setting is that all study assessments were standardized and performed by well trained and motivated SLTs. Furthermore, participants were included by over 20 centers, both rural and metropolitan, which increases the generalizability of the results. Drawbacks of the study are that we could not collect all the predefined assessments for each person with aphasia at 3 and 6 months after stroke and that the sample size is relatively small.
Several previous studies have compared the judgments of people with aphasia and their partner about the communication of the person with aphasia on the level of participation restrictions, but not primarily on the level of activity limitations. We compared the judgment of the expert (SLT) and the partner, the two persons having the best opportunities to monitor the communication of the person with aphasia in the rehabilitation stage. We used the Partner Communication Questionnaire, which very well matches the ANELT-A, the expert assessment. Both measure the level of activity limitations of the person with aphasia.
The only comparable study using the same test and questionnaire is that of Blomert (Reference Blomert1995). Although this study used a 20-item precursor of the current PCQ, the correlation between the ANELT-A and the PCQ in the chronic stage (r = .69) very well accords with our correlation at 3 and 6 months (.66 and .57). Blomert found that 71% of the ratings (n = 28) concurred and that in case of a discrepancy, 62% of the proxies underestimated and 38% overestimated the verbal communication of people with aphasia compared with the expert. In contrast, we found that only 26% of the ratings concurred and that of the remaining 74%, only 41% of the proxies underestimated and 59% overestimated. A difference between our study and that of Blomert is the time elapsed since onset: the proxies in our sample had had less time to obtain sufficient rating expertise than those in Blomert's study. Still, the correlations between the ANELT and the PCQ in both studies are similar, which does not support the assumption that a minimum of 5 months post onset might not be enough time for proxies to adapt (Shewan & Cameron, Reference Shewan and Cameron1984).
We found no conclusive evidence for the influence of time post onset on the association between expert and proxy rating. The correlations at 3 and 6 months were comparable (though little less strong at 6 months), whereas the discrepancy between both ratings slightly decreased: the difference was statistically significant at 3 months and (just) not at 6 months. Possibly, in a more chronic stage of aphasia, for example, 1 year or more after stroke onset, better acceptance by proxies might lead to higher agreement between proxies’ and experts’ view on the communicative abilities of the people with aphasia.
Our finding that the proxies rated the verbal communicative ability of people with aphasia slightly higher than the SLTs is in line with previous studies (Helmick et al., Reference Helmick, Watamori and Palmer1976; Taylor, Reference Taylor1965) and provides some support for the notion that people with aphasia communicate better in natural contexts than would be anticipated by results of standardized tests (Manochiopinig et al., Reference Manochiopinig, Sheard and Reed1992; Rautakoski et al., Reference Rautakoski, Korpijaakko-Huuhka and Klippi2008). It is unknown whose judgment most accurately appraises the actual communication of the person with aphasia, the expert's or the proxy's. There are several explanations for the finding that expert ratings were on average slightly below proxy ratings. First, SLTs might be more sensitive to the linguistic deviations than proxies because proxies probably do not recognize and label formal deviations in the verbal communication such as phonemic paraphasias. Also, proxies probably cannot disentangle verbal and nonverbal communication as efficiently as experts and therefore rate the verbal communicative ability higher than experts. In addition, the SLT does not witness the person with aphasia during communication in everyday situations, like the proxy does. Also, the ANELT contains 10 selected scenarios that are played—it is an artificial measure—in contrast to the many scenarios in the PCQ which are easy to imagine for the proxy. Finally, the ANELT requires some level of abstraction by the people with aphasia; they have to imagine being in the given situation.
A possible explanation for the finding that a lower (Goodglass) Aphasia Severity Rating of the person with aphasia was associated with less agreement between proxy and expert judgment is that in more severely impaired persons with aphasia, nonverbal communication may play a larger role. Another explanation might be that people with a less severe aphasia are involved in more communicative situations than people with worse communicative abilities, which could make a realistic rating easier for their proxy.
Aphasia Severity Rating had a strong predictive value. Addition of the Barthel Index, EQ-5D and type of relation to Aphasia Severity Rating in our regression analysis demonstrated that these variables did not have any additional predictive value.
In randomized trials of aphasia treatment, a functional outcome measure like the ANELT is often used. However, the results of the present study suggest that one should verify whether the rating corresponds to the judgment of the proxy. This is vital for the validity of the findings in an intervention study: if the test indicates a significant improvement after intervention, the proxy should recognize this improvement in daily life. Proxy views of the abilities of people with aphasia are important for planning rehabilitation goals and therapy and are crucial for the motivation and active involvement of people with aphasia and their relatives (Oxenham et al., Reference Oxenham, Sheard and Adams1995). It is promising that we found moderate agreement between expert and proxy judgment. The ANELT appears to be a fair test with significant ecological validity, particularly in people with milder aphasia. The fact that all participants received therapy in our study might have influenced the level of agreement: expert and proxy ratings have been shown to align better in couples of whom the person with aphasia was receiving aphasia treatment (Rautakoski et al., Reference Rautakoski, Korpijaakko-Huuhka and Klippi2008; Shewan & Cameron, Reference Shewan and Cameron1984).
In conclusion, our findings suggest that the judgments of verbal communicative ability by experts and proxies align reasonably well, especially in milder cases. The greater divergence between ratings of experts and proxies for persons with severe aphasia may reflect greater sensitivity of experts to formal deviations in verbal communication; the inability of the experts to witness the communication of the person with aphasia in everyday life situations; the requirement of the ANELT for some level of abstraction by the person with aphasia; and the difficulty of proxies in excluding nonverbal communication in formulating their judgments. Future research should focus on gaining more insight into mechanisms behind the diverging judgments.
Acknowledgments
The Rotterdam Aphasia Therapy Study-2 was funded by the Stichting Nuts Ohra (T-07-71). The authors declare that they have no conflict of interest. We gratefully acknowledge the people with aphasia and their proxies for the invested time and effort. Permission to include items of the Partner Communication Questionnaire (PCQ) and the Amsterdam-Nijmegen Everyday Language Test (ANELT) in the Appendix was obtained from the author of both, prof. Dr. L. Blomert and from the publisher of the ANELT: Hogrefe Uitgevers B.V.
Appendix
Examples From the Amsterdam-Nijmegen Everyday Language Test (ANELT)
- You have just moved in next door to me and you would like to meet me. You ring my doorbell and say …
- You are in a store and you want to buy a television. I am the salesperson. “Can I help you?”
- You have an appointment with the doctor but something else has come up. You call and what do you say?
- You are at the bakery and you find this (present glove) on the floor. What do you say?
- You are at the florist and you want to have a bouquet delivered to a friend. I am the florist, what do you say to me?
Examples From the Partner Communication Questionnaire (PCQ)
- Your partner worries about the future. Can your partner tell you what bothers him/her?
- Can your partner ask for directions when he/she is lost?
- Your partner runs into a friendly couple. They ask if they can visit you that evening. That is possible! Can your partner answer that it is okay?
- You want to go on holiday. You partner does not yet feel fit enough. When you ask your partner his/her opinion, can your partner tell you what his/her thoughts are about that?
- You are sad about something. Can your partner ask you what is the matter?
