INTRODUCTION
Chronic memory impairments are among the most reported cognitive complaints after stroke, affecting up to one-third of survivors (das Nair & Lincoln, Reference das Nair and Lincoln2007; Lamb, Anderson, Saling, & Dewey, Reference Lamb, Anderson, Saling and Dewey2013; Taylor & Broomfield, Reference Taylor and Broomfield2013), and are associated with reduced independence in daily activities, and decreases in occupational performance and quality of life (Brown, Mapleston, Nairn, & Molloy, Reference Brown, Mapleston, Nairn and Molloy2013; Sheldon & Winocur, Reference Sheldon, Winocur, Schweizer and Loch Macdonald2014). Benefits to everyday memory are achievable from rehabilitative intervention, particularly through strategy training to compensate for deficits (Elliott & Parente, Reference Elliott and Parente2014).
Recent findings have provided support for a group-based compensatory memory intervention program developed by Radford, Say, Thayer, & Miller (Reference Radford, Say, Thayer and Miller2010), incorporating psychoeducation, memory strategy training, and lifestyle improvements. On examining the program’s impact on functional memory for patients with stroke and neurological disorders, Radford, Lah, Thayer, Say, & Miller (Reference Radford, Lah, Thayer, Say and Miller2012) reported gains in list learning and delayed recall, subjective measures of prospective memory, and use of internal and external memory strategies, consistent with findings of Miller and Radford (Reference Miller and Radford2014) in a non-acute stroke group.
More recently, Withiel, Wong etal. (Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019) investigated impact on everyday memory performance using the Monash Memory Skills program, a group-based intervention adapted from the program by Radford etal. (Reference Radford, Say, Thayer and Miller2010). Compared with computerized cognitive training and waitlist control, Withiel, Wong etal. (Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019) found that the memory group achieved greater attainment of participant goals and use of internal memory strategies. Although immediate gains were also found in self-reported everyday memory complaints post-intervention, these benefits were not seen at a six-week follow-up. This finding is consistent with a meta-analysis of memory intervention studies by das Nair, Cogger, Worthington, & Lincoln (Reference das Nair, Cogger, Worthington and Lincoln2016), describing short-term gains on subjective measures of memory functioning. In addressing the possible short-term nature of benefits, a potential role of booster sessions has been suggested, with evidence supporting consolidation of memory skills training and maintenance of associated gains for patients with chronic stroke (House etal., Reference House, Burdea, Grampurohit, Polistico, Roll, Damiani, Keeler, Hundal and Pollack2016) and mild cognitive impairment (Optale etal., Reference Optale, Urgesi, Busato, Marin, Piron, Priftis, Gamberini, Capodieci and Bordin2010).
Some members of the stroke community face barriers to engagement with rehabilitation due to limited availability of services in rural areas (Allott & Lloyd, Reference Allott and Lloyd2009; Hill & Theodoros, Reference Hill and Theodoros2002; Jia, Cowper, Tang, Litt, & Wilson, Reference Jia, Cowper, Tang, Litt and Wilson2012; Joubert etal., Reference Joubert, Prentice, Moulin, Liaw, Joubert, Preux, Ware, Medeiros de Bustos and McLean2008). Remote delivery of rehabilitation through internet-enabled videoconferencing represents a potential alternative to face-to-face therapy. Encouraging evidence around use of videoconferencing (e.g., Skype, Zoom) in healthcare is emerging (Armfield, Bradford, & Bradford, Reference Armfield, Bradford and Bradford2015; Hill & Theodoros, Reference Hill and Theodoros2002; Russell, Reference Russell2009), but comprehensive guidance for clinicians in use of telehealth is still lacking (Armfield, Gray, & Smith, Reference Armfield, Gray and Smith2012). Although positive results and high patient satisfaction have been found from internet-based memory rehabilitation following traumatic brain injury (Bergquist, Thompson, Gehl, & Pineda, Reference Bergquist, Thompson, Gehl and Pineda2010; Ownsworth, Arnautovska, Beadle, Shum, & Moyle, Reference Ownsworth, Arnautovska, Beadle, Shum and Moyle2017), such options remain unexplored for stroke populations.
Our study had four aims. We first aimed to explore the feasibility of individual (i.e., one-on-one) face-to-face and telehealth-delivered applications of the Monash Memory Skills program, including documentation of technological impediments to telehealth delivery. Second, we aimed to explore the effectiveness of each mode of program delivery in improving memory functioning, measured by personal attainment of goals, and subjective and objective measures of memory functioning. Third, we aimed to evaluate whether the effectiveness of individual telehealth and face-to-face formats were at least comparable to the existing successful group-based memory skills program (Withiel, Wong etal., Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019). Finally, we aimed to examine the role of a booster session after a six-week follow-up point, in maintaining potential gains in memory functioning. We hypothesized that both face-to-face and telehealth programs would represent feasible program formats, and that participants in both programs would show significant intervention-related gains in goal attainment and subjective outcome measures although not on objective memory tests (based on findings from Withiel, Wong etal., Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019). We also hypothesized that the benefits of individual telehealth and face-to-face formats would not be inferior to the existing group-based formats. Finally, we hypothesized that participants randomized to receive a booster session would show greater improvements on subjective measures of functional memory than those not receiving the booster session, at 12-week follow-up.
METHOD
Study Design
This study was a pilot two-arm, parallel, non-randomized clinical trial comparing individual telehealth and individual face-to-face intervention delivery methods, with outcomes measured at pre- and post-intervention, and six-week follow-up. Allocation to delivery condition was determined by participant self-selection, geographic location (e.g., living locally or interstate), or availability to attend intervention sessions in person. We included an additional randomized component to assess effectiveness of booster sessions, with outcomes measured at twelve-week follow-up, and blinding of outcome assessors to booster condition.
Participants
Participants were invited via community advertising through Australian online stroke support forums including enableme.com.au, the Stroke Foundation online newsletter and podcast, and various Facebook community groups. Clinicians were also contacted through national neuropsychology email listserv NPinOz and invited to refer patients. Ineligible participants from a related study (Withiel, Wong etal., Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019) were also referred. Referrals and expressions of interest were followed up via telephone and email, and a screening interview was conducted to establish eligibility. Inclusion criteria were age at least 18 years, confirmed diagnosis of stroke at least three months prior, post-stroke memory difficulties confirmed by self, close-others, or referring clinicians and availability for weekly sessions and pre- and post-intervention assessments. Exclusion criteria were confirmed diagnosis of a neurodegenerative disorder, or severe language or cognitive deficits precluding engagement with the intervention or assessment measures. Participants were not paid but were offered intervention free of charge.
The study was completed in accordance with the Declaration of Helsinki and with the approval of the Monash Health Human Research Ethics Committee. The project was registered with the Australian New Zealand Clinical Trials Registry (ACTRN 12617000618358).
Materials
We adopted principally the same intervention program and outcome measures employed by Withiel, Wong etal. (Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019).
Intervention
The intervention was a modified version of the Monash Memory Skills Group program used in Withiel, Wong etal.’s (Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019) study, adapted from an earlier manualized group program entitled Making the Most of your Memory: An Everyday Memory Skills Program (Radford etal., Reference Radford, Say, Thayer and Miller2010). Minor modifications were made to group-based exercises to ensure suitability for individual and telehealth delivery (the same modifications were made for both formats) and are summarized in Table 1. Weekly sessions of two-hour duration were delivered by trained provisional psychologists under supervision from an experienced neuropsychologist over six weeks; however, flexibility in program length was permitted to accommodate participant availability. Program content (detailed in the study by Withiel, Stolwyk etal., Reference Withiel, Stolwyk, Ponsford, Cadilhac and Wong2019) included psychoeducation regarding memory functioning, practical training in internal and external compensatory memory strategies, and information about relevant impacts from lifestyle factors including diet, sleep, and exercise. Interactive in-session exercises and between-session homework tasks were included to encourage practice and regular application of skills and strategies in everyday contexts. Face-to-face participants saw a therapist either at a university clinic or at the participant’s home. Telehealth sessions were conducted via Zoom, a freely available software program facilitating secure internet-enabled video calls with features such as screen sharing for presentation of visual content.
Table 1. Memory Group program session content and modifications for Individual Face-to-Face and Individual Telehealth Formats
Note: * denotes content sections in which no major modifications were made, although any group discussions were replaced by individual discussions between participant and clinician.
Booster sessions
A booster session was included in the intervention program as a novel addition for randomly allocated participants. This component was not included in the original group program (Radford etal., Reference Radford, Say, Thayer and Miller2010) or in the related RCT conducted by Withiel, Wong etal. (Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019). A single booster session was offered after a six-week follow-up assessment, administered by the same clinician who delivered the main intervention program, in the same delivery format (face-to-face or via videoconferencing). The content in the booster session covered a summary of the main program, with discussion around application of trained strategies in a broader range of contexts.
Baseline measures
To characterize groups at baseline, the Montreal Cognitive Assessment cognitive screen (Nasreddine etal., Reference Nasreddine, Phillips, Bedirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005) and Test of Premorbid Functioning (Wechsler, Reference Wechsler2009) were administered. The Nottingham Extended Activities of Daily Living (Nouri & Lincoln, Reference Nouri and Lincoln1987) was used to characterise functional independence. Telehealth participants also completed the 12-item Computer Proficiency Questionnaire (CPQ-12; Boot etal., Reference Boot, Charness, Czaja, Sharit, Rogers, Fisk, Mitzner, Lee and Nair2015) as a self-report screen for general technological competence.
Feasibility measures
Measures of program feasibility were the rates of (1) agreement to participate among those informed and eligible after screening, (2) overall recruitment, and (3) treatment adherence, for each delivery format. Treatment adherence was measured as a rate of completion of all six program sessions. Additionally, telehealth delivery feasibility was measured as the incidence of technological issues impacting intervention session delivery. Technological issues were tracked using a brief online clinician survey, completed following administration of each telehealth-delivered program session. Clinicians were asked to specify the nature of any technology-related issues impacting the quality or completion of the session.
Primary Outcome Measure
Goal Attainment Scaling (GAS) is a person-centred measure of rehabilitation progress, commonly used with clinical populations including acquired brain injury, to define and assess attainment of personal therapy goals (Turner-Stokes, Williams, & Johnson, Reference Turner-Stokes, Williams and Johnson2009). Participants were invited to nominate two types of memory lapses experienced in everyday activities, in which improvements would represent realistic participation goals. Typical goals included remembering names of newly-met people or improving recall for details of conversations. Measures of improvement or deterioration from baseline were determined collaboratively. We used the GAS scoring method outlined by Turner-Stokes (Reference Turner-Stokes2009), which measures goal attainment on a 5-point scale, where “−1” represents baseline, “0” represents a targeted level of achievement, and “+1” and “+2” signifying improvement beyond expectation, with “−2” allowing for the possibility of deterioration from baseline. Progress was reviewed at each assessment time point. To facilitate comparison and analysis, a composite score was calculated to determine an overall GAS score at each time point, using T scores formulas and online calculators (also outlined by Turner-Stokes, Reference Turner-Stokes2009).
Secondary Outcome Measures
Subjective measures
Self-reported lapses in memory were assessed using the Everyday Memory Questionnaire-Revised (EMQ-R; Royle & Lincoln, Reference Royle and Lincoln2008), a 13-item scale to measure memory failures in everyday activities. Items include forgetting to do things you planned to do, with responses on a five-point Likert scale from Once or less in the last month to At least once a day. Raw total scores were used for analysis, with higher scores representing higher frequency of memory lapses.
Part A of the Comprehensive Assessment of Prospective Memory (CAPM) is a 39-item questionnaire used to assess the incidence of commonly-reported lapses in prospective memory functioning for individuals with acquired brain injury (Fleming etal., Reference Fleming, Kennedy, Fisher, Gill, Gullo and Shum2012). Items such as forgetting to buy an item at the grocery store are rated on a five-point scale, from Never to Very Often (daily), with higher scores representing more frequent memory failures. Mean scores were used for analysis.
A self-report Strategy Use Checklist was used to record participant use of external strategies (e.g., lists, smartphone applications) and internal strategies (e.g., mental rehearsal, face-name association). The frequency of use of each strategy was estimated, from “Daily” to “Not at all”, with total scores used for analysis, ranging from 0 to 44 for external strategies and 0 to 40 for internal strategies.
Objective measures
The Royal Prince Alfred Prospective Memory Test (RPA-ProMem) provided an objective measure of prospective memory function using naturalistic time-based (e.g. “In 15 min’ time, tell me it’s time for a coffee break”) and event-based tasks (e.g. “At the end of our session, ask me for an information sheet on note-taking strategies”; Radford etal., Reference Radford, Lah, Say and Miller2011). Raw scores were used for analysis. The Rey Auditory Verbal Learning Test (RAVLT) was used as an objective measure of capacity for learning and delayed memory (Schmidt, Reference Schmidt1996). Total number of words repeated over five trials, and total recalled after a 30-min delay, were converted to standardized Z scores. Parallel forms of the RPA-ProMem and RAVLT were used for repeated test administrations.
Procedure
After initial interview and informed consent, eligible participants were assigned to either the face-to-face or telehealth condition. Baseline screening and assessment were conducted within three weeks before program commencement, by a researcher independent of intervention delivery. Post-intervention assessment was administered by another independent researcher within three weeks following the final session and repeated six and twelve weeks after intervention completion. All assessments were completed within three-week time-windows. Assessment sessions were conducted either in person or via videoconferencing, consistent with group allocation. A meta-analysis by Brearly etal. (Reference Brearly, Shura, Martindale, Lazowski, Luxton, Shenal and Rowland2017) supported the equivalence of videoconferencing administration of synchronous-dependent tests (such as wordlist-learning) used in this study. Due to practical factors (such as the interstate location of some telehealth group participants), blinding of assessment staff to group condition was not feasible.
Participants were randomized into booster and non-booster conditions at screening, after allocation into telehealth or face-to-face delivery. Randomization was performed by a separate researcher, using an online randomization program (www.randomization.com) generating random permuted blocks. Randomized lists were prepared for face-to-face and telehealth groups separately to ensure balanced numbers of booster and non-booster allocations within delivery mode groups. The researcher conducting follow-up assessments was blind to booster condition. Booster sessions occurred as early as possible following the six-week follow-up for allocated participants, and subjective and objective measures were repeated at twelve-week follow-up.
Data Analyses
For Aim 1 (to establish the feasibility of individual and telehealth intervention delivery), descriptive statistics were compiled summarizing rates of agreement to participate, recruitment, and treatment adherence for both delivery formats. A summary of clinician questionnaire responses was also compiled regarding technical barriers impacting telehealth sessions.
For Aim 2 (to determine the effectiveness of individually delivered interventions in improving memory functioning), multilevel analysis was conducted (Howell, Reference Howell2012), with three time points (baseline, post-intervention, and follow-up) as a repeated measures factor and one between-groups factor (two groups: individual telehealth and individual face-to-face). Baseline scores and age were entered into the model as potential confounders (Jager, Zoccali, MacLeod, & Dekker, Reference Jager, Zoccali, MacLeod and Dekker2008). We employed a first-order autoregressive structure (Howell, Reference Howell2012) for repeated measures, as this led to a good fit using the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) (Weakliem, Reference Weakliem2016) for all measures. Treatment effect size estimates were calculated using Cohen’s d, comparing baseline measures to post-intervention and baseline to follow-up based on the mixed model output, whereby an effect size of .2 denotes a small effect, .5 denotes a medium effect, and .8 denotes a large effect (Cohen, Reference Cohen1988).
For Aim 3, a non-inferiority design was selected to examine whether the individual delivery modes offered comparable treatment effects to previously observed benefits of a similar, existing group-based program. Comparison data were taken from the RCT recently conducted by our research team (Withiel, Wong etal., Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019) to calculate a treatment effect for the memory group intervention compared with a healthy control group on the GAS primary outcome measure. Results indicated a statistically significant mean treatment effect in standardized GAS scores (M = 16.29, 95% CI [9.26, 23.32]). As a comparison point for the current analysis, the lower end of the above 95% CI (i.e., 9.26 GAS T score points) was taken as a conservative estimate of the treatment effect. Following guidelines for non-inferiority analysis (Rothman & Tsou, Reference Rothman and Tsou2003; Schumi & Wittes, Reference Schumi and Wittes2011), a non-inferiority margin, or limit for acceptable loss of effect (Δ), was set at 50% of the established treatment effect, i.e. 9.26/2, or 4.64. One-sided 90% confidence intervals, as customarily employed in non-inferiority analyses, were calculated for the difference in post-intervention treatment effect between memory group and the individual delivery mode groups, where a value of zero indicates no difference in intervention-related gains between delivery formats. Non-inferiority was considered as established for each delivery format if the lower limit of the 90% confidence interval, as the most conservative measure of treatment effect, lay within the non-inferiority margin (-Δ).
For Aim 4 (addressing the effect of booster sessions in maintaining intervention-related gains), multilevel analysis (Howell, Reference Howell2012) was conducted with two time points (six-week and twelve-week follow-up) comprising the repeated measures factor and two groups (booster condition and non-booster condition) comprising the between-groups factor. Baseline scores and age were entered into the model as covariates. In the interests of preserving statistical power with our limited group sizes, telehealth and face-to-face participants were combined within the booster and non-booster conditions. In examining the effect of a booster session on outcome measures between the six-week and twelve-week time points, focus was centred on statistically significant interactions between the groups.
Due to the exploratory nature of this research, a conventional alpha level of .05 was employed where relevant for each of the above analyses, although greater credence should be given to findings where p ≤ .01. An initial power analysis was conducted using G*Power 3.1 (Faul, Erdfelder, Lang, & Buchner, Reference Faul, Erdfelder, Lang and Buchner2007). Assuming medium correlation between time points of .3 (Cohen, Reference Cohen1988) with an alpha of .05, 80% power, and two groups, a total minimum sample of 38 was indicated. All statistical analyses were conducted using IBM SPSS 25 (IBM Corp, 2017).
RESULTS
Participants
Of the 94 clinician referrals or expressions of interest, 34 declined or could not be recontacted, and a further 14 were excluded. Forty-six stroke survivors satisfied inclusion criteria, with 18 allocated to the face-to-face condition and 28 in the telehealth delivery condition. A recruitment and procedure flowchart outlining reasons for exclusion is shown in Figure 1. Within the face-to-face delivery group, 12 attended the clinic for intervention and assessment sessions and the remaining six were visited at home. Participant demographic variables are summarized in Table 2. There was a statistically significant difference in age between the groups, with telehealth participants being younger than those in the face-to-face condition.
Fig. 1. Recruitment and procedure flowchart with reasons for exclusion.
Table 2. Participant and Stroke variables for face-to-face and telehealth condition
Note: IQ = intelligence quotient; MoCA = Montreal Cognitive Assessment; NEADL = Nottingham extended activities of daily living; L = Left; R = Right; B = Bilateral.
a t test.
b Chi-square test for independence.
c Cohen’s d.
d Cramér’s V.
Subjective memory difficulties were reported for all participants, either by the participants themselves, close-others, or referring clinicians. Twenty-three participants (50.0%) further demonstrated memory impairment on objective measures at baseline. Fifteen participants (32.6%) scored at or below 23 on the MoCA, used as a cutoff score to suggest possible impairment (Carson etal., Reference Carson, Leach and Murphy2017).
Feasibility
Rates of agreement to participate were similar for both delivery conditions, with 94.7% of face-to-face participants and 96.6% of telehealth participants agreeing to participate after screening. Final recruitment rate was higher for the telehealth program option, with 28 telehealth participants recruited over the 24-month recruitment period, compared to 18 face-to-face participants. There was a single occurrence of mid-intervention dropout for one telehealth participant due to stroke reoccurrence. Otherwise there was full treatment adherence across both conditions, with remaining participants completing all six sessions. There was no statistically significant difference in program completion period between telehealth delivery (mean 7.41 weeks, SD 1.69; range 6–12 weeks) and face-to-face delivery (mean 8.12 weeks, SD 2.66; range 6–15 weeks) (t(45) = 1.17, p = .25). Of the 166 telehealth program sessions conducted, clinicians completed 76 questionnaires regarding technical issues. Although the majority of reported sessions indicated no technological concerns, issues were reported for some sessions. Most resolvable issues included short-term delays in establishing online connections, temporary connection instabilities affecting video or sound quality, or brief connection dropouts. A small number of sessions encountered more significant difficulties including persistent connection errors requiring rescheduling of sessions. One session encountered issues with the participant’s microphone, overcome using a telephone in conjunction with the videoconferencing session. All reported issues were able to be resolved. A summary of technical issues encountered is presented in Table 3.
Table 3. Summary of technology-related issues impacting telehealth sessions
Note: Total n = 76 sessions.
Effectiveness Analyses
Primary Outcome Measure (GAS)
Analysis of intervention effectiveness on the primary outcome measure revealed that, controlling for baseline scores and age, there were no statistically significant main effects for group and no interaction effect between delivery mode and time point. There was a statistically significant main effect of time, (F(2, 62.48) = 70.58, p < .001), indicating an overall increase in GAS scores over time for both telehealth and face-to-face delivery groups. Estimated marginal means and standard errors based on the model are presented in Table 4. Pairwise comparisons indicated that GAS scores improved significantly at post-intervention compared to baseline (p < .001, d = 1.92 signifying a very large effect) and remained significantly improved from baseline at 6-week follow-up (p < .001, d = 2.29: an even larger effect), indicating that gains were extended at follow-up. Descriptive analysis confirmed that post-intervention, 92.9% of participants in the face-to-face condition reported attaining at least one goal, reducing slightly to 85.7% at six-week follow-up. For telehealth participants, 90.9% reported attaining at least one goal following intervention; by the six-week follow-up time point, 100% reported attaining at least one goal, and 85% had achieved all of their set goals.
Table 4. Descriptive statistics for outcome measures for face-to-face and telehealth conditions
Notes: Marginal means are estimated from the multilevel model; SE=standard errors; F2F=Face-to-face condition; TH=Telehealth condition; Overall means represent combined F2F and TH scores; GAS=Goal Attainment Scaling; EMQ=Everyday Memory Questionnaire; CAPM=Comprehensive Assessment of Prospective Memory; RPA-ProMem=Royal Prince Alfred Prospective Memory Test; RAVLT=Rey Auditory Verbal Learning Test; * denotes significance at an alpha level of .05; ** denotes significance at an alpha level of .01.
Secondary outcome measures
Subjective measures. For everyday memory, there was a statistically significant group by time interaction (F(2, 62.59) = 3.73, p = .030). Both conditions showed improvements post-intervention, with the telehealth condition demonstrating greater improvement (a medium to large effect size compared to a small to medium effect size in the face-to-face condition). At follow-up, the telehealth condition reported further improvements while the face-to-face participants reported a relapse in memory failures. For self-reported prospective memory, there was no significant group by time interaction; however, there was a statistically significant main effect of time (F(2, 73.11) = 4.54, p = .014). Pairwise comparisons demonstrated that prospective memory lapses were significantly improved at post-intervention (p = .018) and follow-up (p = .006), compared to baseline for both groups. Effect sizes (d = .50 and d = .63, respectively) were moderate at both time points. For internal strategy use, there was a statistically significant group by time interaction (F(2, 66.68) = 3.43, p = .038), whereby telehealth participants improved markedly at post-intervention (d = .74, demonstrating a moderate to large effect), but reported a loss of a proportion of those gains at six-week follow-up (d = .31). Face-to-face participants reported improvements at both post-intervention and follow-up. There were no statistically significant main effects or interactions regarding the use of external strategies. Significant interactions are presented in Figure 2.
Fig. 2. Marginal means (±SE) by Telehealth and Face-to-face groups for measures showing group by time interactions. Note: EMQ = Everyday Memory Questionnaire; Lower scores for EMQ = fewer lapses.
Objective measures. For prospective memory, there was a statistically significant main effect of time (F(2, 85.11) = 9.48, p < .001), whereby post-intervention and follow-up scores (p = .002 and p < .001, respectively) were significantly higher than baseline. A medium to large effect at post-intervention was indicated (d = .68), increasing to a large effect by follow-up (d = .93). In contrast, there were statistically significant main effects of time indicating decreases in verbal new learning (F(2, 79.31) = 8.69, p < .001) and delayed memory (F(2, 76.87) = 9.33, p < .001). Both new learning and delayed memory measures recorded significantly poorer scores at post-intervention (p = .002, and p = .008, respectively, with medium effect sizes of .68 and .51) and at follow-up (p < .001 and p < .001, large effect sizes of .86 and .91), compared to baseline.
Comparison with existing group delivery: Non-inferiority analysis
The results of the non-inferiority comparisons are shown in Figure 3, demonstrating that the entire confidence interval for the telehealth comparison, 90% CI [-4.07,7.49], lay above the value of Δ (-4.64), establishing non-inferiority to group delivery for the individual telehealth delivery mode on the primary measure of GAS. In contrast, the lower end of the confidence interval for the face-to-face comparison, 90% CI [-5.45, 8.26], fell outside the non-inferiority margin, hence non-inferiority to group delivery was not established for the face-to-face condition.
Fig. 3. Ninety percent confidence intervals of the mean difference in GAS T Scores between Memory Group and Individual Delivery Modes.
Note: MG = Memory Group; F2F = Individual Face-to-face; TH =Individual Telehealth.
Effect of booster sessions
Descriptive statistics for pre-intervention participant and stroke variables for the booster and non-booster groups are presented in Table 5, although caution in the interpretation of pre-intervention significance tests is noted as described by Egbewale, Lewis, & Sim (Reference Egbewale, Lewis and Sim2014). Controlling for baseline scores and age, significant group by time interactions were observed for subjective everyday memory (F(1, 35.98) = 9.93, p = .012) and prospective memory (F(1, 37.91) = 14.54, p < .001), whereby participants who received a booster session reported significantly greater improvement in memory functioning at twelve-week follow-up than those who had not received a booster. There was a trend toward significance for the group by time interaction for goal attainment (F(1, 30.55) = 4.12, p = .051) in the opposite direction, whereby participants who had not received the booster session reported greater levels of attainment of goals than those who had. No other statistically significant group by time interactions were found on the remainder of subjective or objective outcome measures. Significant interactions are presented in Figure 4. Marginal means and significance values for both groups are presented in Table 6.
Table 5. Participant and stroke variables, and baseline group differences for booster and non-booster groups
Note: IQ = intelligence quotient; MoCA = Montreal Cognitive Assessment.
a t test.
b Chi-square test for independence.
c Cohen’s d.
d Cramér’s V.
Fig. 4. Marginal means scores (±SE) by booster and non-booster groups for statistically significant group by time interactions (EMQ and CAPM) and trend (GAS).
Note: GAS = Goal Attainment Scaling; EMQ = Everyday Memory Questionnaire; CAPM = Comprehensive Assessment of Prospective Memory. Higher scores for GAS = higher attainment of goals; lower scores for EMQ and CAPM = fewer lapses.
Table 6. Descriptive statistics for outcome measures by booster or no booster conditions
Notes: Marginal means are estimated from the multilevel model; SE = standard errors; Overall means represent combined Booster and Non-Booster scores; GAS = Goal Attainment Scaling; EMQ = Everyday Memory Questionnaire; CAPM = Comprehensive Assessment of Prospective Memory; RPA-ProMem = Royal Prince Alfred Prospective Memory Test; RAVLT = R; * denotes significance at an alpha level of .05; ** denotes significance at an alpha level of .01.
Discussion
This study aimed to investigate whether a compensatory memory rehabilitation intervention program would be feasible and effective when delivered individually and remotely using videoconferencing. As hypothesized, our results demonstrated that both individual and telehealth formats were feasible and effective in improving subjective and functional memory performance for stroke survivors. Crucially, we found support for our hypothesis that individual telehealth delivery of memory rehabilitation is at least comparable in effectiveness to face-to-face group-based methods and would not create an unacceptable loss of treatment effect. Our exploratory examination of the role of booster sessions in maintaining gains over time also found encouraging preliminary evidence centred around subjective memory measures.
Overall, high rates of participation agreement, recruitment, and treatment adherence supported feasibility for both face-to-face and telehealth programs. Recruitment rate was higher for the telehealth condition as expected, given (a) the wider geographic scope for participants and (b) regular findings of unmet need for rehabilitation services in regional areas. Treatment adherence was assisted by the flexibility to reschedule sessions to accommodate participant availability, which is less possible with group-based programs. While technological problems were present for a moderate proportion of telehealth sessions, most amounted to minor connectivity issues and barriers to delivering sessions were negligible overall.
Participants in both telehealth and face-to-face conditions exhibited significant intervention-related improvements in attainment of personally relevant goals, with demonstrated maintenance of gains at six-week follow-up. Both delivery modes resulted in similar improvements in subjective measures, including reductions in lapses of prospective memory, again with gains maintained at follow-up. Regarding everyday memory lapses however, only the telehealth condition demonstrated continued improvement through to follow-up, while face-to-face participants relapsed at follow-up. Interestingly, the opposite was demonstrated in internal strategy use. Face-to-face participants demonstrated a consistent trajectory of increased strategy use, while telehealth participants reported marked increase in implementing internal strategies at the conclusion of the program, but this increase was not maintained at 6-week follow-up. In contrast, external strategy use did not change significantly for either group. Further research is needed to explore the mechanisms of change for each delivery mode and investigate the most effective ways to sustain the various gains from memory rehabilitation over time.
These findings of overall subjective improvements are consistent with our hypothesis and with some previous investigations in rehabilitation of memory (Radford etal., Reference Radford, Lah, Thayer, Say and Miller2012; Withiel, Wong etal., Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019). The results diverge from earlier reviews (das Nair etal. Reference das Nair, Cogger, Worthington and Lincoln2016; das Nair, Cogger, Worthington, & Lincoln, Reference das Nair, Cogger, Worthington and Lincoln2017; Lockwood, Reference Lockwood2017) finding that intervention-related gains were limited to the short-term, while our participants reported continued improvements on most subjective measures at follow-up. However, the studies included in these meta-analyses mostly evaluated either group-based programs (Aben etal., Reference Aben, Heijenbrok-Kal, van Loon, Groet, Ponds, Busschbach and Ribbers2013; Radford etal., Reference Radford, Lah, Thayer, Say and Miller2012) or computerized memory training (Akerlund, Westerberg etal., Reference Westerberg, Jacobaeus, Hirvikoski, Clevberger, Ostensson, Bartfai and Klingberg2007). The remainder of studies taking a strategy training approach in a one-on-one setting similar to ours differed in the scope of the intervention (Doornhein & de Haan, Reference Doornhein and de Haan1998) or in only examining immediate benefits (Chen, Jiang, Liu, Huang, & Ding, Reference Chen, Jiang, Liu, Huang and Ding2006).
Results on objective measures of memory functioning were more mixed. There were improvements in prospective memory at both post-intervention and follow-up, consistent with findings across the range of subjective measures. Our measure used for prospective memory, while considered an objective test, measures memory performance on a functional level and allows for participants to employ compensatory strategies that were among those trained during the intervention program. Interestingly, both groups showed significantly worse performance on list learning and delayed recall measures at both post-intervention and follow-up. Explanation for this deterioration in word list recall over time, in parallel with improvements in functional memory on other measures, is not immediately clear. It is possible that the absence of everyday context for the test content, and the restriction of compensatory strategies to assist recall, was more challenging and anxiety-provoking after intensive training in the use of such strategies to support memory performance in everyday contexts. Evidently, and unsurprisingly, improvements in objective memory capacity do not underlie gains made in response to compensatory strategy training.
Non-inferiority to the existing group-based intervention format was established for the telehealth condition on the primary measure of goal attainment, whereas comparison of individual face-to-face with the group-based format narrowly fell beyond the non-inferiority margin. While mean difference in treatment effect favored individual face-to-face over the group-based format, the variability in outcome for the individual condition influenced the statistical comparison. The result may be an artifact of smaller sample size under-powering the analysis and producing a wider confidence interval in the face-to-face group.
A number of factors could have contributed to the effectiveness of the one-on-one program formats in our study. While the structural content of the individual formats was held consistent with the group-based intervention and modifications were minor, the personalized nature of individual delivery may have been an advantage, permitting greater tailoring of strategy training and allowing participants to focus on their particular needs and goals. The results for the telehealth condition were particularly encouraging. In considering factors underpinning these positive results, exposing the clinician to participants’ home environments through telehealth sessions may provide extra context to their everyday functioning and assist in training and generalization of skills. The results may also be reflective of the telehealth group, notably including rurally dwelling stroke survivors with limited rehabilitation options, being more engaged with rehabilitation goals on account of their rarer opportunities for therapy and connection with stroke clinicians. These results call for further research investigating active ingredients for different intervention formats.
Regarding the role of booster sessions, greater improvements were seen for participants in the booster session condition on subjective everyday memory and prospective memory functioning, while a trend suggested improvements in goal attainment in favor of the non-booster group. The latter result may appear contradictory; however, it is possible that booster sessions may have shifted participants’ focus toward applying strategies in a wider range of real-world contexts, reflected in the variety of everyday memory problems captured by the questionnaire measures, while the participants not receiving boosters may have maintained a more specific focus on their goals. Similarly, the absence of an effect on strategy use may reflect that beyond the 6-week follow-up, participants are likely aware of the trained strategies that serve their individual purposes and may use those more exclusively (although in a wider range of situations following a booster session). Nonetheless, these results suggest a positive effect on everyday memory for those participants having difficulty maintaining their gains over time. Future studies are needed to replicate these results and could examine alternative booster formats to maximize effectiveness in consolidating trained skills.
We acknowledge certain methodological limitations. Notably, our delivery method groups were not randomized due to pragmatic factors, consequently we are unable to discount the potential for bias or sampling imbalances between our groups. Furthermore, due to restricted resources we did not have a control group; however, this was partly addressed by using non-inferiority analysis to compare our treatment effect on the primary measure with that obtained by Withiel, Wong etal. (Reference Withiel, Wong, Ponsford, Cadilhac, New, Mihaljcic and Stolwyk2019), which did use a control group. The subjective nature of self-report measures may also be accompanied by issues of accuracy and reliability, related in this population to factors such as possible deficits in self-awareness or memory difficulties themselves. Nevertheless, the importance of measuring goal-attainment and subjective experience in rehabilitation is well established. Although our sample size was considered adequate to power our effectiveness analyses, it remained modest particularly for non-inferiority analysis, which may limit the generalizability of findings. Lastly, we were not able to fully characterize our sample due to limited availability of information in a diverse community group. We assume that our convenience sample is representative of the community of stroke survivors seeking rehabilitation of memory difficulties, although further replication and extension of the findings with greater detail regarding medical, neurological, and neuropsychological history would be needed in order to confirm this.
Our study was also unable to investigate the potential for a telehealth, group-based version of the intervention. Withiel etal. (Reference Withiel, Sharp, Wong, Ponsford, Warren and Stolwyk2018) reported that certain aspects unique to group-based rehabilitation were reported by participants as positive and valuable elements of the experience, including opportunities for social connection and learning from the experiences of fellow survivors about application of strategies in everyday life. These factors may be applicable to group-based videoconferencing, presenting a key opportunity for future researchers in this area.
This study provides preliminary evidence in support of individual and telehealth delivery of memory rehabilitation. The results are consistent with previous suggestions that functional gains in memory performance are possible through compensatory training of memory strategies. Our relatively small sample size calls for restraint in interpretation of results; however, our findings contribute uniquely to existing research and are encouraging for the development of telehealth options for memory rehabilitation services. With further research into clinical and cost-effectiveness, service providers may maximize the potential of home-based technology to overcome barriers to accessing traditional face-to-face rehabilitation services and optimize the everyday functioning of stroke survivors.
ACKNOWLEDGEMENTS
This study was supported by a Stroke Foundation Small Project Grant (SPG1712) and a seed grant from the Monash Institute of Cognitive and Clinical Neurosciences (now the Turner Institute for Brain and Mental Health). We would like to thank Dr Toni Withiel for providing the comparison dataset from her randomized controlled trial, and for her contribution to the research process. We also extend our thanks to the clinicians who delivered the intervention program, and especially to Joanna Tran and Dr Megan O’Neill for their work in administering follow-up assessments and supporting the project. Finally, many thanks to our participants and their families.
CONFLICTS OF INTEREST
The authors have nothing to disclose.
