Abstract

In recent years, several multi-user virtual environments (VEs) have been developed to promote motivation and exercise intensity in motor rehabilitation. While competitive VEs have been extensively evaluated, collaborative and competitive rehabilitation VEs have seen relatively little study. Therefore, this article presents an evaluation of a VE for post-stroke arm rehabilitation that mimics everyday kitchen tasks and can be used either solo or collaboratively. Twenty subacute stroke survivors exercised with the VE for four sessions, with the first and third sessions involving solo exercise and the other two involving collaborative exercise. Exercise intensity was measured using inertial sensors while motivation was measured with questionnaires. Results showed high motivation and exercise intensity over all four sessions, and 11 of 20 participants preferred collaborative over solo exercise while only 4 preferred solo exercise. However, there were no differences in motivation, exercise duration, or exercise intensity between solo and collaborative sessions. Thus, we cannot currently claim that collaborative exercises are beneficial for upper limb rehabilitation. Future studies should evaluate other collaborative VE designs in different settings (e.g., at home) and with different participant pairs (e.g., patient-unimpaired) to find effective ways to utilize collaborative exercises in motor rehabilitation.

1 Introduction

Stroke is the fifth leading cause of death in the United States. On average, someone has a stroke every 40 seconds, resulting in 795 thousand people with a new or recurrent stroke (ischemic or hemorrhagic) every year (Mozaffarian et al., 2016). While stroke survival rate is high, most survivors experience some level of motor impairment that requires rehabilitation (Bonita & Beaglehole, 1988). With increased demand for therapy, rehabilitation technologies have become an important tool in the treatment of these motor impairments. The most popular technological approach is to combine motion trackers or rehabilitation robots with intelligently designed virtual environments (VEs), which has been shown to be beneficial for improving upper limb functions after stroke (Laver et al., 2017).

One essential aspect of successful rehabilitation is motivation to exercise, which is correlated with long-term improvement in functional motorabilities (Maclean, Pound, Wolfe, & Rudd, 2000b, 2000a; Putrino et al., 2017). It has been shown that VEs can effectively increase patient motivation for motor rehabilitation using design strategies such as performance-based scoring, in-game rewards, entertaining graphics, etc. (Colombo et al., 2007; Mihelj et al., 2012; Saposnik & Levin, 2011; Zimmerli, Jacky, Lünenburger, Riener, & Bolliger, 2013). Since a sense of real-world relevance and usefulness is also an important motivator for exercise (Demers, Chan Chun Kong, & Levin, 2019; Lewis & Rosie, 2012), some rehabilitation VEs also aim to mimic everyday activities such as cooking or shopping (Guidali et al., 2011; Laver et al., 2012). Additionally, to ensure appropriate exercise intensity (amount of movement, muscle activation, or heart rate) that is neither overwhelming nor boring for the patient, these VEs generally include intelligent difficulty adaptation algorithms that adapt the environment to the patient's skills and abilities over time (Colombo et al., 2007; Koenig et al., 2011; Zimmerli et al., 2012). Supported by flow theory (Csikszentmihalyi, 1990) and challenge point theory (Guadagnoli & Lee, 2004), such difficulty adaptation is considered essential for rehabilitation.

One way to increase patient motivation in rehabilitation is via social interaction, which is an important aspect of intrinsic motivation in general (Malone, 1981) and was suggested for use in VEs for motor rehabilitation as early as 2006 (Johnson et al., 2006). The last few years have thus seen the development of many multiplayer VEs that allow patients to exercise together with other patients or unimpaired people (e.g., friends, significant others, etc.). Such multiplayer VEs can be classified into four main types: competition, co-activity, cooperation, and collaboration (Jarrasse, Charalambous, & Burdet, 2012). Competition, where players compete against each other, has been extensively studied for rehabilitation; both single-session (Ballester, Bermúdez i Badia, & Verschure, 2012; Goršič, Cikajlo, & Novak, 2017; Novak, Nagle, Keller, & Riener, 2014; Pereira, Bermúdez i Badia, Ornelas, & Cameirão, 2019) and multisession (Goršič, Cikajlo, Goljar, & Novak, 2017; Maier, Ballester, Duarte, Duff, & Verschure, 2014) studies have shown that competition can increase exercise intensity and motivation compared to exercising alone. However, the same studies have shown that competition is not suitable for everyone. For example, our previous work showed that a subset of patients does not enjoy competition and would prefer to work together instead (Goršič, Cikajlo, & Novak, 2017; Novak et al., 2014); another recent study observed overall higher motivation when patients worked together than when they competed (Pereira et al., 2019). Thus, multiplayer VEs that allow people to work together have the potential to improve patient motivation, especially in noncompetitive patients.

As mentioned, there are three ways to work together in multiplayer VEs: co-activity, cooperation, and collaboration (Jarrasse et al., 2012). In co-activity, two people perform a task together, but each person simply does his or her own subtask and does not interact with the other person. For example, two people may play Pong on parallel fields without being able to affect the other player's field (Goršič, Cikajlo, & Novak, 2017; Novak et al., 2014); alternatively, two people may collect objects together, with one person's performance having no effect on the other person (Pereira et al., 2019). In both collaboration and cooperation, on the other hand, each player must take the other player's behavior into account. The difference is that, in cooperation, different roles are ascribed to the players prior to the task and cannot be altered during the task; in collaboration, on the other hand, the distribution of roles is spontaneous and can be negotiated by the players during the task itself. A mostly cooperative rehabilitation VE was presented by Mace et al. (2017): both players hold an object (one on each end) and move it around, but their roles are preassigned—each player's holding point is preassigned and neither player can release the object. Elements of collaboration were present (players can still discuss how to move the object around), but the task was (in our opinion) largely cooperative. Another example of a cooperative rehabilitation VE for patient-unimpaired pairs was presented by Baur et al. (2018): the patient is meant to complete as much of the task as possible, with the unimpaired person providing assistance; while the unimpaired person can vary the amount of assistance, their roles are predefined. A similar third example of a cooperative VE is found in our previous work, with unimpaired participants providing a “backup” for patients (Goršič, Cikajlo, & Novak, 2017). These studies have shown that cooperative tasks are more motivating than exercising alone (Baur et al., 2018; Goršič, Cikajlo, & Novak, 2017; Mace et al., 2017); our previous study also found cooperation to be more motivating than co-activity (Goršič, Cikajlo, & Novak, 2017).

Unlike previous work on competition and cooperation, collaborative rehabilitation VEs remain largely unexplored. They do, however, have potential: a recent single-session study found higher motivation during collaboration than during competition or co-activity, though only in unimpaired older adults (rather than patients) and only over single 8-minute sessions (Pereira et al., 2019). Thus, the goal of the present study was to compare a collaborative and single-player VE with regard to stroke survivors’ motivation and exercise intensity (defined as the amount of movement) over multiple sessions. Our hypothesis was that the collaborative VE will be more enjoyable than an equivalent single-player VE and will not be less intense than an equivalent single-player VE. Furthermore, the VEs will maintain high levels of motivation and exercise intensity over multiple sessions (i.e., there will be no decrease in enjoyment or exercise intensity as a result of habituation).

The tested collaborative rehabilitation VE was designed to provide players with a single common goal but give them freedom to develop their own tactics regarding how to achieve the goal. Furthermore, unlike most co-active and cooperative rehabilitation VEs, which are set in entirely game-like environments with no real-world relevance (Pong, collecting stars with a jointly held object) (Goršič, Cikajlo, & Novak, 2017; Mace et al., 2017; Novak et al., 2014), we designed our collaborative VE to visually resemble a gamified food preparation scenario similar to that of Baur et al. (2018), hopefully providing a sense of purpose and real-world relevance (Guidali et al., 2011; Laver et al., 2012). Both the collaborative and single-player versions of the VE were also augmented with automatic difficulty adaptation to promote high motivation over time. An early version of this VE with no difficulty adaptation was developed and tested with only unimpaired pairs in a single session as described in our conference paper (Goršič, Tran, & Novak, 2018). In the current article, the final version of the VE was evaluated with 20 subacute stroke survivors in a 4-session protocol. In future work, the VE will be combined with competitive, cooperative, and solo exercises in a longer-term clinical trial.

2 Method

2.1 Participants

Twenty stroke survivors (53.1 ± 9.8 years old; 11 male, 9 female) were recruited. All were undergoing inpatient rehabilitation at the University Rehabilitation Institute of the Republic of Slovenia at the time of the study, and all were screened prior to inclusion in the study by an occupational therapist who otherwise did not participate in the study protocol. Participants had to be able to move their impaired arm left and right and at least partially open and close the hand. Their cognitive abilities were tested with the Mini-Mental State Examination (Crum, Anthony, Bassett, & Folstein, 1993), with a minimum score of 24 out of 30 required for inclusion. Of the 20 participants, 17 had Mini-Mental State scores of 28 or 29; the other three had scores of 25, 26, and 27. Furthermore, the Box & Block Test (BBT) (Mathiowetz, Volland, Kashman, & Weber, 1985) was performed with each arm in order to evaluate their functional abilities. Participants were paired together, resulting in 10 pairs; the two participants in each pair always experienced the collaborative VE together. These pairings were made largely based on convenience and participants’ own preferences (i.e., participants who had free time simultaneously and liked each other), though efforts were made to match participants by impairment level and age if possible. Table 1 shows participants’ ages (rounded to nearest year), genders, time since stroke (rounded to nearest month), stroke type, and BBT scores.

Table 1.
Characteristics of the 20 Participants, Who Were Divided into 10 Pairs
Box & Block Test Score
PairAgeGenderMonths since strokeStroke typeImpaired limbImpairedUnimpaired
72 male ischemic left 23 38 
65 male ischemic left 17 46 
68 male hemorrhagic right 50 
50 female ischemic left 32 62 
56 female hemorrhagic left 25 58 
50 male hemorrhagic left 48 57 
50 male ischemic right 45 56 
53 female ischemic right 48 59 
46 female hemorrhagic right 41 70 
54 male ischemic right 50 53 
53 female hemorrhagic left 13 64 
42 male hemorrhagic right 48 53 
55 female hemorrhagic left 43 46 
75 female hemorrhagic left 56 52* 
42 female ischemic right 44 60 
46 male 10 ischemic left 26 46 
52 male ischemic right 39 69 
43 male surgery** right N/A N/A 
10 44 female ischemic right 23 60 
10 45 male ischemic left 59 67 
Box & Block Test Score
PairAgeGenderMonths since strokeStroke typeImpaired limbImpairedUnimpaired
72 male ischemic left 23 38 
65 male ischemic left 17 46 
68 male hemorrhagic right 50 
50 female ischemic left 32 62 
56 female hemorrhagic left 25 58 
50 male hemorrhagic left 48 57 
50 male ischemic right 45 56 
53 female ischemic right 48 59 
46 female hemorrhagic right 41 70 
54 male ischemic right 50 53 
53 female hemorrhagic left 13 64 
42 male hemorrhagic right 48 53 
55 female hemorrhagic left 43 46 
75 female hemorrhagic left 56 52* 
42 female ischemic right 44 60 
46 male 10 ischemic left 26 46 
52 male ischemic right 39 69 
43 male surgery** right N/A N/A 
10 44 female ischemic right 23 60 
10 45 male ischemic left 59 67 

Notes.* = Dominant arm was impaired, resulting in a higher Box and Block score for the impaired arm. ** = Stroke occurred as a result of complications during brain surgery.

All participants filled out a pre-study questionnaire that asked about their age and gender as well as how often they play computer games (never, less than 2 h/week, 2–5 h/week, 5–10 h/week, 10+ h/week), how difficult they prefer computer games to be (range 1–7; 1 meaning not at all, 7 very difficult), and how much they enjoy collaborating with other people (range 1–7; 1 meaning not at all, 7 very much). This questionnaire showed that the average participant plays games less than 2 hours per week (average answer 1.8 ± 1.0; 1 meaning never, 2 meaning less than 2 h/week), prefers medium game difficulty (4.2 ± 1.5), and weakly enjoys collaborating with other people (4.8 ± 1.5).

All participants also filled out the Ten Item Personality Inventory, which measures the Big Five personality scales: extraversion, agreeableness, conscientiousness, emotional stability (neuroticism) and openness to experiences (Gosling, Rentfrow, & Swann Jr., 2003). Each scale consists of two items that are answered on a 7-point Likert scale, resulting in a range of 2–14 for each scale. Self-reported values across the 20 participants were 9.5 ± 2.8 for extraversion, 5.4 ± 2.3 for agreeableness, 10.9 ± 2.0 for conscientiousness, 6.0 ± 2.3 for emotional stability, and 10.1 ± 2.1 for openness to experiences. These were collected for use as covariates since prior research showed that interest in competition and collaboration is influenced by the Big Five scales (Goršič, Cikajlo, & Novak, 2017; Novak et al., 2014).

2.2 Materials

2.2.1 Arm Rehabilitation System

The device used in the study was the Bimeo arm rehabilitation system (Kinestica d.o.o, Slovenia). It consists of 3 inertial measurement units: one on the upper arm, one on the forearm, and one in a hand module placed on a table (see Figure 1). Additionally, a force sensor is integrated into the hand module to detect pressing forces (pushing the hand module downward). The VE requires participants to tilt the hand module left, right, forward, and backward up to 20° from the center position using their wrist and forearm; while such fine distal movements are not optimal for all patients, they are commonly used in rehabilitation (Hsieh et al., 2018), and we chose to use the same motions for all patients to allow easier comparison. Furthermore, it requires participants to press the hand module with a vertical force of at least 5 N in order to pick up or release virtual objects; this threshold was selected as the lowest that can be reliably detected by the force sensor without false positives or false negatives. Participants were not allowed to lean forward onto the module in order to apply the force. A similar setup was used in our previous studies (Goršič, Cikajlo, Golijar, & Novak, 2017; Goršič et al., 2018). The single-player version of the VE uses a single Bimeo while the collaborative version of the VE uses two Bimeo systems (one for each player). Each player's exercise intensity is estimated as the root-mean-square of the angular velocity of the inertial sensor in the hand module; this has been shown to be a good estimate of exercise intensity (van der Pas, Verbunt, Breukelaar, van Woerden, & Seelen, 2011) and was previously used in our research (Goršič, Cikajlo, Goljar, & Novak, 2017; Goršič et al., 2018).

Figure 1.

The Bimeo arm rehabilitation system. It consists of three inertial sensors: one attached to the upper arm, one to the forearm, and one integrated in the hand module together with a force sensor.

Figure 1.

The Bimeo arm rehabilitation system. It consists of three inertial sensors: one attached to the upper arm, one to the forearm, and one integrated in the hand module together with a force sensor.

2.2.2 Collaborative Rehabilitation Environment

Our collaborative rehabilitation VE visually resembles two everyday tasks: preparing meals and sorting dishes, though with simplified movements. It consists of seven different scenarios that players perform in random order. In each scenario, players are required to move 8 objects to a designated spot; once this is accomplished, they advance to the next scenario. The scenarios involve either preparing different meals (pizza, two different salads, and two different buffets) or sorting dishes (loading dirty dishes in a dishwasher or placing clean dishes in a cupboard) as shown in Figure 2.

Figure 2.

Screenshots of two-player scenarios in the collaborative rehabilitation environment. Two hand pointers represent each of the Bimeo handles used by the participants. Each scenario consists of eight items that need to be moved to the designated spot in order to complete a cooking or dish sorting task. Simultaneously, flies appear randomly from the edges of the screen and fly towards the designated target spot (bowl, basket, dishwasher, etc.). Players must swat the flies away using their pointers while completing the scenarios.

Figure 2.

Screenshots of two-player scenarios in the collaborative rehabilitation environment. Two hand pointers represent each of the Bimeo handles used by the participants. Each scenario consists of eight items that need to be moved to the designated spot in order to complete a cooking or dish sorting task. Simultaneously, flies appear randomly from the edges of the screen and fly towards the designated target spot (bowl, basket, dishwasher, etc.). Players must swat the flies away using their pointers while completing the scenarios.

Players use wrist and forearm motions to move the Bimeo's hand module in two dimensions (left-right, forward-backward) to move a hand-shaped pointer on the screen, with each player's position represented by a different pointer (blue for one player, white for the other). Tilting the Bimeo's hand module forward/backward moves the pointer up/down (with tilts of ±20° from the center corresponding to the top and bottom edges of the screen) while tilting the module left/right moves the pointer left/right (with tilts of ±20° from the center corresponding to the left and right edges of the screen). Items are picked up or dropped onto the designated spot (bowl, basket, dishwasher, etc.) by pushing the hand module downward with a vertical force of at least 5 N. The position of the designated spot on the screen differs between scenarios to promote arm movements in various directions. Additionally, in every scenario, flies appear randomly from the edge of the screen and move towards the designated spot. Participants need to swat the flies by “touching” them with their pointer before they reach the designated spot; if the flies do reach that spot, the scenario is considered failed and is automatically restarted.

The VE can be used in a single-player (SP) version or a two-player (2P) collaborative version. The difference between the two versions is that the two-player version includes an additional hand pointer representing the second Bimeo system. While the 2P VE can be played either with no preassigned roles (each player can do anything) or with preassigned roles (one collects items, the other swats flies), our previous study with unimpaired participants (Goršič et al., 2018) found that participants strongly preferred not to have preassigned roles. Not preassigning roles increased interaction between players and encouraged them to develop tactics for more efficient scenario completion. In this study, participants thus always used the VE without preassigned roles.

The VE automatically adapts the difficulty of the current scenario based on the completion time of the previous scenario. The difficulty level is manipulated via the number of flies and the speed with which they move towards the designated spot. The VE starts with 2 slow flies (speed = 1/20 of screen width per second). Increasing or decreasing the difficulty by 1 level then increases or decreases the number of flies by 1. Once players have successfully completed a scenario in the presence of more than 4 flies (difficulty level > 4), higher difficulty levels also increase the flies’ speed (speed = 1/15 of screen width per second in difficulty levels 5 & 6). The highest difficulty level (7) consists of 7 flies that move with the highest possible speed (speed = 1/10 of screen width per second). This is the maximum level that was successfully completed by unimpaired pairs in an unpublished pilot study.

2.3 Procedure

The study was set at the University Rehabilitation Institute of the Republic of Slovenia, where participants were undergoing inpatient rehabilitation. It was approved by the Institute's ethics committee. The protocol consisted of four sessions performed within a week. All sessions took place in a designated room next to the room where participants engaged in their regular occupational therapy. At the start of the first session, all participants filled out the pre-study questionnaire and personality inventory, described in the Participants section. In each session, participants were required to exercise with the rehabilitation system for at least 10 minutes, with no upper time limit. In the first and third sessions, participants exercised with the single-player version of the VE; in the second and fourth sessions, they exercised with the two-player version of the VE. Participants were paired together—the same two participants exercised together in both two-player sessions. This protocol is similar to our previous study of a competitive game (Goršič, Cikajlo, Goljar, & Novak, 2017); in that study, three two-player sessions were followed by one single-player session (AAAB), but this was shown to be a study weakness and was thus replaced with an alternating (ABAB) protocol for the current study. A similar alternating protocol was also recently used with cooperative and single-player rehabilitation VEs by Baur et al. (2018). We acknowledge that our alternating protocol is nonetheless prone to bias since the first session is always a single-player one; we considered using both ABAB and BABA protocols (which would have been scientifically more valid), but eventually only used an ABAB protocol since this was a feasibility study where the number of participants was expected to be limited.

The experimental setup for a two-player session is shown in Figure 3. During each session, three measurements were automatically taken by the VE: session duration, difficulty level, and exercise intensity (measured as described in section 2.2.1). At the end of each session, participants filled out the Intrinsic Motivation Inventory (IMI) which measures four aspects of motivation: enjoyment/interest, effort/importance, perceived competence, and pressure/tension. The IMI is a standard questionnaire commonly used with rehabilitation VEs (Colombo et al., 2007; Mihelj et al., 2012; Nijenhuis et al., 2015; Pereira et al., 2019) and has been extensively validated in other settings (McAuley, Duncan, & Tammen, 1989). The specific IMI version used was the same 8-item version used in our previous study (Goršič, Cikajlo, Goljar, & Novak, 2017), with two 7-point Likert items for each aspect of motivation. This resulted in a possible range of 2–14 for each motivation aspect. If participants were unable to fill out the IMI, the text was shown to them and an experimenter verbally asked for their answer to each item.

Figure 3.

Experimental setup. Two participants exercise with the collaborative VE on a common screen, each controlling their Bimeo system with their impaired arm.

Figure 3.

Experimental setup. Two participants exercise with the collaborative VE on a common screen, each controlling their Bimeo system with their impaired arm.

At the end of the last (4th) session, participants also filled out an overall comparison questionnaire that asked participants to compare the single-player and two-player VE. It consisted of three questions: “Which of the two game conditions did you prefer?” (5 possible answers: strongly/weakly preferred playing alone, no preference, weakly/strongly preferred playing with someone else), “Which of the two game conditions was more fun?” (7 possible answers: playing alone was slightly/moderately/much more fun, both were equally fun, playing with someone else was slightly/moderately/much more fun), and “Which of the two game conditions was more stressful?” (possible answers as in the previous question, but with “stressful” instead of “fun”). The same questionnaire was used to compare a competitive and single-player rehabilitation game in our previous work (Goršič, Cikajlo, Goljar, & Novak, 2017); the phrase “game” was retained since it was considered simpler and easier to understand for patients than “virtual environment.”

3 Data Analysis

For purposes of data analysis, all 20 participants were treated as independent, though we acknowledge that they were not truly independent since the second and fourth two-player sessions were played by two participants together. Across all 20 participants, one-way repeated-measures analyses of variance (RMANOVA) followed by post-hoc Holm-Sidak tests were conducted on IMI aspects, maximum difficulty reached, exercise intensity, and session duration to identify differences between sessions. When normality requirements for RMANOVA were not met, one-way RMANOVA on ranks followed by post-hoc Tukey tests was conducted instead. Furthermore, if RMANOVA showed significant differences between sessions, within-subjects contrasts were calculated to assess discrepancies between SP and 2P sessions. Finally, the participants’ ages, BBT scores, and five personality scales were used as covariates in all RMANOVA to identify effects of age, personality and impairment level. All statistical tests were conducted in SigmaPlot 12.0. The threshold for significance was set at p = 0.05, and the Huynh-Feldt correction was used in RMANOVA to adjust for lack of sphericity.

4 Results

Measurements taken by the VE across all 20 participants for all four sessions are presented in Table 2 as mean ± standard deviation. IMI results across all participants for all four sessions are presented in Table 3.

Table 2.
Measurements Taken by the Virtual Environment (Session Duration, Exercise Intensity, and Maximum Difficulty Achieved in Session) for All Four Sessions
Session1 (SP)2 (2P)3 (SP)4 (2P)
Duration (min) 12.0 ± 2.8 12.8 ± 1.6 13.7 ± 3.1 14.0 ± 2.9 
Exercise Intensity left-right (rad/s) 0.17 ± 0.05 0.18 ± 0.05 0.20 ± 0.05 0.20 ± 0.05 
Exercise Intensity forward-backward (rad/s) 0.29 ± 0.08 0.29 ± 0.10 0.33 ± 0.08 0.33 ± 0.08 
Maximum Difficulty (1-7) 2.2 ± 1.8 4.7 ± 1.9 3.8 ± 1.8 6.2 ± 1.1 
Session1 (SP)2 (2P)3 (SP)4 (2P)
Duration (min) 12.0 ± 2.8 12.8 ± 1.6 13.7 ± 3.1 14.0 ± 2.9 
Exercise Intensity left-right (rad/s) 0.17 ± 0.05 0.18 ± 0.05 0.20 ± 0.05 0.20 ± 0.05 
Exercise Intensity forward-backward (rad/s) 0.29 ± 0.08 0.29 ± 0.10 0.33 ± 0.08 0.33 ± 0.08 
Maximum Difficulty (1-7) 2.2 ± 1.8 4.7 ± 1.9 3.8 ± 1.8 6.2 ± 1.1 

Note. Sessions 1 and 3 involved the single-player (SP) environment while sessions 2 and 4 involved the two-player (2P) environment.

Table 3.
Results from the Intrinsic Motivation Inventory for All Four Sessions
Session1 (SP)2 (2P)3 (SP)4 (2P)
Enjoyment/Interest 12.7 ± 1.4 12.2 ± 1.4 12.7 ± 1.3 12.1 ± 2.3 
Effort/Importance 13.0 ± 1.8 12.7 ± 1.4 13.3 ± 0.9 13.2 ± 1.3 
Perceived Competence 10.7 ± 2.5 11.1 ± 2.7 11.9 ± 2.0 11.9 ± 2.7 
Pressure/Tension 6.7 ± 3.5 6.6 ± 3.5 6.5 ± 3.4 5.7 ± 3.3 
Session1 (SP)2 (2P)3 (SP)4 (2P)
Enjoyment/Interest 12.7 ± 1.4 12.2 ± 1.4 12.7 ± 1.3 12.1 ± 2.3 
Effort/Importance 13.0 ± 1.8 12.7 ± 1.4 13.3 ± 0.9 13.2 ± 1.3 
Perceived Competence 10.7 ± 2.5 11.1 ± 2.7 11.9 ± 2.0 11.9 ± 2.7 
Pressure/Tension 6.7 ± 3.5 6.6 ± 3.5 6.5 ± 3.4 5.7 ± 3.3 

Note. Sessions 1 and 3 involved the single-player (SP) environment while sessions 2 and 4 involved the two-player (2P) environment.

4.1 Results of Repeated-Measures Analyses of Variance

RMANOVA found an effect of session on exercise intensity (p < 0.001). Post-hoc tests found that exercise intensity in the left-right axis was significantly different between the 1st and 3rd sessions (p < 0.001), between the 1st and 4th sessions (p = 0.001), the 2nd and 3rd sessions (p = 0.014), and the 2nd and 4th sessions (p = 0.016). Similarly, for exercise intensity in the forward-backward axis, RMANOVA found an effect of session (p < 0.001). Post-hoc tests found significant differences between the 1st and 3rd sessions (p = 0.005), between the 1st and 4th sessions (p = 0.007), the 2nd and 3rd sessions (p = 0.020), and the 2nd and 4th sessions (p = 0.024). This reflects an increasing trend in exercise intensity over time, as seen in Table 2. There was no significant within-subjects contrast between SP and 2P sessions.

Maximum difficulty achieved in the VE was the same for both participants in each pair for the 2P (2nd and 4th) sessions. RMANOVA found an effect of session on maximum difficulty (p < 0.001). Post-hoc tests found significant differences between the 1st and all other 3 sessions (p < 0.001), between the 2nd and 3rd sessions (p = 0.016), 2nd and 4th sessions (p < 0.001), and 3rd and 4th sessions (p < 0.001). There was a significant within-subjects contrast between SP and 2P sessions (t = 8.104, df = 48, p < 0.001), indicating that higher maximum difficulty was reached in 2P sessions than SP sessions.

Similar to maximum difficulty, duration was equal for the two participants in the pair for the 2nd and 4th sessions. As the normality test failed, the Tukey test on ranks was performed, showing significant differences between the 1st and the 4th sessions (p < 0.05). This again reflects an increasing trend in session duration over time, as seen in Table 2, and no significant within-subjects contrast was observed.

RMANOVA showed no effect of session on any aspect of the IMI, and no contrasts were thus calculated. Finally, only one significant effect of covariates was found: extroversion was a significant covariate for effort/importance (p = 0.042).

4.2 Results of Overall Comparison Questionnaire

According to the overall comparison questionnaire, 7 participants strongly and 4 weakly preferred the 2P VE, 5 were indifferent, and 1 strongly and 3 weakly preferred the SP VE. When asked which condition was more fun, 8 participants strongly and 1 moderately experienced more fun with the 2P VE, 9 were indifferent, and 1 moderately and 1 strongly experienced more fun with the SP VE. When asked about experienced stress, most of the participants (12) were indifferent, 1 strongly, 1 moderately, and 3 weakly experienced more stress with the 2P VE while 1 moderately and 2 strongly experienced more stress with the SP VE.

5 Discussion

5.1 Overall Effects on Motivation and Exercise Intensity

The 4-session protocol was successfully completed by all 20 participants. On average, session duration was always at least 2 minutes above the minimum requirement. Furthermore, session duration, exercise intensity, and maximum achieved difficulty all increased from the first to the fourth sessions. This indicates that participants did not lose their interest in the VE over time and that they improved their skills over time.

5.2 Differences between Single-Player and Collaborative Sessions

Most measurements found no difference between the SP (1st and 3rd) and 2P (2nd and 4th) sessions. Exercise intensity, session duration, and self-reported competence all primarily increased over time rather than in only 2P sessions; enjoyment/interest and effort/importance were very similar across all four sessions. Thus, neither motivation nor exercise intensity appear to be enhanced by collaboration. Only two measurements found a difference between SP and 2P. First, maximum difficulty reached was consistently higher in the 2P sessions (with a significant within-subjects contrast), but this is to be expected given that two participants can complete the tasks more easily than one participant, and has been demonstrated in other human–human interaction studies (Sawers & Ting, 2014). Second, on the overall comparison questionnaire, most participants (11 of 20) preferred the 2P VE over the SP VE while only 4 preferred the SP one. This indicates an overall subjective preference for 2P over SP.

Lack of a measurable increase in motivation (other than the subjective preference) indicates that collaboration may not be beneficial in rehabilitation from a motivational perspective. In our opinion, there are two possible explanations: either motivation truly did not increase, or it did increase but this was not captured by our measurements (other than the overall comparison questionnaire). If there was indeed no increase in motivation, it is possible that the basic VE was designed poorly, that collaboration was designed poorly, or that only limited motivational gains are possible given the VE. Since self-reported enjoyment and effort were high in both the solo and collaborative versions of the VE, we believe that the basic VE was not poorly designed. It is possible that collaborative aspects (dividing food preparation and fly swatting) were designed poorly, reducing their motivational effects. We previously pilot-tested different collaboration designs within this VE and found good results with unimpaired participants (Goršič et al., 2018); however, stroke survivors may have different reactions to collaboration, necessitating different VE designs. Finally, it is possible that, since participant motivation was already very high in solo sessions (e.g., mean 12.7 out of 14 in session 1), there was little room to improve it further using collaboration. This would imply that the motivational effects of different rehabilitation VE elements are not necessarily additive, which could have broad implications for VE design. However, based on the results on our study, we cannot say which of the above explanations might be accurate.

On the other hand, if motivation did increase but was not measurable using our self-report methods and sensors, we may wonder whether different measurements should be taken. We believe that session duration measurements should definitely be retained, as they provide perhaps the most objective (though indirect) measurement of participant motivation and have shown good results in long-term trials (Nijenhuis et al., 2015). The IMI, though extensively validated (McAuley et al., 1989) and commonly used with rehabilitation VEs (Colombo et al., 2007; Mihelj et al., 2012; Nijenhuis et al., 2015; Pereira et al., 2019), may be poor at differentiating between similar scenarios presented to the same participant. This was previously observed for rehabilitation VEs in our previous studies (Goršič, Cikajlo, & Novak, 2017; Novak et al., 2014); it has also been observed in studies outside rehabilitation and has led to the development of more situation-focused questionnaires such as the Situational Motivation Scale (Guay, Vallerand, & Blanchard, 2000) that may provide more positive results in future studies. However, we acknowledge that an increase of motivation that cannot be easily measured with questionnaires or objective measures may be too small to meaningfully affect the rehabilitation process.

5.3 Extension to Other Populations

Our study was conducted with 20 stroke survivors undergoing inpatient rehabilitation. Furthermore, as the personality questionnaire showed, the participants tended to be low on agreeableness and emotional stability, which may have influenced their preferences regarding collaboration. In the future, we believe that collaborative exercises should be evaluated with participants in the chronic phase of stroke (who may have different attitudes toward rehabilitation) as well as in other settings—for example, participants who exercise at home with a loved one may have different attitudes toward collaboration than participants who exercise in the clinic with another patient, as shown for competitive exercises in our previous work (Goršič, Cikajlo, & Novak, 2017). A similar VE was recently developed for patient-spouse exercises at home (Baur et al., 2018); while it has not been extensively tested, we believe that it is highly promising. Finally, the work could be extended to other patient populations, such as people with cerebral palsy (who are typically much younger).

6 Conclusions

Our study compared a collaborative VE for upper limb rehabilitation to a single-player version of the same VE in a 4-session protocol. We found consistently high motivation and exercise intensity in all four sessions, and the final overall comparison questionnaire showed that most participants subjectively preferred the collaborative VE over the single-player one. However, no measurable differences in self-reported motivation, exercise duration, or exercise intensity were found between the collaborative and single-player sessions. Thus, we cannot currently claim that collaborative exercises are beneficial for enhancing patient motivation in upper limb rehabilitation. Future studies should evaluate other collaborative VE designs and potentially use other motivation measurements in order to identify effective ways for patients to collaborate during rehabilitation. Furthermore, future studies should evaluate collaborative VEs in other settings (e.g., in home rehabilitation for people with chronic stroke), with different participant pairings (e.g., stroke survivors and unimpaired loved ones), and with different age groups.

Acknowledgments

This research was supported by the National Science Foundation under grant no. 1717705, by the National Institute of General Medical Sciences of the National Institutes of Health under grant no. P20GM103432, and by the Slovenian Research Agency (research core funding No. P2-0228 and project BI-US/18-19-082). The authors would like to thank occupational therapists Katarina Košir, Tinkara Jeras, Julija Ocepek, and Slavi Kotnik for their help with data collection, Minh Tran for his help with developing the VE, and Joshua D. Clapp for data analysis advice.

References

Ballester
,
B. R.
,
Bermúdez i Badia
,
S.
, &
Verschure
,
P. F. M. J.
(
2012
).
Including social interaction in stroke VR-based motor rehabilitation enhances performance: A pilot study
.
Presence: Teleoperators and Virtual Environments
,
21
,
490
501
.
Baur
,
K.
,
Wolf
,
P.
,
Klamroth-Marganska
,
V.
,
Bierbauer
,
W.
,
Scholz
,
U.
,
Riener
,
R.
, &
Duarte
,
J. E.
(
2018
).
Robot-supported multiplayer rehabilitation: Feasibility study of haptically linked patient-spouse training
.
Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
. doi: 10.1109/IROS.2018.8593769
Bonita
,
R.
, &
Beaglehole
,
R.
(
1988
).
Recovery of motor function after stroke
.
Stroke
,
19
,
1497
1500
.
Colombo
,
R.
,
Pisano
,
F.
,
Mazzone
,
A.
,
Delconte
,
C.
,
Micera
,
S.
,
Carrozza
,
M. C.
, …
Minuco
,
G.
(
2007
).
Design strategies to improve patient motivation during robot-aided rehabilitation
.
Journal of NeuroEngineering and Rehabilitation
,
4
,
3
.
Crum
,
R. M.
,
Anthony
,
J. C.
,
Bassett
,
S. S.
, &
Folstein
,
M. F.
(
1993
).
Population-based norms for the Mini-Mental State Examination by age and educational level
.
JAMA: The Journal of the American Medical Association
,
269
(
18
),
2386
2391
.
Csikszentmihalyi
,
M.
(
1990
).
Flow: The psychology of optimal experience
.
New York
:
HarperCollins
.
Demers
,
M.
,
Chan Chun Kong
,
D.
, &
Levin
,
M. F.
(
2019
).
Feasibility of incorporating functionally relevant virtual rehabilitation in sub-acute stroke care: Perception of patients and clinicians
.
Disability and Rehabilitation: Assistive Technology
,
14
(
4
),
361
367
.
Goršič
,
M.
,
Cikajlo
,
I.
,
Goljar
,
N.
, &
Novak
,
D.
(
2017
).
A multisession evaluation of an adaptive competitive arm rehabilitation game
.
Journal of NeuroEngineering and Rehabilitation
,
14
,
128
.
Goršič
,
M.
,
Cikajlo
,
I.
, &
Novak
,
D.
(
2017
).
Competitive and cooperative arm rehabilitation games played by a patient and unimpaired person: Effects on motivation and exercise intensity
.
Journal of NeuroEngineering and Rehabilitation
,
14
,
23
.
Goršič
,
M.
,
Tran
,
M. H.
, &
Novak
,
D.
(
2018
).
Cooperative cooking: A novel virtual environment for upper limb rehabilitation
.
Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society
,
3602
3605
.
Gosling
,
S. D.
,
Rentfrow
,
P. J.
, &
Swann
Jr.,
W. B.
(
2003
).
A very brief measure of the Big-Five personality domains
.
Journal of Research in Personality
,
37
,
504
528
.
Guadagnoli
,
M. A.
, &
Lee
,
T. D.
(
2004
).
Challenge point: A framework for conceptualizing the effects of various practice conditions in motor learning
.
Journal of Motor Behavior
,
36
(
2
),
212
224
.
Guay
,
F.
,
Vallerand
,
R. J.
, &
Blanchard
,
C.
(
2000
).
On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS)
.
Motivation and Emotion
,
24
(
3
),
175
213
.
Guidali
,
M.
,
Duschau-Wicke
,
A.
,
Broggi
,
S.
,
Klamroth-Marganska
,
V.
,
Nef
,
T.
, &
Riener
,
R.
(
2011
).
A robotic system to train activities of daily living in a virtual environment
.
Medical & Biological Engineering & Computing
,
49
(
10
),
1213
1223
.
Hsieh
,
Y.
,
Lin
,
K.
,
Wu
,
C.
,
Shih
,
T.
,
Li
,
M.
, &
Chen
,
C.
(
2018
).
Comparison of proximal versus distal upper-limb robotic rehabilitation on motor performance after stroke: A cluster controlled trial
.
Scientific Reports
,
8
,
2091
.
Jarrasse
,
N.
,
Charalambous
,
T.
, &
Burdet
,
E.
(
2012
).
A framework to describe, analyze and generate interactive motor behaviors
.
PLOS One
,
7
(
11
),
e49945
.
Johnson
,
M. J.
,
Feng
,
X.
,
Johnson
,
L. M.
,
Ramachandran
,
B.
,
Winters
,
J. M.
, &
Kosasih
,
J. B.
(
2006
).
Robotic systems that rehabilitate as well as motivate: Three strategies for motivating impaired arm use
.
Proceedings of the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics
,
254
259
.
Koenig
,
A.
,
Omlin
,
X.
,
Bergmann
,
J.
,
Zimmerli
,
L.
,
Bolliger
,
M.
,
Müller
,
F.
, &
Riener
,
R.
(
2011
).
Controlling patient participation during robot-assisted gait training
.
Journal of NeuroEngineering and Rehabilitation
,
8
,
14
.
Laver
,
K. E.
,
Lange
,
B.
,
George
,
S.
,
Deutsch
,
J. E.
,
Saposnik
,
G.
, &
Crotty
,
M.
(
2017
).
Virtual reality for stroke rehabilitation
.
Cochrane Database of Systematic Reviews
,
11
,
CD008349
.
Laver
,
K.
,
Lim
,
F.
,
Reynolds
,
K.
,
George
,
S.
,
Ratcliffe
,
J.
,
Sim
,
S.
, &
Crotty
,
M.
(
2012
).
Virtual reality grocery shopping simulator: Development and usability in neurological rehabilitation
.
Presence: Teleoperators and Virtual Environments
,
21
(
2
),
183
191
.
Lewis
,
G. N.
, &
Rosie
,
J. A.
(
2012
).
Virtual reality games for movement rehabilitation in neurological conditions: How do we meet the needs and expectations of the users
.
Disability and Rehabilitation
,
34
(
22
),
1880
1886
.
Mace
,
M.
,
Kinany
,
N.
,
Rinne
,
P.
,
Rayner
,
A.
,
Bentley
,
P.
, &
Burdet
,
E.
(
2017
).
Balancing the playing field: Collaborative gaming for physical training
.
Journal of NeuroEngineering and Rehabilitation
,
14
,
116
.
Maclean
,
N.
,
Pound
,
P.
,
Wolfe
,
C.
, &
Rudd
,
A.
(
2000a
).
A critical review of the concept of patient motivation in the literature on physical rehabilitation
.
Social Science & Medicine
,
50
(
4
),
495
506
.
Maclean
,
N.
,
Pound
,
P.
,
Wolfe
,
C.
, &
Rudd
,
A.
(
2000b
).
Qualitative analysis of stroke patients’ motivation for rehabilitation
.
British Medical Journal
,
321
(
7268
),
1051
1054
.
Maier
,
M.
,
Ballester
,
B. R.
,
Duarte
,
E.
,
Duff
,
A.
, &
Verschure
,
P. F. M. J.
(
2014
).
Social integration of stroke patients through the multiplayer Rehabilitation Gaming System
.
Proceedings of GameDays 2014
,
100
114
.
Malone
,
T. W.
(
1981
).
Toward a theory of intrinsically motivating instruction
.
Cognitive Science
,
5
(
4
),
333
369
.
Mathiowetz
,
V.
,
Volland
,
G.
,
Kashman
,
N.
, &
Weber
,
K.
(
1985
).
Adult norms for the Box and Block test of manual dexterity
.
American Journal of Occupational Therapy
,
39
,
386
391
.
McAuley
,
E.
,
Duncan
,
T.
, &
Tammen
,
V. V.
(
1989
).
Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis
.
Research Quarterly for Exercise and Sport
,
60
(
1
),
48
58
.
Mihelj
,
M.
,
Novak
,
D.
,
Milavec
,
M.
,
Ziherl
,
J.
,
Olenšek
,
A.
, &
Munih
,
M.
(
2012
).
Virtual rehabilitation environment using principles of intrinsic motivation and game design
.
Presence: Teleoperators and Virtual Environments
,
21
(
1
),
1
15
.
Mozaffarian
,
D.
,
Benjamin
,
E. J.
,
Go
,
A. S.
,
Arnett
,
D. K.
,
Blaha
,
M. J.
,
Cushman
,
M.
,
Stroke Statistics Subcommittee
. (
2016
).
Executive summary: Heart disease and stroke statistics—2016 update
.
Circulation
,
133
(
4
),
447
454
.
Nijenhuis
,
S. M.
,
Prange
,
G. B.
,
Amirabdollahian
,
F.
,
Sale
,
P.
,
Infarinato
,
F.
,
Nasr
,
N.
, …
Rietman
,
J. S.
(
2015
).
Feasibility study into self-administered training at home using an arm and hand device with motivational gaming environment in chronic stroke
.
Journal of NeuroEngineering and Rehabilitation
,
12
,
89
.
Novak
,
D.
,
Nagle
,
A.
,
Keller
,
U.
, &
Riener
,
R.
(
2014
).
Increasing motivation in robot-aided arm rehabilitation with competitive and cooperative gameplay
.
Journal of NeuroEngineering and Rehabilitation
,
11
,
64
.
Pereira
,
F.
,
Bermúdez i Badia
,
S.
,
Ornelas
,
R.
, &
Cameirão
,
M. S.
(
2019
).
Impact of game mode in multi-user serious games for upper limb rehabilitation: A within-person randomized trial on engagement and social involvement
.
Journal of NeuroEngineering and Rehabilitation
,
16
,
109
.
Putrino
,
D.
,
Zanders
,
H.
,
Hamilton
,
T.
,
Rykman
,
A.
,
Lee
,
P.
, &
Edwards
,
D. J.
(
2017
).
Patient engagement is related to impairment reduction during digital game-based therapy in stroke
.
Games for Health Journal
,
6
(
5
),
295
302
.
Saposnik
,
G.
, &
Levin
,
M.
(
2011
).
Virtual reality in stroke rehabilitation: A meta-analysis and implications for clinicians
.
Stroke
,
42
(
5
),
1380
1386
.
Sawers
,
A.
, &
Ting
,
L. H.
(
2014
).
Perspectives on human--human sensorimotor interactions for the design of rehabilitation robots
.
Journal of NeuroEngineering and Rehabilitation
,
11
,
142
.
van der Pas
,
S. C.
,
Verbunt
,
J. A.
,
Breukelaar
,
D. E.
,
van Woerden
,
R.
, &
Seelen
,
H. A.
(
2011
).
Assessment of arm activity using triaxial accelerometry in patients with a stroke
.
Archives of Physical Medicine and Rehabilitation
,
92
(
9
),
1437
1442
.
Zimmerli
,
L.
,
Jacky
,
M.
,
Lünenburger
,
L.
,
Riener
,
R.
, &
Bolliger
,
M.
(
2013
).
Increasing patient engagement during virtual reality-based motor rehabilitation
.
Archives of Physical Medicine and Rehabilitation
,
94
(
9
),
1737
1746
.
Zimmerli
,
L.
,
Krewer
,
C.
,
Gassert
,
R.
,
Müller
,
F.
,
Riener
,
R.
, &
Lünenburger
,
L.
(
2012
).
Validation of a mechanism to balance exercise difficulty in robot-assisted upper-extremity rehabilitation after stroke
.
Journal of NeuroEngineering and Rehabilitation
,
9
,
6
.