PORTFOLIO
Exerbike
My Role: Research, Art, Qualitative Design, Participants Recruiting, Experiment
The Exerbike project starts from the cognitive health of the elderly and studies methods to improve the functional cognitive health of the elderly by combining aerobic physical activity (PA), cognitive training (CT) and foreign language learning (FLL). We recommend using game design, immersive technology, and user testing to investigate novel and high-acceptance intervention and training modalities, understand older adults' perspectives, and increase community engagement through PA+FLL-CT.
1. Problem
Over the years, Cognitive decline is a common concern among older adults, while maintaining good cognitive function is crucial for a better quality of life as we age. As a result, finding effective interventions to protect cognitive function is critical for successful aging.
Although physical activity has been shown to promote learning through brain plasticity, there is limited human research on the benefits of combining aerobic physical activity (PA) with cognitive training. A lack of evidence hinders our understanding of the potential cognitive benefits of such combined interventions. Based on the concept that PA itself can promote learning through brain plasticity, a combination of PA + some type of CT may lead to greater cognitive performance benefits.
This project introduces an integrated tool that combines aerobic exercise and second language learning (L2) for the purpose of enhancing adult cognitive health. The tool utilizes virtual reality (VR) technology to focus on auditory recognition and content comprehension aiming to improve language proficiency in potential adult L2 learners. The study investigates participants’ experiences, cognitive load demands, and usability of the integrated tool within a VR-based multitasking environment. We see a promising future in serious game that integrate physical exercise to motivate the uptake of health-promoting behaviors. The motivation for this experiment was to make cognitive interventions accessible, engaging and enjoyable for older adults.
2. Project Overview
While traditional cognitive training focusing on specific skills has limited real-life impact, research has found that engaging in tasks like second language learning(L2), which require a range of skills including attention and working memory, can have positive effects on brain function and help protect against cognitive decline associated with aging.
Here we consider L2 as a possible candidate type of cognitive training for mitigating cognitive decline. Adult L2 learners face difficulties in terms of maintaining motivation and engaging sufficient cognitive resources to succeed in L2 learning. Any reductions in the friction for adult L2 learning would be a substantial improvement.
VR offers benefits for language learning as it immerses learners in real-life scenarios. Studies have shown that VR-based language learning can enhance cultural understanding, boost motivation and engagement, and improve language outcomes in adults. Our approach to adult second language (L2) learning aligns with the principles of first language acquisition and lifelong learning.
In this study, we propose a tool, called Integrated Aerobic Physical Activity and Language Learning (IPALL), to promote L2. Based on the notion that aerobic exercises can, n its own, instigate learning via neuroplasticity, which affects learning. A combination of aerobic exercise and immersive language learning experience may confer greater learning retention benefits. The proposed tool focuses on auditory recognition and content comprehension to improve language proficiency in potential adult language learners using VR. IPALL is built on a stationary bike and equipped with a cadence sensor, providing a fully immersive VR Spanish language learning experience. The pilot study explores participants’ experience and cognitive load demand in this VR-based multi-tasking game environment and examines the utility and usability of the integrated tool.
3. Related Research
3.1. Adult 2nd language learning
Adult learners of foreign languages often grapple with difficulties such as first language interference and overanalyzes. Evidence shows that students who are exposed to the language they’re learning in an immersive way exhibit higher levels of fluency. Our approach is based on the recognition that learners can become aware of patterns in the foreign language as they learn, even without explicitly articulating them, which is well-established in first-language acquisition research.
Our language learning paradigm involves verbal repetition of auditory stimuli, as research supports the benefits of articulatory rehearsal for vocabulary learning. Differing from text-based exchanges, our auditory method mitigates negative transfer and enables learners to discern word segmentation and intonation patterns. By engaging the phonological loop, we support the encoding of new word forms, similar to how children acquire language and how foreign language learners acquire syntactical patterns. This strategy optimally suits adults and maximizes implicit learning.
3.2. Aerobic Exercise and Learning
Moderate aerobic exercise, keeping the heart rate within 64% and 76% of the maximum—endorsed by the American Heart Association (AHA)—is advocated for adults. Such exercise amalgamates the advantages of light and intense workouts while curbing injury risk. It has been correlated with cognitive health improvements. Existing research corroborates that both sustained and acute aerobic activities can amplify cognitive aculties such as working memory and attention. Although the specific mechanisms remain partially unexplained, the role of enhanced cerebral blood flow and neurotrophic factors is speculated. Alongside direct cognitive gains, aerobic exercise fosters neuroplasticity, essential for learning.
Empirical evidence suggests that physical activity can boost academic achievement and vocabulary acquisition in children. However, further research is warranted to ascertain the impact of aerobic exercise on learning amid cognitively challenging tasks in adults.
3.3. Immersive VR and Learning Outcomes
Immersive virtual reality (VR) has gained considerable attention for its educational applications, including language learning and second language (L2) teacher training. Previous studies have demonstrated the positive impacts of immersive VR on learning outcomes, including motivation, effectiveness, and learning performance. Furthermore, immersive VR has been successfully utilized in L2, as evidenced by commercial products like Mondly, which have shown significant improvements in learning outcomes for young learners. In-depth investigations have explored the influential factors affecting learning outcomes in immersive VR, such as usability, VR features, and motivation. These factors have been found to play a mediating role in the learning process.
4. Experiment Design
We developed a Spanish Language stationary bike tour that is accessible, immersive, and designed in training episodes that leverage exergame concepts to keep people enthusiastically engaged. We use immersive experience for the biking visual and audio tour information, while monitoring heart rate to ensure that participants engage moderate to vigorous PA.
The experimental design for stationary bike tour pilot phase incorporates pre- and post-intervention assessments, language learning evaluation, user experience testing, and data analysis to assess the intervention's effectiveness in improving Spanish language skills and cognitive function. Findings from this pilot phase will inform the development of a full-scale study and help refine the intervention for broader implementation.
4.1. Overview of the System
The IPALL system was developed by modifying the open world game Grand Theft Auto 5 using the C# programming language. Third-party mods, including gta5-real-mod, were incorporated to create a virtual city bike tour for participants. The system utilized ANT+ compatible Garmin cadence sensor and Polar H10 for monitoring participants’ cadence and heart rate, with data being polled every 250ms in-game and recorded to a csv file every second. The VR environment was presented using an HTC Vive Pro headset with lighthouses for tracking. The narrative script and Spanish audio were generated using the Voicemaker website.
4.2. Learning in Game
The experiment consists of a 20-minute virtual bike route featuring narrative audio in both English and Spanish. Participants control the bike’s movement in the game by pedaling, guiding it to desired waypoints. At each waypoint, a visual landmark is displayed along with corresponding Spanish audio, which participants are encouraged to repeat. The bike path is fixed to ensure consistent learning for all participants, eliminating the need for steering control in the game. The 20-minute path includes ten carefully selected landmarks, such as churches, commercial streets, banks, and gold courts, replicating real-world experiences. The goal is to help learners recognize patterns in Spanish vocabulary as they listen and engage with the landmarks.
4.3. UI Design
The participant’s date of birth is used to calculate the target heart rate range for moderate aerobic exercise (220-age*64% to 220-age*76%) recommended by AHA. This range is displayed through UI elements, providing feedback to participants about their pedaling speed. The UI elements include a green rectangle for being within the target range, a gray rectangle for being below the target range, and a red rectangle for being above the target range. The UI elements include corresponding text that reads “maintain this pace,”“pedal harder,”and “slow down,”, respectively. Participants can also view their heart rate and time in the game.
Fig 1. Experimental display
5. Methods
5.1. Subject Recruitment
Eighteen students (7 female, 11 male) aged 18 to 32 (M=25, SD=3.6) were recruited for the study through social media, posters, and university email distribution. A screening questionnaire collected data on age, gender, motion sickness tendency, VR experience, vision, neurological and heart conditions, falling history in the past year, and prior exposure to Spanish language learning. Participants were in good health, without significant physical impairments or visual acuity issues. Those with a history of neurological diseases (Alzheimer’s, Parkinson’s, epilepsy, chronic migraines, multiple sclerosis, stroke) or heart-related conditions (unstable angina, heart attack, heart failure, arrhythmia, valve diseases) were excluded, as were individuals with a history of more than two falls in the past year. This pilot study is in preparation for larger study in older adults, so we used the same criteria.
5.2. Measurements
1) Usability: Usability is defined as a quality attribute that assesses how easy the user interfaces are to use. The usability testing goal varies by study, in our case, the scales we used are adapted from previous studies measuring the learning experiences and outcomes of using technology and VR.
The scale we adapted includes four subscales in ’Motivation’, ’Usefulness’, ’Perceived Ease of Use’, and ’Satisfaction’.
• The motivation for using this tool is to exercise and learn a foreign language.
• The usefulness of supporting learning.
• The perceived ease of use during the learning process.
• The satisfaction of the immersive learning experience.
Fig 2. Means of Usability Quality Components (N=18
A 5-point Likert scale was used to examine the quality components mentioned above. The scale responses were stored on Qualtrics and include both positive and negative questions, the score of negative questions was inverted before data analysis. We did a mode analysis on each scale item and a mean analysis on each sub-scale under the whole evaluation of learning outcomes of using this IPALL tool.
2) Performance Data: Participants’ performance data, including heart rate (maximum, minimum, and average) and language performance from Pre- and Post-Spanish tests, were collected. The Pre-Spanish and Post-Spanish are identical test and targets the same vocabulary. We examined correlations between performance data and data collected by a 5-point Likert scale in usability, and utility to explore the possible factors that impact learning outcomes and motivation.
3) Utility: NASA-TLX, a workload measurement, was used to assess the cognitive loads induced by the tool [19]. It encompasses six dimensions: Physical Demand, Mental Demand, Performance, Effort, Temporal Demand, and Frustration. NASA-TLX is widely employed to understand users’ experiences in multitasking environments, both physical and digital. While NASA-TLX involves rating and weighting, weights are not commonly used due to individual differences in dimension importance. Some studies have shown that rating scores alone are as sensitive or even more effective in measuring workload. In our study, we used rating scores to gauge the participants’ level of cognitive involvement.
5.3. Experiment Procedure
Participants were first asked to complete a pre-Spanish proficiency test upon arrival, followed by being seated on stationary bikes. They were then equipped with an HTC Vive Pro VR headset and a Polar H10 Heart Rate sensor. To ensure familiarity, participants were shown a trial environment where they pedaled to activate the cadence sensor and move in the game. If motion sickness occurred, the researcher paused the trial and assisted the participant. Once ready, the actual experiment began, with participants aiming to maintain their heart rate in the green range. After twenty minutes gameplay, participants took a post-Spanish proficiency test and completed a 5-point Likert scale on usability. Additionally, the NASA-TLX workload assessment was given to the last nine participants before the post-Spanish test.
6. Results
6.1. Usability
Four quality components scored above 3.5 (1=strongly disagree, 5=strongly agree), as shown in Figure 2(N=18). Perceived ease of use (M=4.18, SD=0.47) received the highest score, indicating a high level of agreement with the tool’s ease of use. Most statements related to motivation, usefulness, satisfaction, and perceived ease of use received high agreement scores (4=agree or 5=strongly agree) from the majority of participants. Using this type of application enhances/will enhance the effectiveness of my learning.’ received the most neutral level of agreement.
6.2. Utility
The NASA-TLX workload assessment was as added after we collected nine participants’ data, and we were interested to understand which tasks in this multi-tasking game environment took the most cognitive workload. Thus, only the second half of the participants’ data was collected. Table I (N=9) illustrates the mean values and standard deviations for physical demand (M=75, SD=16.58), mental demand (M=61, SD=21.3), and effort (M=64, SD=14.45) in the multitasking environment. Participants were required to pedal for 20 minutes while maintaining their heart rate at a targeted aerobic level, and also listen to and repeat Spanish that they heard. The mean values for temporal demand (M=48, SD=22.65), frustration (M=24, SD=22.69), and performance (M=21, SD=15.36) were lower.
Table 1. Means by Cognitive Load Measured by NASA-TLX (N=9), the Scale Range Is From 0-100.
6.3. Exploration
Additional analyses were conducted on the participants’ learning outcomes to identify potential correlations and differences. As expected, no significant differences were observed between the participants’ pre- and post-Spanish accuracy after a single 20-minute gameplay section. However, of the 18 participants, 8 (44%) improved their Spanish recognition accuracy after a single section, 6 (33%) maintained the same performance, and 4 (22%) had a lower recognition accuracy performance on the post-Spanish test.
7. Discussion
We present pilot data on a combined L2 and aerobic exercise game. Our aim was to assess the feasibility of this combination for a future larger study. To this end, we examined user experience within this integrated tool, with the goal of improving future versions for community settings involving potential beneficiary populations, such as older adults. Usability results show a trend towards positive attitudes regarding the usefulness of this game tool, with agreement on its potential for motivating exercise and learning, supporting learning in an easier way, and providing perceived ease of use during the learning process. Overall, participants expressed satisfaction with their learning experience during a virtual bike tour.
However, we agree that there is room for improvement. The lack of feedback on language learning in the game may be related to the reflection on ’perceived usefulness,’ preventing users from tracking their progress and monitoring their learning outcomes. The workload results indicate high physical and mental demands during multitasking. While we expected that a single 20-minute training session would not lead to measurable language learning, some participants did improve their Spanish recognition accuracy after gameplay, while others showed a decline. We do not have a definitive explanation for this, but previous studies suggest that task switching and attention withdrawal can impair memory and lower performance. It is possible that some participants did not grasp the learned word patterns, and the credits obtained in the Pre-Spanish test could be attributed to lucky guesses rather than understanding. The interaction between cognitive load demands and learning performance in a multitasking environment warrants further investigation to inform the design of a learning game.
Despite the limitations of our preliminary study, such as homogenous participant backgrounds and restricted sample size, the outcomes derived provide valuable insights for iterative design enhancements. Future adaptations include implementing audio recordings of player repetitions and scrutinizing speech pitch for real-time feedback on pronunciation and learning progress assessment. We also aim to delve deeper into the correlation between the game environment and learning outcomes, and contemplate the inclusion of a multimodal sensor for gauging cognitive load during gameplay. Exploring the use of authentic voices versus AI-generated ones for heightened engagement and user experience is a prospective area of interest. By continuing to work on game environments like this, we believe it can have a positive impact on adult implicit language learning and potentially improve motivation for exercise.