Age-related Electrophysical Correlates of Cross-modal Attention Switching

The human experience is inherently multisensory, demanding seamless attentional switches between different sensory modalities. As people age, basic auditory and visual processing abilities decline, such as visual acuity (Spear, 1993; Cerella, 1985) and peripheral auditory processing (Schneider, Speranza, & Pichora-Fuller, 1998; Alain, Ogawa, & Woods, 1996; Pichora-Fuller, Schneider, & Daneman, 1995). Multiple studies also suggest that sensory modalities are crucial in age-related selective attention (Guerreiro, Murphy, & Van Gerven, 2010). This raises the question of whether the cross-modal attention-switching ability changes when aging. Performance in cross-modal attention switching was accessed using a cued cross-modal attention-switching paradigm, which offers greater ecological validity by closely mirroring the complex and dynamic nature of real-world attentional demands.

The task-switching paradigm utilizing stimuli or task sets across various modalities has become a valuable tool for investigating the flexibility required to switch attention between sensory modalities (Lukas, Philipp, & Koch, 2010a, 2010b). In contrast, the traditional task-switching paradigm, explored for over 3 decades, has typically involved adapting the stimuli or task sets (i.e., the stimulus–response mapping rules) within the same sensory modality (i.e., unimodal attention-switching paradigm). It has yielded fruitful findings (see, e.g., Koch & Kiesel, 2022; Koch, Poljac, Müller, & Kiesel, 2018, for a review) and shown that older adults, relative to younger adults, developed performance deficits specifically in task switching (i.e., increased mixing costs and switch costs, see, e.g., Wasylyshyn, Verhaeghen, & Sliwinski, 2011; Karbach & Kray, 2009).

The cross-modal attention-switching paradigm (see Figure 1) involved the simultaneous presentation of lateralized visual and auditory stimuli (i.e., bimodal stimuli). A unimodal visual or auditory cue indicated the relevant target modality for each trial. Participants were tasked with determining whether the stimulus in the currently relevant target modality appeared on the left or right side (i.e., a spatial location task). The cue enabled participants to know when to switch to the other modality or continue attending and responding to the same modality (i.e., mixed task). This approach allowed us to investigate the dynamics of attentional switches systematically (see Lukas et al., 2010a, 2010b) and explore how individuals, particularly across different age groups, manage cross-modal demands in a modality-switching context. In addition, the stimuli in the target modality were either spatially congruent to the stimulus in the current distractor modality (i.e., a right-sided visual target and a right-sided auditory distractor [or vice versa] would lead to the same response) or incongruent (i.e., a right-sided visual target and a left-sided auditory distractor would lead to different response). This design allowed us to estimate cross-modal interference by the congruency effect. We also included a single task in which the participants were required to attend to only one modality of the bimodal stimulation in the block (e.g., Rubin & Meiran, 2005).

It is essential to define the concept of a task as it pertains to our study. A widely accepted notion is that each task necessitates the activation of a “task-set.” Monsell (2003) described a task-set as the mental representation of task goals, encompassing the set of stimuli (such as cues, targets, and distractors) and their associated rules (i.e., which response corresponds to which target). In most task-switching experiments (e.g., Hirsch, Schwarzkopp, Declerck, & Reese, 2016; Meiran, 1996; Rogers & Monsell, 1995), the rules typically differ from one task to another, allowing each response to be linked to multiple rules. However, distinguishing between tasks does not require differing rules; as long as one component of the task-set varies, the tasks can be considered distinct. For instance, in spatial tasks, participants indicate the side on which a stimulus is presented. With bimodal stimuli (simultaneously presented auditory and visual stimuli, i.e., bivalent stimuli), each modality can be presented unilaterally on the left or right in each trial. Therefore, we consider a switch in target modality as a task switch. This perspective guides our approach, where a target modality switch is treated as a task switch.

In this cross-modal attention-switching paradigm, one can uncover three crucial behavioral performance indicators. First, the mixing costs refer to decreased performance when repeating the same modality in a mixed task compared with a single task. The mixing costs are associated with the working memory load required to simultaneously hold and update multiple tasks and the corresponding task sets in memory (Kray & Lindenberger, 2000). Second, the switch costs denote decreased performance when switch trials within a mixed task are compared with repetition trials within a mixed task. The switch costs are related to the ability to flexibly switch from one task to another by activating the currently relevant task while inhibiting the currently irrelevant task (see Koch et al., 2018, for a review). Finally, the congruency effect highlights performance differences between incongruent and congruent trials, which, in our case, involves distractor suppression processes. Lukas and colleagues (Lukas et al., 2010a, 2010b) showed that younger adults exhibit impaired performance when switching modalities compared with repeating the same modality and when dealing with incongruent trials compared with congruent trials. Schils and colleagues (Schils, Koch, Huang, Hsieh, & Stephan, 2024), using the same paradigm, found age-related increased mixing costs but no specific age-related increase of switch costs. In addition, the congruency effect was only slightly larger in younger adults especially for the switch trials. In the current study, we focused on the electrophysical correlates of such cross-modal attention switching.

Electrophysiological measures like ERPs can improve our understanding of the neural processes contributing to age-related cross-modal attention switching. ERPs have been employed to investigate the processing stages in the task-switching paradigm that usually involves one modality (see Gajewski, Ferdinand, Kray, & Falkenstein, 2018, for a review), but to our knowledge, no study ever examined whether the cross-modal attention-switching paradigm also elicits ERP components similar to those observed in previous task-switching studies. Thus, our first aim was to investigate whether cross-modal attention switching exhibits similarities to unimodal attention switching. Second, we aimed to assess how aging impacts cross-modal attention-switching performance, namely, age-related mixing/switch costs and their interactions with the congruency effect, in behavioral and ERP measures.

Previous ERP studies using unimodal attention-switching paradigms (typically focusing on the visual modality) revealed ERP components associated with mixing costs and switch costs during both the cue processing and target processing stages (see Gajewski et al., 2018, for a review). The prominent findings from the previous research regarding mixing- and switch-related ERP components include P3 and lateralized readiness potential (LRP). The P3 component, often associated with attention and working memory processes, reflects the allocation of attentional resources and the updating of working memory (Polich, 2007; Donchin, 1981). The LRP is an electrophysiological marker of motor preparation that reflects the preparation and execution of motor responses (Masaki, Wild-Wall, Sangals, & Sommer, 2004; Leuthold & Jentzsch, 2002). It has been shown that mixing- and switch-related ERP components demonstrated increased cue-locked P3 amplitude (P3 increased positivity), prolonged cue-locked P3 latency, decreased target-locked P3 amplitude (P3 decreased positivity), and prolonged onset latency of the target-locked LRP. Therefore, in the current cross-modal attention-switching study, we likewise focused on the cue-locked P3, target-locked P3, and target-locked LRP components.

The cue-locked P3 is believed to reflect the update of relevant task sets and preparation for an upcoming task (Kieffaber & Hetrick, 2005). Previous research has indicated a larger amplitude for the mixed task than for the single task and a further increase in amplitude for switch trials compared with repetition trials in the mixed task. The magnitude of positivity is inversely related to the behavioral switch costs, showing that a larger amplitude is associated with smaller switch costs (Kieffaber & Hetrick, 2005; Nicholson, Karayanidis, Poboka, Heathcote, & Michie, 2005). Regarding the age effect, no age-related differences were observed in cue-locked P3 amplitude when comparing single and mixed tasks (Karayanidis, Provost, Brown, Paton, & Heathcote, 2011; Kray, Eppinger, & Mecklinger, 2005; see also West & Travers, 2008, for the opposite findings). However, the cue-locked P3 latency was significantly delayed in older adults under the mixed task (Kray et al., 2005). This implied that both younger and older adults utilized the cues for updating a representation of the relevant task sets, and this updating processing is slower for older adults. The age effect was also evident in switch and repetition trials within the mixed task. Specifically, the cue-locked P3 was larger on switch trials than on repetition trials in younger adults, whereas this difference diminished in older adults (Karayanidis et al., 2011; Kray et al., 2005; West & Moore, 2005). Given that congruency is defined by the target–distractor relation and only determined after cue presentation, we did not expect any cue-locked P3 modulation based on congruency.

The target-locked P3 can serve as a valuable measure for assessing the cognitive workload involved in a task (Donchin, 1981). The amplitude of target-locked P3 typically reduces under high task demand. Consistent with this observation, previous research has shown a larger amplitude for the single task than the mixed task, with a further decrease in amplitude for switch trials compared with repetition trials (Karayanidis et al., 2011; Kieffaber & Hetrick, 2005; Karayanidis, Coltheart, Michie, & Murphy, 2003). Regarding the age effect, Karayanidis and colleagues (2011) discovered distinct P3 activation for mixing costs and switch costs in older adults. Many studies have demonstrated smaller target-locked P3 positivity associated with mixing costs and switch costs, mainly found in younger adults and high-performing older adults. Conversely, low-performing older adults showed similar amplitude for switch and repetition trials (Gajewski & Falkenstein, 2011; Karayanidis et al., 2003, 2011). The latency of P3 is sensitive to task demands and related to the processing time for evaluating and categorizing a presented stimulus (Polich, Howard, & Starr, 1983; Duncan-Johnson, 1981). Hsieh and Liu (2005) showed a similar latency of P3 for switch and repetition trials and to the best of our knowledge, no comprehensive studies have reported on age-related target-locked P3 latency during task switching for the unimodal task-switching paradigms.

The LRP is believed to mirror the preparation for motor activity on a specific body side. The target-locked LRP is measured concerning the moment the triggering stimulus is presented and reflects the initial time of response selection (Masaki et al., 2004). A delayed target-locked LRP onset latency has been frequently reported in unimodal task-switching paradigms in which target-locked LRP onset was delayed for switch trials compared with repetition trials (e.g., Whitson et al., 2014; Hsieh & Liu, 2005; Hsieh & Yu, 2003a, 2003b). Delayed target-locked LRP onset may result from slower stimulus encoding and/or a delayed decision because of stimulus-level interference (Whitson et al., 2014), suggesting that unimodal task switching influenced stages preceding LRP onset, such as stimulus identification and response selection. Regarding the age effect, older adults showed similarly delayed target-locked LRP onset for repetition and switch trials (Whitson et al., 2014).

Previous studies have largely focused on unimodal attention switching, often neglecting cross-modal attention switching. The present study emphasizes cross-modal attention-switching performance, reflecting real-world scenarios where individuals frequently switch their attention between different sensory modalities to navigate complex and dynamic environments. Two primary objectives were examined. We investigated whether cross-modal attention switching demonstrates similarities in behavioral performance and ERP components compared with traditional unimodal attention switching. Specifically, we examined ERP components related to mixing/switch-related increased positivity in cue-locked P3, mixing/switch-related decreased positivity in target-locked P3, and mixing/switch-related longer latency in target-locked LRP components.

In our current study where switch tasks span different modalities (i.e., visual vs. auditory) rather than switching between different stimulus–response mapping rules within the same modality, the focus may switch toward identifying stimulus modality rather than the decision making. Accordingly, the switch processes across different modalities may involve more stimulus-set biasing processes than response-set adjustments, as per Meiran's (2000) model.

Moreover, we aimed to assess how aging impacts cross-modal attention-switching performance in behavioral and ERP measures. Our exploration involved examining distinct ERP components mentioned above, allowing us to discern the influence of aging on neural correlates associated with attentional flexibility across sensory modalities in cross-modal attention switching. Older adults have been observed to face difficulties in integrating visual and auditory information (Yang et al., 2022), as well as in maintaining memory for multiple task sets (Kray & Lindenberger, 2000). This indicates potential age-related deficits in the stimulus identification stage of processing, as well as in working memory. Consequently, we anticipated observing larger age-related mixing and switch costs on the target-locked P3 latency than on the target-locked LRP onset latency, as previously reported for switching between task sets within the same visual modality (e.g., Whitson et al., 2014; Hsieh & Liu, 2005; Hsieh & Yu, 2003a, 2003b).

In addition, limited research explores the influence of age on cross-modal (auditory and visual) distractor processing, necessitating a deeper exploration. It has been shown that older adults tend to be impaired in ignoring distractors (Hasher & Zacks, 1988) and that the congruency effect was slightly larger in younger adults, especially for the switch trials in behavioral data (Schils et al., 2024). We expected that congruency may further interact with age-related mixing and switch costs in target-lock P3 and LRP.

We recruited 36 younger adults (21 male participants, mean age = 21.61 ± 2.1 years, age range: 18–26 years) and 36 older adults (12 male participants, mean age = 70.3 ± 4.7 years, age range: 63–80 years) from the community near National Cheng Kung University in Tainan City, Taiwan. All participants reported normal or corrected-to-normal hearing and vision, with no history of neurological disorders or cognitive impairments. Before the experiment, each participant provided signed informed consent, approved by the National Cheng Kung University Research Ethics Committee for Human Behavioral Sciences (REC109–074-2). After the experiment, each younger participant received NT$200 per hour, and each older participant received NT$250 per hour as compensation. Each participant underwent two sessions lasting approximately 2.5 hr per session, including ERP setup, conducted on separate days. The study was conducted following the ethical standards of the Declaration of Helsinki.

Visual stimuli were presented on a 24.5-in. LCD monitor (ROG Strix XG258Q), and the auditory stimuli were delivered through a headset (ER-3C insert earphones). The experimental program was controlled by a personal computer compatible with the Psychophysics Toolbox (Brainard, 1997) within the MATLAB environment (MathWorks Inc.). The monitor operated at a refresh rate of 85 Hz and a resolution of 1920 × 1080 pixels. A chinrest was utilized to minimize head movements and ensure participants maintained a straight gaze at the screen's center. The viewing distance was set at 61 cm. The entire experiment took place in a sound-attenuated darkroom, with the sole illumination source being the monitor.

The visual target was a white diamond (1.5 cm in width and height, corresponding to a visual angle of 1.41°) displayed against a black background. The visual target appeared either on the left or right side of the monitor, positioned 2.5 cm (equivalent to 2.35°) from the fixation (a white cross measuring 1 cm × 1 cm, or 0.94°, located at the center of the screen). The fixation was consistently presented throughout the experiment. The auditory target consisted of a 600-Hz monaural tone played on either side of the headset, with the sound volume fixed at 60 dB.

The experimental procedure is depicted in Figure 1. Before the target appeared, either a visual or an auditory cue was presented to indicate the target modality to which the participants should respond. The visual cue consisted of two white asterisks (1.5 cm × 1.5 cm, 1.41°) appearing on both sides of the monitor, located at the same position as the visual target. The auditory cue was a 400-Hz binaural tone. Each trial began with the modality cue displayed for 100 msec, followed by the appearance of the target after 300 or 600 msec, lasting until a response or up to 1500 msec, resulting in cue-to-target intervals (CTI) of 400 msec or 700 msec. Simultaneously presented visual and auditory targets (i.e., bivalent target) required participants to identify the target modality's location based on the preceding cue. Participants used their left (“Z”) and right (“/”) index fingers to indicate the cued target location. Correct responses triggered the subsequent trial's modality cue after a 500-msec response-cue interval. Responses exceeding 1500 msec prompted a 500-msec reminder text, excluding the trial from the analysis. Incorrect responses led to the display of two red crosses at the visual cue locations, accompanied by an 800-Hz binaural tone for 100 msec. Following the disappearance of warning signals, the modality cue for the next trial appeared after 400 msec. Participants were instructed to respond as fast and correctly as possible.

To ensure a sufficient number of trials for ERP data analysis, participants completed two sessions, one per day, each including a single task and a mixed task with a conducting order of the single task, the mixed task, and the single task. Each single task contained four blocks, with each containing 80 trials, where both the cue and target belonged to the same modality (either visual or auditory). Participants were assigned to one of two sequences of the single task: AVVA or VAAV (V for visual, A for auditory). The first and third single task involved different modality orders. For example, if the first single task was AVVA, the third single task was VAAV, and vice versa. Within the blocks, CTI of 400 msec or 700 msec were randomly presented. The single task on the first day featured an unimodal stimulus, where each stimulus (visual or auditory) was presented alone. On the second day, the single task involved a bimodal stimulus, with simultaneous visual and auditory stimuli presented either on the same side (congruent trial) or on different sides (incongruent trial).

The mixed task contained eight blocks, each containing 80 trials with a bimodal stimulus. A repetition trial indicated that the cued modality in the current trial was the same as in the previous trial, whereas a switch trial indicated a different cued modality. The sequence of repetition and switch trials was randomized, and the number of each type of trial was predetermined to ensure balance. The sequence of CTI (400 vs. 700 msec), cued modality (visual vs. auditory), modality congruency (congruent vs. incongruent), and target-modality transition (repetition vs. switch) was randomly presented within the blocks. The experiment comprised 640 trials each for unimodal and bimodal stimulation in the single task and 1280 trials for the mixed task. In the main text, we report only the bimodal stimulus condition, as we analyzed bimodal stimulation conditions for the single task and the mixed task. Participants underwent initial practice trials for task comprehension. The sounds used during the experiment, including cue, target, and error, were also played individually to the participants to ensure they knew these three types of sounds before the experiment.

Participants sat comfortably in a sound attenuation room for the experimental sessions. A Neuroscan Q-cap with 32 AgCl scalp electrodes was used to record continuous EEG activity. Two electrodes, one 2 cm below and one 2 cm above the left eye, were used to record VEOGs. HEOGs were obtained using electrodes positioned at the outer canthi of each eye. Initially, the left mastoid was the online reference for the electrodes, and the average of the left and right mastoids was the offline reference. They were maintained below five kiloohms to preserve appropriate electrode impedances. EEG and EOG signals were amplified using NuAmps amplifiers, sampled at 500 Hz, and online-filtered with a high-pass filter with a cutoff frequency of 0.1 Hz and a low-pass filter with a cutoff frequency of 50 Hz.

Blink-related ocular artifacts were corrected using the ocular reduction command in the ERPLAB Toolbox (Lopez-Calderon & Luck, 2014), MATLAB Version 2018. Further artifact removal involved an algorithm that rejected epochs if the signal fell below −100 μV, if the EEG drift from the baseline exceeded 100 μV, or if the analog-to-digital converts became saturated. The overall epoch rejection rate distributed roughly equally across conditions. For the cue-locked analysis, the average rejection rate was 22% ± 3% for the younger adults and 23% ± 3% for the older adults. For the target-locked analysis, the rejection rate was 19% ± 2% for younger adults and 19% ± 2% for older adults. Consequently, the mean number of trials contributing to each averaged ERP waveform (per trial type) ranged from 111 to 133 for the cue-locked analysis and 116 to 143 for the target-locked analysis. ERP epochs were established by extracting 1024 msec (from −200 to 824 msec) epochs around the onset of the cue or target, incorporating a 200 prestimulus (cue or target) interval. For the cue-locked analysis, baseline correction was performed by averaging signals between −200 and 0 msec of cue separately for each electrode site and subtracting this average from each sample point in the electrode site. For the target-locked analysis, baseline correction was performed by averaging signals between −50 and 50 msec.

To ensure consistency, the physical appearance and frequency of stimulus presentation were standardized across all ERP comparisons. In this study, we evaluated the cue-locked P3 (200–400 msec after cue onset), the target-locked P3 (200–400 msec after target onset), and the target-locked LRP (0–600 msec). The P3 components were measured at the Pz site. The averaged epoch waveform for each condition was subjected to a low-pass filter of 30 Hz to derive the P3 amplitude and latency. The P3 latency was measured using the peak latency method, defined as the duration between cue/target onset and the peak, which ranges from 200 to 400 msec after cue/target onset. Before determining the peak latency, we smoothed the P3 waveforms using a 30-Hz refiltering to mitigate biases from local maxima influenced by high frequencies. In addition, we manually retrieved each P3 peak latency to avoid any possible biases from the automatic detection tool provided by ERP Lab. The LRP takes advantage of the fact that a preparatory (or readiness) potential can be recorded before initiating the response. This is maximally recorded over the central site contralateral to the hand used for responding (e.g., C3 in the case of the right-hand response and C4 in the case of the left-hand response). Conventionally, there are two steps in measuring the LRP (see Appendix A for the steps). To measure the onset latency of the LRP, we adopted the method of identifying the latency of the 50% maximum amplitude, with the LRP waveform subjected to a low-pass filter of 4 Hz.

We excluded data from individuals (two younger adults and eight older adults) whose error rate (ER) was above 75% in one of the experimental manipulations (i.e., a long CTI switch incongruent trial for an auditory target). Consequently, the analysis involved 62 participants in this cross-modal attention-switching ERP study: 34 younger adults (20 male participants, mean age = 21.5 ± 2.0 years, age range: 18–26 years) and 28 older adults (11 male participant s, mean age = 69.4 ± 4.5 years, age range: 63–80 years).¹ Ultimately, the ER after combining visual and auditory trials ranged from 0% to 39.49% for older adults and from 0% to 32.69% for younger adults, which was a reasonable range for participant's performance.

The variability in the CTI was introduced to examine its impact on task performance. However, it was observed that the short CTI could potentially contaminate the cue-locked ERP components that overlapped with target-locked ERP components. Therefore, the main focus of the analysis in the main text was on the CTI = 700 msec. This allowed for a more appropriate comparison between behavioral and ERP analyses. Data filtering involved applying time-outs as nonresponses and categorizing incorrect spatial responses as errors. Exclusions comprised practice trials and the initial trial of each block. In addition, trials with RTs less than 100 msec or error responses were excluded. Subsequently, mean and standard deviation calculations were performed for each participant. Trials with RTs differing by more than 3 SDs based on each participant's mean RT (separately for the single task and the mixed task) were excluded from further analysis. The exclusion rate for the single task was 2.35% ± 0.21% for younger adults and 1.24% ± 0.19% for older adults. For the mixed task, the exclusion rate was 1.55% ± 0.17% for younger adults and 2.81% ± 0.58% for older adults.

Mixing costs and switch costs were analyzed in separate contrasts. For the mixing costs analysis, the between-subject independent variable was Age (older vs. younger adults), and the within-subject independent variables were Transition (repetition trials in the mixed task vs. trials in the single task) and Congruency (incongruent vs. congruent). For the switch costs analysis, the between-subject independent variable was Age, and the within-subject independent variables were Transition (switch of target modality vs. repetition of target modality) and Congruency.

Relevant to the research objectives outlined in the introduction, the analysis primarily focused on the following effects. We focused on analyzing the main effect of Transition (mixing or switching) to determine if cross-modal attention-switching behaves similarly to unimodal task-switching in terms of behavioral performance and ERP components. To understand how aging influences cross-modal attention-switching performance, we focused on the interaction between Age and Transition. In addition, we explored whether Congruency interacts with age-related mixing costs and switch costs by investigating the three-way interaction involving Age, Transition, and Congruency. If interactions exist, the following multiple comparisons were corrected using simple interaction and simple main effect tests, following higher-order interaction effects with adjusted error terms and degrees of freedom (Howell, 2010).

RT, ER, and amplitude/latency of the ERP components were the measured dependent variables, with the significance level set at α = .05. With testing 62 participants, a power = 0.90 leads to a detectable effect size of f = 0.21 (a medium effect) in a two-way interaction. Hence, we felt that enough participants were tested to demonstrate adequate statistical power.

Table 1 shows the RT, ER, and ERP components under various conditions. In the main text, we report the results of the statistical analyses that aligned with our hypothesis testing. Tables B1 and B2 summarize the statistical analyses of behavioral data and ERP components of all effects.² Grand mean cue-locked and target-locked P3 at P_Z midline electrodes and target-locked LRP for the mixing costs and congruency effect are shown in Figure 2. Figure 3 illustrates the switch costs and congruency effect. RT and ER data are also presented in both figures.

The main effect of Transition was significant, F(1, 60) = 122.96, p < .001, $ηp2$ = .672, 368 msec versus 414 msec for single task versus repetition trials. In addition, the interaction of Age and Transition was significant, F(1, 60) = 29.65, p < .001, $ηp2$ = .331, indicating larger mixing costs for older adults (84 msec) than for younger adults (29 msec). The three-way interaction effect of Age, Transition, and Congruency was not significant, F(1, 60) = 3.87, p = .054, $ηp2$ = .061, but showed a similar trend as described later in the ER results (see below).

The main effect of Transition, F(1, 60) = 45.36, p < .001, $ηp2$ = .430, 3.60% versus 5.40%, and the interaction of Age and Transition, F(1, 60) = 26.34, p < .001, $ηp2$ = .305 (older: 3.43% vs. younger: 0.46%), were both significant, consistent with prior findings indicating age-related increases in mixing costs. In addition, a significant three-way interaction for Age, Transition, and Congruency was observed, F(1, 60) = 15.62, p < .001, $ηp2$ = .207. Subsequent simple interaction tests provide evidence of age-related increase in mixing costs in incongruent trials, F(1, 120) = 29.85, p < .001, $ηp2$ = .199 (older: 6.51% vs. younger: 1.18%), but not in congruent trials, F(1, 120) = 0.52, p = .473, $ηp2$ = .043.

For the mean amplitude, only a significant interaction between Age and Transition was observed, F(1, 60) = 6.75, p = .012, $ηp2$ = .101 (older: −0.3 μV vs. younger: 0.85 μV). Subsequent simple main effect tests showed that older adults exhibited similar amplitudes for repetition trials in the mixed task and trials in the single task, F(1, 60) = 0.83, p = .367, $ηp2$ = .014 (0.69 μV vs. 0.40 μV), whereas younger adults demonstrated larger amplitudes for the repetition trials in the mixed task, F(1, 60) = 8.21, p = .006, $ηp2$ = .120 (0.43 μV vs. 1.27 μV). These findings suggest that older adults may not utilize the cues as efficiently as younger adults when they selectively attend to only one relevant sensory modality. None of the main effects and other interactions showed significant effects, Fs < 2.95, ps > .091, $ηp2$ < .047. For the peak latency, none of the main effects and interactions showed significant effects, Fs < 2.62, p > .111, $ηp2$ < .016.

For the mean amplitude and the peak latency, no significant main effect of Transition, interaction of Age and Transition, and significant three-way interaction of Age, Transition, and Congruency was found (all ps > .05).

The main effect of Transition was significant, F(1, 60) = 20.87, p < .001, $ηp2$ = .258, (159 msec vs. 180 msec), but neither the interaction of Age and Transition, F(1, 60) = 2.06, p = .157, $ηp2$ = .033, nor the three-way interaction of Age, Transition, and Congruency, F(1, 60) = 1.98, p = .16, $ηp2$ = .032, reached significance. These findings suggest that although mixing costs might entail prolonged response selection, this did not contribute to the magnified age-related behavioral mixing costs.

To summarize the results of the mixing costs analysis, our behavioral findings revealed significant mixing costs, along with age-related increases in mixing costs for both RT and ER. In addition, for ER, there was a further three-way interaction of Age, Transition, and Congruency. This suggests that age-related increases in mixing costs were more prominent for incongruent than congruent trials. Regarding RT results, although the statistical significance for the three-way interaction was not reached, there was a similar trend observed as in ER.

Regarding the ERP findings, the significant main effect of Transition only occurred in the target-locked LRP onset latency, suggesting that working memory load (i.e., maintaining two modalities in mind in the mixed task) in the cross-modal attention switching involves prolonged processing time in the response selection stage, consistent with findings from unimodal task-switching studies. On the other hand, concerning the interaction of Age and Transition, it was exclusively observed in the cue-locked P3 mean amplitudes, indicating the absence of amplitude differences for the repetition trials between the mixed task and the single task solely for older adults. These findings suggest that older adults may not utilize the cues as efficiently as younger adults for exclusively attending to only one relevant sensory modality.

The main effect of Transition, F(1, 60) = 24.95, p < .001, $ηp2$ = .294, and the interaction of Age and Transition, F(1, 60) = 4.78, p = .033, $ηp2$ = .074, were both significant, indicating that both older and younger adults displayed significant switch costs, with older adults exhibiting larger switch costs (24 msec) compared with younger adults (9 msec). No further significant three-way interaction of Age, Transition, and Congruency was observed, F(1, 60) = 2.21, p = .142, $ηp2$ = .036; however, it demonstrated a similar trend as described in the ER findings.

Similarly, there were significant main effects of Transition, F(1, 60) = 27.39, p < .001, $ηp2$ = .313, and the interaction of Age and Transition, F(1, 60) = 10.00, p = .002, $ηp2$ = .143, indicating that both older and younger adults exhibited ER switch costs, with older adults showing larger ER costs compared with younger adults (3.44% vs. 0.85%). Furthermore, a significant three-way interaction was observed for Age, Transition, and Congruency, F(1, 60) = 9.25, p = .003, $ηp2$ = .134. Subsequent simple interaction tests revealed that age-related increases in switch costs were evident on incongruent trials, F(1, 120) = 13.05, p < .001, $ηp2$ = .098, with older adults exhibiting switch costs of 6.64% compared with younger adults' switch costs of 1.72%. However, this age-related increase in switch costs was not apparent on congruent trials, F(1, 120) = 0.05, p = .828, $ηp2$ < .001.

For the mean amplitude, a significant main effect of Transition, F(1, 60) = 34.16, p < .001, $ηp2$ = .363, and an interaction of Age and Transition, F(1, 60) = 28.85, p < .001, $ηp2$ = .325, were observed and consistent with the behavioral findings. Subsequent simple main effect tests revealed that older adults exhibited no significant difference in amplitudes for repetition and switch trials, F(1, 60) = 0.10, p = .751, $ηp2$ = .002 (0.40 μV vs. 0.47 μV), whereas younger adults showed significantly larger amplitudes for switch trials compared with repetition trials, F(1, 60) = 69.66, p < .001, $ηp2$ = .537 (1.27 μV vs. 2.90 μV). These findings suggest that older adults may not utilize the cues as efficiently as younger adults for updating a representation of the relevant target modality for a switch.

As for the peak latency, a significant main effect of Transition, F(1, 60) = 15.22, p < .001, $ηp2$ = .047, and an interaction of Age and Transition, F(1, 60) = 10.10, p < .001, $ηp2$ = .031, were observed. The cue-locked P3 latency was longer on switch trials than on repetition trials (283 msec vs. 265 msec), but this effect was evident only in younger adults and diminished in older adults (32 msec vs. 3 msec). The results suggest that older adults may exhibit less efficient utilization of the cue to update target representation for a switch.

For the mean amplitude, a significant main effect of Transition was observed, F(1, 60) = 32.71, p < .001, $ηp2$ = .353, indicating the typical switch-related decreased positivity for switch trials, 5.92 μV versus 5.02 μV. However, there was no interaction of Age and Transition, F(1, 60) = 1.70, p = .197, $ηp2$ = .028, and no further three-way interaction of Age, Transition, and congruency, F(1, 60) = 0.110, p = .741, $ηp2$ = .002.

Regarding the peak latency, a significant main effect of Transition was observed, F(1, 60) = 11.43, p = .001, $ηp2$ = .023, demonstrating a longer latency for switch than repetition trials (305 msec vs. 286 msec). In addition, we observed a significant interaction of Age and Transition, F(1, 60) = 5.84, p = .019, $ηp2$ = .012, indicating larger switch costs on the P3 latency for older adults (19 msec) than for younger adults (13 msec). Yet, there was no further three-way interaction of Age, Transition, and Congruency, F(1, 60) = 0.06, p = .813, $ηp2$ = .000. The results suggest that the behavioral finding of increased age-related switch costs may partially be because of the re-identification of the target modality following a switch.

No significant main effect of Transition and interaction of Age and Transition were found, Fs < 0.98. However, a significant three-way interaction was observed for Age, Transition, and Congruency, F(1, 60) = 7.16, p = .010, $ηp2$ = .107. Subsequent simple interaction tests of LRP onset latencies showed age-related differences in switch costs for incongruent trials, F(1, 120) = 5.98, p = .016, $ηp2$ = .048, yet with numerical (but non-significant) switch benefits for older adults (−24 msec) compared with younger adults (8 msec), whereas there were no age-related differences in switch costs for congruent trials, F(1, 120) = 1.28, p = .260, $ηp2$ = .011.

To summarize the switch costs results, significant RT and ER switch costs were observed, consistent with findings from unimodal task-switching studies. This suggests similar additional processing time and more errors incurred during modality switching. Importantly, older adults exhibited increased RT and ER switch costs compared with younger adults, indicating that older adults find switching between sensory modalities more challenging than younger adults, as predicted.

The ERP findings revealed that the transition effect was observed in cue-locked P3 amplitudes, cue-locked P3 latency, target-locked P3 amplitude, and target-locked P3 latency, but not in the target-locked LRP onset latency. Regarding the interaction of Age and Transition, it occurred in cue-locked P3 amplitude, cue-locked P3 latency, and target-locked P3 latency. These findings suggest that older adults may not efficiently utilize cues for updating a representation of the relevant target modality during a switch and that the increased age-related switch costs observed behaviorally may partly result from the re-identification of the target modality following a switch.

The current study had two primary aims. Unlike previous studies that focused solely on unimodal attention switching, we examined cross-modal attention switching, providing greater ecological validity by incorporating the complexities of multisensory environments. First, we aimed to determine if cross-modal attention switching behaves similarly to unimodal attention switching in terms of behavioral performance and ERP components related to mixing and switch costs. We focused on analyzing the main effect of Transition (mixing or switch) to achieve this. Second, we sought to understand how aging influences cross-modal attention-switching performance, examining both behavioral and ERP measures related to mixing and switch costs. By analyzing distinct ERP components, we aimed to identify the impact of aging on neural correlates associated with attentional flexibility across sensory modalities. We focused on examining the interaction of Age and Transition to address this objective. In addition, we explored whether congruency interacts with age-related mixing and switch costs, investigating the possibility of a further three-way interaction involving age, transition, and congruency.

Our study demonstrated an age-related increase in mixing costs (both in RT and ER) for cross-modal attention switching. The demonstration of age-related differences in mixing costs aligns with findings from traditional unimodal task-switching paradigms (Wasylyshyn et al., 2011, for a meta-analysis) and confirms the results of a recent behavioral study on cross-modal attention switching (Schils et al., 2024). This suggests that despite maintaining consistent stimulus–response mappings for different sensory modalities (as in the current study), selectively attending to one out of two sensory modalities incurs similar additional processing demands (Kreutzfeldt, Stephan, Willmes, & Koch, 2016), akin to paradigms where the sensory modality remains constant but stimulus–response mappings differ between task sets (as in the unimodal task-switching studies).

The ERP findings on mixing costs further indicated that the main effect of transition was significant only in the target-locked LRP onset latency. This suggests that the working memory load in the mixed task prolonged the processing time in response selection, even during cross-modal attention-switching paradigms, as often reported in task-switching paradigms. This result was unexpected, as we originally hypothesized that the switch in sensory modalities would tap more processing demands on the stimulus identification stage (i.e., target-locked P3 effect; Karayanidis et al., 2003, 2011; Kieffaber & Hetrick, 2005) than on the response-selection stage as reflected in the target-locked LRP onset latency (e.g., Whitson et al., 2014; Hsieh & Liu, 2005; Hsieh & Yu, 2003a, 2003b). However, the design of our stimuli and responses was constrained, with a left tone or visual stimulus resulting in a left-key response and vice versa. Therefore, attributing the mixing costs to the response-selection stage is not surprising, as it was difficult to differentiate between stimulus-level and response-level congruency (see also the study's limitations).

Moreover, the interaction of Age and Transition was particularly evident in cue-locked P3 mean amplitudes, indicating reduced efficiency among older adults in utilizing cues to attend to relevant sensory modalities in the mixed task. The cue-locked P3 is believed to reflect the updating of relevant task sets and preparation for an upcoming task (Kieffaber & Hetrick, 2005; Karayanidis et al., 2003). This finding aligns with our prediction that cross-modal attention-switching may require more effort in stimulus identification processes.

Regarding behavioral switch costs, our study demonstrated significant increases in RT and ER when switching between sensory modalities, consistent with findings from unimodal task-switching paradigms (Kiesel et al., 2010). Notably, older adults exhibited longer RT and larger ER switch costs. These behavioral findings were further supported by several ERP components, including cue-locked P3 amplitudes and latency, as well as target-locked P3 amplitude and latency, but not in target-locked LRP onset latency. Previous ERP studies on unimodal task switching have emphasized the decision-response processing demands associated with switch trials. However, our study, which involved switching tasks across different modalities (visual vs. auditory), prioritized identifying stimulus modality over decision-making. Consequently, the switch processes between different modalities may involve more stimulus-set biasing processes than adjustments in response-set, aligning with Meiran's (1996, 2000) model.

Furthermore, the increased behavioral switch costs observed in older adults were reflected in ERP components, notably through decreased cue-locked P3 amplitudes and prolonged target-locked P3 latency compared with younger adults. These ERP findings suggest that older adults may struggle to efficiently utilize cues for updating a representation of the relevant target modality during a modality switch. This inefficacy leads to a longer time to reidentify the target modality, as indicated by the increased target-locked P3 latency. Previous research (Yang et al., 2022) has shown that older adults encounter difficulties in integrating information across audiovisual modalities, highlighting potential age-related deficits in the stimulus identification stage of processing. Therefore, we anticipated observing larger age-related switch costs on target-locked P3 latency rather than on target-locked LRP onset latency, as previously reported for switching between task sets within the same visual modality (e.g., Whitson et al., 2014; Hsieh & Liu, 2005).

About our final prediction regarding whether distractor suppression processes, as measured by the Congruency effect, may interact with age-related increases in mixing and switch costs, the current results appeared to be less clear-cut. Previous cross-modal attention-switching studies using bivalent stimuli have noted modality-related switch costs (Lukas et al., 2010a, 2010b). However, there is only limited research on the influence of age on cross-modal (auditory and visual) distractor processing (Guerreiro, Adam, & Van Gerven, 2012; Guerreiro et al., 2010). Thus, in our study, we included this variable to investigate how congruency modulates attentional flexibility ability across different sensory modalities in different age groups. Although the ERs showed a significant age-related increase of mixing and switch costs in incongruent trials, this specific interaction pattern was not significant in the RTs (but showing the same data pattern). Nevertheless, this finding supports the notion that older adults find it more challenging than younger adults to exclusively attend to the relevant target modality, whether selectively attending to one relevant modality (as in repetition trials in the mixed task) or switching to attend to one relevant modality (as in switch scenarios), particularly when the two modalities imply conflict responses (incongruency).

The ERP results were inconclusive, as only target-locked LRP onset latency for switch costs showed a significant three-way interaction of age, transition, and congruency. However, subsequent simple interaction tests revealed an unclear pattern for older adults, showing numerically (but not significantly) reversed switch costs in incongruent trials (see also the limitations of the study).

Limitations of the study include two unresolved issues that require further research. First, although the current study obtained robust effects of transition and age-related increases in mixing and switch costs, there are complexities related to the interaction effects that need clarification. For example, the three-way interaction related to congruency for ERPs appeared less clear than the behavioral findings, suggesting the need for more precise investigation. The inability to tease apart stimulus-level versus response-level congruency because of the concurrent presentation of bimodal spatial stimuli likely led to conflicts occurring at different processing stages, which may have obscured the ERP findings. Therefore, future research is needed to disentangle the congruency effect.

Second, the study did not manipulate CTI in a way that would have allowed us to better separate shorter versus longer task preparation in the ERP data. This limitation prevents a direct test of switch models proposed in previous studies. For instance, models suggesting additional control reconfiguration for a task switch, such as “goal-shifting” and “rule-activation” (Rubinstein, Meyer, & Evans, 2001), as well as models proposing additional stimulus-set biasing processes and prolonged response selection processing for a task switch by Meiran (1996, 2000), were not directly examined. In addition, models suggesting no additional reconfiguration processes for a task switch, but rather a carry-over effect from the previous task set (Sohn & Anderson, 2001; Sohn & Calrson, 2000; Allport, Styles, & Hsieh, 1994), were not tested either. Despite not being the focus of the current study, the finding that increased age-related switch costs mainly manifested in the stage of stimulus identification aligns more closely with Meiran's model. However, further research is needed to directly test these models and explore their implications in cross-modal attention switching.

In conclusion, this study sheds light on the behavioral and neural correlates of cross-modal attention switching and how aging influences these processes. To our knowledge, this is the first study to examine age-related differences in the neural correlates of cross-modal attention switching. Our findings align with unimodal attention-switch studies and show increased mixing costs and switch costs for older adults compared with younger adults, indicating age-related impairments in cross-modal attentional control. This is further supported by ERP data, revealing reduced efficiency in cue utilization and prolonged processing times for older adults during modality switches. This study offers valuable insights into the cognitive processes underlying attentional flexibility across sensory modalities and implications for understanding age-related changes in attentional control. Our approach and findings enhance greater ecological validity by closely reflecting the complex and dynamic nature of real-world attentional demands, in which individuals must frequently switch their attention between different sensory modalities.

Conventionally, there are two steps in measuring the LRP. Single-trial epochs at C3 were subtracted from those at C4. The LRP is recorded as a negative preparatory potential over the hemisphere opposite to the hand used for responding. If the response were subsequently made with the right hand, the negative-going LRP would be larger at C3 than at C4. The subtraction (C3 − C4) would then result in a negative preparatory potential. If the response were subsequently made with the left hand, the negative-going LRP would be larger at C4 than at C3. However, the subtraction procedure (C3 − C4) would have a positive preparatory potential. To overcome this, the LRP was computed by subtracting left-hand responses from the right-hand: LRP = (right hand (C3 − C4) - left hand (C3 − C4))/2. Accordingly, responses to be made from the right and left hands were collapsed and averaged with the above two-step procedure, yielding a single LRP independent of the hand responding. Trials were then sorted into those in which a correct or error response was subsequently made and averaged separately. Importantly, this procedure should yield a negative-going LRP when correct responses are subsequently made (thus, e.g., the negative LRP would be larger in the hemisphere contralateral to the correct hand used for responding). However, a positive-going LRP would be apparent when the participant subsequently used the wrong hand for responding.

In the main text, we focused on following effects to test our hypothesis: the main effect of Transition (mixing or switching), the interaction between Age, Transition, and Congruency. In Tables B1 and B2, we also provide an overview of the complete ANOVA results. Moreover, ANOVA analyses with sex as a covariate are provided in Tables B3 and B4. The results remained consistent, confirming that sex did not influence the outcomes.

Corresponding author: Shulan Hsieh, Department of Psychology, National Cheng Kung University, No. 1 University Road, Tainan, Taiwan, or via e-mail: [email protected].

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Pi-Chun Huang: Conceptualization; Formal analysis; Software; Visualization; Writing—Original draft. Ludivine A. P. Schils: Conceptualization; Writing—Review & editing. Iring Koch: Conceptualization; Writing—Review & editing. Denise N. Stephan: Conceptualization; Funding acquisition; Writing—Review & editing. Shulan Hsieh: Conceptualization; Formal analysis; Funding acquisition; Software; Writing—Original draft; Writing—Review & editing.

This research is funded by the Deutsche Forschungsgemeinschaft (https://dx.doi.org/10.13039/501100001659) under STE 2466/1-1 granted to Denise N. Stephan and by the National Science and Technology Council (Taiwan, https://dx.doi.org/10.13039/501100020950) under National Science and Technology Council 109-2923-H-006-002-MY3 granted to Shulan Hsieh.

Retrospective analysis of the citations in every article published in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/W = .115, and W/W = .159, the comparable proportions for the articles that these authorship teams cited were M/M = .549, W/M = .257, M/W = .109, and W/W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.