The Differential Contributions of pFC and Temporo-parietal Cortex to Multimodal Semantic Control: Exploring Refractory Effects in Semantic Aphasia

Semantic cognition involves the retrieval of information about the meanings of words, pictures, sounds, and objects and the application of this knowledge to a specific task or context. Evidence from patients suggests that semantic cognition can be impaired in at least three ways. First, patients may have degeneration of information within the semantic store itself, as in semantic dementia (SD; Hodges, Patterson, Oxbury, & Funnell, 1992; Warrington, 1975). Second, patients may be unable to recognize an object in a specific modality (as in visual agnosia) because of damaged connectivity between the sensory input and the semantic store (Catani & Ffytche, 2005). Finally, patients may be unable to control activation within the semantic system, such that it becomes harder for task-relevant aspects to be brought to the fore, as in semantic aphasia (SA; Jefferies & Lambon Ralph, 2006).

According to the “hub-and-spoke” theory (Patterson, Nestor, & Rogers, 2007), semantic information is represented in both modality-dependent cortices (“spokes”) and an amodal “hub” (the anterior temporal lobes [ATL]). This acts as a convergence zone, binding together attributes within domain-specific regions to form an amodal conceptual store with a semantic similarity structure (Pobric, Jefferies, & Lambon Ralph, 2010b; Lambon Ralph & Patterson, 2008; Patterson et al., 2007). In line with this theory, patients with SD have atrophy focused on the ATLs and a selective disorder of semantic memory that leaves other aspects of cognition largely intact (Mion et al., 2010; Rohrer et al., 2008; Patterson et al., 2007; Mummery et al., 2000). These patients show impaired comprehension across the full range of input and output modalities (Piwnica-Worms, Omar, Hailstone, & Warren, 2010; Coccia, Bartolini, Luzzi, Provinciali, & Lambon Ralph, 2004; Bozeat, Lambon Ralph, Patterson, Garrard, & Hodges, 2000) plus a high degree of consistency when the same concepts are probed using words, pictures, and environmental sounds (Garrard & Carroll, 2006; Bozeat et al., 2000). In addition, functional neuroimaging studies show coactivation of ATL in healthy participants when meanings are accessed from both words and pictures (Binney, Embleton, Jefferies, Parker, & Lambon Ralph, 2010; Lambon Ralph, Sage, Jones, & Mayberry, 2010; Visser, Jefferies, & Lambon Ralph, 2010; Vandenberghe, Price, Wise, Josephs, & Frackowiak, 1996), and TMS research demonstrates that a “virtual lesion” in this region disrupts both verbal and nonverbal semantic judgments (e.g., Pobric, Jefferies, & Lambon Ralph, 2007, 2010a; Lambon Ralph, Pobric, & Jefferies, 2009).

SA patients also show multimodal semantic deficits that equally affect word and picture tasks (Corbett, Jefferies, Ehsan, & Lambon Ralph, 2009; Corbett, Jefferies, & Lambon Ralph, 2009; Jefferies & Lambon Ralph, 2006). However, unlike SD, the ATL is spared in SA—instead, these patients show infarction of the left pFC or TP cortex areas (Noonan, Jefferies, Corbett, & Lambon Ralph, 2010; Jefferies & Lambon Ralph, 2006). A number of studies have shown that SA patients are unable to control activation within the semantic system so that it is appropriate to the context—irrespective of whether the task is verbal or nonverbal or whether the task involves a particular semantic category. However, SA patients do not appear to have damage to core semantic representations, unlike those with SD. (1) SA patients do not consistently perform when the same concepts are probed using semantic tasks with differing control requirements, unlike those with SD (e.g., word–picture matching [WPM] and judgments of semantic association; Jefferies & Lambon Ralph, 2006). (2) They are highly sensitive to manipulations of the executive demands of semantic tasks—both when making judgments about word meaning (Noonan et al., 2010) and in the highly nonverbal domain of object use (Corbett, Jefferies, & Lambon Ralph, 2009, 2011; Corbett, Jefferies, Ehsan, et al., 2009). (3) They show strong effects of cues and task constraints that reduce the requirement for internally generated semantic control (Corbett et al., 2011; Noonan et al., 2010; Soni et al., 2009). When naming pictures, SA performance is substantially improved by phonological cues, although SD patients only show a modest benefit (Jefferies, Patterson, & Lambon Ralph, 2008). SA patients are also sensitive to semantic miscues (e.g., /d/ when attempting to name “cat”; Noonan et al., 2010; Soni et al., 2009). Moreover, they show parallel effects of task constraints in object use (Corbett, Jefferies, & Lambon Ralph, 2009, 2011). (4) Unlike SD patients, individuals with SA also have general executive deficits beyond the semantic domain, which correlate with their performance on semantic tasks (Jefferies & Lambon Ralph, 2006). Intriguingly, SA patients with lesions in the left pFC or TP cortex have been found to show highly similar semantic deficits—including equivalent sensitivity to manipulations of executive semantic demands (Noonan et al., 2010; Jefferies & Lambon Ralph, 2006; Berthier, 2001). SA patients with left pFC or TP cortex lesion also show similar impairment on nonsemantic executive tasks such as the WCST (Baldo et al., 2005) and Raven's Colored Progressive Matrices (Baldo, Bunge, Wilson, & Dronkers, 2010; Jefferies & Lambon Ralph, 2006).

Research by several other groups has suggested that the semantic storage deficit in SD can be contrasted with a “semantic access” disorder observed in some stroke–tumor cases (Warrington & Cipolotti, 1996; Forde & Humphreys, 1995; Warrington & McCarthy, 1983; Warrington & Shallice, 1979). Unlike SD cases, “access” patients show inconsistent performance when semantic tests are repeated, and they exhibit refractory effects—their accuracy in WPM declines when a small set of semantically related items is presented repeatedly and rapidly over a number of cycles. This refractory pattern is typically accompanied by strong cueing effects and insensitivity to item frequency—symptoms that again differentiate access patients from SD. Although this work has not focused on the role of executive control, refractory tasks are likely to produce a build-up of competition across cycles, because the items in the set are both targets and distractors on different trials (see also Schnur, Schwartz, Brecher, & Hodgson, 2006). Jefferies, Baker, Doran, and Lambon Ralph (2007) examined the possibility that “semantic access” disorder overlaps with the semantic control deficit in patients with SA. SA patients were found to display all the classic symptoms of access disorder, including effects of item repetition and speed of presentation, and this refractory pattern was linked to poor executive control over semantic activation. However, SA patients with left TP cortex lesions were less sensitive to refractory variables than those with pFC damage in this study. This is in clear contrast to other manipulations of semantic control, which affected both lesion subgroups equally (e.g., Noonan et al., 2010).

Several questions remain from this research. First, refractory effects have largely been explored in the verbal domain, both in the study conducted by Jefferies and colleagues (2007) and in other research (Schnur et al., 2006; Warrington & Cipolotti, 1996; Warrington & McCarthy, 1983). However, the semantic control impairment in SA affects all modalities equally (Corbett, Jefferies, Ehsan, et al., 2009; Corbett, Jefferies, & Lambon Ralph, 2009; Jefferies & Lambon Ralph, 2006). Therefore, we would expect multimodal refractory effects in these patients. In contrast, as noted below, several theories of “access” semantic disorder predict this impairment will be restricted to verbal–auditory tasks. Second, it is important to confirm whether patients with left pFC and TP cortex lesions differ in terms of the influence of refractory variables—and to consider how such a difference could be reconciled with the semantic control deficits, which appear to characterize both subgroups of SA patients.

There are at least two distinct theories of “access” semantic disorders that predict different refractory effects according to modality. The first, proposed by Warrington and Crutch (2004), is one of “multiple semantics.” This idea is again motivated by the comparison of patients with SD versus “refractory access” impairment who typically have stroke aphasia. In contrast to patients with SD, assessment of the refractory patients has most commonly focused on comprehension within the verbal modality (Warrington & Cipolotti, 1996; McNeil, Cipolotti, & Warrington, 1994; Warrington & McCarthy, 1983, 1987). Moreover, the existence of individual cases that show refractory effects on verbal but not visual tasks has been taken as evidence for a cognitive and neural dissociation between verbal and visual semantic systems (Crutch & Warrington, 2008; Warrington & Crutch, 2004). However, testing for visual refractory effects is relatively rare, and where it has been done, there is some debate as to whether tasks in different modalities are equally difficult and whether they control for the intrinsic differences in the nature of mapping from words or pictures to a concept (see Lambon Ralph & Patterson, 2003; Forde & Humphreys, 1997; Shallice, 1987). Therefore, it is still very much open to question whether SA patients have a purely verbal or a multimodal refractory deficit.

Another modality-specific theory suggests that refractory deficits result from impairment of verbal selection, with increases in lexical competition across cycles (Schnur et al., 2006; Belke, Meyer, & Damian, 2005; Damian, Vigliocco, & Levelt, 2001). According to this theory, activation of word nodes spreads to semantic associates, generating competition at the stage of lexical production in picture naming. When sets of semantically related items are presented repeatedly for naming, competition becomes stronger. Therefore, the framework predicts refractory effects in verbal but not nonverbal tasks, and much stronger refractory effects in picture-naming compared with WPM tasks. Jefferies et al. (2007) directly compared naming and matching tasks and found that SA patients showed refractory impairments in both tasks.

In contrast with these two proposals, several theories predict multimodal refractory effects in SA. As discussed above, we have suggested that SA patients have semantic control deficits that produce multimodal impairment (Corbett, Jefferies, Ehsan, et al., 2009; Jefferies & Lambon Ralph, 2006). The control network is required to activate the specific subset of information within the semantic store to generate time- and task-appropriate behavior. This is particularly demanding when there is strong competition or in more open-ended situations and has been associated with regions in both the left pFC and TP cortex (Whitney, Jefferies, & Kircher, 2011; Whitney, Kirk, O'Sullivan, Lambon Ralph, & Jefferies, 2011; Whitney, Grossman, & Kircher, 2009; Badre, Poldrack, Paré-Blagoev, Insler, & Wagner, 2005; Wagner, Paré-Blagoev, Clark, & Poldrack, 2001; Thompson-Schill, D'Esposito, Aguirre, & Farah, 1997). This kind of controlled processing is necessary in both verbal and nonverbal activities (Corbett, Jefferies, Ehsan, et al., 2009; Jefferies & Lambon Ralph, 2006), and so this theory would predict that the SA patients should exhibit refractory effects in all domains, given the correct assessment materials (see below).

Finally, using an implemented model of semantic processing, Gotts and Plaut (2002) demonstrated that refractory effects can result from neuromodulatory deficits, which generate increased synaptic depression, thus reducing the efficiency with which new stimuli can override current processing during the refractory period. Although this theory does not explicitly consider the issue of modality, if extended to an amodal semantic system, it would predict refractory effects for both verbal and nonverbal tasks, with the effect for both arising from the same general neuromodulatory deficit.

Brain regions damaged in SA patients include the left pFC and/or the left TP cortex (Noonan, Jefferies, Visser, & Lambon Ralph, submitted; Jefferies & Lambon Ralph, 2006). Neuropsychological, rTMS, and neuroimaging evidence suggests these two regions work together to underpin semantic control. Lesions of the left pFC and TP cortex produce highly similar patterns of semantic impairment (Noonan et al., 2010; Jefferies & Lambon Ralph, 2006; Berthier, 2001) plus common deficits in attention (Peers et al., 2005). For example, Noonan and colleagues (2010) found no significant differences between the left pFC and TP cortex cases on a range of tasks that manipulated semantic control by varying (i) semantic distance between probes and targets in category matching, (ii) associative strength between probes and distractors in synonym judgment, (iii) the presence of semantic cues and miscues on picture naming, and (iv) semantic ambiguity of the target word. Equally, a recent TMS study (Whitney, Kirk, et al., 2011) found that selective TMS to the pMTG specifically increased response times on a task requiring greater control over semantic retrieval that required participants to retrieve weak associations between probe and target words (e.g., “salt” with either “radio,” “grain,” or “adult”). This effect was indistinguishable from TMS over inferior frontal gyrus (IFG) and did not have any effect on a task involving more automatic semantic retrieval of strong associations (e.g., “salt” with “pepper,” “machine,” or “land”). Moreover, functional neuroimaging studies of healthy participants reveal that both regions show activation modulated by the executive demands of semantic tasks (Noonan et al., submitted; Whitney, Jefferies, et al., 2011). The same findings have been obtained across a wide range of semantic control manipulations, tapping selection between competing responses (Thompson-Schill et al., 1997), controlled semantic retrieval (Badre et al., 2005; Wagner et al., 2001), and semantic judgments to ambiguous words (Whitney et al., 2009; Rodd, Davis, & Johnsrude, 2005).

Despite these similarities, previous studies have suggested that, in refractory tasks, patients with left pFC lesions show stronger effects of stimulus set repetition than those with TP cortex damage, implying a subtle specialization within this control network. In a cyclical picture naming, Schnur et al. (2006) found increased error rates in Broca's aphasics, but not non-Broca patients. Refractory effects were associated with the degree of damage to left inferior frontal cortex—but not with damage to either posterior temporal or inferior parietal regions (Schnur et al., 2009). Researchers argue that spreading activation causes lexical competition within Broca's area (Schnur et al., 2006). By this view, refractory effects should only occur in verbal production tasks and in patients with left inferior frontal lesions. In line with this proposal, Campanella, Mondani, Skrap, and Shallice (2009) studied 20 patients with tumor with posterior damage and found that effects of WPM set repetition and speed of presentation were very weak. Finally, Jefferies et al. (2007) found only weak refractory effects (i.e., in response times and not accuracy) in naming and WPM in SA patients with TP cortex damage, compared with patients whose lesions included left pFC. Differential performance of these subgroups of SA patients is at odds with the hypothesis that both left pFC and TP cortex regions contribute to domain-general semantic control (Noonan et al., 2010; Jefferies & Lambon Ralph, 2006) and that refractory deficits can be understood in terms of semantic selection/competition demands that increase over time (e.g., Jefferies et al., 2007).

In summary, previous work on this topic highlights two controversial issues, which are the focus of the current study: (1) comparison of refractory effects across different modalities and (2) lesion location. The current study subdivides SA patients according to the location of their brain injury and directly compares them using refractory tasks that probe the same items in different modalities across three experiments. In Experiment 1, we contrast WPM and picture–picture matching (PPM; requiring participants to match visually dissimilar exemplars of the same object, for example, vintage-style dial telephone with modern cordless button telephone). In Experiment 2, we compare spoken WPM with environmental sound–picture matching (SPM; requiring the sound of “barking” to be matched with a picture of a dog). In Experiment 3, we compare WPM tasks that tap associative relationships (e.g., the word “train” or a picture of this item, matched to train tracks).

Nine aphasic stroke patients (seven men and two women) were recruited from stroke clubs and speech and language therapy services in Manchester and York, United Kingdom. Following previous studies on SA, patients who showed semantic comprehension deficits affecting both words and pictures were selected. They were not chosen to show refractory effects. All patients had chronic impairment after a cerebrovascular accident (CVA) at least 1 year before testing. Three patients had transcortical sensory aphasia, with fluent speech but poor comprehension. The remaining six patients had less fluent speech and/or poor repetition. Patients were aged between 36 and 83 years, with a mean age of 66 years, as shown in Table 1.

CT/MRI scans were available for eight patients (see Figure 1). Five cases (NY, BB, DB, KA, and LS) had damage to both left pFC and TP cortex areas (pFC+) and three (HN, SC, and ME) displayed infarcts confined to the left TP cortex (TP-only). A scan was not available for PG because of contraindications for MRI; however, a radiological report indicated a left frontal lesion, so in subsequent analyses, he was included in the pFC+ group. Further details of the patients' lesions are shown in Table 2. The TP-only patients all showed some damage extending anteriorly along the temporal lobe, but crucially none of these patients had damage to regions in the ATL, which is implicated in amodal semantic storage (Binney et al., 2010; Mion et al., 2010). Additionally, the damage was more ventral—around the angular gyrus—to regions that have shown category-specific patterns, namely the area around the intraparietal sulcus (Pobric et al., 2010b).

Table 2 shows a breakdown of the brain damage in each patient, focusing on ROIs in the temporal, parietal, and frontal lobes implicated in previous studies (see Noonan et al., 2010). Lesion size did not significantly correlate with background semantic scores (r = −.65, p > .05) or refractory effects in the current task (r = −.14 to −.69, p > .05).

The patients were examined on a range of general neuropsychological tests to assess cognitive ability. These were forward and backward digit span (Wechsler, 1987), Visual Object and Space Perception battery (Warrington & James, 1991), elevator counting with and without distraction from the Test of Everyday Attention (Robertson, Ward, Ridgeway, & Nimmo-Smith, 1994), Brixton Spatial Rule Attainment task (Burgess & Shallice, 1997), and the Ravens Colored Progressive Matrices test of nonverbal reasoning (Raven, 1962). Factor analysis was used to compute a composite executive/attentional score from tasks for which data were available for each patient (digit span, Test of Everyday Attention, Ravens Colored Progressive Matrices, and Brixton Spatial Rule Attainment).

Semantic assessments included three components of the 64-item semantic test battery (Bozeat et al., 2000): WPM with 10 semantically related distractors and picture and word versions of the Camel and Cactus Test. This test of semantic association involves deciding which of four semantically related items has an association to a probe (e.g., does “camel” go with “cactus,” “tree,” “sunflower,” or “rose”?). Additionally, there was a 96-item synonym judgment task that involves matching a probe to a target word with the same meaning, presented with two unrelated distractors (Jefferies, Patterson, Jones, & Lambon Ralph, 2009). Factor analysis of these four semantic tests was used to compute a composite semantic score, with larger values representing better performance. Table 3 provides this background assessment plus the semantic and executive composite scores.

Twelve age-matched control participants (six men and six women) were selected from a participant database at the University of York. Participants had no prior history of brain injury and showed unimpaired cognitive functioning on the Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975). Participants were aged between 35 and 90, with a mean age of 69 years. Independent t tests showed that the age of the controls did not differ from the patients, t(19) < 1.

This study consisted of three experiments, each involving a within-subject manipulation of modality: (1) identity matching of spoken words and pictures to pictures, (2) identity matching of spoken words to pictures and environmental sounds to pictures, and (3) matching a probe item to its associated location, using spoken WPM and PPM.

The experiments were run using E-Prime 1.1 (Psychology Software Tools, Sharpsburg, PA). An array of four semantically related pictures was displayed. Following all past studies of refractory effects, items were presented repeatedly, such that the target on one trial became the distractor on another, until all items within a semantic category had been the target. This completed one cycle. There were four cycles for each set of items, which probed the items in the semantic array in a pseudorandom order.¹ After each set of four cycles, participants had a short break.

The probe item was presented together with the four-item array, either through speakers or as a picture at the top of the screen. SA patients indicated their response by pointing to one of the pictures and the experimenter pressed a key, which advanced the task onto the next trial (this method was used as aphasic participants had difficulty using a stylus with a touch-sensitive screen in a pilot study). The experimenter recorded accuracy (our primary-dependent variable), whereas RT was recorded by the computer. As soon as a response was given, the next trial was presented. Each participant had 10 sec to respond, and if they did not respond within this time, the next trial was presented and an error was recorded. There were four practice items before the start of each block. Each experiment was carried out in four blocks using an ABBA design to control for order effects across the verbal and non-verbal tasks.

This experiment combined Experiments 5 and 6 from Warrington and Crutch (2004). There were two presentation conditions: visual (PPM) and verbal (WPM). The stimuli consisted 40 inanimate objects. These were grouped into 10 semantic sets (tools, electrical items, drink containers, clothes, household appliances, kitchen tools × 2, furniture × 2, and vehicles). In WPM, a spoken voice recording of the object name was used as the probe (Figure 2A). In PPM, two dissimilar pictures of the same item were selected to be the probe and target in an attempt to prevent simple visual matching (as shown in Figure 2B).

This experiment had two presentation conditions: verbal (WPM) and nonverbal auditory (SPM). The stimuli in this experiment consisted 32 inanimate and animate objects. These were grouped into eight sets (farm animals, other animals, birds, tools, vehicles, household objects, humans, and musical instruments). In the SPM task, a recording of an environmental sound produced by the object was presented as the probe, whereas in WPM a spoken voice recording of the object name was used (Figure 3B).

This experiment had two presentation conditions: visual (PPM) and verbal (WPM). The stimuli in this experiment consisted 40 inanimate and animate objects and 40 associated locations. These were grouped into 10 sets (farm animals, pets, exotic animals, clothes, plants, large household objects, small household objects, people, vehicles, and food). On every trial, participants selected the typical location of the probe object from an array of four locations within the set. The probe item was presented as a word (Figure 4A) or picture (Figure 4B).

Across all experiments, control participants' accuracy was close to ceiling levels (the control mean ranged from 92% to 100%, and there were no refractory effects). Repeated-measures ANOVAs of control RT indicated facilitation from repetition; in contrast, there were no significant effects in RT for the patients across cycles (see Table 4 for RT data and analysis). The following analysis, therefore, focuses on response accuracy.

The category (living or manmade) was mixed in Experiments 2 and 3. A paired t test for each experiment was used to confirm that patients showed no difference in accuracy according to category, t(8) < 1. Additionally, an ANOVA assessing the interaction between the effect of Category and Subgroup (TP-only compared with pFC+ patients) revealed no significant interaction, F(7) < 1.

The accuracy data were analyzed using a three-way mixed factor ANOVA, including Group (patients vs. controls), and two within-subject factors—Cycle (Repetitions 1–4) and Modality (words vs. pictures). There was a significant main effect of Group, F(1, 19) = 20.77, p < .001. There was also an interaction between Cycle and Group, F(3, 54) = 3.65, p = .034, indicating that the patients showed greater refractory effects than controls. This is shown in Figure 5. No significant effect of Modality was found, F(1, 19) < 1, and there was no Group × Modality interaction, F(1, 19) < 1, or Cycle × Modality interaction, F(3, 24) = 2.40, p = ns, indicating that refractory effects were equivalent for both tasks. Similarly, the three-way Task × Cycle × Group interaction was not significant, F(3, 54) = 2.67, p = ns.

The effects of Cycle and Modality were examined further in the patient group using a two-way, within-subject ANOVA. The main effect of Cycle was significant, F(3, 24) = 8.18, p = .011, but there was no influence of Modality, F(1, 8) < .1. Again, the interaction between Cycle and Modality was not significant, F(3, 24) = 1.94, p = ns, confirming equal refractory effects for WPM and PPM.

A three-way mixed-factor ANOVA revealed a main effect of Group, F(1, 19) = 15.85, p = .001 and an interaction between Cycle and Group, F(3, 54) = 7.18, p = .003, indicating that refractory effects were stronger in patients than controls. In this experiment, however, the effect of Modality was significant, F(1, 19) = 17.58, p < .001. Accuracy was higher in WPM than in SPM, as shown in Figure 6. There was no Modality × Group interaction, F(1, 19) = 1.93, p = ns, or Modality × Cycle interaction, F(1, 19) = 1.53, p = ns, but the three-way interaction between Task, Group, and Cycle was significant, F(3, 54) = 7.06, p = .003.

These findings were explored further in the patient group using a two-way, within-subject ANOVA. The main effect of Cycle was significant, F(3, 24) = 8.75, p < .001, and the influence of Modality approached significance, F(1, 8) = 4.83, p = .06. However, the interaction between Modality and Cycle was not significant, F(3, 24) = 1.66, p = ns, indicating that the patients showed equal refractory effects for WPM and SPM. The three-way interaction reported above is, therefore, likely to reflect the fact that, in contrast to patients, the control participants showed some improvement in SPM but not WPM over cycles. It may also reflect the fact that the data in the SPM task were noisy, as seen in Figure 6.

A three-way mixed factor ANOVA revealed a main effect of Group, with controls performing at a higher level than patients, F(1, 19) = 29.31, p < .001. There were no significant effects of modality, F(1, 19) < 1, or cycle, F(3, 54) = 2.60, p = ns. There was also no significant interaction between Cycle and Group, F(3, 54) = 1.98, p = ns, or between Modality and Group, F(1, 19) = 2.33, p = ns. However, there was a significant interaction between Modality and Cycle, F(3, 54) = 5.35, p = .009, and the three-way interaction was significant, F(3, 54) = 4.89, p = .012. These data are shown in Figure 7.

The possibility that refractory effects were found in only one task was explored using separate two-way mixed factor ANOVAs for each modality. In WPM, there was a significant main effect of Cycle, F(3, 54) = 4.79, p = .013, and Group, F(1, 19) = 23.30, p < .001. Additionally, there was an interaction between Cycle and Group, F(3, 54) = 3.24, p = .048, as the patients showed stronger refractory effects than controls. In the PPM task, there was a main effect of Group, F(1, 19) = 25.27, p < .001, but no effect of Cycle, F < 1, and no interaction between Cycle and Group, F(3, 54) = 1.13, p = ns. Bonferroni-corrected paired-sample t tests were used to compare the patients' performance on WPM and PPM at each cycle. There was a difference between the two modalities only on the fourth cycle, t(8) = 5.13, p = .004, with no differences between modalities on Cycles 1, 2, or 3, t(8) < 1.

It was predicted that, for TP-only patients, accuracy would not decline over cycles; therefore, significant refractory effects would be shown in the pFC+ group, but not in the TP-only group. Logistic regression was used to establish whether the effect of cycle interacted with lesion location. Overall, lesion subgroup alone explained 14.6% of the variance in the data. Accuracy was higher in the TP-only than in the pFC+ group, as shown in Figure 8. A model that included experiment, distinguishing all six tasks (Wald = 3.24), cycle (Wald = 40.28), individual patient identifiers (Wald = 305.69), and lesion subgroup (Wald = 10.66) found a significant predictive value for each variable (p ≤ .001), except experiment.

When two interactive terms were added to this model, there was a significant effect of Cycle × Lesion subgroup, but no interaction between experiment and lesion subgroup (inclusion of these interactions also led to the main effects of Subgroup and Cycle becoming nonsignificant; see Table 5). pFC+ patients showed significantly greater effects of cycle than TP-only patients, but there were no significant differences between the subgroups in the effects of the experiment variable. The Cycle × Subgroup interaction remained significant (p = .007) with the addition of Cycle × Patient ID in the equation.

Separate logistic regression analysis of each cycle, using the predictor variables Experiment, Subgroup, and Patient ID, revealed that there was no difference in accuracy between subgroups at Cycle 1, but this difference became increasingly significant as the number of cycles increased. These four analyses are shown in Table 6.

Further logistic regressions examining the different input modalities compared (1) pictures and words from Experiments 1 and 3 and (2) environmental sounds and words from Experiment 2. These models included modality, cycle, patient identifier, subgroup, and cycle by subgroup. The first model found a significant predictive value of patient identifier (Wald = 286.16, p < .001), subgroup (Wald = 5.99, p = .014), and cycle by subgroup (Wald = 10.30, p = .001). There was no significant effect of Modality (Wald = .375, p = .54) or Cycle (Wald = 2.87, p = .09).

In the second comparison, examining sound and WPM scores from Experiment 2, the model found a significant effect of Modality (Wald = 35.90, p < .001), Patient ID (Wald = 32.37, p ≤.001) and Subgroup (Wald = 4.84, p = .028), but not Cycle (Wald = .48, p = .49), or Cycle × Subgroup (Wald = .05, p = .83). The significant predictive value of Modality was driven by higher performance in WPM in this experiment, as shown in Figure 6.

McNemar tests were carried out on the data from each patient to determine which individuals showed significant refractory effects. The results are provided in Table 7. All of pFC+ patients showed some degree of refractory impairment, whereas none of the TP-only patients did. There were also substantial individual differences in the refractory effects shown by pFC+ patients. Some patients showed refractory effects at the beginning of the task between the first two cycles (e.g., KA). In contrast, some showed refractory effects between the last two cycles (e.g., LS), whereas others showed subtle but consistent refractory effects, which became significant across the whole task (e.g., DB).

This study assessed the multimodal nature of refractory effects in SA using, for the first time, a case series approach as opposed to analysis of individual cases. Additionally, it explored the effect of lesion location on refractory semantic access. Refractory deficits were found in all modalities—in word-, picture-, and sound-matching tasks. Second, patients with left pFC lesions always showed deterioration in performance across cycles, whereas none of the TP cortex patients did.

Our finding of equivalent refractory effects in verbal and nonverbal modalities is compatible with the view that, in SA, the store of semantic representations remains intact (shown by good performance on the first cycle), whereas executive control over semantic activation is impaired (shown in the reduction of accuracy over cycles). This pattern of impairment might be expected from the brain injury in SA: The ATLs, which are thought to form a key hub for semantic knowledge (Lambon Ralph et al., 2010; Pobric et al., 2010b; Patterson et al., 2007), are intact, whereas there is significant damage to the left pFC and TP cortex regions. These brain areas are associated with semantic control functions in neuropsychological research (Noonan et al., 2010; Jefferies & Lambon Ralph, 2006), functional neuroimaging studies of healthy volunteers (for a review, see Noonan et al., submitted) and TMS work (Whitney, Kirk, et al., 2011). In refractory tasks, the same set of semantically related items is presented repeatedly, such that targets become distractors and vice versa. This should produce significant competition between targets and distractors in later cycles, irrespective of input modality—and consequently SA patients with semantic control impairment show refractory effects across words, pictures, and environmental sounds.

Our findings pose significant challenges to theoretical frameworks that only predict refractory effects in verbal tasks. For example, Warrington and Crutch (2004) argue that there are separate semantic systems, with their patient only showing refractory effects in the verbal modality. Given later evidence that this same patient showed nonverbal refractory effects for environmental sounds, this proposal was modified to suggest that the visual system is qualitatively distinct from the auditory system (Crutch & Warrington, 2008). The current study confirms that refractory effects can emerge in parallel in verbal, picture, and sound tasks. Moreover, some of our patients (e.g., LS and DB) resembled Warrington and Crutch's patient AZ in Experiment 1 (which used Warrington and Crutch's materials) in that they showed refractory effects in the verbal modality that did not extend to the visual task. However, in further experiments, these patients showed the opposite pattern (e.g., stronger refractory effects in nonverbal than verbal judgments). Therefore, it is helpful to consider performance across different tasks that may vary in their sensitivity to refractory effects. In picture–picture identity matching tasks (used in Experiment 1 and by Crutch and Warrington), it is difficult to avoid probe and targets looking somewhat alike (given they are examples of the same object) and even partial visual similarity may be enough to weaken the refractory effect. In contrast, there is no surface similarity issue to consider when matching sounds or words to target pictures.

Our findings are also at odds with another theoretical perspective that accounts for refractory effects in picture naming in terms of lexical competition (Schnur et al., 2006; Belke et al., 2005; Damian et al., 2001). In this theory, coactivation of a word (e.g., “dog”) and its category node (e.g., “animal”) results in activation spreading back to semantically related word nodes (e.g., “cat”; Levelt, 2001), and this gives rise to strong competition within the lexical network when sets of semantically related items are presented. This framework only predicts refractory effects in picture-naming tasks; nevertheless, if similar competition is envisaged in the semantic system, effects of cycle might be expected in semantic judgment tasks (i.e., decisions not requiring spoken output) and across verbal and nonverbal input modalities.

The current study also confirms that lesion location affects the likelihood of refractory deficits, with pFC+ patients showing deterioration in accuracy, whereas TP-only patients maintain performance across cycles. A difference between these patient groups has been predicted by previous research (Campanella et al., 2009; Schnur et al., 2009), although the explanation given for refractoriness is not compatible with the current findings of multimodal refractory effects in pFC+ patients. Indeed, the difference between pFC+ and TP-only patients is perhaps surprising, as several lines of research indicate that both left pFC and posterior temporal/inferior parietal regions make a critical contribution to multimodal semantic control. First, in several investigations of verbal and nonverbal semantic control, SA patients with lesions in these two locations have shown highly similar deficits, characterized by strong sensitivity to manipulations of semantic control demands (Noonan et al., 2010; Corbett, Jefferies, Ehsan, et al., 2009; Corbett, Jefferies, & Lambon Ralph, 2009; Jefferies & Lambon Ralph, 2006). Second, neuroimaging studies frequently reveal activation of both the left pFC and posterior temporal/inferior parietal regions in tasks that load semantic control (Noonan et al., submitted; Whitney, Jefferies, et al., 2011; Badre et al., 2005). Third, a recent TMS study found that a “virtual lesion” in either the left IFG (LIFG) or pMTG disrupted executively demanding semantic judgments to an equal degree (Whitney, Kirk, et al., 2011). Consequently, if refractory effects in SA occur simply because the patients have poor control over activation within the semantic system, then we would expect both lesion subgroups to show parallel deficits.

One possible explanation for this difference between the lesion subgroups is that TP-only patients have a milder deficit of semantic control. A recent meta-analysis of neuroimaging studies revealed the left pFC is strongly and consistently activated in executive semantic tasks, whereas the TP region shows a somewhat smaller peak of activation, which is only significant in some studies/tasks (Noonan et al., submitted). However, in the current study, while two of the TP-only patients had relatively mild semantic impairment, another (patient ME) showed much more substantial deficits.

A second possibility is that both the left pFC and TP cortex regions contribute to semantic control, but their exact roles vary. If so, the TP cortex region may be necessary for aspects of semantic control that do not interact with cycle in refractory tasks. For example, LIFG may be crucially involved in inhibition, especially when activation of previously relevant semantic information must be dampened down (leading to more perseverative errors as well as strong refractory effects in patients with left pFC lesions (see Corbett, Jefferies, & Lambon Ralph, 2008). In contrast, TP cortex areas, alongside LIFG, may help to retrieve nondominant semantic associations and/or bring task-relevant information to the fore in a flexible way. On the basis of the findings from fMRI, Badre and colleagues (2005) proposed a two-step semantic retrieval model involving “controlled retrieval” and “postretrieval selection.” They found that activation in posterior temporal cortex was sensitive to controlled retrieval demands, as measured by the associative strength between a cue and a target or the number of response alternatives, but not manipulations of postretrieval selection demands, such as whether the judgment related to global semantic similarity or a specific attribute. In contrast, regions within LIFG responded to both of these elements of semantic control (although Badre et al. focused on divisions within LIFG that were not testable in our patient sample).

This distinction between controlled retrieval and postretrieval selection could prove to be crucial in understanding refractory performance. The first block in cyclical tasks always demands controlled retrieval, but with stimulus repetition, the items have already been retrieved and postretrieval selection is required. If the TP cortex region plays a key role in controlled retrieval, patients with TP cortex lesions but intact pFC selection processes would not find the last trial any more difficult than the first. In line with this theory, damage to the LIFG has been linked to heightened difficulty in processing words with multiple propositions, which tax semantic selection (often, counterintuitively, high-frequency words; Hoffman, Jefferies, & Lambon Ralph, 2011; Hoffman, Rogers, & Lambon Ralph, 2011). For example, a recent study of 72 brain-injured patients found focal damage to LIFG caused impaired performance on a sentence generation task when the probe word referred to multiple conceptual propositions (Robinson, Shallice, Bozzali, & Cipolotti, 2010). These findings suggest that LIFG may be specifically involved in selection between competing items, which have been retrieved (see also Robinson, Shallice, & Cipolotti, 2005; Robinson, Blair, & Cipolotti, 1998).

Although the current data are consistent with the view that anterior and posterior sites within the semantic control network have varying roles, further research is required to fully specify the control processes that they underpin. Given the differences between Badre et al.'s (2005) semantic selection task and the refractory paradigm, it is not currently known whether IFG makes a greater contribution to all forms of semantic selection or only when previously relevant information must be inhibited. The current data advance knowledge in two ways: (1) they show that refractory impairments resulting from poor semantic control generalize from verbal to nonverbal modalities, and (2) they provide evidence that lesion location is important in determining deficits in the refractory paradigm. This lends further support to the theory that deregulated semantic control results in an amodal “access” impairment (Jefferies & Lambon Ralph, 2006). Additionally, it sparks new interest into the function of the TP cortex and pFC in semantic cognition and executive control.

We thank the patients and their carers for their generous assistance with this study; Rachel Byrne, Linda Collier, and Claire Slinger for referring some of the patients to us; Karen Sage for help with classifying the stroke patients' aphasic syndromes; and Sebastian Crutch and Elizabeth Warrington for the use of their materials. This work was supported by grants from the Medical Research Council (G0501632), the National Institutes of Mental Health (MH64445), and Research into Ageing.

Reprint requests should be sent to Elizabeth Jefferies, Department of Psychology, University of York, York, YO10 5DD, United Kingdom, or via e-mail: ej514@york.ac.uk.