In this study, we bridge the gap between monkey electrophysiological recordings that showed selective responses to informative features and human fMRI data that demonstrated increased and selective responses to trained objects. Human participants trained with computer-generated fish stimuli. For each participant, two features of the fish were informative for category membership and two features were uninformative. After training, participants showed higher perceptual sensitivity to the informative dimensions. An fMRI adaptation paradigm revealed that during categorization the right inferior frontal gyrus and occipitotemporal cortex were selectively responsive to the informative features. These selective cortical responses were experience dependent; they were not present for the entire trained object, but specific for those features that were informative for categorization. Responses in the inferior frontal gyrus showed category selectivity. Moreover, selectivity to the informative features correlated with performance on the categorization task during scanning. This all suggests that the frontal cortex is involved in actively categorizing objects and that it uses informative features to do so while ignoring those features that do not contribute category information. Occipitotemporal cortex also showed selectivity to the informative features during the categorization task. Interestingly, this area showed a positive correlation of performance during training and selectivity to the informative features and a negative correlation with selectivity to the uninformative features. This indicates that training enhanced sensitivity to trained items and decreased sensitivity to uninformative features. The absence of sensitivity for informative features during a color change detection task indicates that there is a strong component of task-related processing of these features.
When you are swimming in the ocean and suddenly a dorsal fin surfaces next to you, you will be getting out of the water as soon as you can, assuming you are dealing with a shark. However, there are a number of other ocean creatures with a dorsal fin, such as whales and dolphins. Therefore, seeing only the dorsal fin will not be informative for the type of fish you encountered. You need to see more features of the fish to categorize it. But given the situation you were in, you probably focused all your attention on this dorsal fin while ignoring other features of the fish. In this study, we investigated if and how our brain distinguishes between features that are informative for categorization and features that are uninformative for categorization. We also looked at whether this is an automatic process or whether attention plays a role. More specifically, we trained participants to discriminate between two types of fish categories. The fish had four features (mouth, tail, dorsal, and ventral fin) that varied between the two categories, but only two of the features were informative for categorization.
Categories can take on several forms, for example, they can be based on simple abstract rules or on the similarity of the members. Depending on the material to be categorized and by which strategy, the brain areas involved in categorization will differ (Seger & Miller, 2010; Smith & Grossman, 2008). In the present experiment, we look into categorization based on the similarity of the exemplars. This is the most widely studied type of categorization when it comes to attempts to unravel the neural correlates of perceptual categorization and how this shapes the cortical representation of object categories in the occipitotemporal cortex.
At first it was found that experience with an existing object category leads to changes at the neuronal level in the occipitotemporal cortex (van der Linden, Murre, & van Turennout, 2008; Gauthier, Skudlarski, Gore, & Anderson, 2000); however, experience with novel objects also leads to changes in this part of the cortex (Weisberg, van Turennout, & Martin, 2007; Moore, Cohen, & Ranganath, 2006; Op de Beeck, Baker, DiCarlo, & Kanwisher, 2006; Gauthier, Williams, Tarr, & Tanaka, 1998). It remained unclear whether specifically categorization itself was responsible for these neuronal changes. Therefore, the nature of the underlying neuronal mechanism of these changes has been closely investigated using fMRI adaptation paradigms (Grill-Spector & Malach, 2001). Normal BOLD fMRI reflects the averaged activity of relatively large neuronal populations and therefore cannot index the functional properties of groups of neurons. The method of fMRI adaptation was developed to get around this problem. Using the adaptation approach, the nature of a neuronal stimulus representation can be revealed by selectively repeating or changing certain stimulus attributes and investigating its effect on the hemodynamic response. This is based on the principle that cortical neurons tend to reduce their activity upon stimulus repetition; this is called the adaptation effect. By examining the sensitivity of the adaptation effect to stimulus manipulation, the invariant and selective properties of its neuronal representation can be exposed (Cohen Kadosh, Cohen Kadosh, Kaas, Henik, & Goebel, 2007; Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier, 2005; Grill-Spector et al., 1999). This makes the adaptation technique a valuable tool in making inferences about neuronal sensitivity of specific cortical regions.
Previous studies showed that regions involved in representing stimuli from a certain category show selective adaptation to repeated presentation of objects from this category. For example, the fusiform face area (FFA) shows sensitivity to repeated presentation of faces (Andrews & Ewbank, 2004) and the parahippocampal place area to the repetition of places (Ewbank, Schluppeck, & Andrews, 2005; Epstein, Graham, & Downing, 2003). Following categorization training with objects, adaptation effects were found in several brain regions including the occipitotemporal cortex (van der Linden, van Turennout, & Indefrey, 2010; Gillebert, Op de Beeck, Panis, & Wagemans, 2009; Jiang et al., 2007) and pFC (Jiang et al., 2007). This suggests that neuronal clusters in these areas became selectively responsive to the trained objects. But what is it exactly about those objects that neurons become selectively responsive to? It seems likely that the task at hand, in this case categorization, determines what parts of the objects induce sensitivity in the neurons. Indeed, Schyns and Rodet (1997) found that object features were flexible and developed through categorization experience, influencing the perception of subsequent category exemplars. In addition, more recently, it has been found that paying attention during training to one dimension over another dimension leads to sensitivity in visual cortex for this dimension (Folstein, Palmeri, & Gauthier, 2013). In their experiment, a dimension contained several object features that differed between dimensions. It is unknown which of these features were indeed informative for categorization—possibly all of them—and, subsequently, which features were used by the participants to categorize the stimuli. For the current experiment, we hypothesize that training induces neurons to become selectively responsive to those object features that are informative for categorization. However, up to now, the features of objects in fMRI studies investigating category formation were not quantified. A few studies have investigated categorization based on informative and uninformative features in macaque inferior temporal cortex (De Baene, Ons, Wagemans, & Vogels, 2008; Sigala & Logothetis, 2002) and found indeed selective responses to informative features. In humans, this has, as far as we know, never been studied on the cortical level.
In this study, we used a stimulus set similar to that of Sigala and Logothetis (2002). Participants trained with fish that they categorized based on two features that were informative for categorization. At the same time, they were exposed to two other features that the fish had, but these features were uninformative for categorization. We combined this training regimen with an fMRI adaptation paradigm to investigate neuronal selectivity to informative and uninformative features. The effects of categorization on feature processing were investigated by having participants perform a categorization task during scanning. We expected that training leads to increased sensitivity of neurons to those features that are informative for categorization. Therefore, repetition of fish with the same informative features will give rise to an adaptation effect, even if the uninformative features are differing. Presenting the same uninformative features should not elicit an adaptation effect, because there should be no training-related sensitivity to these features. Because we used an active categorization task, we also expected, next to the occipitotemporal cortex, frontal areas to respond selectively to the informative features. Selective responses to trained items—with no distinction between features—has been shown in human (Jiang et al., 2007) and macaque (Freedman, Riesenhuber, Poggio, & Miller, 2001, 2002, 2003) pFCs.
To investigate whether the processing of informative features is dependent on active categorization or whether it is also, in part, an automatic process, we presented the same fish stimuli to the participants while they performed a feature color change detection task. In half of the trials, participants attended an informative feature, and in the other half, they attended an uninformative feature. If neuronal selectivity is automatic, we expected to see adaptation for repetition of the informative features, even when participants attended the uninformative features. If neuronal selectivity is in part driven by attention, we expected to see adaptation for those trials where participants attended to the informative features and not when they attended the uninformative features. If, however, neuronal selectivity is task dependent, then we might find no adaptation for the informative features, but only for the same color.
Twenty-four healthy participants (nine men, mean age = 22.4 years, range = 20–25 years) participated in the experiment. All participants had normal or corrected-to-normal visions. Participants were paid for their participation. All participants gave written informed consent.
The stimuli consisted of pictures of computer-generated fish (Pacific Perch by Daz3d, Draper, UT) that were constructed in a 3-D model manipulation program (Poser 7, E Frontier, Scotts Valley, CA). Four features of the fish were manipulated: ventral fin, dorsal fin, tail, and head (see Figure 1A). These four features could each take the shape of two extremes (Figure 1B). To create different fish exemplars, the features were morphed between these two shape extremes. Morphing was done parametrically in 20 steps (corresponding to a 5% physical difference per step), creating 20 possible shapes of a feature. As such, the stimulus space of the informative features contained 400 fish exemplars. For each participant, two features were informative for categorization. A category boundary was placed within this feature space. The boundary was expressed as a linear relation between two informative feature shapes (see Figure 1C). Stimuli that were on or within a distance of less than 5% of the category boundary were not included in the stimulus set and neither were the feature extremes. Each of 400 fish exemplars with morphed informative features would also have morphed uninformative features. The space for the uninformative features also contained 400 exemplars. As such, there were 160,000 possible exemplars, each with a unique feature combination.
We rendered each feature separately for the 20 morph steps. Each feature was colorless and rendered against a black background under the same lighting and camera settings. The software used for presenting the fish stimuli to the participants constructed fish exemplars from the shapes of the four features. The total fish images measured 250 by 250 pixels in both the training and scanning session.
Recently, Folstein, Gauthier, and Palmeri (2012) proposed two different methods of morphing stimuli to create different category members, the blended and factorial models. In both cases, the morph space between the prototypes is shaped as a tetrahedral volume. In the factorial model, the space is populated with stimuli by morphing images factorially along two of the axes whereas the other axes stay constant. In the blended model, the stimuli are created from all possible morphs between all prototypes, using all axes of the volume. Although this provides an interesting conceptual framework, previous studies have only used a very restricted part of the blended model space. The blended model, as Folstein et al. propose it, does not describe the stimulus space used in Jiang et al. (2007), Gillebert et al. (2009), and van der Linden et al. (2010). In these studies, the categories were 2-D and followed the ribs of the tetrahedron and not its volume. As such, an exemplar of a category had only two prototype parents, and no exemplars that were blends of all parents existed. Having said that, the current morph method does follow Folstein's morph model of a factorial morph space. Such a morph space has been shown to increase discriminability along the relevant dimensions (Folstein et al., 2012). In this study, we manipulated the features of the fish independent of each other.
Training included three sessions, each of which lasted approximately 2 hr, on 3 consecutive days. During a training session, participants sat comfortably in a soundproof cabin in front of a 19-in. computer screen. They performed a 1-back task on a series of fish images, in which they indicated with the index and middle fingers of their right hand whether two consecutive fish belonged to the same fish category or not. Participants received feedback to their responses consisting of a printed text centered on the screen in colored Arial font in size 16 (green: “right,” red: “wrong,” and yellow: “too late”). The proportion of “same” and “different” responses was 50:50. In each trial, stimuli were presented for 400 msec, after which a response could be given during 1850 msec. Feedback was presented for 250 msec. SOA was 200 msec. A training session consisted of nine blocks of 150 trials. Each block was followed by a small self-paced pause, after which a participant could continue the experiment by pressing a button.
We used an adaptive training paradigm. Difficulty of trials increased over sessions by selecting fish exemplars using different criteria. The first criterion was based on the distance of the fish exemplars parallel to the category boundary (close, medium, or far; see Figure 2A). The second criterion was the place of the fish exemplars perpendicular to the category boundary (center, medium, or corner; Figure 2A). During the first week of training (six participants), we used only the first selection criterion. For these participants, the first training session started easy (with fish far from the category boundary), and in the next blocks, more difficult fish exemplars (close to the category boundary) and less easy exemplars (far from the category) were added as training progresses (see Figure 2B). However, using this criterion, participants could employ a 1-D strategy, utilizing only one informative feature to categorize the exemplars and still achieve reasonable performance (see Figure 3). To encourage participants to combine information from both informative features, we used the second criterion to select fish exemplars for training. Again, participants would start with more easy fish (centered perpendicular to the category boundary) and would be introduced to more difficult fish during the course of training (from the corners of the fish space; see Figure 2C). For all 24 participants, the first and last block of each training session had identical distributions of fish over the feature space based on the first criterion (1/3 far, 1/3 medium, 1/3 close). Because of this, these blocks could be compared and performance of these blocks was our measurement for training success.
After three training sessions on three consecutive days, the participants participated in two fMRI experiments. In the first experiment (not this study), they were scanned in a 7T scanner in Essen, Germany. They performed a feature color change detection task for about 1 hr followed by the same localizer that was used in the present experiment. The next day, the participants participated in the fMRI experiment in Nijmegen (this study).
An adaptation paradigm with two tasks was used during scanning. The two tasks were a categorization task and a feature color change detection task (Figure 4A). In the categorization task, participants indicated whether the second fish belonged to the same fish category as the first fish. In the other task, we wanted to avoid participants actively categorizing the fish to tap into automatic processing of the features. Therefore, we used a feature color change detection task. In each fish, one feature would be colored. The participants indicated, for each second fish in the pair, whether the colored feature had the same or a different color as in the first fish. For half of the trials, the participants attended an informative feature, and in the other half of the trials, they attended an uninformative feature. Each task was performed in a separate run. The order of the tasks was counterbalanced over participants. In both runs, an adaptation trial started with a fish picture that was shown for 400 msec, followed by a blank screen interval of 400 msec and another picture of a fish for 400 msec. After the onset of the second picture, the participant could respond. The interstimulus interval was randomly jittered between 3500 and 4500 msec. The order of trials was pseudorandom to have an optimal distance between pairs of the same adaptation condition.
The adaptation condition was determined by the relation between the two fish that were rapidly presented in a pair. We used a 2 (same or different informative features) × 2 (same or different uninformative features) design (see Figure 4B). The informative features of the second fish could either be the same or different from the first fish. In addition, the uninformative features of the second fish would also either stay the same or be different. The adaptation conditions were, first, SS, where the informative and uninformative features stay the same (identical repetition); second, SD, where the second fish has the same informative features as the first fish, but different uninformative features; third, DS, where the second fish has different informative features but the same uninformative features; and fourth, DD, where the second fish has different informative and uninformative features. For each adaptation condition, there were 60 trials; 20 trials per distance to the category boundary (close, medium, and far).
Participants responded with the index (“same”) and middle finger (“different”) of the right hand on an MR-compatible response box (Lumitouch by Photon Control, Burnaby, Canada). In the categorization task, the response in the SS and SD conditions was always “same.” In the DS and DD, half of the trials were also from the same category. Therefore, the ratio of “same” and “different” responses was 75:25. We used the same ratio of “same” and “different” responses for the feature color change detection task. To increase sensitivity, we did not use fish stimuli from the corners of the fish space perpendicular to the category boundary. Fish could belong to close, medium, and far distances parallel to the category boundary.
We used a block design using stimuli from five categories: 32 faces (16 male, 16 female), 32 houses, 32 common objects, 32 fish (not computer-generated but natural fish), and 32 scrambled pictures. Images had gray backgrounds and measured 500 × 500 pixels. Images were randomly assigned to blocks of 17 images (each image was displayed for 400 msec and followed by a blank screen of 600 msec). Each block lasted 16.4 sec. Within each block, 16 images were unique and one image was repeated. The participants' task was to detect this repetition by pressing a button with the index finger of the right hand. Each block was followed by a blank screen interval of 10 sec. Each image was presented twice to the participant, but within different blocks of images. There were four blocks for each category of objects. The localizer run lasted 8.7 min.
After participating in the fMRI experiment, the participants judged the perceived similarity of fish stimuli in a behavioral experiment. Participants were instructed to rate the overall similarity of pairs of fish by pressing a key from 1 (very dissimilar) to 5 (very similar). Twenty-five fish exemplars were used for all participants. Fish stimuli consisted of a combination of 5 points from the mouth–tail space and 5 points from the dorsal–ventral fin space (Figure 2A). An asterisk, presented for 250 msec, marked the start of each trial. This was followed by the presentation of the first fish stimulus (400 msec), a scrambled fish image (350 msec), and the second fish stimulus (400 msec). The participants could respond for 1250 msec. Each combination of stimuli was presented one time in each unique order, giving 600 trials.
Whole-brain images (EPI, 35 slices, 3-mm thick with 10% gap, repetition time = 2220 msec, in-plane resolution = 3.3 × 3.3 mm2, echo time = 30 msec, flip angle = 80°, field of view = 21.1 cm, matrix size = 64 × 64) were acquired on a 3T whole-body MR scanner (Magnetom TIM TRIO by Siemens Medical Systems, Erlangen, Germany). In addition, a high-resolution structural T1-weighted 3-D magnetization prepared rapid acquisition gradient-echo sequence image was obtained after the functional scan (192 slices, voxel size = 1 × 1 × 1 mm).
Data analysis was done using BrainVoyager QX (by Brain Innovation, Maastricht, The Netherlands). The first two volumes were discarded to allow for T1 signal equilibrium. The following preprocessing steps were performed: slice scan time correction (using sinc interpolation), linear trend removal, temporal high-pass filtering to remove low-frequency nonlinear drifts of three or fewer cycles per time course, and 3-D motion correction to detect and correct for small head movements by spatial alignment of all volumes to the first volume by rigid body transformations. All volumes were aligned to the first volume of the color change detection task scan session. This ensured between-session alignment of all three functional sessions. Coregistration of functional and 3-D structural measurements was computed by relating T2*-weighted images and the T1-weighted MPRAGE measurement, which yields a 4-D functional data set. Structural 3-D and functional 4-D data sets were transformed into Talairach space (Talairach & Tournoux, 1988) and spatially smoothed with a Gaussian kernel (FWHM = 6 mm).
The expected BOLD signal change was modeled using a gamma function (tau of 2.5 sec and a delta of 1.5) and convolved with the second event (Boynton, Engel, Glover, & Heeger, 1996). Statistical analyses were performed in the context of the general linear model. Both fixed and random-effects group analyses were performed. The statistical threshold was set at p < .001 at the voxel level and a cluster threshold of 50 mm3.
First, we looked at the difference between the two tasks (categorization and feature color change detection), comparing each task with rest and the two tasks with each other (categorization > feature color change detection). Next, we defined areas that showed adaptation following the repeated presentation of the same fish using the contrast SS < DD. The contrasts that we used from the localizer session were faces versus houses and objects versus scrambled images.
Next, significantly activated clusters were selected for a more sensitive ROI analysis. The ROI time courses were standardized, so that beta weights (regression coefficients) of predictors, as indices of effect size, reflect the BOLD response amplitude of one condition relative to the variability of the signal. Beta weights were obtained for all voxels within these ROIs, per participant and per adaptation condition (SS, DS, DS, and DD), per distance (medium, close, or far from the category boundary), and per category (same or different category). Random-effects analyses were performed on the subject-averaged adaptation scores and beta weights by applying t tests, with a threshold set at p < .05. All t tests were two-tailed.
For the correlation analysis, we paired the proportion of correctly categorized fish during scanning or during the third training session (first and last block averaged) with the score for selectivity to the different feature types. Selectivity was defined as the amount of adaptation. So for informative features, this was defined as the subject-averaged beta weight for those trials where the informative features were different minus the subject-averaged beta weights of trials where the informative features remained the same [(DD + DS) minus (SS + SD)], and for the uninformative features, it was defined as [(DD + SD) minus (SS + DS)].
Training Data Analysis
Mean RTs for the correct trials and the percentage of correct trials of the first and last blocks of a training session were computed for each participant. These dependent variables were submitted to a Training Session × Block × Distance MANOVA with repeated measures. Training session consisted of three levels (first, second, and third training sessions), block of two levels (first and last), and distance to the category boundary consisted of three levels (close, medium, and far). Differences between training sessions were explored with MANOVAs with two levels for session and three levels for distance. Significant differences between sessions and blocks were explored with paired t tests. We used a one-sample t test to test the difference of the first block of the first training session with chance level. All reported t tests were two-tailed.
fMRI Behavioral Data Analysis
Percentage of correct responses and RTs on correct trials were computed for both the categorization task and the feature color change detection task. To investigate the effect of distance on RTs and percentage of correct responses for both tasks, we used a Distance × Informative Feature × Uninformative Feature MANOVA with repeated measures. Distance had three levels (close, medium, and far), informative feature had two levels (same and different), and uninformative feature also had two levels (same and different).
In the categorization task, we also investigated differences between fish with different informative features that belonged to either the same or different fish categories. We used an Uninformative Features × Category MANOVA with repeated measures. Uninformative features had two levels (same and different), and category also had two levels (same and different). To investigate the effect of attention for the feature color change detection task, we used an Attention × Informative Feature × Uninformative Feature MANOVA with repeated measures. Attention had two levels (attended informative feature and attended uninformative feature), informative feature had two levels (same and different), and uninformative feature also had two levels (same and different).
Similarity Rating Data Analysis
Similarity ratings (on a scale of 1–5) were entered in a MANOVA with repeated measures with four levels for feature (mouth, tail, dorsal fin, and ventral fin), four levels for the distance between two features (1, 9, 10, or 19 steps), and two levels for feature type (informative or uninformative). Significant interactions were further explored with MANOVAs and t tests (paired and two-tailed).
Participants trained for 3 days with the fish and had one training session per day. The percentage of correct responses increased over training sessions [F(2, 22) = 27.66, p < .001] (see Figure 5A). In the first block of the first session, participants performed above chance [t(23) = 35.00, p < .001] and performance improved from the first to the last block [F(1, 23) = 40.93, p < .001]. Performance also improved from the first to the second training session [F(1, 23) = 23.83, p < .001] and from the second to the third training session [F(1, 23) = 15.12, p < .005]. Within the second and third training session, there were no differences in performance between the first and last blocks.
Performance was significantly modulated by the distance of the fish exemplars to the category boundary [F(2, 22) = 136.65, p < .001]. Participants are better at categorizing fish that are farther away from the category boundary. In the first block of the first training session, the distance effect was already present [F(2, 22) = 3.64, p < .05]. In the last block of the first session, the effect of distance [F(2, 22) = 38.98, p < .001] was significantly greater than in the first session, as revealed by a significant Block × Distance interaction [F(2, 22) = 16.46, p < .001]. In the second and third sessions, the effect of distance remained strong and did not differ between the first and last blocks of a session.
RTs (Figure 5B) became faster over training sessions [F(2, 22) = 6.43, p < .01]. During the first training session, participants were faster in the last block than in the first block [F(1, 23) = 9.71, p < .01], but for consecutive training sessions, no differences between the first and last blocks were observed. RTs improved from the first to second sessions of training [F(1, 23) = 12.71, p < .005], but not from the second to third training sessions.
Participants responded faster to fish that were farthest away from the category boundary [F(2, 22) = 10.03, p < .005]. We observed a significant three-way interaction between training session, block, and distance [F(4, 20) = 4.36, p < .05]. In the first block of the first training session, no effect of distance to the category boundary was present yet. In all subsequent blocks, a significant effect of distance of the fish to the category boundary was obtained.
Participants performed well in the scanner (an average of 88.4% correct responses with a standard deviation of 4.1%). Accuracy was highest for fish that were far from the category boundary [F(2, 22) = 166.65, p < .001] (see Figure 6A). Trials that had the same informative features had a higher accuracy than trials with different informative features [F(1, 23) = 198.36, p < .001]. This effect, albeit smaller, was also present for fish that had the same uninformative features than fish with different uninformative features [F(1, 23) = 4.94, p < .05]. The effect of distance to the category boundary was largest for fish with different informative features, as revealed by an interaction between informative feature and distance [F(2, 22) = 46.12, p < .001].
Participants responded fastest to fish far from the category boundary [F(2, 22) = 12.21, p < .001] (see Figure 6B). Participants responded faster to trials in which the fish had the same informative features than when the fish had different informative features [F(1, 23) = 26.86, p < .001]. When the uninformative features were also the same, the responses were even faster than when the uninformative features were different [F(1, 23) = 4.10, p = .055]. The effect of distance to the category boundary was largest for fish with different informative features [F(2, 22) = 7.44, p < .005].
Fish with the same informative features were from the same category. However, fish with different informative features could belong to the same or different categories. Accuracy was higher for those fish with different informative features that belonged to the same category [F(1, 23) = 10.52, p < .005] than for fish with different informative features that belonged to different categories (see Figure 6C). It did not matter for these fish whether their uninformative features were the same or different. The RTs did not differ between fish from the same or different categories.
Feature Color Change Detection Task
Performance accuracy in the feature color change detection task was very high. The mean percentage of correct responses was 98.4% (standard deviation = 1.8%) but did not differ between conditions. Also no significant effects were found for the RTs for the different adaptation conditions nor for the different distances of the fish to the category boundary. However, a significant effect of attention on RTs was found (Figure 6C). Participants were on average 20 msec faster when they attended the informative features [F(1, 23) = 9.74, p < .01] (Figure 6D).
Categorization versus Feature Color Change Detection
After 3 days of training, the participants were scanned. We used an fMRI adaptation paradigm with two tasks: a categorization task and a feature color change detection task. Both tasks activated largely the same brain areas (occipitotemporal, parietal, and motor cortices [t(23) = 3.77, p < .001]; see Figure 7A). When comparing the categorization task with the feature color change detection task, we found that right frontal cortex was more active for the categorization task than for the feature color change detection task [t(23) = 3.77, p < .001] (see Figure 7A). We used an ROI analysis to test for effects of adaptation (Figure 7B) in the right inferior frontal gyrus (Figure 7C). We found that the inferior frontal gyrus showed adaptation when presented with two identical fish [SS vs. DD: t(23) = 4.73, p < .001], but more importantly only the responses to the informative features were adapted [F(1, 23) = 33.20, p < .001]. Changing the uninformative features did not alter the response [F(1, 23) = 0.88, p = ns]. This area was not active during the color change detection task (see Figure 7B). In addition, we tested whether there was an effect of distance of the fish to the category boundary on the responses of the right inferior frontal gyrus (see Figure 7D). Indeed, we found that responses were higher to those fish that were closer to the category boundary [F(2, 22) = 8.06, p < .005].
Because frontal cortex is usually associated with active categorization, we tested whether there was a category effect in this area (Figure 7E). We compared fish that had different informative features and belonged to different categories with fish that also had different informative features but belonged to the same category. We found that responses were higher for fish from different categories than for fish from the same category [F(1, 23) = 6.42, p < .05] and that it made no difference whether the uninformative features were the same or different [F(1, 23) = 1.01, p = ns]).
We were also interested in seeing whether responses in this area correlated with performance during training or with performance during scanning and whether this holds for the informative features or for the uninformative features. We found a significant correlation between categorization during scanning and mean beta weights (r = .46, p < .05; Figure 7F). Significant correlation was only found for sensitivity to the informative features and not for sensitivity to the uninformative features, and there was also no correlation with performance during training.
Next, we analyzed the two tasks separately.
First, we investigated areas that showed adaptation when presented with the same objects compared with different objects (SS < DD; Figure 8A). We found that this effect was driven entirely by the informative features (Figure 8B). No selectivity was obtained for the uninformative features. We extracted responses from the right occipitotemporal cortex from the area that was obtained by the SS < DD contrast—for this area, the effect driven by informative features and not by uninformative features (Figure 8C). There was no effect of distance [F(2, 22) = 0.93, p = ns] on the responses from the right occipitotemporal cortex (Figure 8D). Because we hypothesized that training increases the sensitivity to informative features, we found that there was indeed a correlation between training success and increased selectivity to informative features (r = .48, p < .05; Figure 8E). What is very interesting is that, for the uninformative features, we obtained the reversed effect. Training led to decreased selectivity to uninformative features (r = −.52, p < .01). We found no correlation of occipitotemporal sensitivity with categorization during scanning.
Because the area we found seems more located in the inferior temporal cortex than in the fusiform or lateral occipital gyrus, we compared its location to the results from our localizer. We contrasted faces with houses, which usually activates an area in the fusiform gyrus, the FFA, and we looked for object-selective cortex, usually found in the lateral occipital complex (LOC), by contrasting objects with scrambled pictures. Indeed our area falls in between the FFA and the LOC (see Figure 8F).
Feature Color Change Detection Task
No areas were found that showed adaptation to the features. We did find areas that showed adaptation for the same color (Figure 9).
After the fMRI experiment, the participants were asked to rate pairs of fish on their similarity (see Figure 10). No rating differences were obtained for the four different features (mouth, tail, dorsal fin, and ventral fin), so we collapsed them. As expected, participants were more likely to rate fish as more similar when there was a small physical difference between the features and as more dissimilar when the fish in a pair had greater physical difference between their features [F(3, 9) = 7.77, p < .01]. This distance effect was strongest for the informative features, as was revealed by an interaction between distance and feature type [F(3, 9) = 5.25, p < .05]. So pairs of fish were rated more similar when their informative features were close together in fish space and were rated as being more dissimilar when they were farther away in fish space [F(3, 9) = 6.61, p < .05]. In addition, informative features were rated as being more similar at Distance 1 than uninformative features at the same distance [t(11) = 4.17, p < .005] and more dissimilar than uninformative features at a distance of 19 [t(11) = −3.37, p < .01]. At Distances 9 and 10, there was no difference in rating between informative and uninformative features. The ratings of the informative features differed between Steps 1 and 9 [t(11) = 4.18, p < .005] and between Steps 10 and 19 [t(11) = 3.55, p < .01], but not between Steps 9 and 10.
We used a training paradigm in which participants learned to discriminate two categories of fish. These categories could be discriminated based on four features, but only two of the features were informative for category membership and two of the features were uninformative. Participants were able to categorize the fish after 3 days of training and performed best on fish that were farthest away from the category boundary. After training, participants were asked to rate fish on their similarity. As expected, informative features were rated more similar when they had a smaller physical difference and rated as being less similar when they had a greater difference. However, the ratings for uninformative features did not differ between small and great differences in features. Also, informative features with a small difference were rated as being more similar than uninformative features with the same difference. In addition, informative features with a greater distance were rated as being more dissimilar than uninformative features with the same difference. This indicates that training led to increased sensitivity to differences in features that were informative for categorization or a decrease in sensitivity for features that the participants were exposed to in the same amount but that were uninformative for categorization. Only the informative features were weighted when participants made a decision about the similarity of the fish. These findings are in line with those by other studies (Sigala, Gabbiani, & Logothetis, 2002; Schyns & Rodet, 1997), demonstrating the influence of categorization on perception and providing further evidence that perceptual features are formed in response to training task demands.
After three training sessions, the participants were scanned. We used an fMRI adaptation paradigm to test for selective responses to informative and uninformative features. Fish were presented in pairs that had identical or different informative and uninformative features. The idea is that if neuronal clusters are sensitive to a certain type of feature they will show a reduced response when this feature is repeated. We used two tasks, a categorization task and a feature color change detection task. We found that both categorization task and color change detection task activated largely the same areas. Only the frontal cortex was more active during categorization. Frontal cortex is assumed to be involved in active categorization in monkeys and humans (Jiang et al., 2007; Freedman et al., 2001, 2002, 2003), which fits with this result. We investigated the selectivity of responses in the right inferior frontal gyrus. We found that the area responded selectively to informative features. Responses in the right inferior frontal gyrus were reduced when informative features were repeated. It was irrelevant whether the uninformative features stayed the same or differed. Interestingly, responses in the right inferior frontal gyrus were greater for those trials that were close to the category boundary. This could mean that the frontal cortex is involved in representing category boundaries, but it seems more likely that this finding indicates categorization effort. Trials close to the category boundary are found to be more difficult to categorize, so more difficulty in categorization leads to more activity in this area. In addition, responses of the right inferior frontal gyrus were category specific. We found a dissociation between responses to fish from the same and different categories. Jiang et al. (2007) also found this in their study. However, this finding can also be explained by categorization effort. Our behavioral data showed that fish with different informative features had lower accuracy scores and longer RTs, another indication that they were more difficult to categorize when they belong to different categories than when they belong to the same category. Furthermore, we found a correlation of frontal adaptation scores with the behavior of the participants during scanning. Better categorization performance was linked with higher adaptation scores. For the uninformative features, no correlation was found, and also no correlation was found between performance during training and adaption effects in right inferior frontal gyrus. Together, these findings provide new and additive evidence that frontal cortex is indeed involved in active categorization with a focus on those features that facilitate categorization, while selectively ignoring those features that participants were exposed to in the same amount during training but that proved to provide no useful category information.
Occipitotemporal cortex is usually found to show training-related changes in activation (van der Linden, van Turennout, & Fernández, 2011; van der Linden et al., 2008; Moore et al., 2006; Op de Beeck et al., 2006). More specifically, categorizing stimuli has been shown to induce cortical selectivity in occipitotemporal cortex. These occipitotemporal areas included the fusiform gyrus (van der Linden et al., 2010) and the lateral occipital gyrus (Gillebert et al., 2009; Jiang et al., 2007). In this study, we show for the first time in human participants that this selectivity is experience dependent and not present for the entire trained object, but specific for those features that are informative for categorization. This finding bridges the gap between monkey electrophysiological recordings that showed selective responses to informative features (De Baene et al., 2008; Sigala & Logothetis, 2002) and human fMRI data that revealed increased and selective responses to trained objects (van der Linden et al., 2008, 2010; Gillebert et al., 2009; Jiang et al., 2007; Weisberg et al., 2007; Moore et al., 2006; Op de Beeck et al., 2006; Gauthier et al., 1998) and trained dimensions (Folstein et al., 2012). The area in occipitotemporal cortex that we found to be sensitive to informative features is a region that appears closer to the lateral part of the inferior temporal cortex than to the fusiform or lateral occipital gyrus. The area did not overlap with the LOC or the FFA. It did overlap to a small extent with an area that we localized to be responsive to natural fish stimuli. We did not localize the extrastriate body area, but based on the Talairach coordinates, the area showing adaptation to informative features is actually close to the extrastriate body area (Downing, Jiang, Shuman, & Kanwisher, 2001) albeit slightly more ventral. Overlap with the extrastriate body would seem logical as the informative features are all part of the fish's body.
Interestingly the occipitotemporal area showed on first glance similar responses as the inferior frontal cortex. However, a closer look into the data provided a different pattern of selectivity. For example, we did not find any relation between occipitotemporal responses and the distance to the category boundary nor did this area show category specificity. Also, performance during scanning had no influence on the selectivity of the occipitotemporal cortex. However, importantly, performance during training did relate to the sensitivity of the area. Participants who were better during training showed greater selectivity to the informative features, and what is most interesting is that this was paired with less selectivity for the uninformative features. This indicates that training can increase sensitivity to informative features but also that at the same time uninformative features are ignored. To take this a bit further, we might speculate that learning to categorize objects is an active process whereby an optimal sharpening of tuning could be achieved by increasing selectivity to informative features and by reducing or suppressing selectivity to uninformative features. This is similar to the interpretation of results that we proposed in an earlier study (van der Linden et al., 2008), where we suggested that the additional decreased responses to objects that were trained with random feedback (thereby obstructing category learning) were because of a suppression of the responses to those features that proved to be uninformative for categorization. However, for those stimuli, we could not disentangle the informative from the uninformative features. In this study, we separated the informative and uninformative features. However, this also provided a certain caveat. Separable features can be attended to separately, and for our fish stimuli, it is easy to attend to, for example, the fins and ignore the rest of the features—so-called selective attention. However, neuronal selectivity to features has and can only be found using separable features and not when using integral features (De Baene et al., 2008; Sigala & Logothetis, 2002; Op de Beeck, Wagemans, & Vogels, 2001). Although selective attention can be put forward as an explanation for adaptation to repeated presentation of identical informative features and not to uninformative features, there is no reason to assume that there would be more attention and therefore greater responses to fish features that are closest to the category boundary as was found in the inferior frontal gyrus. Occipitotemporal cortex did not show such an effect of distance to the category boundary, but this area showed a correlation with behavior outside the scanner; this is also something that cannot be explained merely by selective attention during categorization in the scanner.
Only during the categorization task did we find selective responses to informative features. We found no selectivity to uninformative features during categorization. Also during the color change detection task, no selectivity to any of the repeated features was present; the brain showed only adaptation when presented with features that had the same color. Although the behavioral data suggested that there was slight advantage for the informative features during the color change detection task, we found no evidence for the automatic cortical processing of the informative features. In the same vein, when car experts were performing a task for which car stimuli were task irrelevant, the expertise-related activity drastically dropped, becoming similar to the activity elicited by car stimuli in novices (Harel, Gilaie-Dotan, Malach, & Bentin, 2010). This supports the idea that selective responses that are found in the occipitotemporal cortex to objects of expertise reveal themselves only when participants are processing the objects' informative features when they are task relevant.
To conclude, we showed for the first time in humans that the informativeness of features for categorization is reflected in neuronal selectivity in occipitotemporal and inferior frontal cortex. Both areas seemed to play a different role. The inferior frontal gyrus was involved in active categorization of the stimuli, and the responses were modulated by categorization effort. The occipitotemporal cortex showed selectivity for informative features that was dependent on the performance of the participants during training. Success in training led to greater selectivity for the informative features in combination with less selectivity for the uninformative features. Importantly, these effects were highly task dependent.
The authors thank Hans Op de Beeck and Johan Wagemans for helpful comments on the research and manuscript. This research was supported by an NWO grant (400-03-338).
Reprint requests should be sent to Dr. Marieke van der Linden, Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition, and Behaviour, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands, or via e-mail: firstname.lastname@example.org.