Our sense of number rests on the activity of neurons that are tuned to the number of items and show great invariance across display formats and modalities. Whether numerosity coding becomes abstracted from local spatial representations characteristic of visual input is not known. We mapped the visual receptive fields (RFs) of numerosity-selective neurons in the pFC and ventral intraparietal area in rhesus monkeys. We found numerosity selectivity in pFC and ventral intraparietal neurons irrespective of whether they exhibited an RF and independent of the location of their RFs. RFs were not predictive of the preference of numerosity-selective neurons. Furthermore, the presence and location of RFs had no impact on tuning width and quality of the numerosity-selective neurons. These findings show that neurons in frontal and parietal cortices integrate abstract visuospatial stimuli to give rise to global and spatially released number representations as required for number perception.
In recent years, much progress has been made in understanding how primate brains process numerical quantity. A network of frontoparietal areas in the primate brain is implicated in processing nonsymbolic numerosity, the number of items in a set (Nieder, 2016a). Neurons in these areas respond selectively to the preferred number of items in an array during a number of discrimination task (Viswanathan & Nieder, 2013, 2015; Nieder & Miller, 2004; Nieder, Freedman, & Miller, 2002), a number of ongoing movements within a sequence (Sawamura, Shima, & Tanji, 2002, 2010), or a number of auditory tones (Nieder, 2012). Many numerosity neurons also abstract the presented numerosity from simultaneous and sequential presentation formats (Nieder, Diester, & Tudusciuc, 2006) and encode quantitative rules (Eiselt & Nieder, 2013; Vallentin, Bongard, & Nieder, 2012; Bongard & Nieder, 2010). These findings suggest that a frontoparietal network comprising intensely interconnected dorsolateral pFC and the ventral intraparietal area (VIP) of the posterior parietal cortex (Lewis & Van Essen, 2000) contains an abstract, generalized numerosity code.
At the same time, pFC and the VIP are part of the dorsal visual stream, the so-called where pathway of the brain in which primarily visuospatial information is encoded (Mishkin, Ungerleider, & Macko, 1983). As a consequence, both areas contain neurons that show great spatial selectivity. In the frontal lobe, pFC neurons represent the locations of stimuli as well as their identity (Rainer, Asaad, & Miller, 1998; Rao, Rainer, & Miller, 1997; Funahashi, Bruce, & Goldman-Rakic, 1989) and encode the position of an upcoming goal-directed movement (Shadlen & Newsome, 2001; Hasegawa, Matsumoto, & Mikami, 2000). In the parietal lobe, area VIP contains multimodal neurons that respond to visual, somatosensory, vestibular, and auditory stimuli (Schlack, Sterbing-D'Angelo, Hartung, Hoffmann, & Bremmer, 2005; Duhamel, Colby, & Goldberg, 1998; Colby, Duhamel, & Goldberg, 1993) and plays a crucial role in mapping inputs from different sensory modalities to a common reference frame (Bremmer, 2011; Avillac, Ben Hamed, & Duhamel, 2007; Avillac, Denève, Olivier, Pouget, & Duhamel, 2005). In addition, VIP neurons encode signals relevant to directions of self-motion (Bremmer, Schlack, Graf, & Duhamel, 2004; Schlack, Hoffmann, & Bremmer, 2003). In summary, both pFC and VIP are highly multimodal and associative in nature and encode spatial as well as nonspatial and cognitive attributes (Wimmer, Spinelli, & Pasternak, 2016; Freedman & Assad, 2009).
A prime feature of spatial selectivity is a neuron's receptive field (RF) that dictates its responses in visual space (Viswanathan & Nieder, 2017a, 2017b). From a perceptual point of view, however, encoding the number of items distributed across visual space would require sensory integration beyond the classical RF of a neuron. After all, the enumeration of a spatially dispersed set of items requires an abstraction of quantity from space. The extraction of numerosity from dot arrays therefore is a classic condition to investigate spatial integration processes. Currently, it is not known whether neuronal selectivity to numerosity is confined to set items shown within the classical RF or whether visual information outside the RF becomes integrated to give rise to global numerosity selectivity. We addressed this fundamental question by separately (1) mapping the response of individual neurons to space and (2) evaluating these individual neurons' selectivity to numerosity. Specifically, we asked whether and how selectivity to numerosity depends on a neuron's classical visual RF. To that aim, we compared the locations of visual RFs of neurons within pFC and VIP in the macaque monkey and those neurons' selectivity to numerosity as the monkeys performed a delayed match-to-sample (DMS) task (Nieder & Miller, 2004).
Experimental Model and Subject Details
Two adult male rhesus monkeys (Macaca mulatta), Monkey L and Monkey S, served as subjects for this experiment. They weighed between 5.5 and 6.3 kg for the duration of this study. During experiments, the monkeys sat in primate chairs within darkened experimental chambers. The primate chairs were positioned 57 cm from a flat screen monitor (15 in., 1024 × 768 display resolution at 75 Hz), which was used for stimulus presentation. The monkeys received fluid rewards, the amount of which was controlled by a valve, upon successful completion of each trial. All experimental procedures have been reported previously (Viswanathan & Nieder, 2013, 2015, 2017a, 2017b) and were in accordance with the guidelines for animal experimentation approved by the local national authority, the Regierungspräsidium, Tübingen, Germany.
We trained both monkeys using the NIMH Cortex program to present the stimuli, monitor their behavior, and collect their responses. We used an infrared eye-tracking system (ISCAN). We used a passive fixation (PF) task to map the RFs of single neurons. The blocks of PF task were interleaved with blocks of the DMS task. For each trial, monkeys fixated a central fixation spot as a moving bar appeared at five successive locations. The moving bar was gray in color and had the dimensions of 3° of visual angle in length and 0.20° in breadth. The bar appeared at each position in two orientations moving at a constant speed of 8°/sec in four directions, first oriented vertically moving left and right, then oriented horizontally moving up and down. Each position was explored for 1000 msec, and five positions were tested in each trial (see Figure 1A), such that the whole screen (30.5° × 23°, 80 positions) could be sampled over 16 such trials.
To screen for numerosity selectivity in the neurons, the monkeys performed a DMS task (see Figure 1B) with two kinds of discriminative stimuli: colors and numerosities (see Figure 1C). Each trial began when the monkeys fixated within 1.75° of a central fixation spot for 500 msec, a sample display then came on for 800 msec and disappeared thereafter giving way to a delay period of 1000 msec. Following this, a test display was shown for 1200 msec. During the color discrimination task, the monkeys were trained to respond to test stimulus if it matched the color of the sample stimulus that appeared at the beginning of the trial. The monkeys were then trained to discriminate the stimuli based on the number of colored dots and respond to the test stimulus if it matched the sample stimulus in the number of items. If a nonmatch test display was shown, the monkeys had to withhold their response until a match stimulus appeared. We presented the five colors (purple, yellow, green, blue, and red) in all five numbers (one to five dots) for the color discrimination task. For the numerosity discrimination task, we presented one to five dots in black color. New stimulus images were created for every session with the dots placed at randomized positions and of randomized sizes (“standard” set). To ensure that the lower visual features, like colored area and the density of display, which co-vary with increasing number of dots in randomly generated “standard” images, did not influence the monkeys' behavior, we created an equivalent set of “control” stimuli that equated the colored area and density across numerosities within the 6° gray background circle. At least 20 images of standard and equivalently 20 images of control stimuli were used per numerosity. Trials were presented either with “standard” numerosity or “control” numerosity. The various factors, color in the first task, numerosity, stimulus protocol (“
Surgery and Electrophysiological Recordings
We implanted the monkeys under general anesthesia with a head bolt in the early stages of behavioral training. The head bolt enabled us to immobilize the head during training and recording sessions and monitor their eye movements. After their behavioral performance reached an average of 80% correct discrimination of stimuli, we implanted recording chambers over the dorsolateral pFC, centered on the principal sulcus, and area VIP in depths of the intraparietal sulcus guided by anatomical MRI and stereotaxic measurements (see Figure 2). Both chambers were implanted in the right hemisphere of both monkeys. The surgical procedures were performed under sterile conditions while the monkeys were under general anesthesia and received postsurgical care.
The electrophysiological signals were recorded using arrays of eight glass-coated tungsten electrodes (Alpha-Omega Engineering) for each area. Two electrodes were attached to each screw microdrive and manually lowered to recording depths. No prescreening was performed to select neurons for recording, and typically only one depth/electrode was sampled in a session. The signals were amplified and filtered, and the waveforms of the action potentials sampled at 40 kHz were stored (Plexon Systems) so that they could be sorted off-line. pFC recordings were made superficially at depths of 3–4 mm. VIP recordings were made exclusively at depths of 9–13 mm below the cortical surface. We observed visual, auditory, and somatosensory responses on VIP electrodes that distinguished them from neighboring divisions. We recorded 1186 pFC (507 from Monkey L, 679 from Monkey S) and 944 VIP (388 from Monkey L, 556 from Monkey S) neurons from 52 sessions of color discrimination (23 with Monkey L, 29 with Monkey S) and 60 sessions of numerosity discrimination (33 with Monkey L, 27 with Monkey S), both combined with RF measurements.
Experimental Design and Statistical Analysis
We extended analyses on neuronal recordings that constituted the data set for previous publications about RFs alone (Viswanathan & Nieder, 2017a, 2017b) or numerosity coding alone (Viswanathan & Nieder, 2013, 2015). We analyzed all the neurons that were recorded for at least two trials per position during the PF blocks and 30 correct trials per sample numerosity during the DMS blocks. An additional criterion of a minimum firing rate of 1 Hz in the period from fixation to delay during the DMS trials were imposed on the neurons. For the current study, we analyzed responses in 389 pFC neurons and 338 VIP neurons that passed the criteria for both blocks. Except for the time-resolved analyses, we used a default latency of 50 msec for VIP neurons and 100 msec for pFC neurons to place our analysis windows. The default latency values were based on well-documented response latencies in these two areas (Nieder & Miller, 2004). All neuronal analyses were performed only on correct trials; trials with fixation breaks and matching errors were excluded from all analyses. All data analysis was performed using custom-written scripts in the MATLAB computational environment (The MathWorks).
We have described the procedure of creating RF maps in detail before (Viswanathan & Nieder, 2017a, 2017b). In brief, we selected visually responsive neurons on the basis of a three-factor ANOVA with the factors Position, Direction, and Orientation on firing rates collected in 250 msec windows. For the neurons with a main effect of Position (p < .05), we created RF maps by averaging the responses at each position, interpolating the responses and smoothing the responses in two dimensions with a Gaussian function. We then checked that these RF maps were consistent across trials by creating two maps for each neuron with half the trials: the first half (H1) and the second half (H2). These halves were typically separated by blocks of the DMS task. We compared the correlation between the first and second maps against a distribution of correlations obtained from shuffling the first and second maps (one-tailed, p < .05). For each map, we shuffled the association between firing rates and the spatial location 1000 times to create 1000 shuffles of H1 and H2 each. The distribution of cross-correlations of shuffled maps was centered around zero, and the true cross-correlation of H1 and H2 was to lie above the 95th percentile of the shuffled distribution to show significant consistency across time and trials. After this test was passed, we obtained RF maps for 65% (252/389) pFC neurons and 62% (210/338) VIP neurons. We normalized each map to its maximum across all positions. The RF of each cell was defined as the area where the neuron exhibited more than half-max activity. Average maps across groups of neurons were created by averaging these normalized maps (see Figure 3).
Visual Responsiveness during Discrimination Task
Based on the kinds of spatial selectivity in the population of neurons, we could classify three types: neurons whose RFs overlap with the DMS stimuli (RF-in), neurons whose RFs do not overlap with the center where the DMS task stimuli were always shown (RF-out), and neurons that do not show a significant RF (no RF). As the stimuli in the DMS task were always displayed centrally around the fixation target, if the RF covered the area occupied by the gray circle or if the RF maxima occurred inside the gray circle, these neurons were classified as “RF-in.” The eccentricities of the RFs of the “RF-out” and RF-in classes were significantly different, as verified with a Mann–Whitney U test, p < .01.
Numerosity Selectivity in Single Neurons
We have described the procedure for testing the neurons for stimulus-selective responses in detail (Viswanathan & Nieder, 2013, 2015). For the color discrimination task, we tested neurons for color-selective, numerosity-selective, and stimulus protocol-selective responses with a three-factor ANOVA. For the numerosity discrimination task, we tested neurons with a two-factor ANOVA for the stimulus features of numerosity and protocol. We did this for the sample phase with firing rates collected in 800-msec windows with the area-appropriate offsets and for the delay phase with firing rates collected in the last 800 msec of the delay phase with the area appropriate offsets. The neurons that showed a main effect of Numerosity (p < .01) but not an interaction with the stimulus protocol (p > .01) were thus termed “numerosity-selective neurons” (see Figure 4). Numerosity tuning functions were created by averaging their responses to the corresponding sample number across all trials (for instance, toward all samples with three dots) in the specified analysis window. To determine if the RFs recorded during the PF task were similarly responsive during visual events in the DMS task (color and number matching), we examined the neuronal activity across time in the DMS task (see Figure 5). We constructed a peri-event histogram of averaged responses from neurons belonging to each class. Average firing rates in each task epoch across RF classes were compared with a Mann–Whitney U test. We also compared spontaneous firing rates across RF classes in the 150 msec before fixation circle onset and separately in the 150 msec at the end of the trial when the background circle disappears with a Kruskal–Wallis test.
Comparison of Numerosity Selectivity across RF Classes
We compared the proportions of the three classes of neurons in our recorded population and those that are selective for the various task variables using the χ2 test of independence (see Figure 6). By chance, the frequency of selective neurons in each RF class follows the frequency of selective neurons in the entire population. The probability of numerosity selectivity was described as the proportion of numerosity neurons to total neurons belonging to a certain RF class (category), having an RF at a certain distance from the center (eight bins), or having a certain RF size (eight bins). These values are then fitted with a linear regression model. All correlations reported in this article are Pearson's linear correlation. For the numerosity-selective neurons, we created average tuning curves by centering individual tuning curves at the preferred numerosity and expressing the responses to the other numerosities as the numerical distance from the preferred numerosity. The response to the preferred numerosity was considered as 100% and to the least preferred numerosity as 0%. All other values were normalized to this scale (see Figure 7).
We calculated a receiver operating characteristic (ROC) curve according to signal detection theory (Green & Swets, 1966) to evaluate the strength of numerosity discrimination by the selective neurons. We considered the response (in firing rates during the sample phase) toward the most preferred numerosity as the true positive rate and the response toward the least preferred numerosity as the false positive rate. Using these, we generated an ROC curve for each neuron, and the area under the ROC curve (AUROC) was an indicator of how well the neuron discriminated between its preferred and least preferred numerosity. A value of 0.5 indicated chance-level discrimination, and a value of 1 indicated ideal discrimination (see Figure 7). For a time-resolved analysis, we calculated the AUROC with a sliding window of 50 msec slid by 1 ms (see Figure 8). We used these values to compute the numerical discrimination latency for individual neurons. The latency was described as the first of 50 consecutive windows with an AUROC value greater than 3 SDs of AUROC calculated in the last 200 msec of the fixation epoch. We tested these values across RF classes with a Kruskal–Wallis test.
We recorded the activity of 389 neurons in the dorsolateral pFC around the principal sulcus and 338 single neurons in area VIP in the depths of the intraparietal sulcus in two rhesus monkeys. These recordings were made across two conditions: First, we mapped the visual RFs of individual neurons, and second, we tested their selectivity for visual numerosity in a DMS task.
The details of the RF mapping procedure together with the characterizations of RFs found in both brain areas has been reported previously (Viswanathan & Nieder, 2017a, 2017b). In brief, although the monkeys fixated a central fixation target in a PF task, the visual field corresponding to the entire screen was mapped using moving bars placed systematically at 80 positions (Figure 1A). RF selectivity of single cells was ascertained with an ANOVA (p < .05) and validated by a cross-correlation analysis of the RFs across halves of the recording session (see Methods for details).
Similarly, the details of numerosity stimuli, the behavioral performance of the monkeys, and the neuronal response patterns of neurons to sample numerosity during the DMS task are described in detail elsewhere (Viswanathan & Nieder, 2013, 2015). In short, although the monkeys fixated a central fixation spot, different numbers of dots were presented on a gray background circle spanning 6° of visual angle in the center of the screen (Figure 1B). The monkeys had to memorize the dots during a delay period and respond to following test display(s) if they matched in number. In the color DMS task, monkeys had not been trained to discriminate number, so they matched the color (five colors) of the dots, whereas in the numerosity DMS task, the monkeys were trained to match the number (1–5) of black dots (Figure 1C). To measure both the RFs and the numerosity selectivity of individual neurons, the PF task and the DMS task were interleaved within a given recording session (Figure 1D–E). Numerosity selectivity was tested separately during the sample or delay periods, respectively, using an ANOVA (p < .01; see Methods for details). As described previously, numerosity-selective neurons discharged maximally to one of the five presented numerosities, a neuron's preferred numerosity (Nieder, 2016a). This was true both in monkeys discriminating dot color (Viswanathan & Nieder, 2013) and in numerically trained monkeys judging the number of dots (Ramirez-Cardenas, Moskaleva, & Nieder, 2016; Viswanathan & Nieder, 2015; Jacob & Nieder, 2014). In the current study, 13% (52/389) of pFC neurons (Figure 2A) and 11% (36/338) of VIP neurons (Figure 2B–C) were exclusively selective for the number of items, irrespective of their overall area or density. These are the neurons we describe as numerosity neurons.
Visual RFs and Numerosity Selectivity in the Association Cortex
Because the RF of a neuron determines the spatial location from which it receives its input, our first interest was to see if the location of the visual RFs in these association areas determined neuronal selectivity to numerosity. Our approach is ideally suited to address this issue because we mapped the RFs separately and did not vary the position of the numerosity stimuli within the task. Neurons were classified according to the location of the RFs into three classes: First, neurons whose RFs overlapped with the location of the numerosity display (outlined by the gray background circle) that was always presented in the center of the screen belonged to the “RF-in” class (Figure 3A–B). Second, neurons whose RFs were outside the location of the numerosity stimulus belonged to the “RF-out” class (Figure 3C–D). Third, neurons for which no RF could be determined were sorted to the “no-RF” class (Figure 3E–F). Of all the randomly selected neurons we recorded in pFC, we found 17% (66/389) belonged to the RF-in, 48% (186/389) neurons belonged to the RF-out class, and 35% (137/389) neurons belonged to the no-RF class. In VIP, 24% (80/338), 38% (130/338), and 38% (128/338) belonged to the three classes (RF-in, RF-out, no-RF), respectively.
Mapping the RFs of numerosity-selective neurons showed that some of the numerosity-selective neurons in pFC and VIP had RFs that covered or partly overlapped with the location of the numerosity display (Figure 4A–B). Surprisingly, however, other neurons were numerosity-selective even when the RF was extrafoveal and thus remote from the numerosity display (Figure 4C–D). A third set of numerosity-selective neurons had no visual RFs at all (Figure 4E–F). Neurons in all three classes exhibited peak-tuned responses to the sample numerosity.
Despite this disparity between RF location and numerosity selectivity in individual neurons, the location of visual RFs had a profound impact on the visual responses of the neurons even during the DMS task in both pFC and VIP (Figure 5A–B). We observed higher firing rates in the RF-in neurons from the fixation phase to the delay phase, indicating a consistent response to visual stimulation within the RF. RF-in neurons showed a higher firing rate even in fixation because of the presence of the gray circle within which the numerosity stimuli were subsequently presented. The onset of colored dots stimulated RF-in neurons more strongly compared with the RF-out and no-RF neurons. Despite the RF mapping and DMS task being separated into blocks, neuronal response toward the gray bar in RF strongly predicted the response to gray background circle within the DMS task. In periods without the background circle, the period directly preceeding fixation onset and the period at the end of the trial, absolute firing rates tended to be lower (average firing rate for RF-in = 7 Hz) than when the background circle was present, but still significantly different across RF classes (Kruskal–Wallis, p < 10−5).
Frequency of Numerosity-selective Neurons by Classes of Visual RF
To determine whether the RF location predicts neuronal selectivity to numerosity, we computed the relative proportions of numerosity-selective neurons in each class (Figure 6, pie charts). Numerosity selectivity was found in all three RF classes at levels greater than expected by chance (1% chance for alpha of .01). In pFC, 26% (17/66) of RF-in neurons were selective for the number of dots in the sample phase, but as many as 14% (26/186) of RF-out neurons and 7% (9/137) of no-RF neurons were also number-selective. In VIP, 16% (13/80) of RF-in neurons were selective for numerosity and comparable proportions of RF-out (12%, 16/130) and no RF (5%, 7/128) were also numerosity-selective. In both areas, the proportions of numerosity selectivity were higher for neurons characterized by an RF (either RF-in or RF-out) than by no RF. In pFC, RF-in neurons were more likely to be recruited for numerosity selectivity than the other classes (χ2 = 16.309, p = .0003), whereas in VIP, both RF-in and RF-out neurons processed numerosity more often than the no-RF neurons (χ2 = 6.6225, p = .0365). We compared the distributions of preferred numerosities as a function of RF (Figure 6, bar plots). The preferred numerosity is defined as the sample number eliciting the maximal response of the numerosity-selective neuron. In both areas, pFC and VIP, neurons preferring the different sample numerosities are distributed across RF classes. This indicates that RF classes are not predictive of the preferred numerosity of the neuron.
We quantified the percentage of numerosity neurons across all neurons with RFs (RF-in and RF-out) to compare the effects of the distance of the RF from the numerosity stimuli and RF size. This analysis of RF distance followed the categorical differences captured in the pie charts (Figure 6A–F). We observed that RF distance, but not RF size, had an effect on the probability that a given neuron would be selective to the numerosity stimuli (Figure 6G–H). The percentage of numerosity selective neurons decreased with increasing RF distance from stimuli in both areas (pFC, p = .0047 with a slope of −0.0135; VIP, p = .0003 with a slope of −0.0138). RF size, on the other hand, did not significantly influence the percentage of neurons selective to the centrally presented numerosity (pFC, p = .1514; VIP, p = .1260; Figure 6I–J).
Quality of Numerosity Selectivity
As described above, numerosity neurons with different types of spatial selectivity displayed a preference for a certain numerosity. We next compared each neuron's response to its preferred numerosity against its response to its preferred location. For RF-in neurons, because the preferred numerosity overlaps the preferred location, these responses are expected to be correlated. For RF-out and no-RF neurons, as the numerosity stimuli are not the preferred location, this comparison tells us the extent to which the DMS task influences the RF. In pFC, we found a strong correlation in these responses across all RF classes (Figure 7A) even though responses to preferred numerosity were always smaller than those to the bar in the preferred location (all RF classes: Pearson's correlation, r > .7, p < .01). In VIP, firing rates to the preferred numerosity were on occasion larger than the responses to the preferred location (Figure 7B). Although firing rates to preferred number were strongly correlated with firing rates to preferred location in RF-in neurons (r = .8751, p < .01), they were not correlated in RF-out or no-RF neurons (p > .1). First, this analysis confirmed the suitability of our mapping procedure to assess the influence of RFs on numerosity selectivity because the responses in both conditions are highly correlated. Second, it showed an interesting difference between pFC and VIP neurons. pFC numerosity neurons responded to numerosity stimuli presented outside the RF as they would to their preferred location (RF); in contrast, VIP neurons maintained their spatial preference even during the DMS task (uncorrelated responses in RF-out and no-RF class).
We next examined if and how the presence or the position of visual RFs affected the quality of selectivity to numerosity. To that aim, we evaluated the sharpness (i.e., the width) of the numerosity tuning curves as a function of RF. We created averaged numerosity tuning curves by plotting the tuning curves as a function of numerical distance from the preferred numerosity (numerical distance = 0) for each neuron. These average tuning curves were created separately for each of the three RF classes (Figure 7C–D). The tuning curves for each RF class were strikingly similar, with considerable overlap, regardless of RF presence or location (Kruskal–Wallis, p > .05 for all but numerical distance −4 in VIP). This held true in both pFC and VIP.
In addition, we compared the neurons' tuning quality to discriminate the preferred from the least preferred numerosity using signal detection theory (Green & Swets, 1966). Thus, we used the AUROC as a measure independent of the absolute firing rates of individual neurons. For each selective neuron, we calculated how well separated the distributions of firing rates elicited by a neuron's preferred numerosity and least preferred numerosity were (Figure 7E–F). Across the RF classes, AUROC values of numerosity neurons were indistinguishable in both areas. In pFC, median values of AUROC were well above chance and hovered between 0.69 and 0.71 (Kruskal–Wallis, χ2 = 3.68, p = .1591). In VIP as well, AUROC values were distributed between 0.6 and 0.85 with medians around 0.7 for all three RF classes (Kruskal–Wallis, χ2 = 3.27, p = .195). Additionally, we tested for the presence of a correlation between the AUROC and the distance of the RF from the numerosity stimulus (Figure 7G–H). In pFC, RF-in and RF-out neurons displayed no significant relationship between the strength of their selectivity and the RF location (r = −.17, p = .1679). RF location and AUROC in VIP neurons were similarly uncorrelated (r = −.04, p = .7885). In summary, the presence and location of RFs had no impact on tuning width and tuning quality of the numerosity-selective neurons.
Latency of Numerosity Selectivity
The presence and location of the RFs might influence the response latency of numerosity-selective neurons, as the visual RFs of a neuron indicate the area of space from which the neuron directly receives visual information. We therefore investigate whether numerosity selectivity within or outside RFs might arise based on different computations (such as fast bottom–up vs. slow top–down computations). We determined putative differences in the latency of numerosity discrimination as a function of the RFs. To that aim, we perform a time-resolved analysis of selectivity within a sliding window of 50 msec moved along the sample epoch in 1-msec steps (Figure 8A–B). In both areas, RF-in neurons show a short-lived enhancement in selectivity when compared with the other RF classes (Kruskal–Wallis, p < .01).
Next, we determined the latency of numerosity selectivity, that is, the time point when the neurons became selectively tuned to numerosity after display onset. Note that this measure is not equivalent to the visual response latency of the neurons. The selectivity latency was determined as the first of 50 consecutive time windows showing an AUROC value of greater than 3 SDs of the fixation period AUROC. We observed that pFC neurons had a nonsignificant tendency to discriminate numerosity at different latencies according to RF class (Figure 8C): “RF-in” neurons had a median of 92 msec, “RF-out” neurons had a median of 161 msec, and “no RF” had a median of 294 msec (Kruskal–Wallis, χ2 = 4.88, p = .087). In VIP, however, no such tendency was found for the latency to numerosity discrimination (Figure 8D): “RF-in” had a median latency of 76 msec, “RF-out” had a median latency of 104 msec, and “no RF” neurons had a median latency of 125 msec (Kruskal–Wallis, χ2 = 0.9, p = .64).
In summary, we mapped the RFs of neurons in the primate magnitude network and quantified the influence of RFs on the neurons' selectivity to numerosity. Notably, neurons in both pFC and VIP were numerosity-selective even when the numerosity was displayed outside their RFs, though the frequency of numerosity-selective neurons was highest when the numerosity was shown at the location of the RF. The strength of neuronal selectivity to numerosity was independent of RF location. These results provide a number of insights into how numerosity is processed in the magnitude network.
Investigations into the magnitude network have largely ignored the interactions between the neuronal RF and their responses to numerosity. Numerosity stimuli were presented at the fovea and elicited selective responses in 20–40% of neurons in frontal and parietal areas (Nieder & Miller, 2004; Nieder et al., 2002). Because the RF of a neuron describes the region of sensory space within which a stimulus modulates the neuron's response (Hubel & Wiesel, 1962), presenting the numerosity stimuli within neuronal RFs was expected to elicit selectivity more reliably. Indeed, when numerosity stimuli were exclusively presented in neuronal RFs in a study of implicit numerosity detection, selective responses were found in about 60% of lateral intraparietal (LIP) neurons (Roitman, Brannon, & Platt, 2007), where about 85% of the neurons were selective for the smallest or largest numerosity. However, the relationship between neuronal RFs and their selectivity to numerosity had never been tested. In our study, we tested this for the first time by examining different populations of neurons: RF-in, RF-out, and no RF and their selectivity to numerosity. A major result of our study is that the presence or location of a neuron's RF does not predict numerosity selectivity. Visual information outside the RF becomes integrated to give rise to numerosity selectivity.
Human psychophysics provided the first hint of the link between numerosity perception and neuronal RFs. Numerosity, like other visual properties of color, size, and distance, is susceptible to perceptual adaptation. When a numerosity (called the “adapter stimulus”) is presented repeatedly before a test stimulus, adaptation causes a participant to misjudge the test numerosity: Adaptation to a large numerosity biases participants to underestimate the numerosity of the test stimulus, whereas adaptation to small numerosity caused overestimation (Ross & Burr, 2010; Burr & Ross, 2008). Sensory adaptation effects are said to arise from stimulus-selective neurons adapting to the stimuli repeatedly displayed in their RFs, diminishing the ability of these neurons to respond accurately to the subsequent test stimulus presented in the same location. In the case of numerosity, this diminished neuronal response results in skewing the behavioral estimation of the test numerosity away from the adapter numerosity. Although the link between adaptation and physiology has not been causally demonstrated, it is well supported by findings of cross-format, cross-modality adaptation and, in parallel, abstract numerosity neurons selective for numerosity presented in different formats and modalities (Anobile, Cicchini, & Burr, 2016; Nieder, 2016a, 2016b).
Recent studies have used the phenomenon of adaptation to test the relationship between numerosity and the location of the presented dot arrays, particularly whether adaptation occurs in retinotopic (eye-centered) or spatiotopic (world-centered) coordinates. When a particular feature is extracted by retinotopically organized early visual areas such that neuronal RFs move when the eyes move, adaptation to that feature is strongest when the adapter and test stimuli are in the same retinotopic position (Durgin, 2008). Whether adaptation to numerosity is affected by its spatial location was tested by presenting the adapter stimuli and test stimuli at specific spatial locations. Subjects were asked to move their eyes between the presentation of the adapter and the test stimuli, such that they could either be in the same retinotopic position (related to eye position) or the same spatiotopic position (related to head/world position). This study showed that adaptation to numerosity was strong when the adapter and test were in same spatiotopic coordinates but extremely weak when they shared the same retinotopic location (Arrighi, Togoli, & Burr, 2014). Thus, the adaptation effects do not simply rely on repeated stimulation within a neuron's classical RF.
These adaptation results predict that numerosity is processed primarily by spatiotopically organized visual areas, that is, by areas in which neuronal RFs are linked to body or head position, rather than eye position. Alternatively, retinotopic adaptation could also be weakened by nonspatial coding of information, that is, numerosity selectivity outside neuronal RFs. Our RF mapping procedure allowed us to test the latter prediction and explain why adaptation is weak in eye coordinates. We mapped the RFs of association cortex neurons based on the eye position, and neurons with selectivity to numerosity presented outside their RFs explain why eye position alone does not determine their responses. Moreover, we have recently reported data that support the first prediction. Visual RFs in primate association areas are not organized topographically, such that neighboring neurons received information from different spatial locations (Viswanathan & Nieder, 2017b). These areas are known to be organized in body-centered rather than eye-centered coordinates (Zirnsak, Steinmetz, Noudoost, Xu, & Moore, 2014; Duhamel, Bremmer, Ben Hamed, & Graf, 1997; Duhamel, Colby, & Goldberg, 1992). Thus, spatiotopic adaptation effects could be driven by the organization within these areas. It seems that a greater independence from classical RFs and spatiotopic organization distinguish association cortex from early visual areas and could drive numerosity adaptation effects in conjunction.
Our second result is that the strength of selectivity to numerosity is similar irrespective of whether the numerosity display and the RF of a neuron overlap or not. Categorical responses to nonnumerical visual stimuli outside the RF have been reported before. Studies into the nonspatial encoding of categories in cortex have traditionally examined the responses of the same neurons to stimuli presented within and outside their individual RFs. For instance, when motion stimuli were presented to LIP neurons within their RFs during a categorization task, the neurons showed a strong visual response to the stimuli. The same neurons responded much more weakly to stimuli shown away from their RFs (Freedman & Assad, 2009). However, responses selective to the learned motion categories (dots moving up/left vs. dots moving down/right) were much more comparable irrespective of whether the moving dot stimuli were presented within the neurons' RF. Similarly, when LIP neurons were studied during another motion categorization task involving saccadic responses toward or away from the neurons' RF, neuronal selectivity to motion categories was found to be strong regardless of their RFs (Rishel, Huang, & Freedman, 2013). Selectivity for the nonspatial information (motion category) existed in these neurons largely independently of the spatial information (upcoming saccade: toward RF or away from RF). So much so that inactivation of LIP affected spatial signals but preserved abstract nonspatial information relevant for the ongoing tasks (Balan & Gottlieb, 2009). These nonspatial signals in LIP were thought to represent feedback signals from other areas playing a greater role in processing nonspatial information. In this distributed fashion, association cortex might play a central role in achieving abstraction from space (Freedman & Assad, 2016; Gottlieb & Snyder, 2010). Our data highlight such an abstraction from space for global numerical stimuli.
Lastly, we found that pFC neurons had a tendency to discriminate numerosity earlier when the numerosity was presented within their RFs, but such a tendency was absent in VIP. Latency differences can inform us how information flows along a hierarchy of areas, but also whether information is primarily processed hierarchically (short latencies) or via horizontal connections (long latencies) within a functional area. In visual area MT, when motion stimuli were presented outside the RF, neuronal selectivity for the stimuli was slower relative to when they were presented within the neurons' RF (Zaksas & Pasternak, 2005). This latency difference indicated that another area downstream might be the source of this modulation. In a similar task, pFC neurons were found to represent motion signals across the visual field with similar latencies (Wimmer et al., 2016). This places pFC in a position to provide top–down modulation to an MT neuron for stimuli presented outside its RF. In demanding tasks with category boundaries that need to be learned flexibly, pFC has been directly implicated in providing top–down input to inferotemporal and parietal areas (Crowe et al., 2013; Chafee & Goldman-Rakic, 2000; Tomita, Ohbayashi, Nakahara, Hasegawa, & Miyashita, 1999). We did not find a significant difference in latencies in pFC across RF classes (possibly because of our relatively small sample size), but such a difference might indicate a different direction of information flow for numerosity estimation, that is, bottom–up information about numerosity from parietal areas like VIP.
VIP is considered upstream to pFC in the numerical magnitude system. VIP neurons have been shown to differentiate between numerosities earlier than pFC (Viswanathan & Nieder, 2013; Nieder & Miller, 2004). Our finding of shorter latencies in VIP neurons lends further support to this hierarchy. Other posterior parietal areas like LIP have displayed a similar role in processing motion. LIP neurons displayed selectivity to motion category at similar latencies irrespective of whether the stimuli were presented within the neuron's RF or in the opposite hemifield (Freedman & Assad, 2009). LIP neurons also reflected well-learned motion categories earlier than pFC neurons (Swaminathan & Freedman, 2012). Numerosity, being a natural category, might be extracted in parietal areas before being transmitted to pFC in a bottom–up manner. Further study will be required to determine how this network transmits numerical information and whether the direction of information flow is changed when greater flexibility or cognitive control is required.
Together, these results describe some distinguishing features of association cortex, particularly in processing a natural category like numerosity. Current computational models of numerosity estimation do not capture these features of association cortex. Whether they use a small number of dots or a dense display of numerosity, the relevant layers of neural network models have access to the entire stimulus (Nasr, Viswanathan, & Nieder, 2019; Zorzi & Testolin, 2018; Stoianov & Zorzi, 2012). Future studies should explore where in the hierarchy of the magnitude network independence from visual RF first emerges. It might relate to the processing stage where single neurons become numerosity-selective rather than populations of neurons (DeWind, Park, Woldorff, & Brannon, 2019; Park, DeWind, Woldorff, & Brannon, 2016). Alternatively, the RF independence might relate to the transition from retinotopic to spatiotopic visual processing in the brain (Chen, DeAngelis, & Angelaki, 2018; Melcher & Morrone, 2015).
This work was supported by DFG grant NI 618/2-1 to A. N.
Reprint requests should be sent to Andreas Nieder, Animal Physiology, Institute of Neurobiology, Auf der Morgenstelle 28, University of Tübingen, 72076 Tübingen, Germany, or via e-mail: firstname.lastname@example.org.