We introduce a neural model capable of feature selectiveness by spike-mediated synchronization through lateral synaptic couplings. For a stimulus containing two features, the attended one elicits a higher response. In the case of sequential single-feature stimuli, repetition of the attended feature also results in an enhanced response, exhibited by greater synchrony and higher spiking rates.
Over the past few decades, there has been an ever increasing interest in the neural mechanisms of visual attention. Researchers ceaselessly wonder how visual attention affects spatial and temporal sensitivity, how it influences the selection of relevant stimuli, how responses are modulated, which computations implement these processes, and what sort of interplay between attention and eye movements should be expected (further details and illustrations can be found e.g. in Desimone & Duncan, 1995; Itti & Koch, 2001; Carrasco, 2011).
Three main types of visual attention have been defined: (1) spatial attention, which can be overt, when observers move their eyes to a relevant location and the focus of attention coincides with the movement of the eyes, or covert, when attention is deployed to relevant locations without accompanying eye movements; (2) feature-based attention (FBA), which can be directed covertly to specific aspects (e.g., color, orientation, or motion direction) regardless of space locations (see Herrmann, Heeger, & Carrasco, 2012); and (3) object-based attention, guided by object structure. Whereas spatial attention guides observers to particular locations, FBA leads them to particular features in the visual scene (object recognition usually requires much richer sets of detectors than FBA; see the comments in Hamker, 2006).
Selective attention arises from the brain’s limited capacity to process vast amounts of sensory information (Attwell & Laughlin, 2001; Lennie, 2003) and enables us to gather the relevant part of these inputs, ultimately guiding our behavior. In FBA, the fact of attending to one feature biases the population activity toward that attended feature, that is, toward the behaviorally relevant stimulus, at the expense of behaviorally irrelevant ones. FBA is very helpful because we are often acquainted with the defining feature of an object without knowing where it will appear.
Concerning mechanisms, FBA appears to involve several visual areas where it modulates activity in response to expectations about task-relevant features. In the case of spatial attention, these areas include V1, MT (see Watanabe et al., 1998) and V2, V4, IT (see Fries, Womelsdorf, Oostenveld, & Desimone, 2008). Bosman et al. (2012) studied interareal influences involving V1 and V4.
In the work we present here, we try to construct a model for selective visual attention based on orientation-coded features. The introduced structure incorporates Izhikevich spiking neurons (Izhikevich, 2003, 2007), which are convenient for considering rate differences, firing regimes, and possible latency effects. From these constituents, we wish to study the possibility of obtaining feature-selective responses based on a particular form of lateral coupling among cell stacks capable of producing firing regimes with approximate synchronization. The network structure and working is explained in section 2, and the simulation results are reported in section 3. A discussion followed by our ending comments is included in section 4.
2 Model Description
2.1 Network Constituents and Structure
Our structure is quite classic and can be obtained by maintaining just the most essential elements of models, for example, the one explained in Sompolinsky, Golomb, and Kleinfeld (1991). The basic part consists of two layers of spiking neurons, which we choose to represent by the Izhikevich model (Izhikevich, 2003, 2007).
Different choices of the a, b, c, d parameters give rise to a rich variety of neuron types. We adopt as a starting point the most typical one, regular spiking (RS)—namely, (ms), (current units)/mV, mV, current units. Because we shall be dealing with a system of many neurons, the original a figure (i.e., the recovery time scale) will be taken as a central value of a random distribution, allowing for some dispersion of the form where is a gaussian random variable of zero mean and standard deviation equal to 1 and . Thus, although the natural frequencies of all our neurons are close to each other, they are not exactly equal, to the extent that in the absence of synchronizing interactions, the cells would evolve incoherently. This is a convenient situation for assessing the power of entraining forces, which have to overcome that initial incoherence. In addition to this measure, we are setting random initial values for the recovery variable u, so that the initial rates of increase for the potentials are not uniform.
Regarding the network organization, layer 1 has M stacks or columns of cells, each preferring a specific feature labeled by a j value (). Layer 2 is a single array of N cells with preferences for these N features. While layer 1 sites are described by two indices (), layer 2 sites need just a single index j. There are lateral synaptic connections between all the layer 1 cells in different stacks and with equal preference and also feedforward connections between the layer 1 cells and the layer 2 cells sharing the same preference. Thus, every layer 2 cell collects or integrates synaptic signals attached to one feature and coming from different stacks. The resulting connectivity is illustrated by Figure 1. Although lateral connections are often limited in length, we have to assume that they suffice to cover the extent of the studied area. In addition, the evolution rule for the couplings among units preferring the same features may be interpreted as a manifestation of a special form of feedback from layer 2 to the layer 1 connections.
2.2 Inputs to Layer 1 Neurons
2.2.1 Feedforward Part
2.2.2 Synchronizing Part
Our z variables are efficacy factors that can be ascribed to either the overall strength of this interaction or the synaptic pulses themselves. The latter interpretation is more evident after rewriting the right-hand side of equation 2.4 into the form .
2.2.3 Net Input
2.3 Inputs to Layer 2 Neurons
2.3.1 Feedforward Part
2.3.2 Net Input
3 Simulation Results
We have carried out two types of simulation: simultaneous presence of two features in the same stimulus and sequential presentations of two single-feature stimuli. Our choices were features and stacks. Numerical integration was performed by the updating algorithm of Izhikevich (2003), which involves mixed time jumps of and for the u and V variables, respectively. For a better time resolution, ms was set instead of just 1 ms. The selected parameter values were (if features are regarded as orientations between 0 and rads, this quantity has to be read in angular units of /N rads), ms, and , , , all them in the adopted current units for the neuron model (e.g., A or nA). The derivation of this parameter set was largely heuristic, serving the purpose of attaining reasonable rate values for the network responses.
3.1 Simultaneous Presence of Two Features in the Stimulus
Figure 2A shows a simulation with features and simultaneously present, in the attention-off condition, for a time span of 900 ms. Every row of plots represents the evolutions of the cells associated with a single feature, , from the top down. The rastergrams in the left column display the spikes on layer 1, with every line showing the th stack, , and the plots on the right depict the time courses of the cell potentials on layer 2. Next, the cases of attention directed to or are illustrated by Figures 2B and 2C, respectively. Attention to a given feature increases the degree of synchrony for all layer 1 cells preferring that feature and enhances the spiking rate of the layer 2 cell with the same preference. In fact, nearly synchronous regimes on layer 1 give rise to bursting on layer 2. Such regimes are here evidenced by a scenario of relatively high coherence. Swapping the attention and attention results in an interchange of rows when going from Figure 2B to Figure 2C. The firing rates themselves are represented in the top row of Figure 3, while the values of the synchrony measure, equation 3.1, are displayed in the second row of the same figures. A comparison of mean cross-covariances including nonzero lag values is displayed in Figure 4.
3.2 Sequential Presentations of Two Single-Feature Stimuli
Both stimuli are presented for 300 ms, and the onset of the second is separated from the removal of the first and by another 300 ms interval. The first stimulus, which in this setup is the target or cue, will have already determined the object of attention when the second one appears.
Experimental results indicate that pre-cueing the orientation of a stimulus improves its detection. The plots obtained from our simulations indeed exhibit an enhancement of the response in both spike counts and the observable degree of synchrony (see Figures 5 and 6). This difference is evident when comparing the last time interval of row 6 in Figure 5A and the last interval of row 3 in Figure 5B. If the second stimulus contains the same feature, as happens in Figure 5B, the amount of synchrony for the matching cells of layer 1 and the spiking rate for the associate neuron in layer 2 are higher than in the case of a different feature. Layer 2 rate values are shown in Figure 6. With the present parameter values, our results end up in the gamma band (30–70 Hz), which is the right frequency region for these enhancing effects (see Fries, Reynolds, Rorie, & Desimone, 2001; Fries et al., 2008). Spiking rate values are close to 35 Hz and 22 Hz for the unattended case and to 25 Hz and 55 Hz for the attended case. Going on to our C measure from the zero-lag correlations, equation 3.1, we obtain values of around 165 for the first stimulus, while the second stimulus yields 95 in the unattended case and 490 when it coincides with the attended feature. As for possible parameter changes, by setting a larger value for the constant, the resulting firing rates get higher. On the contrary, when choosing lower values, the rates fall below that frequency range, but it is then possible to observe that the first spike after the second stimulus onset takes place earlier for the repeated feature—that is, repetition also leads to a shorter latency (not shown).
It is interesting to wonder how the variation in the degree of synchrony can correlate to the firing rate change when the stimulus is presented for the second time. We have performed sets of 50 simulations for different values of the parameter with the aim of observing changes in the synchrony of the resulting regimes. The outcome is shown in Figure 7, depicting the peak rate and the synchrony measure C as a function of the strength, for values around the previously employed constant ( current units). Both magnitudes grow as increases, and the general appearance of the curves suggests a behavior dominated by an overall linear dependence. By eliminating we have also represented the rate value as a function of C, finding a similar type of growth.
A straightforward way of modifying the amount of synaptic inhibition is to multiply the negative terms on the right-hand side of equation 2.4 by a coefficient, say , larger than 1. Values like produce responses that do not differ too drastically from (the studied case). On the other hand, values that are too high may lead to artificial situations, but such a choice as appears to be curious enough. Then, in the case of the repeatedly shown feature, the increased rate gets quite high, actually into the 100–200 Hz range, beyond the gamma band, while the C measure becomes much larger—in fact, on the order of 1000 units. It is already known that some amount of extra inhibition may facilitate the achievement of synchronous regimes and that in the case of some spiking models, negative contributions to neural inputs may lead to high rates via rebound excitation.
4 Discussion and Conclusion
4.1 Robustness and Noise
Our system appears to be robust against moderate changes in the , As, parameters (relative to the values quoted in section 3). Further runs of our network (not shown) indicate that variations on the order of 20% in these quantities cause purely quantitative changes in the results but do not affect the main traits of the responses. Keeping As, fixed and changing just , after 25 runs with current units, we obtain typical rate enhancements for the attended feature from 24 Hz to 46 Hz and synchrony increases from C80 to 400. For , firing rates go from 33 Hz to 55 Hz, and C is enhanced from 110 to 480. Similarly, maintaining unchanged and varying only As we get for a rate increase from 28 Hz to 48 Hz and a synchrony enhancement from 80 to 380, while for , the rates rise from 42 Hz to 65 Hz, and C improves by jumping from 270 to 570 (cases for other As values are shown in Figure 7). Analogously, we can freeze , and change the strength. Then, if current units, rates undergo an enhancement from 21 Hz to 42 Hz, and C is increased from 80 to 420. In the case of , the jumps range from 42 Hz to 64 Hz for the firing rates and from 190 to 480 for our synchrony measure.
The particular choice of neuron type does seem to be crucial either. We have switched from the RS model to the intrinsically bursting type (IB in Izhikevich, 2003), with a simple readjustment of the strength from 100 to 50 current units in order to obtain layer 2 rates within the expected band. For the feature to which attention is directed, the responses exhibit increases from 26 Hz to 55 Hz in spiking rates and from 50 to 360 in the C measure. Although these values are somewhat different from the ones obtained for the RS type, the qualitative outcome does not change. Moreover, it is not necessary to employ a single neuron type. A possible modification is to keep the existing RS neurons in layer 1 and switch to the fast spiking (FS) variant in layer 2. Because FS cells spike more often than RS for similar inputs, higher spiking rates are produced, but this effect is easily corrected by lowering the value of the strength. Changing from 100 to 65 current units, we obtain output rates near the upper limit of the gamma band. Thus, after averaging the results of 25 runs, the case of the repeated feature goes from 76 Hz to 108 Hz, while the C measure rises from 300 to 570.
Synaptic noise has so far been omitted, as we felt it was not essential for the points we wished to illustrate. The role of internal noise can sometimes be complex, bringing about new effects that may range from varying degrees of performance impairment to less obvious stochastic phenomena. Thinking about robustness against randomness, some examples of performance degradation resulting from the presence of white gaussian noise are shown in Figure 8. Large noise amplitudes An can reduce the rate difference between the first and second responses in the different condition and may decrease the observed enhancement in the same condition. However, these properties survive on average when noise is not too strong. Moreover, small adjustments in noise level allow for tuning the peak properties (see section 4.2). From another viewpoint, in connection with the exploitation of external noise, the mechanisms of gain and tuning were also characterized using the equivalent-noise paradigm (Ling, Liu, & Carrasco, 2009), which measures sensitivity for a signal embedded in external noise as a function of increasing levels of noise.
4.2 Comparison to Existing Results
The bell-shaped function (see equation 2.3) has the role of smearing out the selective effects, allowing the channels close to (but different from) the selected one to receive a small part of the enhancing signal, as can be easily seen in the plots of Figures 2 and 3 and Figures 5 and 6. More tuning functions, as well as lateral competition terms, might be added in order to reproduce smaller-scale details of particular data sets (as in Hamker, 2004). This has not been done in this work because our purpose was to be mildly realistic while keeping the number of ingredients to a minimum.
Instead, we prefer to concentrate on larger-scale traits, like the values of the peak rate and rate enhancement. In order to obtain more symmetric plots, we increase N from 8 to 9, choosing a stimulus with the feature, and slightly reduce from 0.8 to 0.7 (same units as in section 2). Next, we go back to the case of the same stimulus as explained in section 3.2 and compare unattended (first) and attended (second) cases. The combined effects of adequate changes in the value of the parameter and in the level of synaptic noise An, with the remaining settings unchanged, suffice to produce an outcome quite close to published results. For current units and a synaptic noise amplitude of current units, a peak rate of 23 Hz is obtained in the unattended condition, which becomes enhanced by around 5 Hz when switching to the attended condition (see Figure 9, top left). This is the increase that can be observed in Figure 8A (right) of Ling et al. (2009). Changing the quoted parameters again to , (same units) yields a peak rate of 28 Hz eventually enhanced by 11 Hz (see Figure 9, top right), in agreement with the appearance of Figure 4A by Martínez-Trujillo and Treue (2004). Further, in the bottom row of Figure 9, the averaged zero-lag cross-correlations among cell potentials (see equation 3.1) have also been represented, illustrating the apparent correlation between coherence increase and rate enhancement.
4.3 Outlook and Final Remarks
Attention increases the responses of neurons with preferences close to the attended feature. In the case of motion direction, which relates in many aspects to orientation, a feature-similarity gain model was developed to explain these and other findings (Martínez-Trujillo & Treue, 2004; Treue & Martínez-Trujillo, 1999). The principles underlying that model are at the core of many implementations, including the one we have presented here. Top-down attention modulations changed the gain of individual sensory neurons in a multiplicative way, to an extent proportional to the similarity between the attended feature and the cell’s preference. Responses of cells tuned to similar features were enhanced, while responses of cells that preferred very different features became suppressed. At the level of neuronal populations, these gain changes modulated the selectivity of the population response (Martínez-Trujillo & Treue, 2004), and stimuli sharing features similar to those in the target were represented more strongly.
Other relevant contributions made decisive steps in establishing key ideas about biased competition, from its conceptual explanation (Desimone & Duncan, 1995) to specific mechanisms like the biased competition model (Reynolds, Chelazzi, & Desimone, 1999). In the normalization model (Reynolds & Heeger, 2009; Herrmann et al., 2012), the stimulus drive is multiplied by the attention field and divided by the suppressive drive to yield the output firing rates.
Since synchrony and phase mechanisms have already been investigated from so many perspectives, the related bibliography is very rich and includes a large number of remarkable works. Compte and Wang (2006) explored the mechanisms of attention-induced receptive field (RF) shifts in cortical network models that receive an attentional spotlight. The RF size was enlarged (respectively, reduced) by attentional signal directed near a cell’s RF center in a recurrent network (resp., a feedforward network). Mishra, Fellous, and Sejnowski (2006) simulated a multicompartment model of V4 cortical neurons that received inputs from two V2 excitatory synaptic pools and analyzed the responses to a variety of combined inputs, including the absence and presence of spike correlations. The results of that highly realistic and elaborate model predict that top-down attention may bias the V4 neuron’s response by an inhibitory correlation phase shift mechanism.
Buia and Tiesinga (2008; see also Tiesinga & Buia 2008) proposed synchrony modulation by interneurons. Buelhman and Deco (2008) studied the coexistence of the rate modulation and synchronization modulation using a theoretical framework with a network consisting of inhibitory and excitatory neurons organized in pools. Börgers and Kopell (2008) showed that the competitive advantage of coherent excitatory stimuli was amplified greatly when the target included inhibitory interneurons acting via GABA-receptor mediated synapses and the coherent input oscillated at gamma frequency. Neurophysiological evidence of gating through phase relation–dependent mechanisms is offered by Grothe et al. (2015). Bichot and Desimone (2006) showed how attentional selection appears to be mediated by changes in the synchrony of responses of neuronal populations, in addition to the modulation of the firing rate of individual neurons.
In this letter, we wished to test a relatively simple network organization leading to the reproduction of higher responses for attended cues and involving the appearance of roughly synchronous regimes. For this purpose, we have devised a two-layered neural model that incorporates a spike-mediated form of approximate synchronization by differential lateral couplings among the bottom-layer columns or stacks. The top-layer neurons, which integrate the spiking activity coming from below, increase their responses if their preferences are close enough to the attended feature. To this end, the functioning of our lateral couplings among stacks appears to suffice. In fact, the effect of the synaptic efficacies’ zj’s on the recurrent connections was decisive. This success does not rule out possible solutions based on feedback signals, as lateral interactions can often be implemented, in an equivalent fashion, with the help of new layers and feedback pathways producing similar effects. Besides that general idea, our scheme includes a particular type of feedback from layer 2 to the layer 1 couplings in equation 2.5, which provides a handy evolution rule for the synchronizing couplings and implicitly involves competition through limitation of their sum.
The model we have presented describes featural attention, but the involved effects could possibly be used to model spatial attention by means of spatial clustering (rather than featural clustering) of neural populations. Our findings show that the described form of synchronization is effective. During simultaneous presentations of two different features to our system, the attended one gives rise to a higher response, as expected. For the case of sequential displays of two stimuli, repetition of the orientation also results in an enhanced spiking response within the correct frequency region. The enhancement is altogether evidenced by greater degrees of coherence, higher spiking rates, and if we allowed frequencies to fall below the typical region, a shorter spike latency. Rate modulations resulting from our model agree with published results. Selecting particular values of the layer 2 strength and the synaptic noise levels , we have obtained rate peak values and rate enhancement maxima like those found by Ling et al. (2009) and Martínez-Trujillo and Treue (2004).
It is sensible to wonder whether an artificial increase in the concentration of neurotransmitters could boost what we have represented as the synchronizing strength (depicted in Figure 7). When this strength is doubled by going from 35 to 70, synchrony measures grows from 50 to 350 for first stimuli and from 100 to 650 for cued stimuli (i.e., they are multiplied by a factor on the order of 7 in both cases). The model predicts that as synchrony and firing rate values go through such increases, the height of the rate enhancement after switching from unattended to attended stimuli is maintained around 20 Hz (in the considered range). On the whole, the magnitude of this jump does not seem to depend too much on the initial degree of synchrony. We can thus speculate that for an experimental paradigm yielding measurable synchrony changes, the detected jumps in firing rates ought to result in approximately constant differences, at least within some bounds. To this end, a possible setup could be a variant of the experiment by Supèr, van der Togt, Spekreijse, and Lamme (2003; see also van der Togt, Kalitzin, Spekreijse, Lamme, & Supèr, 2006), now with the animals watching single-feature stimuli, and including some factor capable of conditioning their synaptic efficacies like the gradual administration of some neurochemical substance.
The phenomenon of synchronization is at the heart of some biological functions and constitutes a plausible abstraction for many processes in fields like genetic networks, circadian rhythms or ecology (Elowitz & Leibler, 2000; Pikovsky, Rosenblum, & Kurths, 2001; Strogatz, 2003; Keith, 1963; Earn, Levin, & Rohani, 2000). Although neural dynamics is far more complex than phase representations, some of the steps highlighting the path to synchrony can be viewed as a good starting point. We feel our network is interesting enough, as similar systems of coupled units including these elements might offer an alternative tool for further studies about synchronization and coherence (Fries, 2005), a key issue of neuronal communication.
This work was supported by the Spanish Ministerio de Economía y Comptetitividad, project no. PSI2014-57454-P.