Abstract

We introduce a neural model capable of feature selectiveness by spike-mediated synchronization through lateral synaptic couplings. For a stimulus containing two features, the attended one elicits a higher response. In the case of sequential single-feature stimuli, repetition of the attended feature also results in an enhanced response, exhibited by greater synchrony and higher spiking rates.

1  Introduction

Over the past few decades, there has been an ever increasing interest in the neural mechanisms of visual attention. Researchers ceaselessly wonder how visual attention affects spatial and temporal sensitivity, how it influences the selection of relevant stimuli, how responses are modulated, which computations implement these processes, and what sort of interplay between attention and eye movements should be expected (further details and illustrations can be found e.g. in Desimone & Duncan, 1995; Itti & Koch, 2001; Carrasco, 2011).

Three main types of visual attention have been defined: (1) spatial attention, which can be overt, when observers move their eyes to a relevant location and the focus of attention coincides with the movement of the eyes, or covert, when attention is deployed to relevant locations without accompanying eye movements; (2) feature-based attention (FBA), which can be directed covertly to specific aspects (e.g., color, orientation, or motion direction) regardless of space locations (see Herrmann, Heeger, & Carrasco, 2012); and (3) object-based attention, guided by object structure. Whereas spatial attention guides observers to particular locations, FBA leads them to particular features in the visual scene (object recognition usually requires much richer sets of detectors than FBA; see the comments in Hamker, 2006).

Selective attention arises from the brain’s limited capacity to process vast amounts of sensory information (Attwell & Laughlin, 2001; Lennie, 2003) and enables us to gather the relevant part of these inputs, ultimately guiding our behavior. In FBA, the fact of attending to one feature biases the population activity toward that attended feature, that is, toward the behaviorally relevant stimulus, at the expense of behaviorally irrelevant ones. FBA is very helpful because we are often acquainted with the defining feature of an object without knowing where it will appear.

Concerning mechanisms, FBA appears to involve several visual areas where it modulates activity in response to expectations about task-relevant features. In the case of spatial attention, these areas include V1, MT (see Watanabe et al., 1998) and V2, V4, IT (see Fries, Womelsdorf, Oostenveld, & Desimone, 2008). Bosman et al. (2012) studied interareal influences involving V1 and V4.

In the work we present here, we try to construct a model for selective visual attention based on orientation-coded features. The introduced structure incorporates Izhikevich spiking neurons (Izhikevich, 2003, 2007), which are convenient for considering rate differences, firing regimes, and possible latency effects. From these constituents, we wish to study the possibility of obtaining feature-selective responses based on a particular form of lateral coupling among cell stacks capable of producing firing regimes with approximate synchronization. The network structure and working is explained in section 2, and the simulation results are reported in section 3. A discussion followed by our ending comments is included in section 4.

2  Model Description

2.1  Network Constituents and Structure

Our structure is quite classic and can be obtained by maintaining just the most essential elements of models, for example, the one explained in Sompolinsky, Golomb, and Kleinfeld (1991). The basic part consists of two layers of spiking neurons, which we choose to represent by the Izhikevich model (Izhikevich, 2003, 2007).

Given a neuron receiving an input current I, Izhikevich’s evolution equations for the membrane potential V and for the recovery variable u associated with the same cell read
formula
2.1
with the prime symbol denoting time differentiation. Izhikevich (2003) supplies the numerical values (current units)/(mV), (current units)/mV, and (current units). C indicates the capacitance of the cell membrane—usually 1 (current unit) ms/mV. a is the scale for the recovery process (inverse of time), and b is the sensitivity of the recovery rate to potential values.
A spike is elicited whenever a potential V gets above the peak value (here, mV). Izhikevich’s model includes the after-spike reset rule,
formula
2.2
where c establishes the reset value for the potential, and d is the magnitude of the reset jump that has to be applied to the recovery variable u. We will be using binary variables of the form denoting (absence, presence) of spike for the studied V.

Different choices of the a, b, c, d parameters give rise to a rich variety of neuron types. We adopt as a starting point the most typical one, regular spiking (RS)—namely, (ms), (current units)/mV,  mV, current units. Because we shall be dealing with a system of many neurons, the original a figure (i.e., the recovery time scale) will be taken as a central value of a random distribution, allowing for some dispersion of the form where is a gaussian random variable of zero mean and standard deviation equal to 1 and . Thus, although the natural frequencies of all our neurons are close to each other, they are not exactly equal, to the extent that in the absence of synchronizing interactions, the cells would evolve incoherently. This is a convenient situation for assessing the power of entraining forces, which have to overcome that initial incoherence. In addition to this measure, we are setting random initial values for the recovery variable u, so that the initial rates of increase for the potentials are not uniform.

Regarding the network organization, layer 1 has M stacks or columns of cells, each preferring a specific feature labeled by a j value (). Layer 2 is a single array of N cells with preferences for these N features. While layer 1 sites are described by two indices (), layer 2 sites need just a single index j. There are lateral synaptic connections between all the layer 1 cells in different stacks and with equal preference and also feedforward connections between the layer 1 cells and the layer 2 cells sharing the same preference. Thus, every layer 2 cell collects or integrates synaptic signals attached to one feature and coming from different stacks. The resulting connectivity is illustrated by Figure 1. Although lateral connections are often limited in length, we have to assume that they suffice to cover the extent of the studied area. In addition, the evolution rule for the couplings among units preferring the same features may be interpreted as a manifestation of a special form of feedback from layer 2 to the layer 1 connections.

Figure 1:

Main part of the network structure. The bottom layer, referred to as 1 in the text, has M stacks, labeled by , each of them containing N feature preferences labeled by . The top layer, 2, contains a single array for the N features in question. This diagram is showing the particular case , . Observe the lateral connections among layer 1 stacks and the FF connections from layer 1 to layer 2. Feedback signals (not shown) would travel from layer 2 to the layer 1 connections themselves.

Figure 1:

Main part of the network structure. The bottom layer, referred to as 1 in the text, has M stacks, labeled by , each of them containing N feature preferences labeled by . The top layer, 2, contains a single array for the N features in question. This diagram is showing the particular case , . Observe the lateral connections among layer 1 stacks and the FF connections from layer 1 to layer 2. Feedback signals (not shown) would travel from layer 2 to the layer 1 connections themselves.

2.2  Inputs to Layer 1 Neurons

2.2.1  Feedforward Part

formula
2.3
These inputs involve the orientation-dependent tuning function , with peak position determined by the stimulus. Greek letters are used for stack labels, and roman letters are employed for feature (i.e., orientation) indices. As FBA deployments concern features and not stack locations, the assigned input values in equation 2.3 are independent of . When more than one orientation is simultaneously present in the stimulus, the individual rule, equation 2.3, is replaced with a max-pooling over the results of applying the tuning function for every . is a constant that measures the overall strength of the feedforward interaction.

2.2.2  Synchronizing Part

With the aim of achieving approximate synchronization among the cells sensitive to the attended feature, we introduce spike-mediated interactions like those used in Supèr and Romeo (2014), now multiplicatively activated by the zj coupling strengths as follows:
formula
2.4
The quantity is in fact a lateral input within layer 1 itself. indicates the (absence, presence) of a spike at site of layer 1. These possible spikes represent the action potentials elicited by the Izhikevich neuron located at the considered site. As is a global constant measuring the strength of the synchronizing interaction. Dependence on the j index through the zj variable confers feature-selectiveness to these connections.

Our z variables are efficacy factors that can be ascribed to either the overall strength of this interaction or the synaptic pulses themselves. The latter interpretation is more evident after rewriting the right-hand side of equation 2.4 into the form .

These coefficients gate the lateral signals, evolving just when attention is engaged. We now set a differential equation for their time evolution. We adopt a rule including feedback from layer 2:
formula
2.5
Every zj variable supplies the relative strength of the synchronizing coupling for the jth feature channel, and the nature of their changes may be attributed to neurotransmitter release variability. The constant is a typical timescale for the zj’s to become close to their asymptotic values, and is a binary variable that signals the possible presence of a spike at site j of layer 2 (see below). Thus, the z coupling can change only when spiking activity takes place at its matching layer-2 site. Furthermore, the last factor on the right-hand side stops the sum of all the couplings from growing without limit, causing a form of competition among the z components. At the zj’s will be set to some given zj(0) initial values. The mathematical form of equation 2.5 has similarities to some of the rules used in Grossberg (1976).

2.2.3  Net Input

In addition, we may include a noise term denoted by (e.g., white gaussian noise of zero mean and a given standard deviation, say An). Then the net input to layer 1 cells is
formula
2.6

2.3  Inputs to Layer 2 Neurons

2.3.1  Feedforward Part

This contribution is just proportional to the sum—or mean—of the spike-coded signals from layer 1 to layer 2:
formula
2.7
with being a global factor that defines the strength of this FF interaction.

2.3.2  Net Input

The total input to the th cell in layer 2 will have the form
formula
2.8
where we have included an additional noise term, , similar to the of equation 2.6. Further tuning functions, spike-mediated lateral excitation, and shunting inhibition might in principle be added for the purpose of simulating competitive interactions between populations, analogous to the spatial attention model in Hamker (2004). However, we now choose not to do so, as we wish to incorporate only the most necessary ingredients for achieving the effects we seek. Moreover, note that every is obtained by a sum-pooling and not a max-pooling, which in our situation would not distinguish between one or more simultaneous spikes (a complete parallel to the mechanism in Hamker, 2004, is not in order because that was a rate model, not a truly spiking one, and was devised for spatial attention rather than FBA).

3  Simulation Results

The degree of coincidence in time evolution among the responses for a specific feature channel—say, j—of layer 1 is measured by averaging the pair-wise cross-correlations between potentials of cells sharing the preference for feature j and contained in different stacks. Thus, for a given feature index j and for some time lag value ,
formula
3.1
denotes the application of the usual cross-covariance definition to the pair of functions formed by the potential of cell j in the th stack and the potential of cell j in the th stack . These variables generically denote layer 1 cell potentials obeying Izhikevich’s evolution equations 2.1, 2.2, which originate the spike variables; therefore, the two subscripts have the same meaning as in equation 2.4. Rather than taking smoothed versions of the spike trains, we have opted for the potentials themselves, which are smooth functions of t everywhere in the chosen window except for the set of reset times. The symbols in equation 3.1 indicate time average over the written t variable, and for every evaluation, the t-domain is limited to the interval in which the considered stimulus is present. Our adopted normalization follows the rule for the biased estimation case, where the raw contributions are scaled, dividing them by the length of the considered input data. The P symbol in equation 3.1 indicates the set of possible stack label pairs, and is their number—actually, . For every considered j value, the mean cross-correlations Cj are functions of the lag times . In terms of dimensions, the Cj units should be (mV) unless we assume that these magnitudes are eventually made dimensionless. Rigorously speaking, cross-covariance measures relate, via Fourier transform, to phase coherence rather than phase synchrony, but they are adequate enough for our purposes (more precise discussions on these concepts can be found in Fries, Roelfsema, Engel, König, & Singer, 1997; Fries, 2005; Schelter, Winterhalder, & Timmer, 2007). Often we will be interested in just the zero-lag value of , that is, Cj(0). We will refer to this quantity as C.

We have carried out two types of simulation: simultaneous presence of two features in the same stimulus and sequential presentations of two single-feature stimuli. Our choices were features and stacks. Numerical integration was performed by the updating algorithm of Izhikevich (2003), which involves mixed time jumps of and for the u and V variables, respectively. For a better time resolution, ms was set instead of just 1 ms. The selected parameter values were (if features are regarded as orientations between 0 and rads, this quantity has to be read in angular units of /N rads), ms, and , , , all them in the adopted current units for the neuron model (e.g., A or nA). The derivation of this parameter set was largely heuristic, serving the purpose of attaining reasonable rate values for the network responses.

3.1  Simultaneous Presence of Two Features in the Stimulus

Attention can be engaged on the first, on the second, or on none of them (the “attention-off” or “attention-away” condition). Evidence has shown that the responses to paired stimuli are weighted averages of the individual responses, and since by construction, our system behaves roughly linearly for high rates, this condition is approximately satisfied. In regard to the initial zj values, we now set
formula
3.2
Thus, the feature-similarity hypothesis is implicitly accepted here, as the application of such initial conditions favors the cell preferences for the attended feature (Martínez-Trujillo & Treue, 2004). In the end, attention boosts sensitivity in the psychophysical channels that best represent the target stimulus.

Figure 2A shows a simulation with features and simultaneously present, in the attention-off condition, for a time span of 900 ms. Every row of plots represents the evolutions of the cells associated with a single feature, , from the top down. The rastergrams in the left column display the spikes on layer 1, with every line showing the th stack, , and the plots on the right depict the time courses of the cell potentials on layer 2. Next, the cases of attention directed to or are illustrated by Figures 2B and 2C, respectively. Attention to a given feature increases the degree of synchrony for all layer 1 cells preferring that feature and enhances the spiking rate of the layer 2 cell with the same preference. In fact, nearly synchronous regimes on layer 1 give rise to bursting on layer 2. Such regimes are here evidenced by a scenario of relatively high coherence. Swapping the attention and attention results in an interchange of rows when going from Figure 2B to Figure 2C. The firing rates themselves are represented in the top row of Figure 3, while the values of the synchrony measure, equation 3.1, are displayed in the second row of the same figures. A comparison of mean cross-covariances including nonzero lag values is displayed in Figure 4.

Figure 2:

(A) Simultaneous presence of features , attention off. The time span was 900 ms (time resolution ms, global FF strengths , , synchronizing strength ). Every row corresponds to a single feature, , from the top down. The left column displays the spike rastergrams on layer 1, coming from the variables. Inside each raster, every line comes from a stack ; the ordinates are the values . The right column shows the time courses of the cell potentials on layer 2: for ). (B) Simultaneous presence of features with attention directed to . The differences between the evolutions of all cells with preference here and in panel A are manifest. The fact that was attended causes the amount of coherence and spiking rate values to increase in row 3. (C) Simultaneous presence of features with attention directed to . Predictably, the situation in panel C is almost the same as in panel A after interchanging the row and the row.

Figure 2:

(A) Simultaneous presence of features , attention off. The time span was 900 ms (time resolution ms, global FF strengths , , synchronizing strength ). Every row corresponds to a single feature, , from the top down. The left column displays the spike rastergrams on layer 1, coming from the variables. Inside each raster, every line comes from a stack ; the ordinates are the values . The right column shows the time courses of the cell potentials on layer 2: for ). (B) Simultaneous presence of features with attention directed to . The differences between the evolutions of all cells with preference here and in panel A are manifest. The fact that was attended causes the amount of coherence and spiking rate values to increase in row 3. (C) Simultaneous presence of features with attention directed to . Predictably, the situation in panel C is almost the same as in panel A after interchanging the row and the row.

Figure 3:

(Top): Average layer-2 spiking rates (in Hz), after running 100 repetitions of three processes like the ones shown in Figure 2, which involve the simultaneous presence of two features. Abscissa values are feature labels (feat is the j orientation, ), and ordinate values are the rates themselves in Hz. Left: and features present, none of them was attended. Center: feature was attended; feature was unattended. Right: feature was unattended; feature was attended. Error bars indicate standard deviations. (Bottom) Averaged zero-lag correlations, here called C, found by setting in equation 3.1 for the three described situations.

Figure 3:

(Top): Average layer-2 spiking rates (in Hz), after running 100 repetitions of three processes like the ones shown in Figure 2, which involve the simultaneous presence of two features. Abscissa values are feature labels (feat is the j orientation, ), and ordinate values are the rates themselves in Hz. Left: and features present, none of them was attended. Center: feature was attended; feature was unattended. Right: feature was unattended; feature was attended. Error bars indicate standard deviations. (Bottom) Averaged zero-lag correlations, here called C, found by setting in equation 3.1 for the three described situations.

Figure 4:

Mean cross-covariance (cross-correlations with subtracted mean and normalization for biased estimation) curves among layer 1 cell potential values, represented as functions of the lag time. They were obtained after averaging two potential covariance curves for all the possible pairs of units sharing the considered feature preference according to definition 3.1. The shown case comes from a process in the same conditions as in Figure 2B, where the attended feature was , (top) and the unattended one was (bottom).

Figure 4:

Mean cross-covariance (cross-correlations with subtracted mean and normalization for biased estimation) curves among layer 1 cell potential values, represented as functions of the lag time. They were obtained after averaging two potential covariance curves for all the possible pairs of units sharing the considered feature preference according to definition 3.1. The shown case comes from a process in the same conditions as in Figure 2B, where the attended feature was , (top) and the unattended one was (bottom).

3.2  Sequential Presentations of Two Single-Feature Stimuli

Both stimuli are presented for 300 ms, and the onset of the second is separated from the removal of the first and by another 300 ms interval. The first stimulus, which in this setup is the target or cue, will have already determined the object of attention when the second one appears.

Instead of equation 3.2, we now adopt the initial conditions:
formula
3.3
From this common start, the z components, which are increased sooner, will have a definite advantage over the rest, winning what is actually a competition process. As a result, the strongest coupling will be the one associated with the first presented feature. The null assignments, equation 3.3, can be replaced by small random values, with similar results. In view of this, we may reinterpret the initial conditions of the type 3.2 as the outcome of having previously run the network for a single feature identical to the cue. In general, a sensible choice for the initial zj(0)’s is called for, and a completely arbitrary set might produce an unwanted bias.

Experimental results indicate that pre-cueing the orientation of a stimulus improves its detection. The plots obtained from our simulations indeed exhibit an enhancement of the response in both spike counts and the observable degree of synchrony (see Figures 5 and 6). This difference is evident when comparing the last time interval of row 6 in Figure 5A and the last interval of row 3 in Figure 5B. If the second stimulus contains the same feature, as happens in Figure 5B, the amount of synchrony for the matching cells of layer 1 and the spiking rate for the associate neuron in layer 2 are higher than in the case of a different feature. Layer 2 rate values are shown in Figure 6. With the present parameter values, our results end up in the gamma band (30–70 Hz), which is the right frequency region for these enhancing effects (see Fries, Reynolds, Rorie, & Desimone, 2001; Fries et al., 2008). Spiking rate values are close to 35 Hz and 22 Hz for the unattended case and to 25 Hz and 55 Hz for the attended case. Going on to our C measure from the zero-lag correlations, equation 3.1, we obtain values of around 165 for the first stimulus, while the second stimulus yields 95 in the unattended case and 490 when it coincides with the attended feature. As for possible parameter changes, by setting a larger value for the constant, the resulting firing rates get higher. On the contrary, when choosing lower values, the rates fall below that frequency range, but it is then possible to observe that the first spike after the second stimulus onset takes place earlier for the repeated feature—that is, repetition also leads to a shorter latency (not shown).

Figure 5:

Layer 1 spike rasters and layer 2 potentials for successive presentation of two single-feature stimuli, which lasted 300 ms and were separated by a 300 ms interval. The layout of columns and rows is analogous to Figure 2. (A) Two different features, in particular feature (first) and (second). Responses to the second one are weaker and lack synchrony. (B) Same feature, , in both cases. Firing rates in the last time window are now higher than in the row of panel A, and the presence of approximate synchrony is evident.

Figure 5:

Layer 1 spike rasters and layer 2 potentials for successive presentation of two single-feature stimuli, which lasted 300 ms and were separated by a 300 ms interval. The layout of columns and rows is analogous to Figure 2. (A) Two different features, in particular feature (first) and (second). Responses to the second one are weaker and lack synchrony. (B) Same feature, , in both cases. Firing rates in the last time window are now higher than in the row of panel A, and the presence of approximate synchrony is evident.

Figure 6:

(Top) Average layer-2 spike rates (in Hz), after running 100 times two processes like the ones shown in Figure 5, with the same conventions for abscissa and ordinate values as in Figure 3. Left: Cue feature (solid curve) followed by a new feature (dotted curve). Right: Cue feature (solid curve) followed by the same feature (dotted curve). As usual, error bars indicate standard deviations. Reappearance of the cue results in firing rate enhancement exhibited by the difference in peak values. (Bottom) Mean zero-lag correlations C, from the value of definition 3.1 applied to these two processes.

Figure 6:

(Top) Average layer-2 spike rates (in Hz), after running 100 times two processes like the ones shown in Figure 5, with the same conventions for abscissa and ordinate values as in Figure 3. Left: Cue feature (solid curve) followed by a new feature (dotted curve). Right: Cue feature (solid curve) followed by the same feature (dotted curve). As usual, error bars indicate standard deviations. Reappearance of the cue results in firing rate enhancement exhibited by the difference in peak values. (Bottom) Mean zero-lag correlations C, from the value of definition 3.1 applied to these two processes.

It is interesting to wonder how the variation in the degree of synchrony can correlate to the firing rate change when the stimulus is presented for the second time. We have performed sets of 50 simulations for different values of the parameter with the aim of observing changes in the synchrony of the resulting regimes. The outcome is shown in Figure 7, depicting the peak rate and the synchrony measure C as a function of the strength, for values around the previously employed constant ( current units). Both magnitudes grow as increases, and the general appearance of the curves suggests a behavior dominated by an overall linear dependence. By eliminating we have also represented the rate value as a function of C, finding a similar type of growth.

Figure 7:

Performance comparison under variations of . (Left) Efficacy measure C for different values of the synchronizing strength , obtained after averaging results for sets of 50 runs in which the same feature was presented twice (solid line for first time, dotted line for second time). (Center) Peak rates as a function of , coming from the same simulations. (Right) Rates in terms of C0 from the previous two data sets, after eliminating .

Figure 7:

Performance comparison under variations of . (Left) Efficacy measure C for different values of the synchronizing strength , obtained after averaging results for sets of 50 runs in which the same feature was presented twice (solid line for first time, dotted line for second time). (Center) Peak rates as a function of , coming from the same simulations. (Right) Rates in terms of C0 from the previous two data sets, after eliminating .

A straightforward way of modifying the amount of synaptic inhibition is to multiply the negative terms on the right-hand side of equation 2.4 by a coefficient, say , larger than 1. Values like produce responses that do not differ too drastically from (the studied case). On the other hand, values that are too high may lead to artificial situations, but such a choice as appears to be curious enough. Then, in the case of the repeatedly shown feature, the increased rate gets quite high, actually into the 100–200 Hz range, beyond the gamma band, while the C measure becomes much larger—in fact, on the order of 1000 units. It is already known that some amount of extra inhibition may facilitate the achievement of synchronous regimes and that in the case of some spiking models, negative contributions to neural inputs may lead to high rates via rebound excitation.

4  Discussion and Conclusion

4.1  Robustness and Noise

Our system appears to be robust against moderate changes in the , As, parameters (relative to the values quoted in section 3). Further runs of our network (not shown) indicate that variations on the order of 20% in these quantities cause purely quantitative changes in the results but do not affect the main traits of the responses. Keeping As, fixed and changing just , after 25 runs with current units, we obtain typical rate enhancements for the attended feature from 24 Hz to 46 Hz and synchrony increases from C80 to 400. For , firing rates go from 33 Hz to 55 Hz, and C is enhanced from 110 to 480. Similarly, maintaining unchanged and varying only As we get for a rate increase from 28 Hz to 48 Hz and a synchrony enhancement from 80 to 380, while for , the rates rise from 42 Hz to 65 Hz, and C improves by jumping from 270 to 570 (cases for other As values are shown in Figure 7). Analogously, we can freeze , and change the strength. Then, if current units, rates undergo an enhancement from 21 Hz to 42 Hz, and C is increased from 80 to 420. In the case of , the jumps range from 42 Hz to 64 Hz for the firing rates and from 190 to 480 for our synchrony measure.

The particular choice of neuron type does seem to be crucial either. We have switched from the RS model to the intrinsically bursting type (IB in Izhikevich, 2003), with a simple readjustment of the strength from 100 to 50 current units in order to obtain layer 2 rates within the expected band. For the feature to which attention is directed, the responses exhibit increases from 26 Hz to 55 Hz in spiking rates and from 50 to 360 in the C measure. Although these values are somewhat different from the ones obtained for the RS type, the qualitative outcome does not change. Moreover, it is not necessary to employ a single neuron type. A possible modification is to keep the existing RS neurons in layer 1 and switch to the fast spiking (FS) variant in layer 2. Because FS cells spike more often than RS for similar inputs, higher spiking rates are produced, but this effect is easily corrected by lowering the value of the strength. Changing from 100 to 65 current units, we obtain output rates near the upper limit of the gamma band. Thus, after averaging the results of 25 runs, the case of the repeated feature goes from 76 Hz to 108 Hz, while the C measure rises from 300 to 570.

Synaptic noise has so far been omitted, as we felt it was not essential for the points we wished to illustrate. The role of internal noise can sometimes be complex, bringing about new effects that may range from varying degrees of performance impairment to less obvious stochastic phenomena. Thinking about robustness against randomness, some examples of performance degradation resulting from the presence of white gaussian noise are shown in Figure 8. Large noise amplitudes An can reduce the rate difference between the first and second responses in the different condition and may decrease the observed enhancement in the same condition. However, these properties survive on average when noise is not too strong. Moreover, small adjustments in noise level allow for tuning the peak properties (see section 4.2). From another viewpoint, in connection with the exploitation of external noise, the mechanisms of gain and tuning were also characterized using the equivalent-noise paradigm (Ling, Liu, & Carrasco, 2009), which measures sensitivity for a signal embedded in external noise as a function of increasing levels of noise.

Figure 8:

Averaged spiking rates after running 100 times simulations as in Figure 5 (thus, current units) now under the effect of nonzero noise terms introduced in equations 2.6 and 2.8. (Top) White gaussian noise of current units for both layers. Error bars indicate standard deviations. (Bottom) White gaussian noise with current units in both cases. As can be seen, a large noise amplitude reduces the rate difference between the two responses in the different condition, and could eventually ruin the observed enhancement in the same condition.

Figure 8:

Averaged spiking rates after running 100 times simulations as in Figure 5 (thus, current units) now under the effect of nonzero noise terms introduced in equations 2.6 and 2.8. (Top) White gaussian noise of current units for both layers. Error bars indicate standard deviations. (Bottom) White gaussian noise with current units in both cases. As can be seen, a large noise amplitude reduces the rate difference between the two responses in the different condition, and could eventually ruin the observed enhancement in the same condition.

4.2  Comparison to Existing Results

The bell-shaped function (see equation 2.3) has the role of smearing out the selective effects, allowing the channels close to (but different from) the selected one to receive a small part of the enhancing signal, as can be easily seen in the plots of Figures 2 and 3 and Figures 5 and 6. More tuning functions, as well as lateral competition terms, might be added in order to reproduce smaller-scale details of particular data sets (as in Hamker, 2004). This has not been done in this work because our purpose was to be mildly realistic while keeping the number of ingredients to a minimum.

Instead, we prefer to concentrate on larger-scale traits, like the values of the peak rate and rate enhancement. In order to obtain more symmetric plots, we increase N from 8 to 9, choosing a stimulus with the feature, and slightly reduce from 0.8 to 0.7 (same units as in section 2). Next, we go back to the case of the same stimulus as explained in section 3.2 and compare unattended (first) and attended (second) cases. The combined effects of adequate changes in the value of the parameter and in the level of synaptic noise An, with the remaining settings unchanged, suffice to produce an outcome quite close to published results. For current units and a synaptic noise amplitude of current units, a peak rate of 23 Hz is obtained in the unattended condition, which becomes enhanced by around 5 Hz when switching to the attended condition (see Figure 9, top left). This is the increase that can be observed in Figure 8A (right) of Ling et al. (2009). Changing the quoted parameters again to , (same units) yields a peak rate of 28 Hz eventually enhanced by 11 Hz (see Figure 9, top right), in agreement with the appearance of Figure 4A by Martínez-Trujillo and Treue (2004). Further, in the bottom row of Figure 9, the averaged zero-lag cross-correlations among cell potentials (see equation 3.1) have also been represented, illustrating the apparent correlation between coherence increase and rate enhancement.

Figure 9:

Firing rates (top row) and averaged zero-lag cross-covariance C ( case of equation 3.1 (bottom row), for presentations of feature , now with , when not yet attended (solid lines) and when already attended (dotted lines). The plotted quantities were averages obtained for the indicated and An values after performing 100 runs. (Left column) Setup with peak rate and peak enhancement values similar to Figure 8, right, of Ling et al. (2009). (Right column) Situation yielding peak rate and peak enhancement height similar to those in Figure 4A of Martínez-Trujillo and Treue (2004).

Figure 9:

Firing rates (top row) and averaged zero-lag cross-covariance C ( case of equation 3.1 (bottom row), for presentations of feature , now with , when not yet attended (solid lines) and when already attended (dotted lines). The plotted quantities were averages obtained for the indicated and An values after performing 100 runs. (Left column) Setup with peak rate and peak enhancement values similar to Figure 8, right, of Ling et al. (2009). (Right column) Situation yielding peak rate and peak enhancement height similar to those in Figure 4A of Martínez-Trujillo and Treue (2004).

4.3  Outlook and Final Remarks

Attention increases the responses of neurons with preferences close to the attended feature. In the case of motion direction, which relates in many aspects to orientation, a feature-similarity gain model was developed to explain these and other findings (Martínez-Trujillo & Treue, 2004; Treue & Martínez-Trujillo, 1999). The principles underlying that model are at the core of many implementations, including the one we have presented here. Top-down attention modulations changed the gain of individual sensory neurons in a multiplicative way, to an extent proportional to the similarity between the attended feature and the cell’s preference. Responses of cells tuned to similar features were enhanced, while responses of cells that preferred very different features became suppressed. At the level of neuronal populations, these gain changes modulated the selectivity of the population response (Martínez-Trujillo & Treue, 2004), and stimuli sharing features similar to those in the target were represented more strongly.

Other relevant contributions made decisive steps in establishing key ideas about biased competition, from its conceptual explanation (Desimone & Duncan, 1995) to specific mechanisms like the biased competition model (Reynolds, Chelazzi, & Desimone, 1999). In the normalization model (Reynolds & Heeger, 2009; Herrmann et al., 2012), the stimulus drive is multiplied by the attention field and divided by the suppressive drive to yield the output firing rates.

Since synchrony and phase mechanisms have already been investigated from so many perspectives, the related bibliography is very rich and includes a large number of remarkable works. Compte and Wang (2006) explored the mechanisms of attention-induced receptive field (RF) shifts in cortical network models that receive an attentional spotlight. The RF size was enlarged (respectively, reduced) by attentional signal directed near a cell’s RF center in a recurrent network (resp., a feedforward network). Mishra, Fellous, and Sejnowski (2006) simulated a multicompartment model of V4 cortical neurons that received inputs from two V2 excitatory synaptic pools and analyzed the responses to a variety of combined inputs, including the absence and presence of spike correlations. The results of that highly realistic and elaborate model predict that top-down attention may bias the V4 neuron’s response by an inhibitory correlation phase shift mechanism.

Buia and Tiesinga (2008; see also Tiesinga & Buia 2008) proposed synchrony modulation by interneurons. Buelhman and Deco (2008) studied the coexistence of the rate modulation and synchronization modulation using a theoretical framework with a network consisting of inhibitory and excitatory neurons organized in pools. Börgers and Kopell (2008) showed that the competitive advantage of coherent excitatory stimuli was amplified greatly when the target included inhibitory interneurons acting via GABA-receptor mediated synapses and the coherent input oscillated at gamma frequency. Neurophysiological evidence of gating through phase relation–dependent mechanisms is offered by Grothe et al. (2015). Bichot and Desimone (2006) showed how attentional selection appears to be mediated by changes in the synchrony of responses of neuronal populations, in addition to the modulation of the firing rate of individual neurons.

In this letter, we wished to test a relatively simple network organization leading to the reproduction of higher responses for attended cues and involving the appearance of roughly synchronous regimes. For this purpose, we have devised a two-layered neural model that incorporates a spike-mediated form of approximate synchronization by differential lateral couplings among the bottom-layer columns or stacks. The top-layer neurons, which integrate the spiking activity coming from below, increase their responses if their preferences are close enough to the attended feature. To this end, the functioning of our lateral couplings among stacks appears to suffice. In fact, the effect of the synaptic efficacies’ zj’s on the recurrent connections was decisive. This success does not rule out possible solutions based on feedback signals, as lateral interactions can often be implemented, in an equivalent fashion, with the help of new layers and feedback pathways producing similar effects. Besides that general idea, our scheme includes a particular type of feedback from layer 2 to the layer 1 couplings in equation 2.5, which provides a handy evolution rule for the synchronizing couplings and implicitly involves competition through limitation of their sum.

The model we have presented describes featural attention, but the involved effects could possibly be used to model spatial attention by means of spatial clustering (rather than featural clustering) of neural populations. Our findings show that the described form of synchronization is effective. During simultaneous presentations of two different features to our system, the attended one gives rise to a higher response, as expected. For the case of sequential displays of two stimuli, repetition of the orientation also results in an enhanced spiking response within the correct frequency region. The enhancement is altogether evidenced by greater degrees of coherence, higher spiking rates, and if we allowed frequencies to fall below the typical region, a shorter spike latency. Rate modulations resulting from our model agree with published results. Selecting particular values of the layer 2 strength and the synaptic noise levels , we have obtained rate peak values and rate enhancement maxima like those found by Ling et al. (2009) and Martínez-Trujillo and Treue (2004).

It is sensible to wonder whether an artificial increase in the concentration of neurotransmitters could boost what we have represented as the synchronizing strength (depicted in Figure 7). When this strength is doubled by going from 35 to 70, synchrony measures grows from 50 to 350 for first stimuli and from 100 to 650 for cued stimuli (i.e., they are multiplied by a factor on the order of 7 in both cases). The model predicts that as synchrony and firing rate values go through such increases, the height of the rate enhancement after switching from unattended to attended stimuli is maintained around 20 Hz (in the considered range). On the whole, the magnitude of this jump does not seem to depend too much on the initial degree of synchrony. We can thus speculate that for an experimental paradigm yielding measurable synchrony changes, the detected jumps in firing rates ought to result in approximately constant differences, at least within some bounds. To this end, a possible setup could be a variant of the experiment by Supèr, van der Togt, Spekreijse, and Lamme (2003; see also van der Togt, Kalitzin, Spekreijse, Lamme, & Supèr, 2006), now with the animals watching single-feature stimuli, and including some factor capable of conditioning their synaptic efficacies like the gradual administration of some neurochemical substance.

The phenomenon of synchronization is at the heart of some biological functions and constitutes a plausible abstraction for many processes in fields like genetic networks, circadian rhythms or ecology (Elowitz & Leibler, 2000; Pikovsky, Rosenblum, & Kurths, 2001; Strogatz, 2003; Keith, 1963; Earn, Levin, & Rohani, 2000). Although neural dynamics is far more complex than phase representations, some of the steps highlighting the path to synchrony can be viewed as a good starting point. We feel our network is interesting enough, as similar systems of coupled units including these elements might offer an alternative tool for further studies about synchronization and coherence (Fries, 2005), a key issue of neuronal communication.

Acknowledgments

This work was supported by the Spanish Ministerio de Economía y Comptetitividad, project no. PSI2014-57454-P.

References

Attwell
,
D.
, &
Laughlin
,
S. B.
(
2001
).
An energy budget for signaling in the grey matter of the brain
.
Journal of Cerebral Blood Flow Metabolism
,
21
,
1133
1145
.
Bichot
,
N. P.
, &
Desimone
,
R.
(
2006
).
Finding a face in the crowd: Parallel and serial neural mechanisms of visual selection
.
Progress in Brain Research
,
155
,
147
156
.
Börgers
,
C.
, &
Kopell
,
N. J.
(
2008
).
Gamma oscillations and stimulus selection
.
Neural Computation
,
20
,
383
414
.
Bosman
,
C. A.
,
Schoffelen
,
J. M.
,
Brunet
,
N.
,
Oostenveld
,
R.
,
Bastos
,
A. M.
,
Womelsdorf
,
T.
, … 
Fries
,
P.
(
2012
).
Attentional stimulus selection through selective synchronization between monkey visual areas
.
Neuron
75
,
875
888
. http://dx.doi.org/10.1016/j.neuron.2012.06.037
Buehlmann
,
A.
, &
Deco
,
G.
(
2008
).
The neuronal basis of attention: Rate versus synchronization modulation
.
Journal of Neuroscience
,
28
,
7679
7686
.
Buia
,
C. I.
, &
Tiesinga
,
P. H.
(
2008
).
Role of interneuron diversity in the cortical microcircuit for attention
.
J. Neurophysiol.
,
99
,
2158
2182
.
Carrasco
,
M.
(
2011
).
Visual attention: The past 25 years
.
Vision Research
,
51
,
1484
1525
.
Compte
,
A.
, &
Wang
,
X-J.
(
2006
).
Tuning curve shift by attention modulation in cortical neurons: A computational study of its mechanisms
.
Cerebral Cortex
,
16
,
761
778
. doi:10.1093/cercor/bhj021
Desimone
,
R.
, &
Duncan
,
J.
(
1995
).
Neural mechanisms of selective attention
.
Annu. Rev. Neurosci.
,
18
,
193
222
.
Earn
,
D.J.D.
,
Levin
,
S. A.
, &
Rohani
,
P.
(
2000
).
Coherence and conservation
.
Science
,
290
,
1360
1364
.
Elowitz
,
M.
, &
Leibler
,
S.
(
2000
).
A synthetic oscillatory network of transcriptional regulators
.
Nature
,
403
,
335
338
.
Fries
,
P.
(
2005
).
A mechanism for cognitive dynamics: Neuronal communication through neuronal coherence
.
Trends in Cognitive Science
,
9
,
474
480
.
Fries
,
P.
,
Reynolds
,
J. H.
,
Rorie
,
A. E.
, &
Desimone
,
R.
(
2001
).
Modulation of oscillatory neuronal synchronization by selective visual attention
.
Science
,
291
,
1560
1563
.
Fries
,
P.
,
Roelfsema
,
P. R.
,
Engel
,
A. K.
,
König
,
P.
, &
Singer
,
W.
(
1997
).
Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry
.
Proc. Natl. Acad. Sci. USA
,
94
,
12699
12704
.
Fries
,
P.
,
Womelsdorf
,
T.
,
Oostenveld
,
R.
, &
Desimone
,
R.
(
2008
).
The effects of visual attention on rhythmic neuronal synchronization in macaque area V4
.
Journal of Neuroscience
,
28
,
4823
4835
.
Grossberg
,
S.
(
1976
).
Adaptive pattern classification and universal recording: I. Parallel development and coding of neural feature detectors
.
Biological Cybernetics
,
23
,
121
134
.
Grothe
,
I.
,
Rotermund
D.
,
Neitzel
,
S. D.
,
Mandon
,
S.
,
Ernst
,
U. A.
,
Kreiter
,
A. K.
, &
Pawelzik
,
K. R.
(
2015
).
Attention selectively gates afferent signal transmission to area V4. bioRxiv
. doi:http://dx.doi.org/10.1101/019547
Hamker
,
F. H.
(
2004
).
Predictions of a model of spatial attention using sum- and max-pooling functions
.
Neurocomputing
,
56C
,
329
343
.
Hamker
,
F. H.
(
2006
).
Modeling feature-based attention as an active top-down inference process
.
BioSystems
,
86
,
91
99
.
Herrmann
,
K.
,
Heeger
,
D. J.
, &
Carrasco
,
M.
(
2012
).
Feature-based attention enhances performance by increasing response gain
.
Vision Research
,
74
,
10
20
.
Itti
,
L.
, &
Koch
,
C.
(
2001
).
Computational modeling of visual attention
.
Nature Reviews Neuroscience
,
2
,
194
203
.
Izhikevich
,
E. M.
(
2003
).
Simple model of spiking neurons
.
IEEE Trans Neural Netw.
,
14
,
1569
1572
.
Izhikevich
,
E. M.
(
2007
).
Dynamical systems in neuroscience: The geometry of excitability and bursting
.
Cambridge, MA
:
MIT Press
.
Keith
,
L.
(
1963
).
Wildlife’s 10-year cycle
.
Madison
:
University of Wisconsin Press
.
Lennie
,
P.
(
2003
).
The cost of cortical computation
.
Current Biology
,
13
,
493
497
.
Ling
,
S.
,
Liu
,
T.
, &
Carrasco
,
M.
(
2009
).
How spatial and feature-based attention affect the gain and tuning of population responses
.
Vision Research
,
49
,
1194
1204
.
Martínez-Trujillo
,
J. C.
, &
Treue
,
S.
(
2004
).
Feature-based attention increases the selectivity of population responses in primate visual cortex
.
Current Biology
,
14
,
744
751
.
Mishra
,
J.
,
Fellous
,
J-M.
, &
Sejnowski
,
T. J.
(
2006
).
Selective attention through phase relationship of excitatory and inhibitory input synchrony in a model cortical neuron
.
Neural Networks
,
19
,
1329
1346
.
Pikovsky
,
A. M.
,
Rosenblum
,
J.
, &
Kurths
,
J.
(
2001
).
Synchronization
.
Cambridge
:
Cambridge University Press
.
Reynolds
,
J. H.
,
Chelazzi
,
L.
, &
Desimone
,
R.
(
1999
).
Competitive mechanisms subserve attention in macaque areas V2 and V4
.
Journal of Neuroscience
,
19
,
1736
1753
.
Reynolds
,
J. H.
, &
Heeger
,
D. J.
(
2009
).
The normalization model of attention
.
Neuron
,
61
,
168
185
.
Schelter
,
B.
,
Winterhalder
,
M.
, &
Timmer
,
J.
(
2007
).
Phase synchronization and coherence analysis: Sensitivity and specificity
.
International Journal of Bifurcation and Chaos
,
17
,
3351
3356
.
Sompolinsky
,
H.
,
Golomb
,
D.
, &
Kleinfeld
,
D.
(
1991
).
Cooperative dynamics in visual processing
.
Physical Review
,
A43
,
6990
7011
.
Strogatz
,
S. H.
(
2003
).
Sync: The emerging science of spontaneous order
.
New York
:
Hyperion
.
Supèr
,
H.
, &
Romeo
,
A.
(
2014
).
Approximate emergent synchrony in spatially coupled spiking neurons with discrete interaction
.
Neural Computation
,
26
,
2419
2440
.
Supèr
,
H.
,
van der Togt
,
C.
,
Spekreijse
,
H.
, &
Lamme
,
V.A.F.
(
2003
).
Internal state of monkey primary visual cortex (V1) predicts figure–ground perception
.
J. Neurosci.
,
23
,
3407
3414
.
Tiesinga
,
P. H. E.
, &
Buia
,
C. I.
(
2008
).
Modulation of synchrony by interneurons: Insights from attentional modulation of responses in the visual cortex
. In
I.
Soltesz
&
K.
Staley
(Eds.),
Computational neuroscience in epilepsy
(pp.
306
321
).
New York
:
Elsevier
.
van der Togt
,
C.
,
Kalitzin
,
S.
,
Spekreijse
,
H. Lamme
,
V.A.F.
, &
Supèr
,
H.
(
2006
).
Synchrony dynamics in monkey V1 predict success in visual detection
.
Cerebral Cortex
,
16
,
136
148
. doi:10.1093/cercor/bhi093
Treue
,
S.
, &
Martínez-Trujillo
,
J. C.
(
1999
).
Feature-based attention influences motion processing gain in macaque visual cortex
.
Nature
,
399
,
575
579
.
Watanabe
,
T.
,
Sasaki
,
Y.
,
Miyauchi
,
S.
,
Putz
,
B.
,
Fujimaki
,
N.
,
Nielsen
,
M.
, …
Miyakawa
,
S.
(
1998
).
Attention-regulated activity in human primary visual cortex
.
Journal of Neurophysiology
,
79
,
2218
2221
.