Abstract

Neural information combination problems are ubiquitous in cognitive neuroscience. Two important disciplines, although conceptually similar, take radically different approaches to these problems. Sensory binding theory is largely grounded in synchronization of neurons responding to different aspects of a stimulus, resulting in a coherent percept. Sensory integration focuses more on the influences of the senses on each other and is largely grounded in the study of neurons that respond to more than one sense. It would be desirable to bridge these disciplines, so that insights gleaned from either could be harnessed by the other. To link these two fields, we used a binding-like oscillatory synchronization mechanism to simulate neurons in rattlesnake that are driven by one sense but modulated by another. Mutual excitatory coupling produces synchronized trains of action potentials with enhanced firing rates. The same neural synchronization mechanism models the behavior of a population of cells in cat visual cortex that are modulated by auditory activation. The coupling strength of the synchronizing neurons is crucial to the outcome; a criterion of strong coupling (kept weak enough to avoid seriously distorting action potential amplitude) results in intensity-dependent sensory enhancement—the principle of inverse effectiveness—a key property of sensory integration.

INTRODUCTION

Many key problems in cognitive neuroscience involve combining information carried by different neural channels. Two fields of cognitive neuroscience—sensory binding and sensory integration—address seemingly related information combination problems in distinctive ways. Sensory binding theory addresses the coordination of sensory responses to yield a coherent percept. The dominant paradigm in binding theory is oscillatory synchronization: Neurons driven by common physical stimuli tend to synchronize their spiking activity (Fries, 2009; Singer & Gray, 1995; Gray, Konig, Engel, & Singer, 1989; Gray & Singer, 1989). Gamma-band (ca. 30–80 Hz) synchronization seems important to binding, and electrophysiological signatures of coherent γ-band activity are predictive of image segmentation and perceptual multistability (Tallon-Baudry & Bertrand, 1999). Neural synchronization has also been implicated in memory, attention, sensory discrimination, stimulus–response behaviors, and some neural disorders (Suppes, de Barros, & Oas, 2012; Lakatos et al., 2009; Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008; Uhlhaas & Singer, 2006; Stopfer, Bhagavan, Smith, & Laurent, 1997; Lisman & Idiart, 1995). Conversely, sensory integration deals with interactions and influences between the major sensory categories: primarily visual, auditory, and tactile. Although there are tantalizing clues that sensory integration may make use of binding-like oscillatory synchronization mechanisms (Kanayama, Sato, & Ohira, 2009; Bauer, 2008; King, 2008; Schroeder, Lakatos, Kajikawa, Partan, & Puce, 2008; Senkowski, Scheider, Foxe, & Engel, 2008; Lakatos, Chen, O'Connell, Mills, & Schroeder, 2007; Kisley & Cornwell, 2006; Sakowitz, Quiroga, Schurmann, & Basar, 2001), the dominant paradigm in sensory integration is the study of neurons that are driven by multiple senses (Stein & Meredith, 1993; Meredith & Stein, 1983). Large populations of such bimodal and trimodal cells are found in the superior colliculus/optic tectum and sensory cortices of many animals (especially at the borders between cortical sensory regions; Wallace, Ramachandran, & Stein, 2004). These multimodal cells often show extraordinary relative firing rate augmentations (increases on the order of 1000% are found in superior colliculus) when driven simultaneously by multiple senses (Stein & Meredith, 1993), but only for weak (but not necessarily the very weakest) stimuli (Holmes, 2007). This intensity-dependent sensory enhancement—the principle of inverse effectiveness—is perhaps the most celebrated finding in sensory integration (Alvardo, Stanford, Vaughan, & Stein, 2007; Ross, Saint-Amour, Leavitt, Javitt, & Foxe, 2007; Stanford & Stein, 2007). Lakatos et al. (2007) call the principle of inverse enhancement “one of the best agreed-upon observations about multisensory interactions.”

Recently, there has been interest in neurons and neural assemblies that are driven directly by one sense and modulated by another (Allman, Keniston, & Meredith, 2008, 2009; Meredith & Allman, 2009; Clemo, Allman, Meredith, & Shrama, 2008; Allman & Meredith, 2007; Lakatos et al., 2007; Barraclough, Xiao, Baker, Oram, & Perrett, 2005; Newman & Hartline, 1981, 1982). These modulated unisensory cells1 also obey the principle of inverse effectiveness; their firing rate enhancement is limited to a maximum of about 100%, but because these cells are numerous in some cortical areas, their summed activity may dominate some perceptual behaviors and mass action measures of multisensory integration (Meredith & Clemo, 2010; Allman et al., 2009; Laurienti, Perrault, Stanford, Wallace, & Stein, 2005). A remarkable example of mass action in modulated unisensory interactions is Convento, Vallar, Galantini, and Bolognini's (2013) observation that auditory and tactile stimulation make it easier to induce visual phosphenes using TMS of occipital cortex; this phosphene enhancement obeys the principle of inverse effectiveness, which Convento et al. ties to sensory binding.

If the gap between sensory integration and binding were bridged, it might be possible to marshal insights from each discipline to aid the other. Intriguingly, the enhancement properties of modulated unisensory cells resemble the behavior of some neural synchronization mechanisms, which we explore first in rattlesnake optic tectum and then in mammalian sensory cortex, using the same neural synchronization model. These modeled neural enhancement effects, including intensity-dependent enhancement, resemble those found in human perception.

THEORETICAL METHODS: COUPLED NEURAL OSCILLATOR MODELS

Since its 1655 discovery by Huygens in adjacent pendulum clocks, oscillatory synchronization has been studied in many biological, electrical, mechanical, and chemical systems (Strogatz, 2003; Pikovsky, Rosenblum, & Kurths, 2001; Kelso, 1995). It models central pattern generators in locomotion, pacemaker oscillations, sleep–wake cycles, and sensory binding. Coupling can be mechanical, electrical, chemical, or optical. In many systems, the coupling strength is fixed; coupling is push–pull and fast compared with the oscillations. Such coupled oscillators—with relatively similar frequencies—can synchronize in near-zero phase at a compromise frequency that lies between the two oscillators. However, neural coupling can be either excitatory or inhibitory, and the delays in signaling between neurons are usually a significant fraction of the period of oscillation. These neural oscillators can synchronize at firing rates well above the faster of the two oscillators or well below the slower one. Moreover, coupling strength need not be constant. Wilson (1999a, 1999b) derived a set of coupled differential equations that models the essential neural behaviors.
formula

When these equations are numerically integrated, the V variables describe the voltages of the spiking neurons A and B, and the R variables describe the neurons' recoveries from spiking. The f and g variables describe synaptic coupling and implement Rall's alpha function, resulting in realistic excitatory postsynaptic potentials in the affected neurons. Hstep is a step function that sets the synaptic current to a constant level, and k is the strength of the synaptic coupling. Fourth-order Runge–Kutta integration of the coupled differential equations (Equation 1) was performed in MATLAB using modified and repurposed versions of software that accompanies Wilson (1999a). Wilson (1999a, 1999b) optimized this model for mammalian Class I neurons, but it mimics Class I action potentials in many species, including rattlesnake (Pappas, Motamedi, & Christensen, 2004). (Class I neurons have firing rates that are graded functions of stimulation level above their low threshold; Class II neurons, much studied in squid, jump to a very fast firing rate above threshold.) Although this model could be reoptimized for each species, for our purposes, it is sufficiently generic to serve. The simplicity of the model and the small size of the network are deliberate; we do not claim that the model is truly descriptive of neural assemblies. We use this level of description because it is the simplest synchronization model that is capable of capturing the neural behaviors underlying neural enhancements; we deliberately avoid computational elaborations, however realistically desirable, that might obscure insight into this basic behavior. There were two sets of simulations performed with this model, one simulating light and heat integration in pit vipers and another simulating visual and auditory integration in cat.

THEORETICAL RESULTS

Model 1: Reciprocal Visual and Infrared (IR) Enhancements in Pit Viper Optic Tectum

Background: How the Lowly Rattlesnake Is Poised in the Gap between Sensory Integration and Binding

Pit vipers are named after the two facial pits mounted below their eyes, each pit is a primitive pinhole camera containing a sensory membrane thermally isolated from the rest of the snake, and IR light is transduced by warming heat-gated ion channels exposed on the cell membranes of demylinated trigeminal nerve masses (Pappas et al., 2004; Campbell, Naik, Sowards, & Stone, 2002; Newman & Hartline, 1981, 1982; Hartline, Kass, & Loop, 1978; Kass, Loop, & Hartline, 1978).2 In humans, a similar trigeminal demylination results in excruciatingly painful trigeminal neuralgia; in rattlesnakes this is not a bug but a feature. The modulated unisensory neurons modeled here were first described in rattlesnake optic tectum (Newman & Hartline, 1981, 1982), for the integration of visual information carried by the optic nerve from the rattlesnake's eye and IR information carried initially by the trigeminal nerve from the facial pit. Our focus on rattlesnake is not historical. Rather, we are driven by the ambiguous status of information combination in rattlesnake, which is usually treated as a sensory integration problem, almost identical to the other instances of sensory integration in tectal cells of many species. However, it is also natural to treat visual/IR integration as analogous to the binding of other wavelength-selective mechanisms (Billock & Tsou, 2004, 2005); for example, the binding of hue and luminance into chromatic saturation and chromatic brightness. This similarity to a binding problem leads us to model rattlesnake modulated-unisensory neurons as members of synchronized complementary pairs, each driven by one sense and modulated by synchronization with the other sensory neuron.

Figure 1 shows examples of six categories of rattlesnake tectal neurons (Newman & Hartline, 1981). Two of these are bisensory cells—cell types that have received much deserved attention in the sensory integration literature (Stein & Meredith, 1993; Meredith & Stein, 1983; Hartline et al., 1978). These cells respond to both IR and visual stimulation. The other four types are modulated-unisensory neurons. In other animals, these are found in sensory cortex (Meredith & Clemo, 2010; Meredith & Allman, 2009; Allman et al., 2008; Clemo et al., 2008; Allman & Meredith, 2007; Barraclough et al., 2005). Unlike other animals, in rattlesnake optic tectum there are more modulated-unisensory cells than bimodal cells. The rattlesnake unisensory cell types are IR-driven cells enhanced or suppressed by simultaneous visual stimulation and visual-driven cells enhanced or suppressed by simultaneous IR stimulation. In the optic tectum/superior colliculus of most animals, suppression is often situationally dependent, caused for example, by using stimuli from nonoverlapping regions of the animal's extrapersonal space (Stein & Meredith, 1993). Newman and Hartline (1981, 1982) suggest that suppressed-modulation neurons contribute to discrimination of cold-blooded versus warm-blooded prey by rattlesnakes; similar suppressed-modulation cells of obscure purpose are found in the auditory and visual cortex of several animals (Meredith & Clemo, 2010; Allman et al., 2009; Barraclough et al., 2005). This study addresses only sensory enhancement.

Figure 1. 

Six neurons, each typical of cell classes involved in visual/IR interactions in the optic tectum of rattlesnake (Newman & Hartline, 1981). Each row shows five series of action potential spike trains elicited by visual, IR, and combined stimulation. Cells A and D are true multisensory cells; cell A is typical of multisensory cells found in the superior colliculus in many species. Cells B, C, E, and F are unisensory cells (driven by only one sense) but whose activity can be modulated by the other sense. Only cells B (a visual neuron whose 3.8-spikes/sec average firing rate is increased to 63% by simultaneous IR stimulation) and C (an IR neuron whose 5.6-spikes/sec average firing rate is increased to 100% by simultaneous visual stimulation) lie within the scope (multisensory firing rate enhancement in unisensory cells) of this study. Reproduced by permission of the American Association for the Advancement of Science.

Figure 1. 

Six neurons, each typical of cell classes involved in visual/IR interactions in the optic tectum of rattlesnake (Newman & Hartline, 1981). Each row shows five series of action potential spike trains elicited by visual, IR, and combined stimulation. Cells A and D are true multisensory cells; cell A is typical of multisensory cells found in the superior colliculus in many species. Cells B, C, E, and F are unisensory cells (driven by only one sense) but whose activity can be modulated by the other sense. Only cells B (a visual neuron whose 3.8-spikes/sec average firing rate is increased to 63% by simultaneous IR stimulation) and C (an IR neuron whose 5.6-spikes/sec average firing rate is increased to 100% by simultaneous visual stimulation) lie within the scope (multisensory firing rate enhancement in unisensory cells) of this study. Reproduced by permission of the American Association for the Advancement of Science.

Coupled Neural Oscillator Model of Enhanced Unisensory Cells in Rattlesnake

Firing rate enhancement of rattlesnake unisensory cells is simulated by the coupled neural oscillator model (Equation 1) for positive coupling (k) strengths. Numerical integration of Equation 1 yields the firing rate patterns shown in Figure 2. “Instantaneous” firing rates were computed from the average of the interspike intervals to model the averaged noninteger spike rates given in Newman and Hartline (1981) and Figure 1. In Figure 2A and B, we simulate Newman and Hartline's (1981) IR-enhanced visual neuron (Figure 1B). In Figure 2A, the blue spikes are action potentials (3.8 spikes/sec) of a neuron driven only by visual stimulation. The red trace does not appear in Newman and Hartline; it describes a second neuron that can only be driven by IR stimuli. Note that this cell is very weakly driven—2 spikes/sec—1 spike/sec above its threshold firing rate. In Figure 2A, the two cells are uncoupled to simulate nonsimultaneous stimulation. In Figure 2B, the same cells have been coupled to simulate simultaneous stimulation. The coupled neurons synchronize their action potentials at zero phase, which is significant for binding theory, but the key outcome is that, for strong coupling (k = 10.25), they are synchronized at higher firing rates (6.2 spikes/sec)—the 63% firing rate enhancement shown by the rattlesnake visual neuron with simultaneous IR stimulation (Figure 1B). Similarly, Figure 2C shows action potentials (red) of an unmodulated IR-sensitive neuron, driven to fire at 5.6 spikes/sec. The blue trance is a simulation of a very weakly stimulated (2 spikes/sec) visual neuron. When strongly coupled (k = 10.35), the 100% enhancement that Hartline and Newman found for the IR cell with simultaneous visual stimulation is closely mimicked. Of course, the experimentally unobserved weaker modulating cell would enjoy an even larger enhancement, similar to those observed in superior colliculus in many animals (Stein & Meredith, 1993; Meredith & Stein, 1983).

Figure 2. 

Simulations of rattlesnake optic tectum unisensory enhanced cells using Equation 1. (A) Blue trace shows a visual neuron firing at 3.8 spikes/sec, simulating the visual-stimulation-only condition of Figure 1B. The red trace represents a neighboring IR-stimulated neuron firing at above (2 spikes/sec) its ca. of 1-spike/sec threshold. (B) If the two cells are stimulated simultaneously and are mutually excitatory with strong coupling (k = 10.25), they synchronize in-phase at 6.2 spikes/sec, the 63% firing rate increase reported for the simultaneous stimulation condition of Figure 1B. (C) The red trace represents an IR neuron, firing at 5.6 spikes/sec, simulating the IR-stimulation-only condition of Figure 1C. The blue trace represents a nearby visual neuron, very weakly stimulated, firing at 2 spikes/sec. (D) If the two cells are stimulated simultaneously and are mutually excitatory with strong coupling (k = 10.35), they synchronize in-phase, at 11.2 spikes/sec, the 100% firing rate increase reported for the simultaneous stimulation condition of Figure 1C. (For consistency, all spike counts in this and subsequent figures are computed from the average reciprocal of the interspike intervals—the so-called mean instantaneous firing rate.)

Figure 2. 

Simulations of rattlesnake optic tectum unisensory enhanced cells using Equation 1. (A) Blue trace shows a visual neuron firing at 3.8 spikes/sec, simulating the visual-stimulation-only condition of Figure 1B. The red trace represents a neighboring IR-stimulated neuron firing at above (2 spikes/sec) its ca. of 1-spike/sec threshold. (B) If the two cells are stimulated simultaneously and are mutually excitatory with strong coupling (k = 10.25), they synchronize in-phase at 6.2 spikes/sec, the 63% firing rate increase reported for the simultaneous stimulation condition of Figure 1B. (C) The red trace represents an IR neuron, firing at 5.6 spikes/sec, simulating the IR-stimulation-only condition of Figure 1C. The blue trace represents a nearby visual neuron, very weakly stimulated, firing at 2 spikes/sec. (D) If the two cells are stimulated simultaneously and are mutually excitatory with strong coupling (k = 10.35), they synchronize in-phase, at 11.2 spikes/sec, the 100% firing rate increase reported for the simultaneous stimulation condition of Figure 1C. (For consistency, all spike counts in this and subsequent figures are computed from the average reciprocal of the interspike intervals—the so-called mean instantaneous firing rate.)

Our choice of the weaker channel spiking at 2 Hz for both examples (over 1 spike/sec above threshold) is arbitrary; as Figure 3 shows, a variety of weaker channel strengths (expressed in uncoupled spike rates) all result in similar enhanced spike rates in the synchronized pair; the synchronization frequency is controlled mostly by the firing rate of the stronger channel if (and only if) the coupling strength is held constant. However, strong coupling carries some costs. Note in Figure 2 that the weaker (uncoupled) oscillators have their spikes' 25-mV peak amplitudes (ca. 100-mV range) suppressed by several millivolts when coupled and synchronized. This is not just a feature of Wilson's model or neural systems but is generic to coupled oscillators (Kopell, 1988). It has special neural significance because strongly suppressed spike amplitudes may propagate poorly (e.g., depolarization block) and be less effective for inducing neurotransmitter release (Meeks, Jiang, & Mennerick, 2005; Callaway & Ross, 1995).3 This is especially found in epileptiform activity and may limit seizures by suppressing the spread of the activity (Meeks et al., 2005). Two examples are shown in Figure 3B and C; the runaway behavior of Figure 3C resembles epileptiform activity and Class II excitability. In general, amplitude suppression is worst for stronger coupling strengths, higher spike rates, and bigger differences in the uncoupled frequencies being synchronized. It follows that spike amplitude suppression can be controlled by weakening coupling strength, yet synchronizing noticeably different firing rates requires strong coupling. We sought a “Goldilocks” solution of just-weak-enough adaptive coupling that should have evolved to maximize synchronization's strengths while minimizing its drawbacks—an Ideal Observer model for multisensory enhancement. We adopted a criterion of keeping synchronized peak spike amplitude at least at 95% of its baseline value (measured for very weak coupling), and we computed the maximum tolerable value of k as a function of the firing rate of the more strongly driven neuron. Figure 4 shows five versions of this, computed for coupled oscillators with uncoupled spike rate differences that range from 0.25 octaves to 1.25 octaves. Coupling strength is set to a maximum value of 10.5 at the lowest levels of stimulation; a slightly higher strength of 10.65–10.7 leads to severe spike compression/runaway firing behaviors like those shown in Figure 3C. As Figure 4 shows, the maximum coupling strength (which suppresses spike voltage by no more than 5%) declines monotonically with increasing firing rate of the stronger neuron and also declines with increasing difference in the uncoupled firing rates of two coupled neurons.

Figure 3. 

(A) The weaker channel's firing rate does not matter much if (and only if) k in Equation 1 is constant. For constant coupling strength, the enhanced synchronized firing rates produced by coupling two unequally stimulated neurons are determined by the more strongly stimulated neuron. Here, the stronger neuron fires at 2, 4, and 5.6 spikes/sec when uncoupled, and the weaker neuron is given many different stimulation strengths. The coupling constant is set to a very high value (10.5), which poses few problems for low spike rates. However, strong coupling between oscillators of very different driven frequencies can distort action potentials, especially at high firing rates, suggesting that perhaps coupling strength should not be held constant. (B) Too strong coupling (k = 10.21) of 16- and 4-spike/sec neurons, resulting in synchronization at 44 spikes/sec and drastic distortion of the (blue) spikes of the more weakly stimulated neuron. (C) A slightly stronger coupling (k = 10.22) yields drastic spike distortions for both oscillators and runaway firing rates (234 spikes/sec) resembling Class II excitability or epileptiform activity.

Figure 3. 

(A) The weaker channel's firing rate does not matter much if (and only if) k in Equation 1 is constant. For constant coupling strength, the enhanced synchronized firing rates produced by coupling two unequally stimulated neurons are determined by the more strongly stimulated neuron. Here, the stronger neuron fires at 2, 4, and 5.6 spikes/sec when uncoupled, and the weaker neuron is given many different stimulation strengths. The coupling constant is set to a very high value (10.5), which poses few problems for low spike rates. However, strong coupling between oscillators of very different driven frequencies can distort action potentials, especially at high firing rates, suggesting that perhaps coupling strength should not be held constant. (B) Too strong coupling (k = 10.21) of 16- and 4-spike/sec neurons, resulting in synchronization at 44 spikes/sec and drastic distortion of the (blue) spikes of the more weakly stimulated neuron. (C) A slightly stronger coupling (k = 10.22) yields drastic spike distortions for both oscillators and runaway firing rates (234 spikes/sec) resembling Class II excitability or epileptiform activity.

Figure 4. 

Optimal strength coupling. Coupling strengths that result in maximal synchronized enhancement without reducing the weaker oscillators' action potential peak response below a criterion of 20.45 mV (95% of baseline amplitude). Computed for 22 stronger firing neurons with spike rates between 1 and 46 spikes/sec for five conditions: weaker neuron firing at 0.25, 0.5, 0.75, 1, and 1.25 octaves below the stronger oscillator. Above 5 spikes/sec (uncoupled), the optimal coupling strength declines with increasing differences between spike rates of the two oscillators.

Figure 4. 

Optimal strength coupling. Coupling strengths that result in maximal synchronized enhancement without reducing the weaker oscillators' action potential peak response below a criterion of 20.45 mV (95% of baseline amplitude). Computed for 22 stronger firing neurons with spike rates between 1 and 46 spikes/sec for five conditions: weaker neuron firing at 0.25, 0.5, 0.75, 1, and 1.25 octaves below the stronger oscillator. Above 5 spikes/sec (uncoupled), the optimal coupling strength declines with increasing differences between spike rates of the two oscillators.

Model 2: Auditory Enhancements of Unisensory Neurons in Cat Visual Cortex

Two published rattlesnake cells seem an inadequate foundation for modeling. Fortunately, populations of similar cells have been found in mammalian cortex. A particularly large sample are Allman et al.'s (2009) 37 “subthreshold unimodal cells” found in the extrastriate visual cortex—posterolateral lateral suprasylvian (PLLS)—of cats; these neurons' responses resemble the enhanced unisensory responses of rattlesnake tectal neurons. The data shown in Figure 5 have been transformed from spikes/600 msec trial to spikes/sec for comparison with the model. These data are typical of many studies that show up to 100% multisensory enhancement for weakly stimulated unisensory neurons. For comparison, the model of Equation 1 is superimposed on Allman et al.'s data as a solid line. Because we cannot know the strength of the unmeasured modulating auditory neurons, we modeled five possibilities (previously shown in Figure 4) in which the weaker channel had a firing rate of 0.25, 0.50, 0.75, 1.0, and 1.25 octaves below the firing rate of the stronger channel (simulations for larger separations cover a narrow range of spike frequencies and add little to the analysis). The optimal coupling strengths plotted in Figure 4 are used as the k terms in Equation 1; integrating Equation 1 gives predictions of the synchronized enhanced firing rate of the cat visual neuron. In Figure 5, we average the outcomes of these numeric experiments (roughly equivalent to assuming that the modulating frequencies are uniformly distributed). Figure 6A shows the same data and averaged model, replotted in a percent enhancement framework. Allman et al.'s percent enhancement data beautifully embody the principle of inverse effectiveness, and the synchronized neural model captures its essential properties, with a peak enhancement of circa 100% for neurons whose weak visual stimulation alone produces less than 5 spikes/sec and a much lower enhancement (about 14%) for neurons that are strongly stimulated. Psychophysical studies of humans have been made under circumstances similar to the auditory modulation of visual cells in cat; an auditory modulation (even an irrelevant one) makes test signals look brighter and enhances detection, just as visual modulation enhances auditory detection (Schirillo, 2011; Lovelace, Stein, & Wallace, 2003; Stein, London, Wilkinson, & Price, 1996). Stein et al. (1996) found a 74% enhancement for weak visual stimuli and an 8% enhancement for the strongest stimulus, both of which are in reasonable accord with both the physiological data and the synchronized neural model prediction of Figure 6A and B. It is worth noting that human psychophysical values are more in line with modulated unisensory sensory cortical cells than with the order of magnitude greater enhancements found in true multisensory cells in superior colliculus and implicated in orienting behaviors.

Figure 5. 

Enhancement of visual neuron firing rates by auditory modulation. Comparison of the coupled neuron model (Equation 1) with cortical data from cat (Allman et al., 2009). Triangles show a population of 37 cells in the PLLS area of cat visual cortex that are driven only by visual stimuli but have their firing rates enhanced if auditory stimuli are simultaneously present. The straight dashed line is the case of input-equals-output, providing no amplification reference. The solid line is the average of five simulations of Equation 1, one for each of the auditory modulation conditions shown in Figure 4, and is not fitted or normalized to the data.

Figure 5. 

Enhancement of visual neuron firing rates by auditory modulation. Comparison of the coupled neuron model (Equation 1) with cortical data from cat (Allman et al., 2009). Triangles show a population of 37 cells in the PLLS area of cat visual cortex that are driven only by visual stimuli but have their firing rates enhanced if auditory stimuli are simultaneously present. The straight dashed line is the case of input-equals-output, providing no amplification reference. The solid line is the average of five simulations of Equation 1, one for each of the auditory modulation conditions shown in Figure 4, and is not fitted or normalized to the data.

Figure 6. 

Relative enhancement of visual neuron firing rates by auditory modulation. (A) The same cat neuron firing rate data (Allman et al., 2009) is replotted as percent firing rate enhancement as a function of the unenhanced firing rate (the black line is the replotted average of the same five model simulations used in Figure 5). The principle of inverse enhancement is demonstrated by both theory and data, with a maximum relative enhancement for weak visual stimuli that yield unenhanced firing rates of about 5 spikes/sec and less relative enhancement for anything stronger. (B) The five model simulations averaged in (A) are shown separately here. Note that the variability in cat spike-rate data is mimicked by the five individual simulations (colored lines), which vary only in degree of auditory modulation.

Figure 6. 

Relative enhancement of visual neuron firing rates by auditory modulation. (A) The same cat neuron firing rate data (Allman et al., 2009) is replotted as percent firing rate enhancement as a function of the unenhanced firing rate (the black line is the replotted average of the same five model simulations used in Figure 5). The principle of inverse enhancement is demonstrated by both theory and data, with a maximum relative enhancement for weak visual stimuli that yield unenhanced firing rates of about 5 spikes/sec and less relative enhancement for anything stronger. (B) The five model simulations averaged in (A) are shown separately here. Note that the variability in cat spike-rate data is mimicked by the five individual simulations (colored lines), which vary only in degree of auditory modulation.

The percent enhancement transformation (Figure 6A and B) also emphasizes the variability of the cat data. Although variability in biological systems is expected, it is interesting to compare this variability with the five individual simulations (Figure 6B) that were averaged in Figure 6A. The variability in the strength of the modulating channel generates variable simulated responses in the modulated neurons that cover much of the measured range in Allman et al.'s (2009) data, suggesting that some of the measured variability in the visual neurons may be because of variability in the unmeasured activity level of the modulating auditory channel.

DISCUSSION

Some studies found macroscopic neural indicators of γ-band activity, phase resetting, or binding-like coherence between cortical areas activated during sensory integration (Kanayama et al., 2009; Bauer, 2008; King, 2008; Schroeder et al., 2008; Senkowski et al., 2008; Lakatos et al., 2007; Kisley & Cornwell, 2006; Sakowitz et al., 2001), but little is known about the fine structure of neural synchronization and binding-like mechanisms involved in sensory integration. In synchronized oscillator theory, most studies emphasize weak coupling. In practice, weak neural coupling is problematic because it only synchronizes oscillators with similar uncoupled frequencies—adequate for modeling pacemaker and locomotion rhythms, but not for many sensory pairings. Some theorists suggest that moderate-strength coupling should produce few ill consequences in real systems (Williams & Bowtell, 1997; Kopell, 1988). On the other hand, very strong coupling can lead to undesirable, even pathological, behaviors like those shown in Figure 3C. The simple assumption of using the strongest neural coupling strength that produces a minimal spike-voltage distortion yields intensity-dependent spike rate enhancements similar to those seen in unisensory neurons in optic tectum and cortex. Although the moderate criterion we adopted (peak spike amplitude suppression < 5%—a maximum reduction in peak voltage of ca. 4.5 mV)—is relatively arbitrary, numeric experiments suggest that it is near optimal for our purposes. A “strict criterion” of tolerating no more than 1- to 3-mV reduction in peak voltage requires much weaker coupling and significantly reduces sensory enhancement at low stimulus strengths, where it is most useful. A “loose criterion” of tolerating, say, 8-mV reductions allows coupling so strong that large sensory enhancements would occur at high stimulus strengths. Both alternate criteria violate the principle of inverse effectiveness and are belied by Allman et al.'s (2009) data. Strong coupling at low stimulus strengths allows enhancement of weak signals; this accords with Bayesian theory because the spatio-temporally correlated presence of a second weakly responding sense makes both more likely to arise from signal rather than noise (Rowland, Stanford, & Stein, 2007; Rowland & Stein, 2007; Anastasio, Patton, & Belkacem-Boussaid, 2000). Decreasing coupling at higher stimulus strengths prevents distortions of already-reliable superthreshold signals. It also makes enhancement difficult to sustain for γ-band firing rates, but γ-band synchronization is mediated by inhibitory coupling.

The form of the enhancement predicted by the neural synchronization model is in accord with numerous electrophysiological, imaging, and behavioral studies, which show that the most profound relative enhancements occur for weak—but not the weakest—sensory signals. Allman et al.'s data show this clearly, as does our model overlaid on it (Figure 6A); peak enhancement occurs for stimulus strengths that produce unenhanced firing rates of about 5 spikes/sec. Similarly, Ross et al. (2007) studied visual modulation of auditory stimuli in noisy environments and found the best enhancement not at the lowest signal-to-noise ratio (−24 dB) but rather at a more moderate one (−12 dB). Similarly, Lakatos et al. (2007) found that, for enhancement of auditory transduction by somatosensory stimulation, the maximum enhancement occurred not for the weakest auditory stimulus (20 dB) but for the next weakest (30 dB). As Holmes (2007) points out, the 30-dB signal was more enhanced than the 20-dB signal in 11 of 12 of Lakatos et al.'s cases and tied in the twelfth. The level of enhancement was superadditive only for signals at or below 50 dB. Of course, for enhanced unimodal cells, one cannot directly compute superadditivity, but for models like Equation 1, where we know individual firing rates, we can compute the superadditivity of the coupled system. For example, for the curve labeled 0.25 octaves in Figure 6B, any relative enhancement above 84% in the modulated cell would be superadditive for the coupled system. Only the maximally enhanced stimuli, which elicited uncoupled firing rates of about 5 spikes/sec in Allman et al.'s experiment, are enhanced enough (ca. 100%) in our simulations to be superadditive.

The idea of firing rate enhancement via excitatory coupling was motivated by oscillatory binding theory but does not actually require binding-like zero-phase synchronization. Indeed, for rate-coded “labeled lines,” what may matter most is that firing rates are enhanced by mutual excitatory coupling, regardless of whether synchronization is actually obtained. Even if synchronization fails because coupling strength is not quite strong enough relative to the uncoupled frequency difference, both cells' firing rates are still enhanced and may contribute to stimulus detection and perceptual salience. In this light, consider Biederlack et al.'s (2006) report that brightness and firing rate enhancement occur even when neurons fail to synchronize; they posit that spike rate enhancement and synchronization are complementary codes. In our modeling, these two neural behaviors are both response modes of an enhanced synchronization mechanism, whose behavior fails gracefully. Moreover, between the extremes of in-phase synchronization and enhanced unsynchronized firing, coupled neurons can synchronize at various fixed phases and frequency ratios that depend on the ratio of unsynchronized firing rate differences to coupling strength (Ermentrout, 1981). Also, coupling can be so asymmetric that one oscillator is entrained to the other, for example, via one-way innervation of a distant cortical area by another. There are however some advantages to in-phase oscillatory synchronization over these other possibilities. Neural responses are speeded up for the later responding channel—a virtue in its own right (Rowland & Stein, 2007)—and in-phase synchronization can improve summation at a secondary target neuron whose dendrites are leaky integrators (Salinas & Sejnowski, 2001; Abeles, 1991). This suggests that bimodal sensory mechanisms would benefit by using synchronization-enhanced unisensory cells as inputs, but there is as yet no evidence that the brain takes advantage of this. There is however evidence that phase resetting allows amplification in cortical assemblies by synchronizing inputs to high-excitability phases during neural oscillations (Schroeder & Lakatos, 2008; Schroeder et al., 2008; Lakatos et al., 2007).

Synchronization between cortical areas, including distant areas, has been much studied (e.g., Soteropoulus & Baker, 2006; Roelfsema, Engle, Konig, & Singer, 1997), but the neural nature of the coupling is seldom elucidated. In the case of auditory modulation of visual responses in cat, visual area PLLS borders the auditory dorsal zone (Allman et al., 2009). Tracer studies show that visual area PLLS, which Allman et al. (2009) recorded from, is connected to auditory areas A1, field of the anterior ectosylvian sulcus (FAES), and the posterior auditory field (Meredith & Clemo, 2010; Clemo et al., 2008). Similarly, auditory area of FAES is known to receive inputs from visual areas, especially anterior ectosylvian visual area (Meredith & Allman, 2009). Alternately, it is of course possible for neurons in a visual area to synchronize to nearby relay units from auditory areas, rather than to their remote source, but shear distance is not an obstacle: Simulation studies made with excitatory coupling (similar to Equation 1, but with intermediate relay cells) show that even distant areas can couple with near-zero time lags (millisecond precision) after an exchange of a few action potentials (Vicente, Gollo, Mirasso, Fischer, & Pipa, 2008).

Variable neural coupling strengths are not new, and the monotonic coupling-strength curves produced by the “just-weak-enough” criterion are reminiscent of other intensity-dependent adaptations found in sensory neurons. Perhaps the best neural information to drive this adaptation is the latency of the first spike or the interspike interval between the first spikes, which are good predictors of spike rates (Van Rullen & Thorpe, 2001; Kuffler, 1953). Synaptic gain changes driven by such measures could be induced within tens of milliseconds after the first spikes (Moser et al., 2010). Remote modulation of coupling strength is possible: Enhancement in superior colliculus is abolished when cortex is cooled. This suggests that coupling (in absence of cortical feedback) could be set low and modulated by cortical activity, reminiscent of how cortical activity (even attention) can modulate lateral geniculate nucleus gain (Przybyszewski, Gaska, Foote, & Pollen, 2000; Sherman & Guillery, 1998). Alternately, there are local coupling adjustment mechanisms available. The firing rate dependence of the coupling strength is reminiscent of the firing rate dependence of synaptic depression, which is driven in part by neurotransmitter depletion at high spike rates (Abbott, Varella, Sen, & Nelson, 1997; Varela et al., 1997). Yet, another way to implement adaptive coupling would be through an inhibitory neuron that samples the firing rates of the excitatory mechanism and shunts the excitatory synapses, reducing the effective coupling.

We do not assert that this oscillatory enhancement mechanism supplants other neural enhancement mechanisms (e.g., multiplicative N-methyl-d-aspartate synapses on multisensory cells; for a review, see Rowland, Stein, & Stanford, 2011). Rather, we point out that this network-based mechanism is available for multisensory interactions between unisensory neurons; is likely to occur when mutually excitatory neurons synchronize; and produces enhancements compatible with the principle of inverse effectiveness, as it is found for cortical-modulated and some tectal-modulated unisensory neurons. To the extent that different neural mechanisms converge on similar results at different sensory scales, they may lead to greater overall system consistency and to more readily combined information.

Although some studies show binding-like mass neural signatures during sensory integration, there is as yet no evidence that individual unisensory neurons are synchronized, in-phase, or otherwise during multisensory interactions. The most suggestive evidence is phase resetting—a key element of synchronization—in auditory cortex during visual amplification of speech (Schroeder et al., 2008; Lakatos et al., 2007). What is still lacking is data from individual synchronized cells analogous to the seminal binding studies (Singer & Gray, 1995; Gray & Singer, 1989; Gray et al., 1989). Rattlesnake optic tectum might be a good place to gather such data in enhanced unimodal cells; Newman and Hartline found that IR-enhanced visual cells were concentrated 150–800 μm below the surface, whereas visual enhanced IR neurons are at depths of 500–1300 μm (Newman & Hartline, 1981). Readily accessible and regularly layered, it might be possible to use tetrodes or multielectrode arrays with various sampling depths to seek signs of correlated firing between different tectal neurons. Alternately, because python optic tectum is organized similarly to rattlesnake (Newman & Hartline, 1982), it offers a nonvenomous alternative. Finally, we note that several neuronal network models result in response enhancements in multisensory and other systems (Ahmadian, Rubin, & Miller, 2013; Hoshino, 2011; Lim et al., 2011; Rowland et al., 2011; Cuppini, Ursino, Rowland, & Stein, 2010). If the underlying units have realistic oscillatory spiking behaviors, it would almost be surprising if some of the units were not synchronizing; it would be interesting to know the relationship between synchronization and enhancement in such models. In any case, the description of a key principle of sensory integration, using a binding-like mechanism, raises the possibility that additional insights from each of these fields may profitably illuminate the other.

Acknowledgments

We thank Brian Allman, Leslie Keniston, and Alex Meredith for providing single-cell data; Bard Ermentrout, Nancy Kopell, Benjamin Rowland, and Jim Schirillo for useful suggestions; and Michael Loop, Eric Newman, John Rinzel, and Pieter Roelfsema for helpful discussions. This work was supported by an AFOSR/NRC Senior Associate award to Vincent Billock at the U.S. Air Force Research Laboratory and by a Spring 2013 Visiting Scholar appointment for Vincent Billock at the Ohio State University Mathematical Biosciences Institute.

Reprint requests should be sent to Vincent A. Billock, College of Optometry, The Ohio State University, 338 W. 10th Ave., Columbus, OH 43210, or via e-mail: billock.3@osu.edu.

Notes

1. 

In the literature, these cells go by various names, including subthreshold multisensory cells.

2. 

Pythons have up to 26 less-sophisticated thermal pits arrayed about the jaws, and vampire bats have three pits with independent pointing and aperture control. Whereas snakes use their pits to locate prey, vampire bats are believed to use their pits to localize vulnerable veins (Campbell et al., 2002).

3. 

Similar problems in coupled pendulum clocks occur if low-amplitude swings fail to engage the clock's escapement mechanism.

REFERENCES

Abbott
,
L. F.
,
Varella
,
J. A.
,
Sen
,
K.
, &
Nelson
,
S. B.
(
1997
).
Synaptic depression and cortical gain control.
Science
,
275
,
220
224
.
Abeles
,
M.
(
1991
).
Corticonics: Neural circuits of the cerebral cortex.
Cambridge
:
Cambridge University Press
.
Ahmadian
,
Y.
,
Rubin
,
D. B.
, &
Miller
,
K. D.
(
2013
).
Analysis of the stabilized supralinear network.
Neural Computation
,
3
,
1994
2037
.
Allman
,
B. L.
,
Keniston
,
L. P.
, &
Meredith
,
M. A.
(
2008
).
Subthreshold auditory inputs to extrastriate visual neurons are responsive to parametric changes in stimulus quality: Sensory-specific verses non-specific coding.
Brain Research
,
1242
,
95
101
.
Allman
,
B. L.
,
Keniston
,
L. P.
, &
Meredith
,
M. A.
(
2009
).
Not just for bimodal cells anymore: The contribution of unimodal neurons to cortical multisensory processing.
Brain Topography
,
21
,
157
167
.
Allman
,
B. L.
, &
Meredith
,
M. A.
(
2007
).
Multisensory processing in “unimodal” neurons: Cross-modal subthreshold auditory effects in cat extrastriate visual cortex.
Journal of Neurophysiology
,
98
,
545
549
.
Alvardo
,
J. C.
,
Stanford
,
T. R.
,
Vaughan
,
J. W.
, &
Stein
,
B. E.
(
2007
).
Cortex mediates multisensory but not unisensory integration in superior colliculus.
Journal of Neuroscience
,
27
,
12775
12786
.
Anastasio
,
T. J.
,
Patton
,
P. E.
, &
Belkacem-Boussaid
,
K.
(
2000
).
Using Bayes' rule to model multisensory enhancement in the superior colliculus.
Neural Computation
,
12
,
1165
1187
.
Barraclough
,
N. E.
,
Xiao
,
D.
,
Baker
,
C. I.
,
Oram
,
M. W.
, &
Perrett
,
D. I.
(
2005
).
Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions.
Journal of Cognitive Neuroscience
,
17
,
377
391
.
Bauer
,
M.
(
2008
).
Multisensory integration: A functional role for inter-area synchronization?
Current Biology
,
18
,
R709
R710
.
Biederlack
,
J.
,
Castelo-Branco
,
M.
,
Neuenschwander
,
S.
,
Wheeler
,
D. W.
,
Singer
,
W.
, &
Nikoloc
,
D.
(
2006
).
Brightness induction: Rate enhancement and neural synchronization as complementary codes.
Neuron
,
52
,
1073
1083
.
Billock
,
V. A.
, &
Tsou
,
B. H.
(
2004
).
A role for correlated crosstalk in the binding problem: Stimulus-driven correlations that link color, form, and motion.
Journal of Cognitive Neuroscience
,
16
,
1036
1048
.
Billock
,
V. A.
, &
Tsou
,
B. H.
(
2005
).
Sensory recoding via neural synchronization: Integrating hue and luminance into chromatic brightness and saturation.
Journal of the Optical Society of America A
,
22
,
2289
2298
.
Callaway
,
A. C.
, &
Ross
,
W. N.
(
1995
).
Frequency dependent propagation of sodium action potentials in dendrites of hippocampal CA1 pyramidal neurons.
Journal of Neurophysiology
,
74
,
1395
1403
.
Campbell
,
A. L.
,
Naik
,
R. R.
,
Sowards
,
L.
, &
Stone
,
M. O.
(
2002
).
Biological infrared imaging and sensing.
Micron
,
33
,
211
225
.
Clemo
,
H. R.
,
Allman
,
B. L.
,
Meredith
,
M. A.
, &
Shrama
,
G. K.
(
2008
).
Auditory projections to extrastriate visual cortex: Connectional basis for multisensory processing in “unimodal” visual neurons.
Experimental Brain Research
,
191
,
37
47
.
Convento
,
S.
,
Vallar
,
G.
,
Galantini
,
C.
, &
Bolognini
,
N.
(
2013
).
Neuromodulation of early multisensory interactions in the visual cortex.
Journal of Cognitive Neuroscience
,
25
,
685
696
.
Cuppini
,
C.
,
Ursino
,
M.
,
Rowland
,
B. A.
, &
Stein
,
B. E.
(
2010
).
An emergent model of multisensory integration in superior colliculus.
Frontiers in Integrative Neuroscience
,
4
,
1
15
.
Ermentrout
,
G. B.
(
1981
).
n:m phase-locking of weakly coupled oscillators.
Journal of Mathematical Biology
,
12
,
327
342
.
Fries
,
P.
(
2009
).
Neuronal gamma-band synchronization as a fundamental process in cortical computation.
Annual Review of Neuroscience
,
32
,
209
224
.
Gray
,
C. M.
,
Konig
,
P.
,
Engel
,
A. K.
, &
Singer
,
W.
(
1989
).
Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties.
Nature
,
338
,
334
337
.
Gray
,
C. M.
, &
Singer
,
W.
(
1989
).
Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex.
Proceedings of the National Academy of Sciences, U.S.A.
,
86
,
1698
1702
.
Hartline
,
P. H.
,
Kass
,
L.
, &
Loop
,
M. S.
(
1978
).
Merging of modalities in the optic tectum: Infrared and visual integration in rattlesnakes.
Science
,
199
,
1225
1229
.
Holmes
,
N. P.
(
2007
).
The law of inverse effectiveness in neurons and behavior: Multisensory integration versus normal variability.
Neuropsychologia
,
45
,
3340
3345
.
Hoshino
,
O.
(
2011
).
Neuronal responses below firing threshold for subthreshold cross-modal enhancement.
Neural Computation
,
23
,
958
983
.
Kanayama
,
N.
,
Sato
,
A.
, &
Ohira
,
H.
(
2009
).
The role of gamma band oscillations and synchrony on rubber hand illusion and crossmodal integration.
Brain & Cognition
,
69
,
19
29
.
Kass
,
L.
,
Loop
,
M. S.
, &
Hartline
,
P. H.
(
1978
).
Anatomical and physiological localization of visual and infrared cell layers in tectum of pit vipers.
Journal of Comparative Neurology
,
182
,
811
820
.
Kelso
,
J. A. S.
(
1995
).
Dynamic patterns.
Cambridge, MA
:
MIT Press
.
King
,
A. J.
(
2008
).
Multisensory integration: Strategies for synchronization.
Current Biology
,
18
,
R339
R341
.
Kisley
,
M. A.
, &
Cornwell
,
Z. M.
(
2006
).
Gamma and beta neural activity during a sensory gating paradigm: Effects of auditory, somatosensory and cross-modal stimulation.
Clinical Neurophysiology
,
117
,
2549
2563
.
Kopell
,
N.
(
1988
).
Towards a theory of modeling central pattern generators.
In A. H. Cohen, S. Rossignol, & S. Griller (Eds.)
,
Neural control of rhythmic movements in vertebrates
(pp.
369
413
).
New York
:
Wiley
.
Kuffler
,
S. W.
(
1953
).
Discharge patterns and functional organization of mammalian retina.
Journal of Neurophysiology
,
16
,
37
68
.
Lakatos
,
P.
,
Chen
,
C. M.
,
O'Connell
,
M. N.
,
Mills
,
A.
, &
Schroeder
,
C. E.
(
2007
).
Neuronal oscillations and multisensory interaction in the primary auditory cortex.
Neuron
,
53
,
279
292
.
Lakatos
,
P.
,
Karmos
,
G.
,
Mehta
,
A. D.
,
Ulbert
,
I.
, &
Schroeder
,
C. E.
(
2008
).
Entrainment of neuronal oscillations as a mechanism of oscillatory selection.
Science
,
320
,
110
113
.
Lakatos
,
P.
,
O'Connell
,
M. N.
,
Barczak
,
A.
,
Mills
,
A.
,
Javitt
,
D. C.
, &
Schroeder
,
C. E.
(
2009
).
The leading sense: Supramodal control of neurophysiological content by attention.
Neuron
,
64
,
419
430
.
Laurienti
,
P. J.
,
Perrault
,
T. J.
,
Stanford
,
T. R.
,
Wallace
,
M. T.
, &
Stein
,
B. E.
(
2005
).
On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies.
Experimental Brain Research
,
166
,
289
297
.
Lim
,
H. K.
,
Keniston
,
L. P.
,
Shin
,
J. H.
,
Allman
,
B. L.
,
Meredith
,
M. A.
, &
Cios
,
K. J.
(
2011
).
Connectional parameters determine multisensory processing in a spiking model of multisensory convergence.
Experimental Brain Research
,
213
,
329
339
.
Lisman
,
J. E.
, &
Idiart
,
M. A.
(
1995
).
Storage of 7 ± 2 short term memories in oscillatory subcycles.
Science
,
267
,
1512
1515
.
Lovelace
,
C. T.
,
Stein
,
B. E.
, &
Wallace
,
M. T.
(
2003
).
An irrelevant light enhances auditory detection in humans: A psychophysical analysis of multisensory integration in stimulus detection.
Cognitive Brain Research
,
17
,
447
453
.
Meeks
,
J. P.
,
Jiang
,
X.
, &
Mennerick
,
S.
(
2005
).
Action potential fidelity during normal and epileptiform activity in paired soma-axon recordings in rat hippocampus.
Journal of Physiology
,
556
,
425
441
.
Meredith
,
M. A.
, &
Allman
,
B. L.
(
2009
).
Subthreshold multisensory processing in cat auditory cortex.
NeuroReport
,
20
,
126
131
.
Meredith
,
M. A.
, &
Clemo
,
H. R.
(
2010
).
Corticocortico connectivity subserving different forms of multisensory convergence.
In M. J. Naumer & J. Kaiser (Eds.)
,
Multisensory object perception in the primate brain
(pp.
7
20
).
New York
:
Springer
.
Meredith
,
M. A.
, &
Stein
,
B. E.
(
1983
).
Interactions among converging sensory inputs in the superior colliculus.
Science
,
221
,
389
391
.
Moser
,
E. I.
,
Corbetta
,
M.
,
Desimone
,
R.
,
Fregnac
,
Y.
,
Fries
,
P.
,
Graybiel
,
A. M.
,
et al
(
2010
).
Coordination in brain systems.
In Von der Malsburg, W. A. Phillips, & W. Singer (Eds.)
,
Dynamic coordination in the brain
(pp.
193
214
).
Cambridge, MA
:
MIT Press
.
Newman
,
E. A.
, &
Hartline
,
P. H.
(
1981
).
Integration of visual and infrared information in bimodal neurons of the rattlesnake optic tectum.
Science
,
213
,
789
791
.
Newman
,
E. A.
, &
Hartline
,
P. H.
(
1982
).
The infrared vision of snakes.
Scientific American
,
246
,
116
127
.
Pappas
,
T. C.
,
Motamedi
,
M.
, &
Christensen
,
B. N.
(
2004
).
Unique temperature-activated neurons from pit viper thermosensors.
American Journal of Physiology, Cell Physiology
,
287
,
C1219
C1228
.
Pikovsky
,
A.
,
Rosenblum
,
M.
, &
Kurths
,
J.
(
2001
).
Synchronization: A universal concept in nonlinear sciences.
Cambridge
:
Cambridge University Press
.
Przybyszewski
,
A. J.
,
Gaska
,
J. P.
,
Foote
,
W.
, &
Pollen
,
D. A.
(
2000
).
Striate cortex increases contrast gain of macaque LGN neurons.
Visual Neuroscience
,
17
,
485
494
.
Roelfsema
,
P. R.
,
Engle
,
A. K.
,
Konig
,
P.
, &
Singer
,
W.
(
1997
).
Visuomotor integration is associated with zero-time lag synchronization among cortical areas.
Nature
,
385
,
157
161
.
Ross
,
L. A.
,
Saint-Amour
,
D.
,
Leavitt
,
V. M.
,
Javitt
,
D. C.
, &
Foxe
,
J. J.
(
2007
).
Do you see what I am saying? Exploring visual amplification of speech comprehension in noisy environments.
Cerebral Cortex
,
17
,
1147
1152
.
Rowland
,
B. A.
,
Stanford
,
T. R.
, &
Stein
,
B. E.
(
2007
).
A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus.
Experimental Brain Research
,
180
,
153
161
.
Rowland
,
B. A.
, &
Stein
,
B. E.
(
2007
).
Temporal profiles of response enhancement in multisensory enhancement.
Frontiers in Neuroscience
,
2
,
218
224
.
Rowland
,
B. A.
,
Stein
,
B. E.
, &
Stanford
,
T. R.
(
2011
).
Computational models of multisensory integration in cat superior colliculus.
In J. Trommershäuser, K. Körding, & M. S. Landy (Eds.)
,
Sensory cue integration
(pp.
333
344
).
Oxford
:
Oxford University Press
.
Sakowitz
,
O. W.
,
Quiroga
,
R. Q.
,
Schurmann
,
M.
, &
Basar
,
E.
(
2001
).
Bisensory stimulation increases gamma-responses over multiple cortical regions.
Cognitive Brain Research
,
11
,
267
279
.
Salinas
,
E.
, &
Sejnowski
,
T. J.
(
2001
).
Correlated neuronal activity and the flow of neural information.
Nature Reviews Neuroscience
,
2
,
539
550
.
Schirillo
,
J. A.
(
2011
).
Cross-modal detection using various temporal and spatial configurations.
Attention Perception & Psychophysics
,
73
,
237
246
.
Schroeder
,
C. E.
, &
Lakatos
,
P.
(
2008
).
Low-frequency neuronal oscillations as instruments of sensory selection.
Trends in Neurosciences
,
32
,
9
18
.
Schroeder
,
C. E.
,
Lakatos
,
P.
,
Kajikawa
,
Y.
,
Partan
,
S.
, &
Puce
,
A.
(
2008
).
Neuronal oscillations and visual amplification of speech.
Trends in Cognitive Sciences
,
12
,
106
113
.
Senkowski
,
D.
,
Scheider
,
T. R.
,
Foxe
,
J. J.
, &
Engel
,
A. K.
(
2008
).
Crossmodal binding through neural coherence: Implications for multisensory processing.
Trends in Neurosciences
,
31
,
401
409
.
Sherman
,
S. M.
, &
Guillery
,
R. W.
(
1998
).
On the actions that one nerve cell can have on another: Distinguishing drivers from modulators.
Proceedings of the National Academy of Sciences, U.S.A.
,
95
,
7121
7126
.
Singer
,
W.
, &
Gray
,
C. M.
(
1995
).
Visual feature integration and the temporal correlation hypothesis.
Annual Review of Neuroscience
,
18
,
555
586
.
Soteropoulus
,
D. S.
, &
Baker
,
S. N.
(
2006
).
Cortico-cerebellar coherence during a precision grip task in the monkey.
Journal of Neurophysiology
,
95
,
1194
1206
.
Stanford
,
T. R.
, &
Stein
,
B. E.
(
2007
).
Superadditivity in multisensory integration: Putting the computation in context.
NeuroReport
,
18
,
787
792
.
Stein
,
B. E.
,
London
,
N.
,
Wilkinson
,
L. K.
, &
Price
,
D. P.
(
1996
).
Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis.
Journal of Cognitive Neuroscience
,
8
,
497
506
.
Stein
,
B. E.
, &
Meredith
,
M. A.
(
1993
).
The merging of the senses.
Cambridge, MA
:
MIT Press
.
Stopfer
,
M.
,
Bhagavan
,
S.
,
Smith
,
B. H.
, &
Laurent
,
G.
(
1997
).
Impaired odor discrimination on desynchronization of odour-encoding assemblies.
Nature
,
390
,
70
74
.
Strogatz
,
S. H.
(
2003
).
Synch.
New York
:
Hyperion
.
Suppes
,
P.
,
de Barros
,
J. A.
, &
Oas
,
G.
(
2012
).
Phase-oscillator computations as neural models of stimulus–response conditioning and response selection.
Journal of Mathematical Psychology
,
56
,
95
117
.
Tallon-Baudry
,
C.
, &
Bertrand
,
O.
(
1999
).
Oscillatory gamma activity and its role in object representation.
Trends in Cognitive Science
,
3
,
151
162
.
Uhlhaas
,
P. J.
, &
Singer
,
W.
(
2006
).
Neural synchrony in brain disorders: Relevance for cognitive dysfunctions and pathophysiology.
Neuron
,
52
,
155
168
.
Van Rullen
,
R.
, &
Thorpe
,
S. J.
(
2001
).
Rate coding vs temporal order coding: What the retinal ganglion cells tell the visual cortex.
Neural Computation
,
13
,
1255
1283
.
Varela
,
J. A.
,
Sen
,
K.
,
Gibson
,
J.
,
Fost
,
J.
,
Abbott
,
L. F.
, &
Nelson
,
S. B.
(
1997
).
A quantitative description of short-term plasticity at excitatory synapses in layer 2/3 of rat primary visual cortex.
Journal of Neuroscience
,
17
,
7926
7940
.
Vicente
,
R.
,
Gollo
,
L. L.
,
Mirasso
,
C. R.
,
Fischer
,
I.
, &
Pipa
,
G.
(
2008
).
Dynamical relaying can yield zero time lag neuronal synchrony despite long conduction delays.
Proceedings of the National Academy of Sciences, U.S.A.
,
105
,
17157
17162
.
Wallace
,
M. T.
,
Ramachandran
,
R.
, &
Stein
,
B. E.
(
2004
).
A revised view of sensory cortical parcellation.
Proceedings of the National Academy of Sciences, U.S.A.
,
101
,
2167
2172
.
Williams
,
T. L.
, &
Bowtell
,
G.
(
1997
).
The calculation of frequency-shift functions for chains of coupled oscillators with application to a network model of the lamphrey locomotor pattern generator.
Journal of Computational Neuroscience
,
4
,
47
55
.
Wilson
,
H. R.
(
1999a
).
Spikes, decisions and actions: Dynamical foundations of neuroscience.
Oxford
:
Oxford University Press
.
Wilson
,
H. R.
(
1999b
).
Simplified dynamics of human and mammalian neocortical neurons.
Journal of Theoretical Biology
,
200
,
375
388
.