The direction of others' gaze is a strong social signal to their intentions and future behavior. Pioneering electrophysiological research identified cell populations in the primate visual cortex that are tuned to specific directions of observed gaze, but the functional architecture of this system is yet to be precisely specified. Here, we develop a computational model of how others' gaze direction is flexibly encoded across sensory channels within the gaze system. We incorporate the divisive normalization of sensory responses—a computational mechanism that is thought to be widespread in sensory systems but has not been examined in the context of social vision. We demonstrate that the operation of divisive normalization in the gaze system predicts a surprising and distinctive pattern of perceptual changes after sensory adaptation to gaze stimuli and find that these predictions closely match the psychophysical effects of adaptation in human observers. We also find that opponent coding, broadband multichannel, and narrowband multichannel models of sensory coding make distinct predictions regarding the effects of adaptation in a normalization framework and find evidence in favor of broadband multichannel coding of gaze. These results reveal the functional principles that govern the neural encoding of gaze direction and support the notion that divisive normalization is a canonical feature of nervous system function. Moreover, this research provides a strong foundation for testing recent computational theories of neuropsychiatric conditions in which gaze processing is compromised, such as autism and schizophrenia.

Surface segregation provides an efficient way to parse the visual scene for perceptual analysis. Here, we investigated the segregation of a bivectorial motion display into transparent surfaces through a psychophysical task and fMRI. We found that perceptual transparency correlated with neural activity in the early areas of the visual cortex, suggesting these areas may be involved in the segregation of motion-defined surfaces. Two oppositely rotating, uniquely colored random dot kinematograms (RDKs) were presented either sequentially or in a spatially interleaved manner, displayed at varying alternation frequencies. Participants reported the color and rotation direction pairing of the RDKs in the psychophysical task. The spatially interleaved display generated the percept of motion transparency across the range of frequencies tested, yielding ceiling task performance. At high alternation frequencies, performance on the sequential display also approached ceiling, indicative of perceived transparency. However, transparency broke down in lower alternation frequency sequential displays, producing performance close to chance. A corresponding pattern mirroring the psychophysical data was also evident in univariate and multivariate analyses of the fMRI BOLD activity in visual cortical areas V1, V2, V3, V3AB, hV4, and V5/MT+. Using gray RDKs, we found significant presentation by frequency interactions in most areas; differences in BOLD signal between presentation types were significant only at the lower alternation frequency. Multivariate pattern classification was similarly unable to discriminate between presentation types at the higher frequency. This study provides evidence that early visual cortex may code for motion-defined surface segregation, which in turn may enable perceptual transparency.

Transcranial magnetic stimulation (TMS) is a popular tool for mapping perceptual and cognitive processes in the human brain. It uses a magnetic field to stimulate the brain, modifying ongoing activity in neural tissue under the stimulating coil, producing an effect that has been likened to a “virtual lesion.” However, research into the functional basis of this effect, essential for the interpretation of findings, lags behind its application. Acutely, TMS may disable neuronal function, thereby interrupting ongoing neural processes. Alternatively, the effects of TMS have been attributed to an injection of “neural noise,” consistent with its immediate and effectively random depolarization of neurons. Here we apply an added-noise paradigm to test these alternatives. We delivered TMS to the visual cortex and measured its effect on a simple visual discrimination task, while concurrently manipulating the level of image noise in the visual stimulus itself. TMS increased thresholds overall; and increasing the amount of image noise systematically increased discrimination thresholds. However, these two effects were not independent. Rather, TMS interacted multiplicatively with the image noise, consistent with a reduction in the strength of the visual signal. Indeed, in this paradigm, there was no evidence that TMS independently added noise to the visual process. Thus, our findings indicate that the “virtual lesion” produced by TMS can take the form of a loss of signal strength which may reflect a momentary interruption to ongoing neural processing.