Abstract

The direction of others' gaze is a strong social signal to their intentions and future behavior. Pioneering electrophysiological research identified cell populations in the primate visual cortex that are tuned to specific directions of observed gaze, but the functional architecture of this system is yet to be precisely specified. Here, we develop a computational model of how others' gaze direction is flexibly encoded across sensory channels within the gaze system. We incorporate the divisive normalization of sensory responses—a computational mechanism that is thought to be widespread in sensory systems but has not been examined in the context of social vision. We demonstrate that the operation of divisive normalization in the gaze system predicts a surprising and distinctive pattern of perceptual changes after sensory adaptation to gaze stimuli and find that these predictions closely match the psychophysical effects of adaptation in human observers. We also find that opponent coding, broadband multichannel, and narrowband multichannel models of sensory coding make distinct predictions regarding the effects of adaptation in a normalization framework and find evidence in favor of broadband multichannel coding of gaze. These results reveal the functional principles that govern the neural encoding of gaze direction and support the notion that divisive normalization is a canonical feature of nervous system function. Moreover, this research provides a strong foundation for testing recent computational theories of neuropsychiatric conditions in which gaze processing is compromised, such as autism and schizophrenia.

INTRODUCTION

The ease with which we can identify another's direction of gaze belies the computational challenge that the brain faces in extracting this information from the visual stream. Our perception of gaze direction is derived from multiple sensory characteristics of the eye region and head (Otsuka, Mareschal, Calder, & Clifford, 2014; Langton, 2010; Wollaston, 1824) and rests upon sensory mechanisms that are sensitive to the recent history of stimulation (Jenkins, Beaver, & Calder, 2006), fluctuations in the reliability of sensory information (Mareschal, Calder, & Clifford, 2013), and higher-order expectations about the social context (Teufel et al., 2009). Gaze processing has critical importance to social behavior and development in humans and other primate species (Rosati & Hare, 2009; Baron-Cohen, 1997) and is implicated in neuropsychiatric conditions including autism, schizophrenia, and social anxiety disorder (Green, Horan, & Lee, 2015; Senju, 2013; Den Boer, 2000). Moreover, the representation of gaze direction presents as a model system for the emerging field of social neuroscience, as it is enacted by neural circuitry that is relatively well localized in the primate brain (Carlin & Calder, 2013) and entails a stimulus property that can be defined parametrically.

There is evidence that the direction of others' gaze is encoded in the brain across cell populations that are tuned to distinct angles of gaze. Pioneering electrophysiological recordings in the macaque cortex identified cells in the anterior STS that were maximally sensitive to specific directions of eye gaze and head rotation (Perrett, Hietanen, Oram, & Benson, 1992; Perrett et al., 1985, 1990). The presence of distinct cell populations in the human brain that are tuned to leftward- and rightward-averted gaze has been demonstrated using sensory adaptation. Prolonged exposure to leftward-averted gaze causes an apparent shift away from the left in the perceived gaze direction of subsequent face stimuli (Calder, Jenkins, Cassel, & Clifford, 2008; Jenkins et al., 2006; Seyama & Nagayama, 2006) as well as a reduction in the BOLD response in anterior STS to leftward- compared with rightward-averted gaze stimuli (Calder et al., 2007). The complementary pattern of perceptual and physiological effects is observed after adaptation to rightward-averted gaze, indicating the existence of cell populations that can be selectively adapted based on their differential response to leftward- and rightward-averted gaze directions.

An outstanding question in social neuroscience, however, is how the brain encodes parameters such as gaze direction across an ensemble of cell populations. Here, we take a psychophysical approach to gaze direction that has previously revealed the functional architecture of systems in the occipital cortex that encode orientation and spatial frequency (Regan & Beverley, 1985; Blakemore & Campbell, 1969). We implement, for the first time, a computational model of gaze direction, in which information is combined across sensory channels tuned to different directions of gaze. A key functional mechanism that we implement in modeling this cross-channel response is divisive normalization, which is a form of gain control that has been proposed as a canonical computational feature of neural responses in the visual cortex (Carandini & Heeger, 1994, 2012) but is yet to be demonstrated for gaze processing or social vision more broadly. This mechanism is thought to play a fundamental role in the neural coding of stimulus properties by controlling for extraneous contextual influences on sensory responses. A widespread failure of divisive normalization in sensory processes has very recently been proposed to occur in autism (Rosenberg, Patterson, & Angelaki, 2015), highlighting the importance of examining this mechanism in the context of social function.

Importantly, by proposing a specific computational structure for the neural system encoding gaze direction, we are able to identify a set of novel quantitative predictions regarding the effects of sensory adaptation on the represented direction of gaze. Most notably, we demonstrate that the interaction between neural adaptation and normalization mechanisms predicts a distinctive pattern of changes in the encoded stimulus after sensory adaptation that is yet to be observed empirically. This allows us to test for the operation of divisive normalization in gaze processing by comparing model predictions to the profile of perceptual aftereffects seen in human observers when tested on stimuli that span the continuum of physically realizable gaze directions. We hypothesized that perceived gaze direction in human participants after sensory adaptation would match the predicted cross-channel response of the divisive normalization model in the relative magnitude and direction of perceptual aftereffects. We also compare the predicted effects of sensory adaptation between models of gaze perception that rest on opponent coding (i.e., sensory channels tuned to left vs. right gaze), broadband multichannel coding (e.g., three channels tuned broadly to left, right, and direct gaze), and narrowband multichannel coding (i.e., a continuum of narrowly tuned sensory channels).

METHODS

Overview of Experimental Methods

To test the predictions of the computational models of gaze perception (described below), we examined in human participants how the perception of gaze direction is altered after adaptation to gaze stimuli. A limitation of existing research on gaze adaptation is the use of categorical measures of perceived gaze direction, meaning that sensory adaptation is only apparent when responses cross category boundaries (e.g., from averted gaze to direct gaze). As a result, the effects of adaptation have only been tested on a narrow range of gaze deviations centered on direct gaze, and the magnitudes of these effects have not been directly quantified. In contrast, this study employs a continuous measure of perceived gaze direction, allowing us to measure the effects of adaptation on the perception of stimuli across the full range of horizontal eye deviations and to quantify the magnitude and direction of these effects. As we will see, this expanded paradigm for measuring the effects of adaptation is critical to discern the operation of divisive normalization in gaze processing and to distinguish between opponent coding and multichannel models.

Participants were six healthy adults (three women, three men), who completed all procedures in a repeated-measures design. This included the two authors and four naive observers. The UNSW Sydney human research ethics committee approved experimental procedures, and all participants gave written informed consent.

Stimuli

All stimuli were computer-generated faces seen from a frontal view (Figure 1). Face textures and 3-D face models were generated using Face-Gen Modeller 3.5 and manipulated using Blender 2.70, as described previously (Otsuka, Mareschal, & Clifford, 2015). The eyes were modeled separately from other face components, and their rotation was set to precisely control both the direction of eye gaze relative to the viewer and the vergence of the two eyes. The head rotation and vertical eye gaze angle were constant across all stimuli and directed at the viewer.

Figure 1.

Task design. (A–D) See Experimental Procedures. (E) Mean pointer response across stimulus gaze directions for one example participant. These data are from a leftward adaptation session, with the difference in mean pointer response between baseline and adapted conditions (i.e., the sensory aftereffect) plotted in the bottom figure. Error bars indicate ±1 SEM.

Figure 1.

Task design. (A–D) See Experimental Procedures. (E) Mean pointer response across stimulus gaze directions for one example participant. These data are from a leftward adaptation session, with the difference in mean pointer response between baseline and adapted conditions (i.e., the sensory aftereffect) plotted in the bottom figure. Error bars indicate ±1 SEM.

Stimuli were generated for 15 horizontal gaze directions ranging between 35° leftward gaze and 35° rightward gaze in 5° intervals. For each direction of gaze, stimuli were generated for each of the three female and three male face identities. To control for any left–right asymmetries in the stimuli, horizontally flipped versions of each image were also used.

Images were sized such that there was an interpupillary distance of 6.3 cm when head and gaze directions were direct. This is consistent with the average interpupillary distance in adults (e.g., Fesharaki, Rezaei, Farrahi, Banihashem, & Jahanbkhshi, 2012); thus, test images were displayed at approximate life size. The vergence of the stimulus eyes was such that the implied fixation distance of the stimuli was approximate to the participant's viewing distance (50 cm). This was designed so that stimuli with direct gaze would appear to be fixating on the participant rather than looking in front of or behind them.

The angles of gaze for adaptor stimuli were 25° leftward gaze, 25° rightward gaze, and direct gaze. This follows previous studies of gaze adaptation (Calder et al., 2008; Jenkins et al., 2006) and ensured that we could test the effects of adaptation on subsequent gaze directions that were both less averted (i.e., <25°) and more averted (i.e., >25°) than the adapted direction. The effects of adaptation were tested across the full range of gaze directions generated (Figure 1D).

It has been demonstrated previously that gaze aftereffects are due at least in part to adaptation of systems representing gaze direction specifically, rather than being limited to an effect of adaptation to lower-level features of the image (Jenkins et al., 2006; Seyama & Nagayama, 2006) or a general effect on spatial bias or perceived object rotation (Jenkins et al., 2006; Fang & He, 2005). In this study, several measures were taken to ensure that adaptor stimuli and test stimuli had very different lower-level image properties, such that the aftereffects observed could be attributed to adaptation of represented direction of gaze. This included presenting adaptor stimuli at a different size to the test stimuli (with a ratio of areas of 3:4) and jittering the spatial position of the test stimuli randomly between trials (by a maximum of 50 pixels in the horizontal and vertical dimensions). Moreover, in each session, differently gendered images were used for the adaptor and test stimuli—thus, the effect of sensory adaptation was always tested on different face identities to those adapted on. The stimulus gender used for adaptation versus test alternated between participants.

Stimuli were presented on Viewsonic (Walnut, CA) Graphics Series G90f and DiamondDigital CRT monitors (1280 × 1024 pixels, approx. 35 pixels per cm, 85 Hz) with the use of MATLAB PsychToolbox (Brainard, 1997).

Experimental Procedures

Each participant completed three adaptation conditions (leftward gaze, rightward gaze, and direct gaze) in separate sessions conducted on separate days. The order of conditions was counterbalanced across participants.

Each session began with a baseline test of gaze perception, in which participants viewed a series of face images and responded by indicating the direction in which each face was looking (Figure 1A). In each trial, the test stimulus was presented for 500 msec. The stimulus was ramped on and then off over the presentation period by applying a raised-cosine envelope to stimulus contrast. The computer produced a short beep upon onset of the test stimulus. Blank screen ISIs were included before and after presentation of the test stimulus, lasting 200 and 300 msec, respectively.

Responses were made along a continuous dimension, using a spherical on-screen pointer that could be rotated in the horizontal plane between 90° leftward to 90° rightward around direct. The initial position of the pointer in each trial was randomized within this range. The pointer was displayed subsequent to the test stimulus, and participants were given 4 sec to make their response using the mouse. The duration of the interval between trials was fixed such that the period between the beginning of the response period in a given trial and the beginning of the next trial was always 4 sec. If participants failed to respond within 4 sec, the trial was repeated at the end of the block.

Participants completed two blocks for the baseline period. Each block consisted of 90 trials (15 gaze directions × 6 face identities), presented in a random order, for 180 trials. In each trial, the rotation of the pointer response was recorded as a continuous angle ranging from 90° leftward to 90° rightward around direct. For each participant, pointer responses were averaged across the 12 repetitions of each gaze direction. The baseline period was conducted in each of the three sessions that participants completed.

After the baseline period, participants completed an adaptation period that also consisted of two blocks. Depending on the session, participants were adapted to a specific direction of gaze: either 25° leftward gaze, 25° rightward gaze, or direct gaze.

Participants were shown an initial series of adaptor stimuli at the beginning of each block (Figure 1B). This consisted of 15 images displayed for 4 sec each (for a total duration of 60 sec). The adaptor images for each condition (described in Stimuli) were each presented an equal number of times during this period, in a random order. Participants were instructed to attend to the eyes when observing the adaptor images. To ensure that this was the case, participants were required to carry out a task during the adaptation period. For 20% of the presented images, the iris color would briefly turn blue for 200 msec at a random point within the 4 sec that the image was presented. Participants were instructed to press a button each time this occurred.

After adaptation, the perception of gaze direction was again tested using a similar procedure to the baseline period (Figure 1C). The only difference during the postadaptation test was that an adaptor stimulus was presented for 4 sec at the beginning of each trial. This usage of “top-up” adaptor stimuli follows the procedure of previous gaze adaptation studies (Calder et al., 2008; Jenkins et al., 2006) and is intended to maintain the level of sensory adaptation throughout the testing period. The adaptor stimuli were the same as those presented during the initial adaptation period and were presented in a random order, with each image shown an equal number of times across the block.

To compute a profile of sensory aftereffects for each session, the mean pointer response for each of the 15 gaze directions tested during the baseline period was subtracted from that of the adaptation period. An example of individual participant data is displayed in Figure 1E.

Three-channel Model of the Gaze System

Sensory channels are a well-established concept in psychophysics, referring to cell populations in the nervous system that are selectively responsive and differentially tuned along a given stimulus dimension (e.g., to ranges of orientation or spatial frequency). Typically, the percept is modeled as a weighted combination of the stimulus attribute represented by those channels, where the weighting corresponds to the channel activation (Salinas & Abbott, 1994; Georgopoulos, Schwartz, & Kettner, 1986). As described in the Introduction, evidence for this form of functional segregation in the primate brain has been reported in the context of stimulus gaze direction (Calder et al., 2007; Perrett et al., 1985). Importantly, the perceptual effects of selective sensory adaptation can indicate how a stimulus property is represented across a system of sensory channels (i.e., population coded). Psychophysical data reported by Calder and colleagues (2008) suggest that horizontal gaze direction is coded by at least three sensory channels, rather than by a simpler (two-channel) opponent system. In its simplest form, a multichannel system for gaze representation entails channels that are broadly tuned to rightward, leftward, and direct gaze, respectively. Thus, in the current article, we focus first on a three-channel model, which is an example of broadband multichannel coding. For comparison, we also implement an opponent model and a narrowband multichannel model, described in Results. We implement the three-channel model of the coding of horizontal gaze direction as follows.

Leftward gaze directions are defined here as negative; and rightward directions, as positive. Rightward channel sensitivity, R, increases monotonically with gaze direction, d, and is described by a logistic function such that
$Rd=11+e−d−gs$
where g defines the gaze direction at which the channel sensitivity is half its maximum and s determines the steepness of the function. Analogously, leftward channel sensitivity decreases monotonically with gaze direction:
$Ld=11+e+d+gs$
The sensitivities of the three channels are constrained to sum to unity in the unadapted state, such that the sensitivity, C, of the “direct” channel is given by
$Cd=1−Rd+Ld$
To convert the responses of the three channels into an encoded direction of gaze, a multichannel response, M, is computed as the difference between rightward and leftward channel responses divided by the sum of the responses of all three channels. The resulting function gives a measure of gaze direction that increases monotonically with stimulus gaze direction:
$Md=Rd−LdRd+Ld+Cd$
A key functional mechanism that this model of gaze processing incorporates is divisive normalization. When a stimulus parameter is encoded across a population of neurons or sensory channels tuned to specific subsets of the parameter space, normalization occurs when the response of a given neuron or sensory channel is modulated in relation to the overall activation present in the population (Carandini & Heeger, 1994, 2012). Here, the multichannel response encoding gaze direction, which is driven by the difference in activity between leftward and rightward channels, is normalized to the summed activity across leftward, rightward, and direct sensory channels. In general, this form of computation ensures that the relative activation across channels provides a sensory code that is robust to variation in the absolute level of stimulation that might be due to extraneous properties of the stimulus. For instance, although the response of a channel tuned to rightward gaze directions will be dependent on the gaze direction of the stimulus, it may also be modulated by factors such as the illumination (e.g., a high-contrast face stimulus may elicit a greater response than a low-contrast stimulus). As long as these extraneous factors affect the responses of all sensory channels equally, however, their influence will be controlled for by normalization such that a consistent sensory code is maintained across stimulus contexts. Importantly, this process does not require an explicit representation of the particular stimulus properties that are being controlled for, and thus, it is plausible that this computation is repeated across the sensory system, including for the coding of both lower- and higher-level visual properties, even if the nature of the extraneous factors that are being controlled for may differ. On the basis of these conceptual considerations as well as a broad range of empirical findings, divisive normalization is proposed to be a canonical computation of the sensory system (Carandini & Heeger, 2012), although it is yet to be established in the context of gaze processing.
We initially fitted this model of encoded gaze direction to data reported by Calder and colleagues (2008), which examined the perceived direction of gaze for face stimuli under typical viewing conditions and after sensory adaptation. This experiment required participants to report their perceived direction of gaze as a three-alternative categorical response (left/direct/right). Thus, to model these data, we convert the multichannel response described to a perceptual decision. On any given trial, the response is taken to be corrupted by sensory noise drawn from a normal distribution with mean zero and standard deviation, σN, such that the probability that the multichannel response exceeds some decision boundary, b, is given by
$pMd>b=ΦMd−bσN$
where Φ denotes the cumulative distribution function of the standard normal distribution. To match the three-alternative (left/direct/right) decisions made by participants in the experiments by Calder and colleagues (2008), the decision boundaries between direct/right and left/direct are set symmetrically about zero at ±b.
Adaptation to any given gaze direction, dA, is modeled as reducing the sensitivity of each channel in proportion to that channel's response to the adapting stimulus, such that
$RAd=1−αR0dAR0dLAd=1−αL0dAL0dCAd=1−αC0dAC0d$
where α determines the degree of adaptation and the subscripts A and 0 denote adapted and unadapted (baseline) responses, respectively.

We first fitted the model to an ensemble of the data reported by Calder and colleagues (2008) using the MATLAB fminsearch function to optimize parameters g and s determining the channel sensitivity profiles, σN and b governing the transformation into a forced-choice decision, and α determining the degree of adaptation. This included unadapted data and data after adaptation to several different directions of gaze: 25° averted gaze, 10° averted gaze, direct gaze, and interleaved left–right 25° averted gaze. Optimization was implemented as the minimization of the residual variance unexplained across the entire data set.

To map the output of the model system onto the continuous (rather than categorical) measure of perceived gaze direction employed in this study, a constant scaling term, S, was applied to the multichannel response. This treats the mapping between multichannel response and perceived gaze direction as linear. Thus, the scaled multichannel response, MS, was given by
$MSd=MdS$
The scaling factor was optimized by fitting the output of the unadapted model to the unadapted (baseline) data from this study. As above, the fminsearch function was used to minimize the sum of squared errors between the scaled multichannel response and mean participant pointer response for each of the 15 directions of gaze tested.

Model predictions were generated for the effects of sensory adaptation to 25° leftward-averted gaze, 25° rightward-averted gaze, and direct gaze, on stimuli that ranged across the spectrum of horizontal gaze directions. The final step was to fit model predictions to the continuous response data collected in this study. The channel parameters (g, s) were fixed to the values that best fit the data reported by Calder and colleagues (2008), whereas the decision parameters (b, σN) were not relevant to the present data set as responses were continuous rather than categorical. We allowed the degree of adaptation, α, to vary given that differences in the experimental protocol between this study and the study by Calder and colleagues (2008) likely resulted in different degrees of sensory adaptation. As above, the fminsearch function was used to minimize the sum of squared errors between the scaled multichannel responses and mean participant pointer responses, in each case using the difference between unadapted and adapted states. This was done simultaneously across the 15 tested directions of gaze and summed across the three adaptation conditions. Thus, the fit of model predictions to the experimental data collected in this study was, conservatively, a single-parameter fit.

RESULTS

Generating Quantitative Predictions of Sensory Aftereffects for Gaze

The three-channel model of encoded gaze direction was first fit to the categorical response data reported by Calder and colleagues (2008). The resultant channel sensitivities in the unadapted (baseline) state are displayed in Figure 2A. The fit between modeled categorical responses and the corresponding experimental data are shown in Figure 2B–D. The model fits the data very closely, accounting for 96.3% of the total variance. The parameters that best fit the model to the data were as follows: g = 7.78°, s = 5.57°, b = 0.28, σN = 0.15, and α = 0.69.

Figure 2.

Three-channel model fit to psychophysical data from Calder and colleagues (2008). (A) Channel sensitivity as a function of stimulus gaze direction for leftward, rightward, and direct sensory channels. (B) Data from Calder and colleagues (2008) are shown as the average proportion of gaze stimuli categorized as “direct” in the unadapted state (black circles) and after adaptation to 25° leftward gaze (green squares) and 25° rightward gaze (orange triangles). The modeled probability of a “direct” response is indicated by the corresponding plot lines. (C) Same as B, but for adaptation to 10° leftward gaze (green squares) and 10° rightward gaze (orange triangles). (D) Same as B, but for adaptation to direct gaze (green squares) and interleaved 25° leftward and rightward gaze (orange triangles).

Figure 2.

Three-channel model fit to psychophysical data from Calder and colleagues (2008). (A) Channel sensitivity as a function of stimulus gaze direction for leftward, rightward, and direct sensory channels. (B) Data from Calder and colleagues (2008) are shown as the average proportion of gaze stimuli categorized as “direct” in the unadapted state (black circles) and after adaptation to 25° leftward gaze (green squares) and 25° rightward gaze (orange triangles). The modeled probability of a “direct” response is indicated by the corresponding plot lines. (C) Same as B, but for adaptation to 10° leftward gaze (green squares) and 10° rightward gaze (orange triangles). (D) Same as B, but for adaptation to direct gaze (green squares) and interleaved 25° leftward and rightward gaze (orange triangles).

The fitted channel sensitivities were then used to generate predictions regarding the direction and magnitude of perceptual aftereffects across stimulus gaze directions ranging from 35° leftward to 35° rightward. Specifically, we modeled the effects of adaptation to 25° leftward gaze, 25° rightward gaze, and direct gaze. Channel sensitivities after adaptation are shown in the top row of Figure 3. The multichannel response (i.e., encoded gaze direction, computed by combining activation across channels) is shown in the center row of Figure 3, both after adaptation and in the unadapted state. The predicted effect of sensory adaptation on perceived gaze direction (i.e., the perceptual aftereffect) was taken as the difference in the multichannel response after adaptation compared with that in the unadapted state. The predicted direction and magnitude of aftereffects are shown in the bottom row of Figure 3. As can be seen, a characteristic profile of sensory aftereffects emerges for adaptation to each of leftward, rightward, and direct gaze. Importantly, the magnitude of perceptual aftereffects depends on the gaze direction of the stimulus in each case, with a surprising nonuniform and nonmonotonic relationship between these variables (discussed in detail below).

Figure 3.

Modeled effects of sensory adaptation in a three-channel system. The effects of adaptation on channel sensitivities, multichannel response, and perceived gaze direction are shown after adaptation to 25° leftward, direct, and 25° rightward gaze. The top row displays channel sensitivities for leftward, rightward, and direct sensory channels after adaptation. Unadapted (baseline) channel sensitivities are shown with dotted lines. The center row displays multichannel response after adaptation and in the unadapted state. The bottom row displays the difference in the multichannel response between the adapted and unadapted states—this corresponds to the direction and relative magnitude of perceptual aftereffects predicted by the model after adaptation to gaze stimuli.

Figure 3.

Modeled effects of sensory adaptation in a three-channel system. The effects of adaptation on channel sensitivities, multichannel response, and perceived gaze direction are shown after adaptation to 25° leftward, direct, and 25° rightward gaze. The top row displays channel sensitivities for leftward, rightward, and direct sensory channels after adaptation. Unadapted (baseline) channel sensitivities are shown with dotted lines. The center row displays multichannel response after adaptation and in the unadapted state. The bottom row displays the difference in the multichannel response between the adapted and unadapted states—this corresponds to the direction and relative magnitude of perceptual aftereffects predicted by the model after adaptation to gaze stimuli.

Fit between Model Predictions and Continuous Response Data

The present section reports the fit between model predictions generated in the previous section and the continuous response data collected in our experimental task. The value of the scaling parameter, S, that optimized the fit between the multichannel response and perceived gaze direction in the unadapted (baseline) state was 37.0. The degree of adaptation, α, that optimized the fit between the scaled multichannel response and perceptual aftereffects across the three adaptation conditions was 0.51. This corresponds to 74% of the degree of adaptation modeled above for the experimental data reported by Calder and colleagues (2008), suggesting a somewhat lesser degree of adaptation in this study.

The single-parameter fit between model predictions and pointer response data is shown in Figure 4. The model fits the data closely, accounting for 66.7% of the total variance. It is notable that several qualitative aspects of the model predictions are apparent in the experimental data, including (i) the tendency for peak aftereffects in the averted adaptation conditions to occur between the direction of the adaptor stimulus and direct gaze, (ii) the tendency for the magnitude of aftereffects to approach nil both for gaze stimuli more averted than the adaptor stimulus and for gaze stimuli on the opposite side to that adapted, (iii) a degree of normalization of the adapted direction itself after adaptation to averted gaze, (iv) the inverse direction of aftereffects in the leftward and rightward adaptation conditions, and (v) the lesser magnitude of aftereffects in the direct adaptation condition compared with the averted adaptation conditions.

Figure 4.

The effects of sensory adaptation on perceived gaze direction. Response aftereffects are reported as the difference in perceived gaze direction between the unadapted (baseline) state and the adapted state. Predictions of the three-channel model are reported alongside experimental data after adaptation to 25° leftward, direct, and 25° rightward gaze stimuli. Error bars indicate ±1 SEM.

Figure 4.

The effects of sensory adaptation on perceived gaze direction. Response aftereffects are reported as the difference in perceived gaze direction between the unadapted (baseline) state and the adapted state. Predictions of the three-channel model are reported alongside experimental data after adaptation to 25° leftward, direct, and 25° rightward gaze stimuli. Error bars indicate ±1 SEM.

The Effect of Divisive Normalization on Sensory Aftereffects

It is initially surprising that a nonmonotonic relationship exists between stimulus gaze direction and the perceptual aftereffects predicted by the model (and which are apparent in the experimental data). This is because adaptation has a monotonic effect on channel sensitivity, which one might expect to translate to a similarly monotonic effect on the multichannel response such that attenuation of leftward channel sensitivity (for instance) would produce a uniform or global shift in subsequent perception toward rightward gaze. Indeed, the perceptual aftereffects reported by previous studies after adaptation to averted gaze resemble a uniform shift in encoded gaze direction away from the adapted direction (e.g., Calder et al., 2008; Jenkins et al., 2006; Seyama & Nagayama, 2006). However, these previous studies only assessed the effects of adaptation on categorical judgments regarding a limited range of stimulus gaze directions and thus are not definitive regarding how the magnitude and direction of aftereffects present across the spectrum of test deviations. As different gaze deviations engage the three sensory channels to differing degrees, the extent to which a given degree of channel adaptation influences subsequent perception (i.e., the magnitude of the perceptual aftereffect) may depend on the particular test stimulus presented, leading to a nonuniform profile of aftereffects. In addition, it has recently been highlighted that the effects of adaptation on neural responses can be complicated by interactions with both excitatory and suppressive mechanisms in neural systems (Solomon & Kohn, 2014). In this section and the next, we illustrate that the specific profile of aftereffects observed after adaptation is diagnostic of how information is combined across sensory channels in the brain to encode the stimulus parameter.

First, the nonuniform and nonmonotonic pattern of aftereffects predicted by our model of gaze direction comes about because of the operation of divisive normalization in computing the multichannel response from channel activations. Importantly, increased activation of a given channel has both a driving influence on the represented gaze direction (via the numerator of the response function described in the previous section, Three-channel Model of the Gaze System) and an attenuating influence on the represented gaze direction (via the denominator of the response function). The relative strength of these competing influences differs across test stimuli, resulting in the distinctive profile of perceptual aftereffects predicted across the spectrum of test gaze directions.

To help illustrate this point, the encoded gaze direction, M(d), described in the previous section, can be rewritten as follows, where N is the numerator of this equation and D is the denominator:
$Md=ND$
where
$N=Rd−LdD=Rd+Ld+Cd$
The change in encoded gaze direction after adaptation (i.e., the sensory aftereffect) can then be specified by following the quotient rule:
$δMd=DδN−NδDD2$
This expression can be separated into a term that is dependent on the change in the numerator and a term that is dependent on the change in the denominator, which we refer to as δNnormalized and δDnormalized, respectively,
$δMd=δNnormalized+δDnormalized$
where
$δNnormalized=δNDδDnormalized=−NδDD2$
The opposing effects that sensory adaptation has on represented gaze direction, via its operation on each of these terms, are illustrated in Figure 5. As can be seen in Figure 5C, the effects of adaptation on the numerator and denominator have opposing effects on the encoded gaze direction, with the relative magnitude of these opposing effects differing across the spectrum of (test) stimulus gaze deviations. This results in the nonuniform and nonmonotonic pattern of sensory aftereffects illustrated in Figures 3 and 4. Without divisive normalization (i.e., the denominator of the response function), the magnitude of aftereffects after adaptation to averted gaze would vary monotonically as a function of test gaze deviation. Thus, the close fit between model predictions and perceived gaze direction in human observers is evidence for the operation of divisive normalization in the neural encoding of gaze direction.
Figure 5.

Opposing effects of sensory adaptation on perceived gaze direction in the three-channel model. (A) The effect of 25° leftward gaze adaptation on the numerator (N) of the multichannel response function. The unadapted (baseline) response is shown with the dotted line. (B) The effect of 25° leftward gaze adaptation on the denominator (D) of the multichannel response function. (C) The normalized change in the numerator (green line; δN/D) and denominator (orange line; (−NδD)/D2) after adaptation to 25° leftward gaze. The dotted line shows the normalized change in the denominator when flipped in sign, such that the predicted aftereffect is given by the difference between the green and dotted orange lines.

Figure 5.

Opposing effects of sensory adaptation on perceived gaze direction in the three-channel model. (A) The effect of 25° leftward gaze adaptation on the numerator (N) of the multichannel response function. The unadapted (baseline) response is shown with the dotted line. (B) The effect of 25° leftward gaze adaptation on the denominator (D) of the multichannel response function. (C) The normalized change in the numerator (green line; δN/D) and denominator (orange line; (−NδD)/D2) after adaptation to 25° leftward gaze. The dotted line shows the normalized change in the denominator when flipped in sign, such that the predicted aftereffect is given by the difference between the green and dotted orange lines.

Opponent Coding

The profile of sensory aftereffects also distinguishes between opponent and multichannel coding. An opponent (i.e., two-channel) model was implemented with channels tuned broadly to leftward and rightward gaze. The channel sensitivities were constrained such that sensitivity was half its maximum at direct gaze (i.e., g = 0°). The multichannel response was defined as follows:
$Md=Rd−LdRd+Ld.$
The model was fit using the identical procedure as for the three-channel model. The parameters that best fit the model to the data reported by Calder and colleagues were as follows: s = 9.11°, b = 0.33, σN = 0.18, and α = 0.38. The model fit to these data is illustrated in Figure 6 and accounted for 85.07% of the total variance.
Figure 6.

Opponent model fit to psychophysical data from Calder and colleagues (2008). See Figure 2 for comparison with the three-channel model and for description of the figure panels.

Figure 6.

Opponent model fit to psychophysical data from Calder and colleagues (2008). See Figure 2 for comparison with the three-channel model and for description of the figure panels.

The effects of adaptation on channel sensitivities and the multichannel response are shown in Figure 7, together with the resulting aftereffects predicted by this opponent model. Previous work has identified that opponent and multichannel systems make different predictions regarding the effect of adaptation to direct gaze (Calder et al., 2008). This is apparent in the results of the modeling reported here by comparing the central column in Figures 3 and 7; specifically, the opponent model predicts no effect of direct gaze adaptation, whereas the three-channel model predicts a small degree of local repulsion around direct gaze (Figure 7). Surprisingly, however, we find that the two- and three-channel models make even more notably different predictions regarding the effects of adaptation to averted gaze. Specifically, adaptation to averted gaze in the opponent model tends to result in a profile of aftereffects that peaks near to direct gaze and falls off in magnitude at approximately the same rate on either side of direct gaze. In contrast, adaptation to averted gaze in the three-channel model, as we have already seen, is associated with peak aftereffects between the point of the adaptor and direct gaze (i.e., an asymmetrical profile around direct gaze).

Figure 7.

Modeled effects of sensory adaptation in an opponent system. See Figure 3 for comparison with the three-channel model and for description of the figure panels.

Figure 7.

Modeled effects of sensory adaptation in an opponent system. See Figure 3 for comparison with the three-channel model and for description of the figure panels.

The fit between the opponent model and the data collected in this study is shown in Figure 8. Qualitatively, it is clear that the aftereffects that we observe after adaptation to averted gaze resemble those predicted by the three-channel model rather than those predicted by the opponent model. Most notably, peak aftereffects occur between the point of the adaptor and the point of direct gaze, rather than showing the pattern centered around direct gaze that is predicted by the opponent model. The opponent model accounted for 45.35% of the total variance, which is notably less than that of the three-channel model. Thus, the profile of aftereffects that we observe in our sample supports a three-channel model over an opponent model.

Figure 8.

Fit between empirical data and the predictions of the opponent model of perceived gaze direction. See Figure 4 for comparison with the three-channel model and description of figure content. The value of the scaling parameter, S, that optimized the fit between the multichannel response of the opponent model and perceived gaze direction in the unadapted (baseline) state was 38.06. The degree of adaptation, α, that optimized the fit between the scaled multichannel response and data collected in this study was 0.24.

Figure 8.

Fit between empirical data and the predictions of the opponent model of perceived gaze direction. See Figure 4 for comparison with the three-channel model and description of figure content. The value of the scaling parameter, S, that optimized the fit between the multichannel response of the opponent model and perceived gaze direction in the unadapted (baseline) state was 38.06. The degree of adaptation, α, that optimized the fit between the scaled multichannel response and data collected in this study was 0.24.

Figure 9.

Opposing effects of sensory adaptation on perceived gaze direction in an opponent system. See Figure 5 for comparison with the three-channel model and for description of figure contents.

Figure 9.

Opposing effects of sensory adaptation on perceived gaze direction in an opponent system. See Figure 5 for comparison with the three-channel model and for description of figure contents.

Narrowband Multichannel Coding

The three-channel model discussed so far is an example of broadband multichannel coding, with sensory channels tuned broadly across a range of stimulus gaze directions. We focus on this model as the most parsimonious system that accounts for the evidence of a direct sensory channel reported in Calder et al. (2008). However, one might imagine multichannel systems with more than three channels. Thus, we have also simulated perceptual aftereffects in systems with many narrowly tuned sensory channels.

In Figure 10, we display a simulation of 17 sensory channels. The peak sensitivities of these channels are spaced in 5° intervals from −40° to 40°. The sensitivity of each channel was simulated as a raised cosine squared function that responds over a range ±10° from that channel's peak. The multichannel response was computed as the sum of channel activations, each weighted by the gaze direction that the channel is most sensitive to, normalized to the summed activity across channels. The effect of adaptation was modeled as described previously.

Figure 10.

Simulation of a narrowband multichannel model. See Figures 3 and 7 for comparison with the three-channel and opponent models and for description of the figure panels. Note that, unlike for the three-channel and opponent models described earlier in this article, the parameters of the narrowband simulation have not been derived by fitting the model to any empirical data. Rather, the strength of adaptation is set to full adaptation in the present figure (i.e., α = 1), meaning that the magnitude of aftereffects is the maximum that could be obtained with the specified channel structure.

Figure 10.

Simulation of a narrowband multichannel model. See Figures 3 and 7 for comparison with the three-channel and opponent models and for description of the figure panels. Note that, unlike for the three-channel and opponent models described earlier in this article, the parameters of the narrowband simulation have not been derived by fitting the model to any empirical data. Rather, the strength of adaptation is set to full adaptation in the present figure (i.e., α = 1), meaning that the magnitude of aftereffects is the maximum that could be obtained with the specified channel structure.

Thus, the predictions of the narrowband model are not qualitatively consistent with the collected empirical data. This is not to say that a more complex model could not be made to fit the data, but under the basic assumptions that we have made (e.g., regarding how adaptation affects channel sensitivities), our results support the three-channel model over opponent coding or narrowband systems.

DISCUSSION

This article implemented, for the first time, a computational model of gaze processing, in which the horizontal direction of gaze of a seen face is encoded across a set of gaze-selective sensory channels, incorporating the divisive normalization of sensory responses. We demonstrate that this form of sensory encoding predicts a distinctive pattern of perceptual changes after sensory adaptation that differ across the spectrum of stimulus gaze deviations. We then find that these model predictions closely match psychophysical data recorded in human observers when adapted to leftward, rightward, and direct gaze. We focus on a model with sensory channels tuned broadly to leftward, rightward, and direct gaze and find evidence in favor of this form of sensory coding over opponent or narrowband multichannel systems.

The present findings are consistent with electrophysiological research in the macaque visual cortex, which identified individual cells in the anterior STS tuned to distinct directions of gaze (Perrett et al., 1985, 1990, 1992). Specifically, these studies detected cells that were differentially selective to gaze stimuli that were directed at the participant, averted horizontally by 45° or averted vertically by 45°. Psychophysical analyses of gaze perception and functional neuroimaging of gaze processing have built on these electrophysiological findings by demonstrating the existence of neural mechanisms in humans tuned specifically to leftward and rightward gaze (Calder et al., 2007, 2008; Jenkins et al., 2006; Seyama & Nagayama, 2006). The advance of the present article is in demonstrating a plausible scheme for the functional interaction between such cell populations that results in an encoded angle of gaze.

Calder and colleagues (2008) proposed that gaze direction is represented via multichannel encoding rather than a simpler, opponent coding system, as the operation of at least three sensory channels is most consistent with the effects of sensory adaptation on gaze perception that they observed. However, these previous data were unable to distinguish between a system with three broadly tuned sensory channels and a continuum of more narrowly tuned channels. Narrowband multichannel coding is associated with the representation of spatial frequency and orientation (Suzuki, 2005); for instance, an opponent coding system is associated with the representation of color, at least at the early stages of processing (Webster, 2015). Here, we show that opponent, three-channel, and narrowband multichannel systems predict different profiles of aftereffects after adaptation to averted gaze. We find that three broadly tuned channels cohere most closely with both the psychophysical data reported in this study and that reported by Calder and colleagues (2008).

A key functional mechanism that our findings point to is the operation of divisive normalization in the encoding of gaze direction across sensory channels. Divisive normalization is proposed to be a canonical computational feature of sensory processing in the brain (Carandini & Heeger, 2012), accounting, for instance, for the response rates of neurons in the retina (e.g., adaptation to global light intensity; Normann & Perlman, 1979), V1 (e.g., encoding of position and orientation; Carandini & Heeger, 1994), and higher visual areas (e.g., MT, encoding speed, and direction of motion; Simoncelli & Heeger, 1998). This computation has also been demonstrated outside the visual system, such as in olfactory neurons (Olsen, Bhandawat, & Wilson, 2010) and primary auditory cortex (Rabinowitz, Willmore, Schnupp, & King, 2011). It has recently been highlighted that the effect of adaptation on sensory coding and neural responses is complicated by its potential to act on both driving and suppressive mechanisms, including those that implement divisive normalization (Solomon & Kohn, 2014). In the current article, we demonstrate correspondingly that a fine-grained analysis of perceptual aftereffects can reveal the operation of divisive normalization in sensory coding. In this study, we model encoded gaze direction as being normalized by pooled activity across leftward, rightward, and direct sensory channels. The surprising correspondence between model predictions and the profile of aftereffects observed psychophysically is strong evidence that the neural encoding of gaze direction rests on the divisive normalization of sensory responses.

This is an exciting insight in part because of the recent trend toward computational approaches to neuropsychiatric conditions in which gaze processing is implicated, including autism and schizophrenia (Friston, Stephan, Montague, & Dolan, 2014). Indeed, a very recent proposal is that symptoms across sensory, cognitive, and social domains in autism reflect a widespread reduction of divisive normalization in neural processing, corresponding to an increased ratio of neural excitation to inhibition (Lawson, Friston, & Rees, 2015; Rosenberg et al., 2015). The role of divisive normalization in sensory coding is to control for the influence of extraneous contextual factors on sensory responses, and thus, dysfunction in the neural mechanisms that implement this computation suggests that contextual interference will be more disrupting to sensory coding and behavior. The appeal of computational approaches to psychiatry is the possibility of quantitatively describing information processing mechanisms that underlie mental function, aiding the bridge between the cognitive and behavioral levels of description that currently define the diagnosis of these conditions and the biological mechanisms that would ideally subserve diagnosis and treatment. Atypical gaze-based behaviors are a cardinal feature of autism (Pell et al., 2016; Lai, Lombardo, & Baron-Cohen, 2014; American Psychiatric Association, 2013), and thus, the present article provides a foundation for directly testing this computational approach to autism in a domain central to the core diagnostic symptoms.

It is a remarkable fact that the direction in which we perceive a face to be looking can be so markedly affected by the recent history of sensory stimulation. This phenomenon of sensory adaptation to eye gaze direction was first reported in 2006 (Jenkins et al., 2006; Seyama & Nagayama, 2006) and has since been demonstrated for other cues to social attention, namely, head and body rotation (Lawson & Calder, 2016; Lawson, Clifford, & Calder, 2009, 2011; Fang & He, 2005). Here, we report effects of adaptation on a continuous measure of perceived gaze direction, demonstrating that adaptation is not limited to an effect on categorical judgments (as used in all previous studies of adaptation to social cues). We find that these quantitative effects can be plausibly modeled as habituation of the sensory channels encoding gaze direction, consistent with the interpretation that this phenomenon is perceptual in nature, rather than reflecting a change in a more cognitive factor like the response criterion used by participants when making categorical judgments about gaze direction.

Adaptation to direct gaze has been reported to increase the tendency to categorize slightly averted gaze directions as being averted rather than direct (Calder et al., 2008). In this study, however, we do not observe clear evidence for an effect of adaptation to direct gaze on perceived gaze direction (Figure 4, center). This may be a matter of sensitivity: The three-channel model predicts that aftereffects after adaptation to direct gaze are notably smaller in magnitude than those induced by adaptation to averted gaze (Figure 3), and this is consistent with the subtle effects of direct gaze adaptation reported previously. Alternatively, however, the effect of the adaptation protocol on gaze judgments could in principle reflect something other than pure adaptation effects (e.g., an effect on the criteria for what constitutes “direct” gaze in categorical gaze judgments). Such effects may not be apparent in the present paradigm due to the use of a continuous measure of perceived gaze direction. Together, our modeling analyses and empirical data suggest that the effects of adaptation to averted gaze are more robustly diagnostic of the underlying channel structure.

A natural hypothesis is that the encoding of gaze direction that we model here is implemented in the anterior STS. The evidence that this region is central to encoding others' direction of gaze converges across experimental approaches (Carlin & Calder, 2013; Langton, 2010). Consistent with the electrophysiological research reviewed earlier, monkeys with bilateral ablation of the STS demonstrate impaired gaze processing, without showing pronounced impairments in face processing more generally (Heywood & Cowey, 1992; Campbell, Heywood, Cowey, Regard, & Landis, 1990). Accordingly, increased activation in STS regions is consistently observed during functional neuroimaging of humans viewing gaze stimuli (e.g., Pelphrey, Viola, & McCarthy, 2004; Hoffman & Haxby, 2000; Puce, Allison, Bentin, Gore, & McCarthy, 1998), and the pattern of activity across voxels in anterior STS carries information about the direction of others' gaze, independent of the particular combination of head and eye rotation used to signal this gaze direction (Carlin, Calder, Kriegeskorte, Nili, & Rowe, 2011). Finally, sensory adaptation to gaze direction in humans modulates the BOLD response to face stimuli in the anterior STS region (Calder et al., 2007). There is evidence that this region sends upstream connections to regions such as posterior STS, lateral parietal cortex, and medial pFC, which implement social–cognitive functions that are based on encoded gaze direction, such as joint attention (Carlin & Calder, 2013).

In summary, the present article builds on existing electrophysiological, neuroimaging, and psychophysical data that demonstrate the existence of cell populations in anterior STS that are tuned to specific directions of observed gaze. We implement a functional model of encoded gaze direction across gaze-selective sensory channels, in which a multichannel response is computed based on the difference in activation between channels normalized by the pooled activation across the channel population. Importantly, normalization of the sensory response is associated with a distinctive and surprising set of predictions regarding the effects of sensory adaptation on encoded gaze direction. These predictions closely match the profile of perceptual aftereffects exhibited by human observers after adaptation to leftward, rightward, and direct gaze, when tested across a broad range of gaze stimuli. This indicates a role for divisive normalization in the sensory encoding of perceived gaze direction and further demonstrates that the effects of recent sensory stimulation on encoded gaze direction can be accurately modeled as a reduction in the gain on individual channels that is proportional to how strongly the channel is activated by the prior stimulus. Our results also suggest that gaze is encoded across at least three broadly tuned sensory channels, rather than an opponent channel system or a bank of more narrowly tuned channels. This establishes a credible functional architecture for the encoding of gaze direction, which may be important for identifying a computational basis to the aberrant gaze mechanisms that occur in conditions such as autism, schizophrenia, and social anxiety disorder, particularly in relation to the role of divisive normalization in sensory computations (Rosenberg et al., 2015) and the modulation of neural responses by recent sensory history (Pellicano, Rhodes, & Calder, 2013).

Acknowledgments

This work is supported by Australian Research Council Discovery Project DP160102239.

Reprint requests should be sent to Colin J. Palmer, School of Psychology, UNSW Sydney, Sydney 2052, New South Wales, Australia, or via e-mail: Colin.Palmer@unsw.edu.au.

REFERENCES

REFERENCES
American Psychiatric Association
. (
2013
).
Diagnostic and statistical manual of mental disorders: DSM-5
(5th ed.).
Arlington, VA
:
American Psychiatric Association
.
Baron-Cohen
,
S.
(
1997
).
Mindblindness: An essay on autism and theory of mind
.
Cambridge, MA
:
MIT Press
.
Blakemore
,
C.
, &
Campbell
,
F. W.
(
1969
).
On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images
.
Journal of Physiology
,
203
,
237
260
.
Brainard
,
D. H.
(
1997
).
The psychophysics toolbox
.
Spatial Vision
,
10
,
433
436
.
Calder
,
A. J.
,
Beaver
,
J. D.
,
Winston
,
J. S.
,
Dolan
,
R. J.
,
Jenkins
,
R.
,
Eger
,
E.
, et al
(
2007
).
Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule
.
Current Biology
,
17
,
20
25
.
Calder
,
A. J.
,
Jenkins
,
R.
,
Cassel
,
A.
, &
Clifford
,
C. W.
(
2008
).
Visual representation of eye gaze is coded by a nonopponent multichannel system
.
Journal of Experimental Psychology: General
,
137
,
244
261
.
Campbell
,
R.
,
Heywood
,
C. A.
,
Cowey
,
A.
,
Regard
,
M.
, &
Landis
,
T.
(
1990
).
Sensitivity to eye gaze in prosopagnosic patients and monkeys with superior temporal sulcus ablation
.
Neuropsychologia
,
28
,
1123
1142
.
Carandini
,
M.
, &
Heeger
,
D. J.
(
1994
).
Summation and division by neurons in primate visual cortex
.
Science
,
264
,
1333
1336
.
Carandini
,
M.
, &
Heeger
,
D. J.
(
2012
).
Normalization as a canonical neural computation
.
Nature Reviews Neuroscience
,
13
,
51
62
.
Carlin
,
J. D.
, &
Calder
,
A. J.
(
2013
).
The neural basis of eye gaze processing
.
Current Opinion in Neurobiology
,
23
,
450
455
.
Carlin
,
J. D.
,
Calder
,
A. J.
,
Kriegeskorte
,
N.
,
Nili
,
H.
, &
Rowe
,
J. B.
(
2011
).
A head view-invariant representation of gaze direction in anterior superior temporal sulcus
.
Current Biology
,
21
,
1817
1821
.
Den Boer
,
J. A.
(
2000
).
Social anxiety disorder/social phobia: Epidemiology, diagnosis, neurobiology, and treatment
.
Comprehensive Psychiatry
,
41
,
405
415
.
Fang
,
F.
, &
He
,
S.
(
2005
).
Viewer-centered object representation in the human visual system revealed by viewpoint aftereffects
.
Neuron
,
45
,
793
800
.
Fesharaki
,
H.
,
Rezaei
,
L.
,
Farrahi
,
F.
,
Banihashem
,
T.
, &
Jahanbkhshi
,
A.
(
2012
).
Normal interpupillary distance values in an Iranian population
.
Journal of Ophthalmic and Vision Research
,
7
,
231
234
.
Friston
,
K.
,
Stephan
,
K. E.
,
Montague
,
R.
, &
Dolan
,
R. J.
(
2014
).
Computational psychiatry: The brain as a phantastic organ
.
Lancet Psychiatry
,
1
,
148
158
.
Georgopoulos
,
A. P.
,
Schwartz
,
A. B.
, &
Kettner
,
R. E.
(
1986
).
Neuronal population coding of movement direction
.
Science
,
233
,
1416
1419
.
Green
,
M. F.
,
Horan
,
W. P.
, &
Lee
,
J.
(
2015
).
Social cognition in schizophrenia
.
Nature Reviews Neuroscience
,
16
,
620
631
.
Heywood
,
C. A.
, &
Cowey
,
A.
(
1992
).
The role of the “face-cell” area in the discrimination and recognition of faces by monkeys
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
335
,
31
37
;
discussion 37–38
.
Hoffman
,
E. A.
, &
Haxby
,
J. V.
(
2000
).
Distinct representations of eye gaze and identity in the distributed human neural system for face perception
.
Nature Neuroscience
,
3
,
80
84
.
Jenkins
,
R.
,
Beaver
,
J. D.
, &
Calder
,
A. J.
(
2006
).
I thought you were looking at me: Direction-specific aftereffects in gaze perception
.
Psychological Science
,
17
,
506
513
.
Lai
,
M. C.
,
Lombardo
,
M. V.
, &
Baron-Cohen
,
S.
(
2014
).
Autism
.
Lancet
,
383
,
896
910
.
Langton
,
S. R. H.
(
2010
).
Gaze perception and visually mediated attention
. In
R. B.
,
N.
,
K.
Nakayama
, &
S.
Shimojo
(Eds.),
The science of social vision
.
New York
:
Oxford University Press
.
Lawson
,
R. P.
, &
Calder
,
A. J.
(
2016
).
The “where” of social attention: Head and body direction aftereffects arise from representations specific to cue type and not direction alone
.
Cognitive Neuroscience
,
7
,
103
113
.
Lawson
,
R. P.
,
Clifford
,
C. W.
, &
Calder
,
A. J.
(
2009
).
About turn: The visual representation of human body orientation revealed by adaptation
.
Psychological Science
,
20
,
363
371
.
Lawson
,
R. P.
,
Clifford
,
C. W.
, &
Calder
,
A. J.
(
2011
).
A real head turner: Horizontal and vertical head directions are multichannel coded
.
Journal of Vision
,
11
,
pii:17
.
doi:10.1167/11.9.17
.
Lawson
,
R. P.
,
Friston
,
K. J.
, &
Rees
,
G.
(
2015
).
A more precise look at context in autism
.
Proceedings of the National Academy of Sciences, U.S.A.
,
112
,
E5226
.
Mareschal
,
I.
,
Calder
,
A. J.
, &
Clifford
,
C. W.
(
2013
).
Humans have an expectation that gaze is directed toward them
.
Current Biology
,
23
,
717
721
.
Normann
,
R. A.
, &
Perlman
,
I.
(
1979
).
The effects of background illumination on the photoresponses of red and green cones
.
Journal of Physiology
,
286
,
491
507
.
Olsen
,
S. R.
,
Bhandawat
,
V.
, &
Wilson
,
R. I.
(
2010
).
Divisive normalization in olfactory population codes
.
Neuron
,
66
,
287
299
.
Otsuka
,
Y.
,
Mareschal
,
I.
,
Calder
,
A. J.
, &
Clifford
,
C. W.
(
2014
).
Dual-route model of the effect of head orientation on perceived gaze direction
.
Journal of Experimental Psychology: Human Perception and Performance
,
40
,
1425
1439
.
Otsuka
,
Y.
,
Mareschal
,
I.
, &
Clifford
,
C. W.
(
2015
).
Gaze constancy in upright and inverted faces
.
Journal of Vision
,
15
,
15.1.21
.
Pell
,
P. J.
,
Mareschal
,
I.
,
Calder
,
A. J.
,
von dem Hagen
,
E. A.
,
Clifford
,
C. W.
,
Baron-Cohen
,
S.
, et al
(
2016
).
Intact priors for gaze direction in adults with high-functioning autism spectrum conditions
.
Molecular Autism
,
7
,
25
.
Pellicano
,
E.
,
Rhodes
,
G.
, &
Calder
,
A. J.
(
2013
).
Reduced gaze aftereffects are related to difficulties categorising gaze direction in children with autism
.
Neuropsychologia
,
51
,
1504
1509
.
Pelphrey
,
K. A.
,
Viola
,
R. J.
, &
McCarthy
,
G.
(
2004
).
When strangers pass: Processing of mutual and averted social gaze in the superior temporal sulcus
.
Psychological Science
,
15
,
598
603
.
Perrett
,
D. I.
,
Harries
,
M. H.
,
Mistlin
,
A. J.
,
Hietanen
,
J. K.
,
Benson
,
P. J.
,
Bevan
,
R.
, et al
(
1990
).
Social signals analyzed at the single cell level: Someone is looking at me, something touched me, something moved!
International Journal of Comparative Psychology
,
4
,
25
55
.
Perrett
,
D. I.
,
Hietanen
,
J. K.
,
Oram
,
M. W.
, &
Benson
,
P. J.
(
1992
).
Organization and functions of cells responsive to faces in the temporal cortex
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
335
,
23
30
.
Perrett
,
D. I.
,
Smith
,
P. A.
,
Potter
,
D. D.
,
Mistlin
,
A. J.
,
,
A. S.
,
Milner
,
A. D.
, et al
(
1985
).
Visual cells in the temporal cortex sensitive to face view and gaze direction
.
Proceedings of the Royal Society of London, Series B, Biological Sciences
,
223
,
293
317
.
Puce
,
A.
,
Allison
,
T.
,
Bentin
,
S.
,
Gore
,
J. C.
, &
McCarthy
,
G.
(
1998
).
Temporal cortex activation in humans viewing eye and mouth movements
.
Journal of Neuroscience
,
18
,
2188
2199
.
Rabinowitz
,
N. C.
,
Willmore
,
B. D.
,
Schnupp
,
J. W.
, &
King
,
A. J.
(
2011
).
Contrast gain control in auditory cortex
.
Neuron
,
70
,
1178
1191
.
Regan
,
D.
, &
Beverley
,
K. I.
(
1985
).
.
Journal of the Optical Society of America A
,
2
,
147
155
.
Rosati
,
A. G.
, &
Hare
,
B.
(
2009
).
Looking past the model species: Diversity in gaze-following skills across primates
.
Current Opinion in Neurobiology
,
19
,
45
51
.
Rosenberg
,
A.
,
Patterson
,
J. S.
, &
Angelaki
,
D. E.
(
2015
).
A computational perspective on autism
.
Proceedings of the National Academy of Sciences, U.S.A.
,
112
,
9158
9165
.
Salinas
,
E.
, &
Abbott
,
L. F.
(
1994
).
Vector reconstruction from firing rates
.
Journal of Computational Neuroscience
,
1
,
89
107
.
Senju
,
A.
(
2013
).
Atypical development of spontaneous social cognition in autism spectrum disorders
.
Brain Development
,
35
,
96
101
.
Seyama
,
J.
, &
Nagayama
,
R. S.
(
2006
).
Eye direction aftereffect
.
Psychological Research
,
70
,
59
67
.
Simoncelli
,
E. P.
, &
Heeger
,
D. J.
(
1998
).
A model of neuronal responses in visual area MT
.
Vision Research
,
38
,
743
761
.
Solomon
,
S. G.
, &
Kohn
,
A.
(
2014
).
Moving sensory adaptation beyond suppressive effects in single neurons
.
Current Biology
,
24
,
R1012
R1022
.
Suzuki
,
S.
(
2005
).
High-level pattern coding revealed by brief shape aftereffects
. In
C. W.
Clifford
&
G.
Rhodes
(Eds.),
Fitting the mind to the world: Adaptation and aftereffects in high-level vision
.
Oxford, UK
:
Oxford University Press
.
Teufel
,
C.
,
Alexis
,
D. M.
,
Todd
,
H.
,
Lawrance-Owen
,
A. J.
,
Clayton
,
N. S.
, &
Davis
,
G.
(
2009
).
Social cognition modulates the sensory coding of observed gaze direction
.
Current Biology
,
19
,
1274
1277
.
Webster
,
M. A.
(
2015
).
.
Annual Review of Vision Science
,
1
,
547
567
.
Wollaston
,
W. H.
(
1824
).
On the apparent direction of eyes in a portrait
.
Philosophical Transactions of the Royal Society of London
,
114
,
247
256
.