Using behavioral and fMRI paradigms, we asked how the physical plausibility of complex 3-D objects, as defined by the object's congruence with 3-D Euclidean geometry, affects behavioral thresholds and neural responses to depth information. Stimuli were disparity-defined geometric objects rendered as random dot stereograms, presented in plausible and implausible variations. In the behavior experiment, observers were asked to complete (1) a noise-based depth task that involved judging the depth position of a target embedded in noise and (2) a fine depth judgment task that involved discriminating the nearer of two consecutively presented targets. Interestingly, results indicated greater behavioral sensitivities of depth judgments for implausible versus plausible objects across both tasks. In the fMRI experiment, we measured fMRI responses concurrently with behavioral depth responses. Although univariate responses for depth judgments were largely similar across cortex regardless of object plausibility, multivariate representations for plausible and implausible objects were notably distinguishable along depth-relevant intermediate regions V3 and V3A, in addition to object-relevant LOC. Our data indicate significant modulations of both behavioral judgments of and neural responses to depth by object context. We conjecture that disparity mechanisms interact dynamically with the object recognition problem in the visual system such that disparity computations are adjusted based on object familiarity.

The visual system's flexibility in estimating depth is remarkable: We readily perceive 3-D structure under diverse conditions from the seemingly random dots of a “magic eye” stereogram to the aesthetically beautiful, but obviously flat, canvasses of the Old Masters. Yet, 3-D perception is often enhanced when different cues specify the same depth. This perceptual process is understood as Bayesian inference that improves sensory estimates. Despite considerable behavioral support for this theory, insights into the cortical circuits involved are limited. Moreover, extant work tested quantitatively similar cues, reducing some of the challenges associated with integrating computationally and qualitatively different signals. Here we address this challenge by measuring fMRI responses to depth structures defined by shading, binocular disparity, and their combination. We quantified information about depth configurations (convex “bumps” vs. concave “dimples”) in different visual cortical areas using pattern classification analysis. We found that fMRI responses in dorsal visual area V3B/KO were more discriminable when disparity and shading concurrently signaled depth, in line with the predictions of cue integration. Importantly, by relating fMRI and psychophysical tests of integration, we observed a close association between depth judgments and activity in this area. Finally, using a cross-cue transfer test, we found that fMRI responses evoked by one cue afford classification of responses evoked by the other. This reveals a generalized depth representation in dorsal visual cortex that combines qualitatively different information in line with 3-D perception.