Recurrent interactions in the primary visual cortex make its output a complex nonlinear transform of its input. This transform serves preattentive visual segmentation, that is, autonomously processing visual inputs to give outputs that selectively emphasize certain features for segmentation. An analytical understanding of the nonlinear dynamics of the recurrent neural circuit is essential to harness its computational power. We derive requirements on the neural architecture, components, and connection weights of a biologically plausible model of the cortex such that region segmentation, figure-ground segregation, and contour enhancement can be achieved simultaneously. In addition, we analyze the conditions governing neural oscillations, illusory contours, and the absence of visual hallucinations. Many of our analytical techniques can be applied to other recurrent networks with translation-invariant neural and connection structures.

Experimental observations suggest that contour integration may take place in V1. However, there has yet to be a model of contour integration that uses only known V1 elements, operations, and connection patterns.This article introduces such a model, using orient ation selective cells, local cortical circuits, and horizontal intracortical connections. The model is composed of recurrently connected excitatory neurons and inhibitory interneurons, receiving visual input via oriented receptive fields resembling those found in primary visual cortex. Intracortical interactions modify initial activity patterns from input, selectively amplifying the activities of edges that form smooth contours in the image. The neural activities produced by such interactions are oscillatory and edge segments within a contour oscillate in synchrony. It is shown analytically and empirically that the extent of contour enhancement and neural synchrony increases with the smoothness, length, and closure of contours, as observed in experiments on some of these phenomena. In addition, the model incorporates a feedback mechanism that allows higher visual centers selectively to enhance or suppress sensitivities to given contours, effectively segmenting one from another. The model makes the testable prediction that the horizontal cortical connections are more likely to target excitatory (or inhibitory) cells when the two linked cells have their preferred orientation aligned with (or orthogonal to) their relative receptive field center displacements.

This paper demonstrates that much of visual motion coding in the primary visual cortex can be understood from a theory of efficient motion coding in a multiscale representation. The theory predicts that cortical cells can have a spectrum of directional indices, be tuned to different directions of motion, and have spatiotemporally separable or inseparable receptive fields (RF). The predictions also include the following correlations between motion coding and spatial, chromatic, and stereo codings: the preferred speed is greater when the cell receptive field size is larger, the color channel prefers lower speed than the luminance channel, and both the optimal speeds and the preferred directions of motion can be different for inputs from different eyes to the same neuron. These predictions agree with experimental observations. In addition, this theory makes predictions that have not been experimentally investigated systematically and provides a testing ground for an efficient multiscale coding framework. These predictions are as follows: (1) if nearby cortical cells of a given preferred orientation and scale prefer opposite directions of motion and have a quadrature RF phase relationship with each other, then they will have the same directional index, (2) a single neuron can have different optimal motion speeds for opposite motion directions of monocular stimuli, and (3) a neuron's ocular dominance may change with motion direction if the neuron prefers opposite directions for inputs from different eyes.

We explore the hypothesis that linear cortical neurons are concerned with building a particular type of representation of the visual world—one that not only preserves the information and the efficiency achieved by the retina, but in addition preserves spatial relationships in the input—both in the plane of vision and in the depth dimension. Focusing on the linear cortical cells, we classify all transforms having these properties. They are given by representations of the scaling and translation group and turn out to be labeled by rational numbers ‘( p + q )/ p ’ ( p, q integers). Any given ( p, q ) predicts a set of receptive fields that comes at different spatial locations and scales (sizes) with a bandwidth of log 2 [( p + q )/ p ] octaves and, most interestingly, with a diversity of ‘ q ’ cell varieties. The bandwidth affects the trade-off between preservation of planar and depth relations and, we think, should be selected to match structures in natural scenes. For bandwidths between 1 and 2 octaves, which are the ones we feel provide the best matching, we find for each scale a minimum of two distinct cell types that reside next to each other and in phase quadrature, that is, differ by 90° in the phases of their receptive fields, as are found in the cortex, they resemble the “even-symmetric” and “odd-symmetric” simple cells in special cases. An interesting consequence of the representations presented here is that the pattern of activation in the cells in response to a translation or scaling of an object remains the same but merely shifts its locus from one group of cells to another. This work also provides a new understanding of color coding changes from the retina to the cortex.

A previously proposed theory of visual processing, based on redundancy reduction, is used to derive the retinal transfer function including color. The predicted kernels show the nontrivial mixing of space-time with color coding observed in experiments. The differences in color-coding between species are found to be due to differences among the chromatic autocorrelators for natural scenes in different environments.