Recent physiological experiments have shown that the responses of many neurons in V1 and V3a are modulated by the direction of gaze. We have developed a neural network model of the hierarchy of maps in visual cortex to explore the hypothesis that visual features are encoded in egocentric (spatio-topic) coordinates at early stages of visual processing. Most psychophysical studies that have attempted to examine this question have concluded that features are represented in retinal coordinates, but the interpretation of these experiments does not preclude the type of retinospatiotopic representation that is embodied in our model. The model also explains why electrical stimulation experiments in visual cortex cannot distinguish between retinal and retinospatiotopic coordinates in the early stages of visual processing. Psychophysical predictions are made for testing the existence of retinospatiotopic representations.