It is generally believed that the visual system is adapted to the statistics of the visual world. Measuring and understanding these statistics require precise knowledge of the structure of the signals reaching fovea during image scanning. However, despite the fact that eye movements cause retinal stimulation to change several times in a second, it is implicitly assumed that images are sampled uniformly during natural viewing. By analyzing the eye movements of three rhesus monkeys freely viewing natural scenes, we report here significant anisotropy in stimulus statistics at the center of gaze. We find that fixation on an image patch is more likely to be followed by a saccade to a nearby patch of similar orientation structure or by a saccade to a more distant patch of largely dissimilar orientation structure. Furthermore, we show that orientation-selective neurons in the primary visual cortex (V1) can take advantage of eye movement statistics to selectively improve their discrimination performance.