Maintaining a coherent percept of the visual scene while eye position continuously changes requires that saccades be accompanied by remapping of the visual environment. We studied saccadic remapping in patients with unilateral lesions in the intraparietal sulcus and healthy controls, using inhibition of return (IOR)—an inhibitory tag that enables efficient visual search. In healthy controls, IOR was found at both retinal and environmental locations of the cue, indicating that the inhibitory tag had been remapped into environmental coordinates. In contrast, right parietal patients demonstrated IOR only at the retinal location of the cue, indicating that the intraparietal sulcus is involved in remapping of the environment after eye movements to afford a stable, environmentally based reference frame. Note that patients did not show environmental IOR in either visual field. These results also suggest that this region may be the neural substrate for encoding inhibitory spatial tags in an environmentally based reference frame.