Recent studies indicate that covert mental activities, such as simulating a motor action and imagining the shape of an object, involve shared neural representations with actual motor performance and with visual perception, respectively. Here we investigate the performance, by normal individual and subjects with a selective impairment in either motor or visual imagery, of an imagery task involving a mental rotation. The task involved imagining a hand in a particular orientation in space and making a subsequent laterality judgement. A simple change in the phrasing of the imagery instructions (first-person or third-person imagery) and in actual hand posture (holding the hands on the lap or in the back) had a strong impact on response time (RT) in normal subjects, and on response accuracy in brain-damaged subjects. The pattern of results indicates that the activation of covert motor and visual processes during mental imagery depends on both top-down and bottom-up factors, and highlights the distinct but complementary contribution of covert motor and visual processes during mental rotation.