Seeing or hearing manual actions activates the mirror neuron system, that is, specialized neurons within motor areas which fire when an action is performed but also when it is passively perceived. Using TMS, it was shown that motor cortex of typically developed subjects becomes facilitated not only from seeing others' actions, but also from merely hearing action-related sounds. In the present study, TMS was used for the first time to explore the “auditory” and “visual” responsiveness of motor cortex in individuals with congenital blindness or deafness. TMS was applied over left primary motor cortex (M1) to measure cortico-motor facilitation while subjects passively perceived manual actions (either visually or aurally). Although largely unexpected, congenitally blind or deaf subjects displayed substantially lower resonant motor facilitation upon action perception compared to seeing/hearing control subjects. Moreover, muscle-specific changes in cortico-motor excitability within M1 appeared to be absent in individuals with profound blindness or deafness. Overall, these findings strongly argue against the hypothesis that an increased reliance on the remaining sensory modality in blind or deaf subjects is accompanied by an increased responsiveness of the “auditory” or “visual” perceptual–motor “mirror” system, respectively. Moreover, the apparent lack of resonant motor facilitation for the blind and deaf subjects may challenge the hypothesis of a unitary mirror system underlying human action recognition and may suggest that action perception in blind and deaf subjects engages a mode of action processing that is different from the human action recognition system recruited in typically developed subjects.