Distributed brain systems contribute to representation of semantic knowledge. Whether sensory and motor systems of the brain are causally involved in representing conceptual knowledge is an especially controversial question. Here, we tested 57 chronic left-hemisphere stroke patients using a semantic similarity judgment task consisting of manipulable and nonmanipulable nouns. Three complementary methods were used to assess the neuroanatomical correlates of semantic processing: voxel-based lesion–symptom mapping, resting-state functional connectivity, and gray matter fractional anisotropy. The three measures provided converging evidence that injury to the brain networks required for action observation, execution, planning, and visuomotor coordination are associated with specific deficits in manipulable noun comprehension relative to nonmanipulable items. Damage or disrupted connectivity of areas such as the middle posterior temporal gyrus, anterior inferior parietal lobe, and premotor cortex was related specifically to the impairment of manipulable noun comprehension. These results suggest that praxis brain networks contribute especially to the comprehension of manipulable object nouns.