Word frequency is a powerful predictor of language processing efficiency in healthy individuals and in computational models. Puzzlingly, frequency effects are often absent in stroke aphasia, challenging the assumption that word frequency influences the behavior of any computational system. To address this conundrum, we investigated divergent effects of frequency in two comprehension-impaired patient groups. Patients with semantic dementia have degraded conceptual knowledge as a consequence of anterior temporal lobe atrophy and show strong frequency effects. Patients with multimodal semantic impairments following stroke (semantic aphasia [SA]), in contrast, show little or no frequency effect. Their deficits arise from impaired control processes that bias activation toward task-relevant aspects of knowledge. We hypothesized that high-frequency words exert greater demands on cognitive control because they are more semantically diverse—they tend to appear in a broader range of linguistic contexts and have more variable meanings. Using latent semantic analysis, we developed a new measure of semantic diversity that reflected the variability of a word's meaning across different context. Frequency, but not diversity, was a significant predictor of comprehension in semantic dementia, whereas diversity was the best predictor of performance in SA. Most importantly, SA patients did show typical frequency effects but only when the influence of diversity was taken into account. These results are consistent with the view that higher-frequency words place higher demands on control processes, so that when control processes are damaged the intrinsic processing advantages associated with higher-frequency words are masked.