When we refer to an object or concept by its name, activation of semantic and categorical information is necessary to retrieve the correct lexical representation. Whereas in neurotypical participants it is well established that semantic context can interfere with or facilitate lexical retrieval, these effects are much less studied in people with lesions to the language network and impairment at different steps of lexical-semantic processing. Here, we applied a novel picture naming paradigm, where multiple categorically related and unrelated words were presented as distractors before a to-be-named target picture. Using eye tracking, we investigated preferential fixation on the cohort members versus nonmembers. Thereby, we can judge the impact of explicit acknowledgment of the category and its effect on semantic interference. We found that, in contrast to neurotypical participants [van Scherpenberg, C., Abdel Rahman, R., & Obrig, H. A novel multiword paradigm for investigating semantic context effects in language production. PLoS One, 15, e0230439, 2020], participants suffering from mild to moderate aphasia did not show a fixation preference on category members but still showed a large interference effect of ∼35 msec, confirming the implicit mechanism of categorical interference. However, preferential fixation on the categorically related cohort words correlated with clinical tests regarding nonverbal semantic abilities and integrity of the anterior temporal lobe. This highlights the role of supramodal semantics for explicit recognition of a semantic category, while semantic interference is triggered if the threshold of lexical cohort activation is reached. Confirming psycholinguistic evidence, the demonstration of a large and persistent interference effect through implicit lexico-semantic activation is important to understand deficits in people with a lesion in thelanguage network, potentially relevant for individualized intervention aiming at improving naming skills.