An impairment of attentional control in the face of threat-related distracters is well established for high-anxious individuals. Beyond that, it has been hypothesized that high trait anxiety more generally impairs the neural efficiency of cognitive processes requiring attentional control—even in the absence of threat-related stimuli. Here, we use fMRI to show that trait anxiety indeed modulates brain activation and functional connectivities between task-relevant brain regions in an affectively neutral Stroop task. In high-anxious individuals, dorsolateral pFC showed stronger task-related activation and reduced coupling with posterior lateral frontal regions, dorsal ACC, and a word-sensitive area in the left fusiform gyrus. These results support the assumption that a general (i.e., not threat-specific) impairment of attentional control leads to reduced neural processing efficiency in anxious individuals. The increased dorsolateral pFC activation is interpreted as an attempt to compensate for suboptimal connectivity within the cortical network subserving task performance.