Insufficient sleep is a known trigger of anxiety. Nevertheless, not everyone experiences these effects to the same extent. One determining factor is sex, wherein women experience a greater anxiogenic impact in response to sleep loss than men. However, the underlying brain mechanism(s) governing this sleep-loss-induced anxiety increase, including the markedly different reaction in women and men, is unclear. Here, we tested the hypothesis that structural brain morphology in a discrete network of emotion-relevant regions represents one such explanatory factor. Healthy participants were assessed across sleep-rested and sleep-deprived conditions, with brain structure quantified using gray matter volume measures. Sleep loss triggered greater levels of anxiety in women compared with men. Reduced gray matter volume in the anterior insula and lateral orbitofrontal cortex predicted the anxiogenic impact of sleep loss in women, yet predicted resilience in men, and did so with high discrimination accuracy. In contrast, gray matter volume in ventromedial prefrontal cortex predicted the anxiogenic impact of sleep loss in both men and women. Structural human brain morphology therefore appears to represent one mechanistic pathway (and possible biomarker) determining anxiety vulnerability to sleep loss—a discovery that may help explain the higher prevalence of sleep disruption and anxiety in women.