Prenatal alcohol exposure (PAE) is associated with various neurological, behavioral and cognitive deficits, including reading and language. Previous studies have demonstrated altered white matter in children and adolescents with PAE and associations with reading and language performance in children aged 3 years and older. However, little research has focused on the toddler years, despite this being a critical period for behavioral and neural development. We aimed to determine associations between structural brain connectivity and early language skills in toddlers, in the context of PAE. Eighty-eight toddlers (2–3 yr, 56 males), 23 of whom had PAE, underwent a diffusion MRI scan in Cape Town, South Africa, with language skills assessed using the Expressive and Receptive Communication subtests from the Bayley Scales of Infant and Toddler Development, Third Edition (BSID-III). Diffusion scans were preprocessed to create a structural network of regions associated with language skills using graph theory analysis. Linear regression models were used to examine moderation effects of PAE on structural network properties and language skills. Toddlers with PAE had higher structural connectivity in language networks than unexposed children. PAE moderated the relationship between structural network properties and Expressive Communication scores. None of the effects survived correction for multiple comparisons. Our findings show weak moderation effects of PAE on structural language network properties and language skills. Our study sheds light on the structural connectivity correlates of early language skills in an understudied population during a critical neurodevelopmental period, laying the foundation for future research.